Colorless Directionless Contentionless ROADM Networks

Introduction:

Often when we are looking at the DWDM networking we come across the terminologies of CDC. The full form of CDC is Colorless-Directionless-Contentionless. However, transmission engineers who are relatively new to the industry and telecom engineers in general who are exploring the world of the DWDM transmission do not really comprehend the CDC in the essence that it is.  

Few doubts that come in mind are.

DWDM is always colored, so what is colorless?

What is directionless, when we know that traffic is essentially directional?

Last but not the least, what is this term contentionless?

When I was a rookie in this field I also had the same questions and I would like to properly address that for people who are still trying to explore this idea of CDC. 

So let us do step by step dissection of this mystical world of ROADM networks.

ROADM = Reconfigurable Add Drop Multiplexer

 Exploring the ROADM

Before we dive deep in to the CDC level we need to understand what is ROADM. ROADM stands for reconfigurable add drop multiplexer. Essentially in a DWDM network this is an advance version of OADM where you can actually program the channels and frequencies that you want to add and drop in a particular location.

In the figure we are seeing a 3degree ROADM configuration in its most simple arrangement. There are three ROADMs in an optical junction points and these are connected to each other by means of express ports.  The configuration is made such that Channel 21 comes from degree 1 and is dropped over there and the same channel 21 is rerouted as an add towards degree 3.

Similarly from Degree 2 there is a channel 22 coming and dropping and this is added and rerouted to Degree-1. 

General 3 Degree ROADM Site Configuration

This is a very simple arrangement of the ROADM and the optical cross connects are created in the ROADM in the similar fashion. 

While everything seems very simple there are certain operational challenges to this configuration that needs to be addressed. 

Suppose the channel 21 now needs to be nor rerouted through degree 3 but through degree 2 then there needs to be physical presence of an engineer on the site to change the ports. 

As all the channels drop through a Mux-Demux port there will be need to physically be present on the site and change the port allocation. 

Similarly if the same Channel 21 needs to be rerouted to say Deg-3 and Deg2 this will not be possible because the port on Deg-1 for channel 21 has already been reused. Thus this will prevent any sort of channel reusability under such optical junction points. 

What are the things that we are missing out over here?

1. We are always mandated to have a physical presence of an engineer on site in order to do manipulation for port add and drops. 

2. It is not possible to reuse the channels in case we want to do that for efficiency. 

3. The flexibility is less in such kind of configurations. 

Directionless Configuration

In order to mitigate the some of the drawbacks of the simple ROADM configuration we have something called as the directionless configuration. The directionless configuration allows the a particular channel to be rerouted or reprogrammed to any direction across the degrees without the presence of an engineer on site. Let us explore how the directionless configuration actually looks like. 

Directionless Configuration

The figure here shows the directionless configuration. Here we are solving one problem and that is not to invest any physical resources to route the channel from one degree to another. As we can see there is a ROADM termed as ROADM-D. This is connected to a Mux/Demux that is having one input of Channel 21. Because we also have the ROADM-D we are able to re-route the channel 21 without any presence of a person on site to change the ports. Hence the channel routing is now a bit more centralized. 

As you can see over here that in order to manipulate the direction for Channel 21 add and drop there only needs to be manipulation in the optical cross connects in the ROADM. This can be done centrally. 

The directionless configuration works well but then it has its own drawbacks for which it is now become more of an obsolete configuration. 

Which problems are not solved by the Directionless Configuration? 

As I mentioned above that there are some problems that are not solved by the directionless configuration eventhough the manipulation of directions can be done by centralized routing. Let us explore these problems over here. 

1. While it is clear that channel 21 is sorted, if the requirement comes to change the channel of the path say the same transceiver now needs to run on channel 22 then in order to make the changes there needs to be a physical shift on the mux demux port on site from channel 21 to 22. 

2. We are not seeing the benefit of a frequency reusage or a channel reusage over here. In the junction point if I have to send Channel 21 in all the directions simultaneously then I will not be able to do so as my Mux-Demux will have only one port of Channel 21 and not multiple ports of the same channel. 

Colorless-Directionless (CD) configuration:

We saw how partially we could mitigate the physical presence of engineers on site with the directionless approach. However, this was only solving the problem to a small extent. Networks have much bigger problems and there will be more number of junction sites in the network that will need manipulations of channels more often than not. In order to mitigate some of these aspects we have the Colorless and Directionless approach that is taken. 

In the CD configuration we use a ROADM on the site that is apart from the degree and this is called as the collector ROADM.  The collector ROADM has a combination of Degree ports that connect to the degree ROADMs and client ports that can directly connect to a tunable client in this case.

Colorless-Directionless Configuration

The best part of the client port is the fact that they can be tuned centrally to any channels/wavelengths that we want. So if a particular tunable interface decides to change the wavelength of transmission from say channel 21 to channel 22 all we need to do is to manipulate the client port wavelength configuration centrally and the optical cross connect centrally.  

A point to be noted over here is that two of the client  port of the collector ROADM cannot be of the same frequency of wavelength. 

As we can see in the figure that now since the client port is tunable we can route any color to any direction without having any physical presence on the site. 

To understand this simply let us take an example.  Suppose port 1 of the client of collector ROADM-C is connected to a tunable interface of say channel 21. Now due to the fact that the collector ROADM is having properties of optical cross connects we can divert this channel to any direction or any degrees.  This is purely the directionless principle. Now say the user decides to change the wavelength of the interface from channel 21 to channel 22. In the case of Directionless we would have to move ports on the Mux-Demux and send a physical engineer, however in the case of CD we just need to change the color of the port connected to channel 22 (Assuming channel 22 is not used in any other ports of the collector). This prevents any physical presence of the person on the site. 

What are we not able to achieve from the CD Configuration?

For the medium sized meshed networks CD configuration solves most of the problems of centralized control of optical cross connects and channel re-routing. However, there is one case, which is apparent in the case of highly meshed networks that the CD configuration is not able to solve.

 

Suppose we have three interfaces on the site of add and drop and the following case applies.

Ø  Interface 1 wants to send to degree 1 on channel 21

Ø  Interface 2 wants to send to degree 2 on channel 21

Ø  Interface 3 wants to send to degree 3 on channel 21

 

Now we have a problem with the CD configuration. All these three interfaces need to be channel 21 but the collector ROADM can have only one port with channel 21. How are we going to achieve this?

Can multiple collector ROADMs handle this problem? 

A layman way of achieving this would be have multiple collectors. So let us have a look at the figure what happens when we have multiple collectors. In this we have ROADMC1 ROADMC2 and ROADMC3 as three collector ROADMs in the drop site. As impractical as it may seem to be let us have a look at this for understanding. 

Because now we have three collector RAODMs we can reuse the same channels in three collectors and connect our interfaces to different collectors and get them sorted. We will need to create separate optical cross connects over here and this will ensure that the directions are routed appropriately and the frequency reusage is done.  Here there will be central control as well and there will be reusage of frequency in these junction points. So in a way we have been able to solve the problem of the channel reusage which was not mitigated in the CD configuration by adding multiple collectors. 

Channel re-usage using multiple collectors

Although we have solved the problem the question that we need to address is much bigger over here.  And the questions are written as follows.

1.       Is this a practical solution?

2.      What happens when I need to add one more degree to the site?

3.      If I have many such junction points in my network what would be my ROADM investment? 

Well, the answers of these question are tough and definitely the solution is not scalable and commercially feasible let us see how. 

Why the multiple collectors will not work after a particular point?

The multiple collector solution is assuming 1:1 provisioning of collectors per degree ROADM. So a site with say 7 degrees will have 7 collectors in order to achieve the channel reusage function. The channel reusability will then cost the provider a huge amount of cost and heavy footprints considering the network. Also the sites will become bulky and meshy as each collector needs to be connected to all the degree ROADMs that we have. As the number of degrees increase and more number of such sites come into existence this solution becomes awefully expensive and non-viable.

Colorless - Directionless - Contentionless (CDC):

From the CD configuration we have only one drawback to mitigate and that is the channel reusage in all the directions. We tried to do the multiple collector arrangement but then the solution seemed to be non-viable with respect to scale and numbers of a network.  So what we need is a kind of a collector ROADM that is having the internal cascading of many collector ROADMs at the same time. Basically it is an internal ROADM switch that allows to have same channel configuration on different ports ad the same time.

CDC Configuration

This kind of collector ROADM is called the contentionless ROADM.  In the figure that we are seeing we have a collector ROADM which is called as ROADM-CN. This is a kind of contentionless ROADM.  Here we can see we can have many client interfaces with the same frequency aligned.

While it is easier said than done the contentionless ROADM is coming with a lot of complexities inside. Technically a Contentionless ROADM collector is a MxN ROADM where there are N number of collectors with M number of degrees. So the ROADM internally is a collection of many ROADMs and is a kind of a switch. However, the contentionless configuration actually enables the ease of operation to a great degree especially in the field of dynamically changing networks. 

Comparison of different kind of configurations

Now that we have an idea of ROADM sites, CD and CDC, let us make a small analysis in terms of a table as to what configuration is to be used where.

Aspect	Normal ROADM	CD Configuration	CDC Configuration
Cost	$	$$	$$$$
Complexity	Simple	Moderate	Complex
Flexibility	Limited	No frequency reusage	Fully Flexible
Type of sites	Can be used for 2D or in some cases even 3D sites which have less channel re-route	Recommended for 3D – 5D sites with moderate traffic and channels	Recommended for a high mesh 5D onwards with higher possibility of traffic rerouting.
Centralized management	Physical presence needed for change of channels or ports	Can be centrally configured	Can be centrally configured.
GMPLS*	Difficult to achieve	Supported	Supported

 

Summary: 

In order to understand what configuration is suitable to your network it is very essential to make a proper planning of wavelengths. Proper wavelength planning and keeping room for the future makes it very easy to ascertain what technology to go for and in which node.

Most of the time it is at the planning stage where we need to decide if we go for the CD configuration or for the CDC configuration. Depending on traffic, flexibility and multiple add-drop points we need to make this decision. 

Cheers

Kalyan 

Understanding UNI (User Network Interface)

Dear Friends, 

The year 2018 is coming to a close and this is the right time to write something on request of somebody. I had provided an article explaining Provider Bridge and Provider Edge concept where I touch-based upon the aspects of bridging and tunneling. Well I believe the article needed a prequel and that was to understand the different interface types that we have in the data networking. 

The networking that we do for data services are very dynamic and there can be several combinations of port-types and service types with domains realizing one important aspect and that is to carry the traffic from one point to another with reliability, security and prioritization. 

There are two kinds of interfaces that we have

UNI ----- User Network Interface

NNI ------ Network Network Interface. 

Today my post will concentrate only on the UNI aspect as I do not want to make this a very lengthy post. People may get grossly bored to read a lengthy post as for one I have noticed, the new generation lacks some kind of patience. 

What is UNI? 

In a very short and crisp definition we can say that UNI is the interface that connects a customer to the network. UNI port is an entity of the provider, however it interacts with a customer port or an interface. Please note that I have mentioned the terms port and interface separately. The reason being that a particular physical port may carry several logical interfaces that may be of UNI nature. 

Figure-1 Representation of a UNI

The figure above visualizes the definition of UNI. 

However, the concept of UNI is deep and needs more elaboration. For this we need to understand the concept of VLANs. A VLAN is a concept where we can break the broadcast domains to different smaller domains without involving the routing. You must be knowing that a switch breaks collision domains and the router breaks broadcast domains. However, what can be done in order to break a broadcast domain within a switching network without involving routers is to use VLAN. 

The VLAN gives a kind of identification to the packet or frame that is coming to the switching device as to which service that is should follow. VLANs are of 4 bytes and that are appended to a standard 802.3 frame. 

We see the structure of VLAN below. Disected. 

Figure-2 Dissection of the structure of VLAN

The VLAN is appended in the frame and it is providing the identification of the frame to follow a service. Here the main important thing is to see how the VLAN is appended in the frame. The frame can be untagged (without a Vlan) or it can be tagged (with a VLAN ID). Basis on this we have the two different types of interfaces of UNI. 

Figure-3: Types of UNI port

The types of UNI are the the basis of how the VLAN is treated in the service and the traffic is mapped. This is the thing that we will see in detail below. 

1. Access UNI port:

Imagine a situation where there are three customers who are all sending untagged frames. However the WAN which is carrying the traffic is a common WAN. How are we going to segregate this traffic. There is one way to do it by leaving it on the Mac Learning and bridging concept. The concept over here is that every device will have a different MAC and based on the MAC filter there will be communication and we will have one common LAN. However, this aspect has some disadvantages. 

Figure-4: Situation where three customers carry different traffic

> What happens in the unlearned state? 

> How will the traffic be treated? 

> What happens if one customer is sending a broadcast? This will affect the performance of all other customers. 

However, we cannot impose the condition on the customer to tag these frames. If we do that we are putting conditions on the customer and this way we will have to put similar conditions on all other customers, which is not a good idea. 

Here the thing is to have an access UNI port. 

The access UNI port accepts an untagged port on the LAN and then appends the VLAN on the frame of customer and then forwards it to the WAN. 

The flow is explained in the figure below. 

Figure 5: How the access VLAN UNI works

Here you see that the customer 1 sends the untagged frame and then it reaches the access port. Here the access port adds a VLAN ID to the frame and this travels through the WAN. On the other side when the frame is coming out the VLAN is matched and stripped and the original frame is received by the end. 

Points to note: 

1. Access VLAN ports in the UNI append VLAN in the ingress and match and strip the VLAN in teh egress. 

2. This action is like attaching a unique envelope to the untagged frame and taking it out on the other end without distorting any information. 

In our case we can have the following scenario for our three customers. 

Figure-6: How the three customers will be segregated

Here we see that every customer is connected to a different access port and so we put the access VLAN accordingly to identify the customer. Also note that the customer 2 may have a different drop point than customer 1 so this has to be organized in that way. 

2. Trunk Port: 

Once we understand the Access VLAN port functioning understanding trunk port is very easy. The trunk port in this case is our WAN. The WAN port is not adding or stripping any VLAN however it does a validation of VLAN that it has to pass. In our case we will put a validation that the trunk link should only pass VLAN 100, 200 and 300. 

The algorithm is the trunk port accepts frames that are already tagged and then matches them with the validation. Based on that it forwards. 

So friends, this is about the UNI port. We shall see the functioning of the NNI port in a later post. Understanding the UNI interface is very important in order to do selection of interfaces in data traffic planning. Therefore, be careful in deciding the same. 

Till then have a fantastic New Years' Eve.... 

Cheers and a Happy New Year, 

Kalyan 

RSTP/MSTP Part –V How does switching take Place in RSTP and MSTP?

 Dear friends of the Telecom Fraternity,

I was writing a series of RSTP in 2013 and possibly, there are many things to catch up. I would like to divert your attention to this blog that was the fourth part of my RSTP series.

RSTP-MSTP part IV

It is from this blog we continue our journey ahead to this vast topic called RSTP. Generally, people believe that RSTP is for switching, but I had clarified before that this is a Loop avoidance Mechanism. RSTP makes the switching of services in case of Failure more salient because there is a transition of paths.

In this section, we will concentrate on a simple architecture and that is RSTP Ring. As explained before we optimize the ring with a selection of optimal blocking port in the service.

Figure-1: RSTP ring topology example

Now in this ring let us understand that there is a service from the root Bridge to N-2 with SVLAN 200 and another service from Root Bridge to N-3 with SVLAN 300. This can be shown in the figure below.

Figure-2 Service configuration in the RSTP domain

To understand the concept of service switching let us understand a failure scenario. So in our case we imagine that the link between N-1 and N-2 has failed. Definitely, the service will be routed from another direction. However, in our case we will see this in a step-by-step basis.  Please remember that a service switching in RSTP is not as simple as it looks. Because there are no predefined main and protection paths like you have in TDM or MPLS. Here the entire switching of the service from one direction to another is working broadly on two principles.

1. RSTP Re-convergence
2. Mac Learning renewal at all switching points. 

We will see all these happening but you have to remember that all these happen very instantly. Typically the re-routing time of the services in the event of failure in RSTP is 200ms. Note this is not 50ms and that is why it is not recommended to run voice services or any real time services involving voice in the RSTP network. This is the reason why RSTP is regarded to be a Non-Carrier grade method. However, for a normal http service or a https service it does not matter as there is a TCP retransmission always happening and so RSTP works very well.

So we see the figure below to understand the failure scenario. 

Figure -3 Failure Occurrence in the link

 After this failure has occurred the first thing that happens is that N-2 does not get the BPDU packets from the N-1 (its designated bridge). So a Root port transition takes place and the link that is between N-2 and N-3 becomes a forwarding link. One special thing to remember over here is that in the N-2 to N-3 link, which is the blocking one of the ports between N-2 and N-3 will be the discarding port. RSTP in this case will not have two discarding port. So we have two cases over here. 

1. In case the discarding port is N-2 and there is a Root port failure on the N-2 then the Topology Change request of RSTP will immediately come into action and N-2 discarding port will turn to forwarding. 

2. In case the discarding port is N-3 then the TC message is communicated from N-2 to N-3  and N-3 changes the port from discarding to forwarding. 

Here the critical part is the topology change notification message that is carried by the BPDU and this always happen after a minimum hold-off time which is 200ms. The difference between STP and RSTP is over here. In STP there is a wait of three hello intervals which makes the initiation of TCN happens delayed. This results to a switching time that is more than 3 seconds. However, when we talk about RSTP (Rapid Spanning Tree Protocol) the TCN notification are subject to port transitions in any switch. Therefore N-2 and N-1 will both have transition changes and will initiate TCN immediately after the expiry of the hold-off timer. 

Now after the TCN is communicated the new state of RSTP will be as the figure below. Please note we have not yet considered how the service is being rerouted, we are still seeing the first part of the switching and that is RSTP re-convergence. 

Figure-4: RSTP topology change

Now the topology change has occurred. But what is remaining still is the re-routing of the service. I told this earlier that RSTP does not have a pre-defined protection and main path so the service re-routing is happening plainly on the basis of Mac Learning. RSTP is a scheme that is used in the case of Provider Bridge networks. To understand what is a provider Bridge network please refer to my earlier blog post in the permalink given below. 

Provider Bridge and Provider Edge Networks

In this blog post you will find clearly how the traffic moves in the provider bridge networks. So as this is a provider bridge we see that for the service affected, which is the service with SVLAN = 200, the mac learning has been done in the following manner of (Root Bridge - N1-N2). Now the path between N1-N2 has failed and there has to be a sort of notification to the root bridge to send the traffic via the other path. 

The self healing way of such a scenario is that the traffic stops and we wait for the expiry of the aging time of the mac table. The aging time of the mac table is a user configurable parameter, however the minimum value is 10 seconds. So technically if such a failure has occurred the service rerouting should take place after 10 seconds (aging time). 

Phew!!!!!! This is long. So the developers of RSTP thought of another approach and this was to flush the mac-table of every bridge that is involved in the RSTP domain. Therefore, the TCN also sends a command to flush the mac-table of all the bridges involved in a particular RSTP domain. 

Something like the figure below. 

Figure-5 Mac-Flush happening n all the nodes involved in RSTP

Here we see that all the points of the RSTP domain are flushed. 

Now it is anybody's guess what will happen after the flush of the FDB occurs. There will be relearning of mac address for the services. In this case the service with SVLAN 300 will have the same path of mac learning but the service with SVLAN-200 will not have the same path of mac learning. Now N-2 which is the destination point will learn the mac via N-3 and not N-1 and this will make N-3 the Designated Bridge for N-2 and the service will now be re-routed. 

Figure-6 Final Re-routing of the service

So here we see a complete step-by-step process of re-routing of the services. Tough but not so tough to understand. 

In this case please note, now the bandwidth distribution in the ring is not optimized and there can be a scene of congestion between the link of RB to N-4 and N-4 to N-3. Under such scenario the QoS will come to play and the RSTP domain has to be properly traffic engineered. 

What happens when the link restores?

Now we saw about the failure the restoration of the link is also treated like a seperate failure in this case. RSTP recognizes only topology changes and now with the link repaired there is another topology change. A similar TCN will pass through the ring and there will be re-convergence and the block port will now be as per before. The TCN will flush the Mac tables of all the bridges and this will lead to service re-routing again. 

So friends, pretty long blog post, but cannot help. In order to understand the switching part there has to be more description, which I have tried to bring in. But, we have just touched the tip of the ice-berg. There are lot many things happening beneath the skin of the water and to dissect it threadbare it would need another 50 blog posts. We will see the operational aspect of RSTP in multiple topology scenarios as well and dual homing cases. 

Till then 

See you.

Regards, 

Kalyan 

Keep thinking!!!! Keep Reading!!!! Keep Evolving!!!!

How to do things differently in Telecom?????

I remember the last blog-post that I had written. Telcos do not need engineers. This invited a lot of censure and on the other side showed the mirror of reality to many telco veterans and contemporaries. I had ended the blog by saying what is needed then? Well here I am to make an effort to answer these things.

Change is a continuous process and not that what will be valid today will be valid after five years. Technology changes as we speak and so does the methods to manage technology in a great deal.  So let us understand what are the new things that we should adapt to. I believe the members of the telecom fraternity should have reptilian tendencies these days in order to survive and be a positive contributor of this industry for a long time. We will classify these things as What is In and What is out?

1.    IN: “Work with me”  OUT: “Work under me”

The most critical point of management is the degree of authority. Authority is a responsibility and not just a right of seniority. However when authority becomes dogmatic then there is definitely a problem in the management style. There is an old saying “People join companies, people leave bosses.”; The saying unfortunately is true till date. Instead of this, it is required to behold a sense of "espirit-de-corps" that should prevail across the ranks. Vertical conflicts are a thing of past. Intellectual conflicts are the new things to watch out for. Ideas are nobody’s monopoly and definitely not a privilege of seniority. Therefore it is always necessary to take your juniors with you. Working with has a lot of advantages than making work under.

2.    IN: “Opinions of Juniors” OUT: “Approvals from Seniors”

Another very rebellious statement, but true! Telecom is an industry that is always shaped by fresh ideas. Fresh ideas cannot be expected from veterans of 10-20 years old, unless there is an unbelievable streak of innovation, which is rare in the Indian context. Any fresher that comes to your company is an asset. Not because you can make him/her laboriously, toil over mundane works but harvest new ideas from him/her. A fresher has a lot of imagination because he/she is undaunted by experiential difficulties and obstructions. A fresher or your junior is just scared to speak because of the weight of position that you throw to him/her knowingly or unknowingly. They are not outspoken sometimes because they feel they will be judged. An approval is a formality but an opinion can lead to a breakthrough and this can only come from the new generation. Experiences are good but they are good to guide the new generation and make them aware of the mistakes that had been made in past. Not to scare them with positional authority. Approvals are needed, only for leaves and protocols.

3.    IN: “Affinity for Automation” OUT: “Justifying the Manual Cause”

Times have changed and so have the approaches to a problem. Gone are the days were we needed a lot of manual staff to manage things that are routine. These have to be automated so that the expectations from human resources can be raised. Contrary to the feeling that automation is a killer of human opportunities, I believe it is an opportunity to raise the bar of human involvement in the field of telecom from being grossly mundane to being innovative and creative. Today’s telecom demands innovation and creativity. This has to come from every levels of the organization. For being creative the human mind needs something which is essential and that is time to think. Of course, there is a corollary which says that work keeps humans busy, but definitely mundane work makes them morose and more like zombies who after some years become of no use. Every human asset of the telecom company has to be a generator of creative and productive idea and that is why automation is required to shun the monotony that is prevalent in the industry.

4.    IN: “Leaders”    OUT: “Bosses”

Explained this before and now again explaining it. It has been long this discussion of leaders vs bosses have taken place and the toying of this idea has gained a lot of popularity. Now is the time to implement and evaluate this idea. A leader is a person who gives direction and takes the initiative to walk the talk. A boss on the other side is a generator of instructions and orders. Industry definitely needs leaders. In war-time you need leaders who can take you safe from the cross-fires and inflict minimum casualties rather than a boss who is not on the field and just instructing from a safe haven.

5.    IN: “Evaluating Technical Edge”   OUT: “Compliance”

This is especially to the people who are evaluating vendors or technology for their implementation. Most of the time they are obsessed about compliance overlooking the technical edge. This leads to a lower shelf-life of the network. There will be solutions that are unique in the market and are innovative, processes have to be bypassed for absorbing those. Regular processes make regular companies. To be a company that stands out there has to unconventionality in the processes. Because unconventional evaluation makes unconventional companies. The idea of providing internet through hot-air balloons was not evolved from a telco. Because a Telco always focuses of compliance, obliterating anything that is unique and outstanding. This definitely has to change.

6.    IN: “Process Management”  OUT: “People Management”

This is the generation of the millennials. They are focused. They know what they want and how they want it. They are coming inbuilt with a management guidance system in their personality. They are intrepid and they are also adventurous. They are passionate at times and at times they are totally detached. A traditional people management approach is futile in managing this generation of people. There is a need to accept the fact that this is the generation with most of the ideas and that too the creative ones. They are a treasure and a treasure is not people managed but protected and preserved. So the requirement is to have a process and a management of the process that lets the company and the industry at large harvest from this amazing pools of knowledge.

So there are a lot of things to change. These changes will come. Today it may seem to be a giggle factor or some philosophy in book, but they will certainly come and I am hopeful. I am betting on this change because of a simple philosophy. “If there was somebody who could stop progress and change we would be still living in the caves.”

On that note, I leave you now to decide and comment on this post of mine.

Cheers,

Kalyan

TELCOS DO NOT NEED ENGINEERS!!!!!!

Yes, you heard it right. Telcos do not need engineers anymore. Infact the engineers that the telcos have are not more than a liability. The engineers, heads, managers, technical team….. a mere cost item, precisely the Right hand side of the balance sheet.

Surprised! You ought to be. I, myself am an engineer in this industry for the last 18 odd years and I am saying that. So why such a change? Why such an iconoclastic statement?

The reason is the definition of “Engineer” that the conventional telco understands is way outdated and way under the requirement. An engineer in today’s telco industry is a person who has to tread a line of processes, forgetting creativity and unconventional thinking. There are sets of ground rules that have to be adhered, sometimes breaking those ground rules is not only a flouting of policy but like a criminal offence.

Result! The same old thing repeating again and again. So called engineers become human robots following monotonous orders and doing the same stuff. Working in this industry in the conventional way does not seem interesting anymore and this affects the overall performance and nature of the company. Every technical staff, of the 100% effort that he/she makes, 90% is devoted to saving himself/herself from something. The attitude that shouts out loud, “I AM NOT RESPONSIBLE” is written everywhere in the company and finally we have “NOBODY RESPONSIBLE” in the entire structure.

The thing that happens is that nothing new comes out of the industry and we have same old services again and again without a shade of change. Scary job environment just mandates one thing and that is to keep your job. The best thing over here is when the planning guy feels insecure he does a charade of taking out a tender as if he/she were to design a NASA space shuttle. Tender process goes on for six to eight months and this is the time the planning guy looks out for another job. The outcome is either the planning guy has moved out to a new company or the tender is postponed for another six months.

Operations is funnier. Sometimes in order to see that they have work, they pray for some fault to come, and when it really comes and they are helpless, they just pick up their phones and make calls to the managed service partner, without applying any significant brains on the problem. As if they were hired in the company to just be telephone operators, they just drain the batteries of their phones and their energy bags shouting at the managed service partner and vendor. Once the fault is fixed, a thank you note to the vendor and a gallantry award from the operations manager who is equally scared of his/her job. Finally, a breather for another three months and life goes on.

IN ALL THIS CIRCUS THE BIGGEST CASUALTY IS THE CONCEPT OF “ENGINEERING”…. RIP

It is much better that such mundane activities are now being outsourced to machines through Artificial Intelligence and automation. Automation today can replace most of the monotonous work in the telecom industry and believe me most of the work is monotonous even planning. Artificial intelligence and automation comes as a boon and not bane. At least if not everything, it shows the mirror to the engineers at all level that they have to be “engineers”, because robots can be made by synthetic things as well.

Well after all this acerbity that I have spat out, I am surely expected to be trolled. After all telling the truth comes with its occupational hazards. However, I have spoken out.

So what is needed? What do we need to do in order to come out of this vicious circle?

Will speak about this in the next article.

Cheers,

Kalyan

IoT and Machine Learning

It was the year 1996 when I first brushed through the concept of statistics. I never knew that a boring field like this could very well be useful over making graphs. I always thought statistics to be a tool of a good argument to start with eg when I start a debate on poverty I throw a statistic that as per last survey around 27% of the population is below the poverty line. Never I could imagine that statistics was such a powerful tool until I studied it not as a subject but with more interest some months ago in my second semester of MBA.  Today's premise of my blog actually is more centered around statistics. Yes Mean, Median, Variance, Standard Deviation, Co-relation etc etc etc. I would like to simplify another aspect of IoT and that is Machine learning. 

Before I do that let me give you a hypothetical situation. 

Suppose you have a house that has a toaster, an alarm on your mobile, a microwave, a refrigerator, a dishwasher, a washing machine and an air-conditioner.  Now everyday more or less you wake up at 6 when the alarm rings you turn off the air-conditioner, then you go for a shower not before you have started the washing machine and the dishwasher at around say 6:15 AM. Post the shower you come out and you toast two breads in the toaster and have some baked potatoes in the microwave. These event happen around 6:45 AM and by 7:15 AM as you are having your breakfast you turn on your television to see the news in say NDTV.  At 8:00 AM you leave for your office.  More or less your routine remains the same from Monday to Friday and you are usually following this unless some untoward incident happens or it is a weekend or it is a holiday. 

Imagine that all your machines would get this pattern, well close to it with some anomalies and one fine day they themselves do the routine without your intervention. At 6:00 AM the Air-Con turns off on its own, the geyser turns on for the hot water. As you go into the bathroom say at around 6:15 AM the washing machine and the dishwasher turn on on their own to do the chores. As you are out of the bathroom by  6:45 AM you have your breads toasted and potatoes baked in the Microwave (Assuming you have them already in the toaster before) and as you take the plate to the drawing room you have your television turned on to NDTV talking you through the important breaking news of the day. 

Seems very very spooky, actually horrifying and to some extent like a sci-fi movie. Well, this not entirely, but partly is possible by means of Machine Learning. 

What is Machine Learning? 

Experts have talked a lot about this and if I would be adding some more technical details to it, I will not do any good. It would be more like adding a drop of water in an ocean. So let me do the reverse. Let me take a drop from the ocean and explain you how the entire ocean water is like. Machine Learning is an innovative way of making a machine aware of the patterns of events based on past events, timings and trends.  Basically if you turn on a machine every day say at 7:00 AM and turn it off at 7:00 PM and this is continuously fed to the machine as two events with respect to time then the machine would  learn a response some-what like the picture below. 

As you can see this is a pretty flat picture of the machine getting turned on and off. This is a fixed pattern and this can be fed to the machine and one fine day the machine can work on its own to turn on at 7AM and turn off at 7PM.  The information that is fed to the machine is the knowledge that you give it to work as per the inputs automatically and to take a decision of turning off or turning on. In short you make the machine learn about a pattern of it working and taking decision automatically.  The process in which the machine is able to make sense of the data that is fed to it by means of some external source or its own internal operations so as to take a decision of its own about some of its processes is called "Machine Learning". 

The example above is a pretty basic example of machine learning and in a practical world this is not the way it always happens. Also time is not the only dimension on which the decision of turning on a process or turning off a process depends on. There are many dimensions into place. 

Eg a thermostat of an airconditioner responds to the ambient temperature and the expected temperature settings automatically. In this case the AC is learning about the ambient temperature and is adjusting the compressor working as per that constantly and this is an example of machine response to machine learning automatically. Same goes with the refrigerator. A washing machine can take a decision on the amount of water that it needs to take in for washing based on the load of the clothes and this is also an example of machine learning. So you see my friends machine learning is not a very alien subject. It is not a subject that we are not knowing or we are not aware off. It is the same as taking feedback of an output and then adjusting the input as per the expectation and go on automatically. 

So then why so much brouhaha over Machine Learning? Well here is the difference. 

Whatever examples I have given you till now are about machines that are stand alone. A washing machine can take a decision for itself, and it cannot intrude or pass on the message that it has stopped to the microwave or some other machines for them to use the washing machine result as an input. A level of machine learning in this case is about automation of an entire set of machines in a house or in an enterprise as if it is a part of the same process. So machines do not work in Silos but in a collective way. 

Since this is somewhat like a symphony or like an orchestra you can also call this as Machine Orchestration. 

Randomness: 

Needless to say that when so many machines are involved in making the decisions based on many machines that are integrated the pattern will not be a constant pattern all the time. I mean the example that I had given before of your routine would not be same clock to clock every day. You have to be a robot to do that every day. Some day you may wake up at 6 and some day say even at 5:30 or some day at 6:30 and if the entire sequence of machine orchestration is based on your waking up time at 6:00 then things may go wrong. Like when you woke up at 5:30 you may be just going to the bathroom to see the geyser not on and having a cold shower or if you wake up at 6:30 you may see your washing machine started but no clothes inside. Basically randomness in a pattern can create havoc and so anomalies have to be accounted for. That is why machine learning does not depend on any one dimensional variable.  as you can see that if the X axis is the independent variable and the Y axis is the dependent variable then due to randomness of events there cannot be any fixed pattern that could be established. 

This randomness is much higher as and when the time duration is short. 

How to overcome randomness? 

As we discussed above randomness in machine learning is definitely a big problem. We will not be able to establish patterns. So how to remove these randomness so that we have a fully automated process by the machines so that there is virtually no error of them working automatically and there is a full fledged accuracy to the level of 99% that they work on their own. Well there are many ways of doing that. 

1. Introducing more independent variables: In a two dimensional plot we would not be able to show this however if the outcome of a dependent variables depends on many independent variables with conditions of any one or all of them have to be true then the probability of randomness would decrease. Let us take our example that we had taken before about you getting up for the office. Now in this case we do not keep only time as an independent variable. We also have a motion sensor in the house and some CCTV cameras in your house that record your movement. Now we put a condition that when you get up the motion sensor gives an input and looking at the time the AC decides that you have got up and then turns the AC Off and turns the geyser on. However if you are delayed going to the bathroom the temperature of the geyser is lowered so that you have the perfect temperature. It is only when you drop your clothes in the washing machine and you go to the washroom and turn on the shower the washing machine turns on. So in this case the shower is telling the washing machine that you are already under the shower so washing machine can turn on. Washing machine tells this to the dishwasher to turn on. In this case we see that the operation of all the machines are relative to the operations of others. This process goes on to create a kind of a chain reaction process till you are out of the house for the office. 

2. Studying the pattern over the days: Another way of reducing randomness is not to increase the number of variables that are independent but then to study the pattern for a long time and develop a kind of an algorithm inside the machine with some error margins. As the number of samples and inputs become higher the accuracy increases and the margin or error reduces substantially. It is a common rule in statistics that with the increase of the sample size the standard level or error reduces. Sampling error due to randomness comes down. In our case the sample size increases as we see this routine happening daily again and again and again. Every day the machines record something in them and develop a pattern. When they see that the relative error of two consecutive events have become considerably less than what it was before then it develops an algorithm of automation. 

However, nowadays we want immediate result of everything. We live in an age where we believe that 9 women can give birth to a child in one month, well on a lighter note. So option -2 alone is not viable as the entire system would take sometimes a lot of time to just learn the patterns. Also the moment it sees a spike or a variance the learning process would start again till there is normalization of the error margin. This would mean that the machines would more and more number of times just keep on learning and never do the things we want to do them on their own. So we need a combination of option -1 and option-2 to reduce the randomness in the machine learning techniques. 

Machine Orchestration and Integration:

A central process that manages the working of the machines on their own based on the independent variable inputs and respective machine algorithms is called as a machine orchestrator and the process is called machine orchestration.  Look at the picture below to understand more in detail about the same. 

There is a central Machine Orchestrator of the brain that is controlling different machines by giving customary commands for the processes. However the Orchestrator is not self sufficient. It gets inputs from the independent variables. Also the machines interact with each other with their controllers so that each of them can talk to themselves and to the central orchestrator. The custom algorithm that is developed in the machine is communicated to the orchestrator. The custom algorithm, in red is a result of all the independent variables and the orchestrator also has a feedback of the same. So in case there is an anomaly the orchestrator has the intelligence to change the process flow or to even shut the process for a manual override. 

Summary:

So all in all machine learning is a process where the machines learn to become self sufficient in a step by step manner. Just like us humans from the day we are born to as we gradually age learn different things and responses to various situations. Machine learning is the same way in which a machine can be made this way of course with a great help from one of the most effective sciences, statistics and predictive analysis. 

So you see my friends, a time is coming when machines are going to be if not more intelligent in decision making then at least capable of decision making at least for the routine processes. Who knows one fine day they might even give us a competition. 

We will have more topics on this as I keep on exploring the fascinating world of Internet of Things, IoT. 

Till then keep evolving. 

Cheers, 

Kalyan