The Amazon Web Services (AWS) pace of innovation over the past eight years amazes me.  Every year their cloud gets wider and deeper!  The breadth of their public cloud offering reveals a count of over sixty services each with regular innovations of their own. It seems as though every day new services get pushed out and new mash-ups can be created. An example would be the myriad ways services can be jigsawed to each other using SQS and Lambda! All of this change and innovation makes AWS more powerful but it also increases complexity.

Size and complexity combined inflicts cognitive overload in humans. Successful and high-performing organizations are turning to visualization to reduce how this complexity looks without taking away any of the power. A picture is worth a thousands lines of configuration.

In this article we example the top five challenges and what to do about them:

  1. Human Difficulties With Hierarchical Text-based Consoles
  2. Human Difficulties with Communication and Collaboration
  3. The CMDB Can’t Cope!
  4. Hybrid Cloud Is Coming
  5. uture Technologies Will Need a New Kind of Visualization

    1. Human Difficulties with Hierarchical Text-based Consoles

    Humans are visual creatures. It is hard for people to build a mental map from hierarchical pages of text such as the configuration forms on AWS console. The console has Resource Groups to make this easier, including some charts, but more help is needed.

    Just looking at one small part of the AWS Console, the all important Virtual Private Cloud (VPC) section, shows how many elements there are. Could you draw a picture from this? The chances are you might if you can get an expert, but the world is not awash with AWS experts just yet.

    The challenge with text-based systems is in making the connections between the elements, understanding the topology, and in a VPC this is crucial to see how virtual machines are connected to gateways and all the other network elements. If you get your VPCs wrong, then everything above it won’t work properly.

    Drawing this manually from text information is time consuming and error prone and it can’t keep up with changes in configuration or system state.

    A cloud visualization system like Hyperglance converts this text into an attractive live map in your browser. Instead of using the Console, Hyperglance uses the live AWS API as a real-time information feed.

    Human Difficulties with Communication and Collaboration

    It is very unlikely that your cloud account operates in a bubble. Even in the most simple of architectures there is a mixture of network engineering, infrastructure engineering and application development skills.

    people standing around whiteboard

    Each of these three teams need to have the same current state and future state model, and this is hard to put together from their individual jigsaw pieces. It doesn’t have to be this hard if you use visualization, though.

    Picture the scene. A team of people that don’t have one visual map to look at instead try to manually jigsaw a map together on the whiteboard. The team splits up and pulls data from different parts of the AWS console and uses colored pens to draw a manual map on the whiteboard.

    In my experience, if you need more than three pens to draw a whiteboard map then it’s going to be messy.

    Then… someone makes a change in the system. Your smudgy multi-coloured map is out of date. That’s if you detected the change in the AWS console. This is not a professional approach.

    Live 3D models of your AWS cloud in real-time, via the AWS APIs and presented in Hyperglance eliminate this whiteboard challenge. Changes detected and model updated automagically. No jigsaw required!

    3. The CMDB Can’t Cope!

    The five essential characteristics are what makes a cloud, and perhaps chief amongst these is self-service. Gone are the days of everyone filling in a change ticket and eventually one of a small handful of operators makes the change with great ceremony.

    Nowadays, users with different roles such as developers and administrators can make the changes themselves on the AWS console. This breaks the old operational model of (a) update our model of the truth in the CMDB, (b) change reality to match our CMDB.

    If the CMDB doesn’t work anymore in a cloud context, how do you know what’s happening in AWS? Is it a wild west out there? This friction between users and operators is a problem in enterprise use of cloud and has given rise to a market of ‘cloud brokers’ that seek to put central IT control back in between users and cloud, though their success is not guaranteed.

    Don’t waste time comparing CMDB models to reality, then doing gap analysis and working out a resolution plan, instead use the AWS configuration as your CMDB and a live 3D visualization platform like Hyperglance to present your model back to you.

    4. Hybrid Cloud Is Coming

    A standalone public cloud is one thing, but just about everyone in the technology industry is talking about hybrid cloud. Analysts such as Gartner expect more than fifty percent of enterprises to be using hybrid cloud in 2017.

    Hybrid cloud is connecting two or more clouds together and these clouds can be heterogeneous. Connecting different clouds means skills and understanding of one cloud clashes with the skills and understanding of another. The pressure on the people on process to manage two clouds as one will be high. Visualization across the hybrid cloud will be a significant benefit to these teams in understanding the joint topologies and connection points in real time.

    A visualization platform like Hyperglance can be the independent, objective third pair of eyes for the hybrid cloud.

    5. Future Technologies Will Need a New Kind of Visualization

    Startups like the UK’s A Cloud Guru have built their online education and training business without a single server. No infrastructure. Instead they use AWS services such Lambda to codify their business process into a series of connected AWS-managed events. All of these new AWS services have APIs which means they can be visualized.

    Serverless architectures are new but industry leader Adrian Cockroft of Battery Ventures predicted them to be a hot area for the next few years.

    Serverless architectures, such as Docker, will need a new kind of visualization to extract information from their API and model intelligently. Hyperglance will add these features as serverless becomes more mainstream in 2016.