IT Pros Use Cloud Visualization

There’s a problem lurking in the cloud-era IT industry.

Nineteen years since the 20th century and documentation techniques such as runbooks from that era still persist today.

21st century technologies and methodologies such as cloud services, containers, microservices, APIs and DevOps are more dynamic than data centers of old because they are more virtual, ephemeral, lightweight and their pace of innovation is startling.

All this new power comes with new complexities that is causing headaches for IT staff that have to try and keep up with them. Not keeping up means operational problems and missed business opportunities.

How do IT staff manage these new systems with with their tried and tested techniques of Visio for maps, Excel for IPs and password, and Notepad for configuration files?

Part of the answer is using new kinds of configuration management systems, but these text-based systems are complex in their own right and often text-based: unless you’re a specialist, that’s a difficult mix for everyone else.

Operations staff are turning to visualization platforms, like Hyperglance, to give them real-time interactive maps of their cloud in the browser on their laptop.

In this post we look at the changes that are happening in the IT industry that cause these operational headaches, and how to use live visualization of systems to soothe a hot, furrowed brow.

1. The Role of Visualization in 21st Century IT Operations
2. Everyone Loves Runbooks. Not!
3. Manual Configurations Aren’t Secure
4. APIs are Replacing User Interface
5. You Don’t Know What You Don’t Know

1. The Role of Visualization in 21st Century IT Operations

There is an emerging class of visualization technologies to help make sense of today’s complex and changing systems.

A typical enterprise uses more than one cloud service and these are connected, creating a complex system.

The larger the enterprise the more changes in flight, especially in Continuous Delivery cultures that have frequent releases.

The early types of visualization are hangovers from the runbook days.

There are products that will take an AWS configuration and draw a static picture to put in a runbook.

This doesn’t fix the runbook problem of keeping up. The picture isn’t live, it isn’t data rich and it isn’t interactive.

Live Interactive Data-rich Diagrams with Hyperglance

For active operations the best method is to use a new visualization platform such as Hyperglance in the Operations environment. Hyperglance plugs into a growing range of cloud and on-premises services via their APIs and creates beautiful, interactive diagrams available in your browser.

Live interactive diagrams are ideal for the human brain to visualize what is really happening in their cloud. Pictures are better than words. IT is not a solo sport.

When getting a team of people to understand the same thing, having them look at the same map on a visualization platform like Hyperglance is a game-changer.

2. Everybody Loves Runbooks. Not!

A look through a typical runbook in any IT organization will reveal pages and pages of screenshots of user interfaces (UIs) and advice on what the user should type in, and where.

Problems happen when live configurations drift away from the documentation, increasing technical debt and system entropy. Configuration drift has been accused of causing 99% of outages, though you might have heard it called “human error”.

Why does this keep happening? Changes are made to the live system but they aren’t recorded in the documentation.

People don't like documentation, and this is endemic across IT.

Have you ever heard, or said, any of the following seven reasons not to do documentation?

1. It’s not my job to do documentation
2. I’ve got more important things to do than documentation
3. I’ll remember that change (with my eidetic memory)
4. I’m terrible at writing documentation
5. I don’t know where to put the documentation
6. I don’t know what to document
7. Nobody reads documentation anyway

There is another, multiplying, problem on the immediate horizon: today’s modern IT systems have APIs, not User Interfaces, so screenshots are replaced by code. And those static lumps of tin called servers are being replaced by ephemeral containers and event-driven, cloud systems.

Use a live visualization platform like Hyperglance, plugged into your AWS cloud, to in effect self-document the system. The less human interference the better.

3. Manual Configurations are not Secure

Configurations are really best kept in the systems built for them. AWS keeps all your settings for you, there is no need to pull them out of AWS and store them in a spreadsheet.

Worse still, whilst AWS has industry-leading security to protect your configurations, your laptop has no such strength. If you lose that spreadsheet, your company can be in trouble.

According to a Benchmark Research Report by the IT Policy Compliance Group released in February 2007, “in one form or another, human error is the overwhelming cause of sensitive data loss, responsible for 75% of all occurrences.”

If you track information like passwords, permissions, and shares manually, and then record them in spreadsheets or logbooks, you can almost guarantee the information will be exploited.

It is better to leave the configurations in AWS and use a visualization platform like Hyperglance to remotely view the model at your convenience. No configurations moved or change, just access via a secure API.

4. APIs are Replacing User Interfaces

Cloud services have APIs first, User Interfaces (UIs) second. This culture can be traced back to a legendary internal Amazon executive directive from none other than Jeff Bezos himself in 2002.

This was leaked in an also legendary online rant by Steve Yegge, who worked at Google and Amazon.

The API Evangelist, Kin Lane, summarizes it well here.

1. All teams will henceforth expose their data and functionality through service interfaces.
2. Teams must communicate with each other through these interfaces.
3. There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
4. It doesn’t matter what technology they use.
5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

Jeff’s mandate closed with:

Anyone who doesn’t do this will be fired.  Thank you; have a nice day!

APIs are self-documenting and accessible via a browser. Modern configuration management systems such as Chef and Puppet interface with APIs to stop configuration drift and replace runbooks.

Visualization platforms such as Hyperglance provide human-friendly visual maps of these APIs and the services and elements they represent, further eliminating the need for manual runbooks.

5. You Don’t Know What You Don’t Know

It may sound simplistic, but you don’t know what you don’t know; most enterprises have only about 10% visibility into system changes at any given point in time.

It’s not unusual to ask an IT manager or administrator how many servers they have and have them answer, 'around 50' or 'between 100 and 110'.

Does your mental picture of your cloud have black holes in it?

If you don’t know how many you have, how can you possibly document them? Especially in decentralized environments, it is easy to overlook servers or other hardware.

As infrastructures continue to grow in complexity, it is inconceivable that you could track all the changes to these systems manually. If you can’t find them, you can’t document them.

By using a visualization platform like Hyperglance that plugs into the cloud API and reveals all of your elements, the guesswork is eliminated.

Visualize Your Cloud Architecture

Trying to understand how your cloud is configured (let alone related), is becoming increasingly complex.

Within minutes, Hyperglance can show you your aggregated cloud inventory in an intuitive, interactive diagram:

• See all your AWS, Azure, & Kubernetes cloud architecture in one scalable view that updates continuously
• Overlay metadata onto your cloud diagram, including cost, to add important context
• Explore resource dependencies and links, and view/search detailed metadata
• Switch to see your cloud inventory & search results with one-click
• Export your diagrams, as often as you like, to VSDX (Visio), PNG, or CSV; automate exports using our RESTful API
• Hyperglance is self-hosted, deployed  through the AWS & Azure Marketplaces, in Kubernetes, or installed on your own instance/VM