AIOps and a Single Understanding of the Enterprise Environment

When I started my career in IT, networks and applications were often much simpler than they are today and the biggest problem was often lack of basic telemetry and alerting. Today, this challenge has flipped on its head – every component in every server, network, cloud platform, and end-user device wants to tell you exactly what’s happening at the same time. Making matters worse, this data is being reported into a myriad of disconnected tools run by different parts of the organization, creating a situation where reconstructing what is happening with your most mission-critical services often feels a bit like trying to get the smoke back into a cigarette.

AIOps - Getting monitoring back under control

Artificial Intelligence for IT Operations (AIOps) began as a response to some of the most fundamental challenges to the monitoring and collection of enterprise data. One of the biggest of these challenges is the sheer number of tools to monitor that data. One study estimates that nearly a quarter of all large enterprises today have at least eight or more monitoring tools, while some have as many as 25.

Consequently, when something goes wrong, monitoring tools often complicate the situation. The challenge today is not in monitoring and collecting data, it’s what to do with all that collected data. The common goal is the need for gaining insights from that data that inform the enterprise what it doesn’t know yet about its environment, possible problem scenarios, or even end user activity that might affect the business. These kinds of insights are the goal of AIOps.

Garbage in. Garbage out.

Data flows into and around systems in a dizzying variety of ways and data types. Structuring and normalizing that data is key to its usefulness and accuracy. Normalizing that data is organizing it so that all the data appears to be the same regardless of its source. Normalizing data also serves to remove redundant or unstructured data that can negatively affect its accuracy or validity. And there lies a major challenge.

The volume of collected data led to the adoption of what are commonly referred to as data lakes. Data lakes are basically enormous reservoirs into which all the collected data, structured or unstructured, are deposited. There is no filter to remove bad data. This is a problem for most AIOps systems because they are not designed to weed out the good or bad data. And once you have trash in a data lake, it’s very difficult to make sense of it.

A single funnel

A Broadcom innovation solves this problem. Rather than attempting to correlate the data first, Broadcom’s approach flips the process on its head. It starts with understanding the IT environment first. Our AIOps solution is designed not as a traditional data lake, but more like multiple different storage vehicles, or distributed data lakes that automatically normalize that data already at ingestion. In other words, Broadcom AIOps brings all the collected data into one solution.

Now, for the first time, enterprises have one holistic view and reporting of their IT environment. A holistic view that allows the enterprise to visualize the data in their environment as components, or Lego pieces of a multi-dimensional, topographical model.

Visualizing data in 3D

By normalizing the data on a 3D model, the Broadcom AIOPs solution adds a whole new dimension to understanding the IT environment.  And just as significantly, it allows enterprise IT teams to visualize all the data in their environment on dedicated, easy-to-understand dashboards.

The MOM advantage

A second Broadcom AIOps innovation is that our solution acts as a monitor of monitors (MOM). It collects and integrates data directly from any monitoring tool or solution, regardless of the source type or origin. It creates a single funnel for all data collection and correlation, consolidating event handling into one problem context across all the monitoring tools data — greatly accelerating the time to insights and analysis.

The Holy Grail

One of the transformative aspects of AIOps is that, for the first time, an enterprise can cut through all the noise and zero in on problems that were formerly difficult to troubleshoot. The combined insights gained from the AIOps’ holistic view allows enterprises to solve technical issues far faster than before, greatly limiting the damage technical problems can do to the business. AIOps can also correlate technical problems to business outcomes and end user experiences. That’s the Holy Grail of monitoring that AIOPs achieves.

AIOps and the future

AIOps gives the enterprise control over its monitored environment. Where that will lead is tremendously exciting. Self-driving cars. Automated remediation. SecOps. Software life cycle integration.

More immediately, some of our priorities include using AIOps to improve our customers’ monitoring capabilities. Another is to use it for customer optimization of their IT environments. The full picture AIOps provides allows enterprises to speak the same language and use the same data when it comes to monitoring.  It gives organizations one single version of truth in terms of data accuracy.

Having personally faced these challenges earlier in my IT career, I’m very proud of the work our team has done to reign in the chaos and am looking forward to the challenges and opportunities that lie ahead. Broadcom has a long and storied history of innovation and I’m excited to continue our significant investment in the products that our AIOps customers are using to solve these challenges in their organizations.  Read more about Broadcom’s AIOps solution in a new report by Research in Action.