A Semantic Graph Unlocks HyperIntelligence | MicroStrategy
BI Trends

A Semantic Graph Unlocks HyperIntelligence

Analytics, with a few exceptions, derives its data from other sources: this is “used” data. It is used in the sense that it was captured and stored for some other purpose than the business at hand—sales data, human resource data, maintenance data, and more.

Analytics, with a few exceptions, derives its data from other sources: this is “used” data. It is used in the sense that it was captured and stored for some other purpose than the business at hand—sales data, human resource data, maintenance data, and more.

Historically, this "used" data has been mostly hand-me-downs within the organization—something we now call "operational exhaust." But for reasons everyone is familiar with—digital transformation, cloud, big data, massive computing resources—it is often referred to as “digital exhaust.”

Early data warehouse practitioners learned a painful lesson: even internal and structured data can be challenging to understand and work with based on just technical metadata such as table name, column name, and datatype. Everything is considerably more complicated than that. 

The creators of MicroStrategy understood this when no one else did. Finding the latent value in used data was only possible by creating an abstraction layer between the data sources and the MicroStrategy engine. I didn’t discover this on my own; MicroStrategy introduced me to the idea more than 25 years ago. In 1996, MicroStrategy contracted with me to develop a definitive document, "The Case for Relational ROLAP." If you are interested in the original concept, my old friend and former MicroStrategy CTO Jeff Bedell filed the patent (US701051881) for it in 2001, granted in 2006. You can look it up if you like at http://patft.uspto.gov/

In 2019, MicroStrategy took the semantic "layer" to the next level: the semantic graph. I like the way Saurabh Abhyankar described it at MicroStrategy World 2019 as a set of features: 

Of particular interest is how the "Last 30 Years" provided not only semantic maps for the data itself, but many other important artifacts of the analytical environment. When you look at the list and get below "derived metrics," everything else is there to manage the entire stack, and 2019 extends that to dossiers, usage, synonyms, collaboration, and even HyperIntelligence™ Cards. The future looks pretty impressive. 


The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise. —Edsger Dijkstra

Today, getting the models together relies on at least some of the business users understanding the location, naming conventions, semantics of the data, if not the intricacies of the crafting of the analytical derivatives such as metrics, hierarchies, grants, and workflows. This is a considerable barrier to progress.

Business people need to define their work in their terms. A business modeling environment is necessary for designing and maintaining structures. It is especially important to have business modeling for the inevitable changes in those structures. It is likewise essential for leveraging the latent value of those structures through analytical work enhanced by understandable models that are relevant and useful to business people.

The solution is some form of abstraction: a way for consumers of data to understand the data resources available to them, but insulated from the physical complexity of it all. Abstraction is applied routinely to systems that are, to some degree, complex and especially when they are subject to frequent change.

A 2020 model car contains more MIPS of computer processing than most computers only a decade ago. Driving the car, even under extreme conditions, is a perfect example of abstraction. Stepping on the gas doesn't pump gas to the engine. It alerts the engine management system.  As a result, it increases speed by sampling and alerting dozens of circuits, relays, sensors, and devices to achieve the desired effect subject to many constraints, such as limiting engine speed and watching the fuel-air mixture for the maximum economy or lowest emissions.

If the driver needed to attend to all of these things directly, she would not get out of the driveway.


For example, a 1971 Audi had virtually no electronics at all. A 2020 Audi S8 practically drives (and stops) itself. It's claimed that the S8 has a teraflop of computing power. That's one trillion (1012) floating-point operations per second. To put that in perspective, Los Alamos Labs developed a single teraflop supercomputer in 1996 to simulate the effect of nuclear weapons designs and testing (not one of the high points of my career).

Until recently, working with the mass of data available was a lot like driving a 1971 Audi. You had to do everything yourself, or rely on someone else to do it for you. A semantic graph is like driving a 2020 Audi S8.

Semantic graphs over data give you the understanding that truth is relative and fleeting, and that well-formulated contexts can be powerful without being perfectly clear. Obviously, for regulatory reporting, launching a Mars probe that doesn't crash into the surface from semantic dissonance, or making a soufflé, precision is required. But rapid decision making with incomplete and imperfect information is the hallmark of intellect. Any fool can make decisions with all the information in front of him—and many do.

Semantic abstraction benefits HyperIntelligence by moving information integration to a new level, where intelligence can more easily and swiftly proliferate throughout an organization. Productivity will increase because machines will make everyone less reliant on a small number of go-to super-users and help us get beyond the rigid, often brittle schemas and thin metadata models that characterize multi-tiered data warehousing. Physical implementation decisions always belong to technologists. However, those most familiar with business objectives, models, and processes will ultimately control information resources.

Read my first guest post in the series, and stay tuned for the next post.

Comments Blog post currently doesn't have any comments.
Security code