Cray Graph Engine (CGE): Graph Analytics for Big Data

Graph Analysis Finds Relationships in Data

Yes, the data deluge is here — but even more significant is the explosion of highly related data. This information is generated in areas such as the internet of things (IoT), social ecosystems, and of course natural or biological systems.

To unleash the value in this related data, you need a different way of looking at it. Graph is this necessary different approach. A graph database does not replace relational databases, which are the foundation of analytics today. It’s an add-on. Because graph is designed to work with unstructured data, it doesn’t require joins or schemas. Adding it to a Hadoop® and Spark™ environment lets you run iterative and pattern queries that you couldn’t run previously.

Graph databases store data in nodes (vertices) and edges rather than tables, as in relational databases. They are the most efficient way of looking for relationships between data items, patterns of relationships or interactions between multiple data items.

This graph structure is very different from the table structure of a standard relational database. While traditional relational database shine at queries looking for information about some item, or sums or averages of many items of the same type of information, they struggle with queries that care more about how the data is connected than the data points themselves

There are several methods for graph databases. On the CGE overview page, we go into detail on the how and the why of our approach. 

• Hypergraphs: Can connect any number of nodes. Work well for domains with mostly many-to-many relationships.
• Triples (semantic graph databases): Use a subject-predicate-object data structure based on W3C standards to represent every known fact, such as “George likes steak.” Works well for combining many varieties of data sources for analysis.   
• Property graphs: Any pair of nodes can be connected by an edge. In addition, any node or edge can have a table of additional data attached to it which works well when you have some data that doesn’t need to be linked to anything else. These graphs represent a compromise between semantic and relational databases to connect small amounts of table data to one point on a graph.

So when should you consider adding this cutting-edge approach to big data analytics? When your analytics needs meet at least two of the below circumstances, a graph database is the solution.    

• Problems and queries have become too complex for the current environment to process
• You care as much about the data relationships and patterns as the data items
• You’re encountering “unrunnable” queries: The queries will not complete or they take too much time
• Current data models and schemas will not support your queries
• You have lots of new and disparate data sources (IoT, web, natural) that are inherently unstructured

Graph Delivers for Cybersecurity, Precision Medicine and Much More

What kinds of applications can make good use of a graph database? Applications where it’s useful to find patterns of relationships between data items. This is not every database query — it’s no accident that relational databases are so popular — but there are many significant applications that can only be analyzed accurately and efficiently via a graph database:

• In many intelligence and law enforcement applications, it’s important to look for a pattern of events. Phone calls, courier messages, travel, money transfers — any one of these events may look innocuous, but the view of all of them together and how they are directly or indirectly related to each other may be ominous.
• Similarly, investment banks guarding against insider trading have to look for a suspicious pattern of actions not necessarily any single action. An investment banker gets insider information about a stock. She emails an information technologist at the same bank. The IT worker phones the banker. At close of business, the two badge out within seconds of each other. That night, the IT employee buys the stock.
• The bioinformatics research community has largely adopted graph databases and the SPARQL query language. They are a natural fit to the huge network of relationships between all the chemicals present in the human body. One of Cray’s bioinformatics customers traced a chain of such relationships from a drug designed to combat AIDS out to a discovery that the same drug might be effective against breast cancer.
• Many vendors of consumer products have become interested in social network analysis, which involves constructing a graph of relationships between people. For example, among Facebook users, graph searches might reveal which person is probably influential over his or her friends, which groups of friends share a common interest and so on — which could lead to some sophisticated targeted marketing strategies.