DataStax | Graph Data Modeling in DataStax Enterprise (Artem Chebotko) | Cassandra Summit 2016
2 likes1,058 views
The document discusses DataStax Enterprise Graph, focusing on its property graph data model and various data modeling frameworks. It outlines schema optimizations, integration with backend technologies, and provides examples of graph applications such as customer 360 and fraud detection. Additionally, it emphasizes the importance of schema generation, data organization, and performance optimization techniques in graph databases.
![Multi- and Meta-Properties
© DataStax, All Rights Reserved. 14
movieId: m267
title: Alice in Wonderland
year: 2010
duration: 108
country: United States
production: [Tim Burton Animation Co.,
Walt Disney Productions]
budget: [$150M, $200M]
m267
movie
source: Bloomberg Businessweek
date: March 5, 2010
source: Los Angeles Times
date: March 7, 2010](https://image.slidesharecdn.com/graphdatamodelingindatastaxenterprise-160926210558/85/DataStax-Graph-Data-Modeling-in-DataStax-Enterprise-Artem-Chebotko-Cassandra-Summit-2016-14-320.jpg)
![Property Indexes
schema.vertexLabel("movie")
.index("movieBudgetBySource")
.property("budget")
.by("source")
.add()
g.V().has("movie","movieId","m267")
.properties("budget")
.has("source","Los Angeles Times").value()
© DataStax, All Rights Reserved. 34
movieId: m267
title: Alice in Wonderland
year: 2010
duration: 108
country: United States
production: [Tim Burton Animation Co.,
Walt Disney Productions]
budget: [$150M, $200M]
movie
source: Bloomberg Businessweek
date: March 5, 2010
source: Los Angeles Times
date: March 7, 2010](https://image.slidesharecdn.com/graphdatamodelingindatastaxenterprise-160926210558/85/DataStax-Graph-Data-Modeling-in-DataStax-Enterprise-Artem-Chebotko-Cassandra-Summit-2016-34-320.jpg)
DataStax | Graph Data Modeling in DataStax Enterprise (Artem Chebotko) | Cassandra Summit 2016
- 1. A r t e m C h e b o t k o Graph Data Modeling in DataStax Enterprise
- 2. 1 DataStax Enterprise Graph 2 Property Graph Data Model 3 Data Modeling Framework 4 Schema Optimizations 2© DataStax, All Rights Reserved.
- 3. DSE Graph • Real-time Graph DBMS • Very large graphs • Many concurrent users • Proven technologies and standards • OLTP and OLAP capabilities © DataStax, All Rights Reserved. 3
- 4. DSE Graph Design © DataStax, All Rights Reserved. 4 Graph Applications DSE Graph
- 5. DSE Graph Property Graph and Gremlin DSE schema API DSE Graph Design © DataStax, All Rights Reserved. 5 Graph Applications
- 6. DSE Graph Property Graph and Gremlin DSE schema API DSE Graph Design © DataStax, All Rights Reserved. 6 Fully integrated backend technologies Graph Applications
- 7. Property Graph and Gremlin DSE schema API DSE Graph DSE Graph Design © DataStax, All Rights Reserved. 7 Schema, data, and query mappings OLTP and OLAP engines Fully integrated backend technologies Graph Applications
- 8. DSE Graph Use Cases © DataStax, All Rights Reserved. 8 Customer 360 Internet of Things Personalization Recommendations Fraud detection
- 9. 1 DataStax Enterprise Graph 2 Property Graph Data Model 3 Data Modeling Framework 4 Schema Optimizations 9© DataStax, All Rights Reserved.
- 10. Property Graph Data Model • Instance • Defined in Apache TinkerPop™ • Vertices, edges, and properties • Schema • Defined in DataStax Enterprise • Vertex labels, edge labels, and property keys © DataStax, All Rights Reserved. 10
- 11. Vertices © DataStax, All Rights Reserved. 11 movie user user genremovie person
- 12. Edges © DataStax, All Rights Reserved. 12 movie user rated rated user knows genre belongsTo belongsTo actor movie person
- 13. Properties © DataStax, All Rights Reserved. 13 movieId: m267 title: Alice in Wonderland year: 2010 duration: 108 country: United States rating: 6rating: 5 genreId: g2 name: Adventure userId: u75 age: 17 gender: F movieId: m16 title: Alice in Wonderland year: 1951 duration: 75 country: United States userId: u185 age: 12 gender: M movie user rated rated user knows genrebelongsTo belongsTo actor movie personId: p4361 name: Johnny Depp person
- 14. Multi- and Meta-Properties © DataStax, All Rights Reserved. 14 movieId: m267 title: Alice in Wonderland year: 2010 duration: 108 country: United States production: [Tim Burton Animation Co., Walt Disney Productions] budget: [$150M, $200M] m267 movie source: Bloomberg Businessweek date: March 5, 2010 source: Los Angeles Times date: March 7, 2010
- 15. Graph Schema © DataStax, All Rights Reserved. 15 movieId :text title :text year :int duration :int country :text production :text* personId:text name :text genreId :text name :text userId :text age :int gender :text rating :int genrebelongsTomovieuser rated person cinematographer actor director composer screenwriter knows
- 16. Importance of Graph Schema • DSE needs a graph schema to generate a C* schema • Vertex labels → tables • Property keys → columns • Graph indexes → materialized views secondary indexes search indexes • Additional data validation benefits © DataStax, All Rights Reserved. 16
- 17. Schema Mapping Example Property Table CREATE TABLE user_p ( community_id int, member_id bigint, "~~property_key_id" int, "~~property_id" uuid, age int, gender text, "userId" text, "~~vertex_exists" boolean, PRIMARY KEY (community_id, member_id, "~~property_key_id", "~~property_id")) © DataStax, All Rights Reserved. 17 userId :text age :int gender :text rating :int user rated knows
- 18. Schema Mapping Example Property Table CREATE TABLE user_p ( community_id int, member_id bigint, "~~property_key_id" int, "~~property_id" uuid, age int, gender text, "userId" text, "~~vertex_exists" boolean, PRIMARY KEY (community_id, member_id, "~~property_key_id", "~~property_id")) Adjacency Table CREATE TABLE user_e ( community_id int, member_id bigint, "~~edge_label_id" int, "~~adjacent_vertex_id" blob, "~~adjacent_label_id" smallint, "~~edge_id" uuid, "~rating" int, "~~edge_exists" boolean, "~~simple_edge_id" uuid, PRIMARY KEY (community_id, member_id, "~~edge_label_id", "~~adjacent_vertex_id", "~~adjacent_label_id", "~~edge_id")) © DataStax, All Rights Reserved. 18 userId :text age :int gender :text rating :int user rated knows
- 19. 1 DataStax Enterprise Graph 2 Property Graph Data Model 3 Data Modeling Framework 4 Schema Optimizations 19© DataStax, All Rights Reserved.
- 20. Data Modeling • Process of organizing and structuring data • Based on well-defined set of rules or methodology • Results in a graph or database schema • Affects data quality, data storage and data retrieval © DataStax, All Rights Reserved. 20
- 21. Traditional Schema Design Data Model • Conceptual Data Model (CDM) • Logical Data Model (LDM) • Physical Data Model (PDM) Purpose • Understand data and its applications • Sketch a graph data model • Optimize physical design © DataStax, All Rights Reserved. 21
- 22. knows User userId age gender Movierated rating movieId title year duration country production Genre Person belongsTo involved Actor Director Composer Screen-‐ writer Cinema-‐ tographer IsA personId name genreId name Conceptual Data Model • Entity types • Relationship types • Attribute types © DataStax, All Rights Reserved. 22
- 23. Transition from CDM to LDM • Both CDM and LDM are graphs • Entity types → Vertex labels • Relationship types → Edge labels • Attribute types → Property keys • Mostly straightforward with a few nuances © DataStax, All Rights Reserved. 23
- 24. movieId :text title :text year :int duration :int country :text production :text* personId:text name :text genreId :text name :text userId :text age :int gender :text rating :int genrebelongsTomovieuser rated person cinematographer actor director composer screenwriter knows Logical Data Model © DataStax, All Rights Reserved. 24 • Vertex labels • Edge labels • Property keys
- 25. Keys • Entity type keys → Property keys • Uniqueness is not enforced • Vertex IDs are auto-generated • Entity type keys → Custom vertex IDs • Uniqueness is enforced • Overriding default partitioning • Advanced feature © DataStax, All Rights Reserved. 25 User userId :text age :int gender :text user userId age gender
- 26. Symmetric Relationships © DataStax, All Rights Reserved. 26 User Movie rated movieuser rated wasRatedBy movieuser wasRatedBy movieuser rated wasRatedBy
- 27. Bi-Directional Relationships © DataStax, All Rights Reserved. 27 User knows user knows user knows user user knows user user knows user knows
- 28. Qualified Bi-Directional Relationships © DataStax, All Rights Reserved. 28 strength :int User likes user likes user likes user user likes user user likes user likes strength strength: 7 strength: 9 strength: 7 strength: 9
- 29. Hierarchies © DataStax, All Rights Reserved. 29 Movie involved Person IsA Actor Director movie person directoractor involved isA isA movie person involved role:text movie person actor director movie person directoractor isA isA involved involved
- 31. 1 DataStax Enterprise Graph 2 Property Graph Data Model 3 Data Modeling Framework 4 Schema Optimizations 31© DataStax, All Rights Reserved.
- 32. Optimizing PDM for Performance • Indexing data • Controlling partitioning • Materializing aggregates and inferences • Rewriting traversals © DataStax, All Rights Reserved. 32
- 33. Vertex Indexes schema.vertexLabel("movie") .index("moviesById") .materialized() .by("movieId") .add() g.V().has("movie","movieId","m267") © DataStax, All Rights Reserved. 33 movieId :text title :text year :int duration :int country :text production :text* movie
- 34. Property Indexes schema.vertexLabel("movie") .index("movieBudgetBySource") .property("budget") .by("source") .add() g.V().has("movie","movieId","m267") .properties("budget") .has("source","Los Angeles Times").value() © DataStax, All Rights Reserved. 34 movieId: m267 title: Alice in Wonderland year: 2010 duration: 108 country: United States production: [Tim Burton Animation Co., Walt Disney Productions] budget: [$150M, $200M] movie source: Bloomberg Businessweek date: March 5, 2010 source: Los Angeles Times date: March 7, 2010
- 35. Edge Indexes schema.vertexLabel("user") .index("toMoviesByRating") .outE("rated") .by("rating") .add() g.V().has("user","userId","u1") .outE("rated").has("rating",gt(6)).inV() © DataStax, All Rights Reserved. 35 rating: 7 movieuser rated rating: 9 movie rated rating: 7 movie rated
- 36. movie_p year K country K movieId C↑ ~~property_key_id C↑ ~~property_id C↑ duration title ~~vertex_exists Custom Partitioning schema.vertexLabel("movie") .partitionKey("year","country") .clusteringKey("movieId") .properties("title","duration") .create() © DataStax, All Rights Reserved. 36 movie_e year K country K movieId C↑ ~~edge_label_id C↑ ~~adjacent_vertex_id C↑ ~~adjacent_label_id C↑ ~~edge_id C↑ ~~edge_exists ~~simple_edge_id
- 37. movieId :text title :text year :int duration :int country :text production :text* avg :float movie Materializing Aggregates g.V().hasLabel("movie") .property("avg",_.inE("rated") .values("rating") .mean()) © DataStax, All Rights Reserved. 37
- 38. Materializing Inferences g.V().has("person","name","Tom Hanks").as("tom") .in("actor").out("actor").where(neq("tom")).dedup() .addE("knows").from("tom") © DataStax, All Rights Reserved. 38 movie tom person actor person person actor actor knows knows
- 39. Rewriting Traversals • Equivalent results • Different execution plans • Different response times © DataStax, All Rights Reserved. 39 g.V().has("movie","year",2010).out("actor") .has("name","Johnny Depp").count() g.V().has("person","name","Johnny Depp").in("actor") .has("year",2010).count()
- 40. Profiling Traversals © DataStax, All Rights Reserved. 40
- 41. Thank You © DataStax, All Rights Reserved. 41 Artem Chebotko [email protected] www.linkedin.com/in/artemchebotko
- 42. The End