Emil Eifrem, profile picture
Uploaded byEmil Eifrem
4,241 views

NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)

The document provides an overview of NoSQL databases, specifically graph databases like Neo4j, highlighting their advantages such as handling large datasets, connectedness, and flexibility in semi-structured data. It discusses four key trends driving the need for NoSQL solutions and categorizes different types of NoSQL databases including key-value stores, document databases, and graph databases. It further illustrates the usage of Neo4j with examples of data modeling, traversal, and practical applications in enterprise content management.

No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
Neo4j the benefits of graph databases + a NOSQL overview QCon London 2010 #neo4j Emil Eifrem @emileifrem CEO, Neo Technology emil@neotechnology.com
What's the plan? Why now? – Four trends NOSQL overview Graph databases && Neo4j A production example of Neo4j Conclusions
Trend 1: data set size 40 2007 Source: IDC 2007
988 Trend 1: data set size 40 2007 2010 Source: IDC 2007
Trend 2: connectedness Giant Global Information connectivity Graph (GGG) Ontologies RDF Folksonomies Tagging User- Wikis generated content Blogs RSS Hypertext Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020
Trend 3: semi-structure Individualization of content! In the salary lists of the 1970s, all elements had exactly one job In the salary lists of the 2000s, we need 5 job columns! Or 8? Or 15? Trend accelerated by the decentralization of content generation that is the hallmark of the age of participation (“web 2.0”)
Aside: RDBMS performance Relational database Salary List Performance Majority of Webapps Social network Semantic } Trading custom Data complexity
Trend 4: architecture 1990s: Database as integration hub
Trend 4: architecture 2000s: (Slowly towards...) Decoupled services with own backend
Why NOSQL 2009? Trend 1: Size. Trend 2: Connectivity. Trend 3: Semi-structure. Trend 4: Architecture.
NOSQL overview
First off: the name NoSQL is NOT “Never SQL” NoSQL is NOT “No To SQL”
NOSQL is simply Not Only SQL!
Four (emerging) NOSQL categories Key-value stores Based on Amazon's Dynamo paper Data model: (global) collection of K-V pairs Example: Dynomite, Voldemort, Tokyo* BigTable clones Based on Google's BigTable paper Data model: big table, column families Example: HBase, Hypertable, Cassandra
Four (emerging) NOSQL categories Document databases Inspired by Lotus Notes Data model: collections of K-V collections Example: CouchDB, MongoDB Graph databases Inspired by Euler & graph theory Data model: nodes, rels, K-V on both Example: AllegroGraph, Sones, Neo4j
NOSQL data models Data size Key-value stores Bigtable clones Document databases Graph databases Data complexity
NOSQL data models Data size Key-value stores Bigtable clones Document databases Graph databases (This is still billions of 90% nodes & relationships) of use cases Data complexity
Graph DBs & Neo4j intro
The Graph DB model: representation Core abstractions: name = “Emil” age = 29 Nodes sex = “yes” Relationships between nodes 1 2 Properties on both type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
Code (1): Building a node space GraphDatabaseService graphDb = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
Code (1): Building a node space GraphDatabaseService graphDb = ... // Get factory Transaction tx = neo.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
Code (1b): Defining RelationshipTypes // In package org.neo4j.graphdb public interface RelationshipType { String name(); } // In package org.yourdomain.yourapp // Example on how to roll dynamic RelationshipTypes class MyDynamicRelType implements RelationshipType { private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; } } // Example on how to kick it, static-RelationshipType-like enum MyStaticRelTypes implements RelationshipType { KNOWS, WORKS_FOR, }
Whiteboard friendly owns Björn Big Car build drives DayCare
The Graph DB model: traversal Traverser framework for name = “Emil” high-performance traversing age = 31 sex = “yes” across the node space 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
Example: Mr Anderson’s friends name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
Code (2): Traversing a node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity” $ bin/start-neo-example Mr Anderson's friends: At depth 1 => Morpheus friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, At depth 1 => Trinity StopEvaluator.END_OF_GRAPH, At depth 2 => Cypher ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, At depth 3 => Agent Smith Direction.OUTGOING ); $
Example: Friends in love? name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 7 3 KN O WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity”
Code (3a): Custom traverser // Create a traverser that returns all “friends in love” Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
Code (3a): Custom traverser // Traverse the node space and print out the result System.out.println( "Who’s a lover?" ); for ( Node person : loveTraverser ) { System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) ); }
name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS KN O CODED_BY 1 7 3 WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity” $ bin/start-neo-example new ReturnableEvaluator() Who’s a lover? { public boolean isReturnableNode( TraversalPosition pos) At depth 1 => Trinity { $ return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } },
Bonus code: domain model How do you implement your domain model? Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive: // In package org.yourdomain.yourapp class PersonImpl implements Person { private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; } public String getName() { return (String) this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); } }
Domain layer frameworks Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Defines Entities / Associations / Properties Sound familiar? Nodes / Rel’s / Properties! Neo4j is an “EntityStore” backend Jo4neo (http://code.google.com/p/jo4neo) Annotation driven Weaves Neo4j-backed persistence into domain objects at runtime
Neo4j system characteristics Disk-based Native graph storage engine with custom binary on-disk format Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc Scales up Many billions of nodes/rels/props on single JVM Robust 6+ years in 24/7 production
Social network pathExists() 12 ~1k persons 3 7 1 Avg 50 friends per person pathExists(a, b) limit 36 41 77 depth 4 5 Two backends Eliminate disk IO so warm up caches
Social network pathExists() 2 Emil 1 5 7 Mike Kevin 3 John Marcus 9 4 Bruce Leigh # persons query time Relational database 1 000 2 000 ms Graph database (Neo4j) 1 000 2 ms Graph database (Neo4j) 1 000 000 2 ms
Pros & Cons compared to RDBMS + No O/R impedance mismatch (whiteboard friendly) + Can easily evolve schemas + Can represent semi-structured info + Can represent graphs/networks (with performance) - Lacks in tool and framework support - Few other implementations => potential lock in - No support for ad-hoc queries +
Query languages SPARQL – “SQL for linked data” Ex: ”SELECT ?person WHERE { ?person neo4j:KNOWS ?friend . ?friend neo4j:KNOWS ?foe . ?foe neo4j:name “Larry Ellison” . }” Gremlin – “perl for graphs” Ex: ”./outE[@label='KNOWS']/inV[@age > 30]/@name”
The Neo4j ecosystem Neo4j is an embedded database Tiny teeny lil jar file Component ecosystem index meta-model graph-matching remote-graphdb sparql-engine ... See http://components.neo4j.org
Example: Neo4j-RDF Neo4j-RDF triple/quad store OWL SPARQL RDF Metamodel Graph match Neo4j
Language bindings Neo4j.py – bindings for Jython and CPython http://components.neo4j.org/neo4j.py Neo4jrb – bindings for JRuby (incl RESTful API) http://wiki.neo4j.org/content/Ruby Neo4jrb-simple http://github.com/mdeiters/neo4jr-simple Clojure http://wiki.neo4j.org/content/Clojure Scala (incl RESTful API) http://wiki.neo4j.org/content/Scala
Grails Neoclipse screendump
Scale out – replication Rolling out Neo4j HA... soon :) Master-slave replication, 1st configuration MySQL style... ish Except all instances can write, synchronously between writing slave & master (strong consistency) Updates are asynchronously propagated to the other slaves (eventual consistency) This can handle billions of entities... … but not 100B
Scale out – partitioning Sharding possible today … but you have to do manual work … just as with MySQL Great option: shard on top of resilient, scalable OSS app server , see: www.codecauldron.org Transparent partitioning? Neo4j 2.0 100B? Easy to say. Sliiiiightly harder to do. Fundamentals: BASE & eventual consistency Generic clustering algorithm as base case, but give lots of knobs for developers
Production example
Case: Enterprise Content Management Background: Enterprise Content Management (think: “CMS but also with non-web content,” or “big filesystem on the webotubes”) Thousands of users Various content types: PDFs, images, videos, doc files, organization-specific XML formats “Multi-tentant SaaS”
Outline A saga in three parts Part I: we're a file system on the web Part II: sharing is caring Part III: profit
Part I: We're a file system on the web Let's get something out there We shall store files in folders Ya know, versions are kinda cool
Part I: concept model Root Folder Sub v. 2 File 1 File 2 v. 1
Part I: SQL? NOSQL? So hmm, this whole relational database thingie... Modeling hierarchies? Doable but kinda painful. Sucky code. And hmm, quite a lot of joins. Activity feeds Wouldn't it be cool if you could subscribe to a folder and get changes fed to you. Whoa, massive amount of joins! Denormalization, write explosion, code complexity.
Part I: concept model Root Folder Sub v. 2 File 1 How do you represent this in a graph database? File 2 v. 1
Tadaa! Root Folder How do you implement activity feeds? Sub v. 2 File 1 Easier when you do ~1M traversals per second. :) File 2 v. 1 No need to denormalize and aggregate events at each folder level.
Part II: Sharing is caring We're oh-so SaaS and multi-tentant Would be useful if we could share content between organizations Since we're all kinda running on top of the same system (not just same software) anyway
Part II: concept model (a) Trifork Root Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2
Part II: Security Whoa, guys, security? Customer sez: we need to model organizations And suborganizations And hierarchical user groups Customer sez: and add some security to all that So add ACLs And incremental security
Admin Group U2 U1 Trifork Org +R +W Trifork Root -W Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2 +RW U3
Part II: Keyword translations Customer sez: we need to cut costs Ouch But we spend a lot of time on manually translating keyword lists and things like that Let's model that in the graph! Also, this whole graph thing is really kinda flexible... so let's throw in some topologies while we're at it!
Admin Group U2 U1 Tree Trifork Woods Org EN EN +R Forest EN +W Plant Träd EN Skog SV Trifork SV Root -W Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2 +RW U3
Part III: profit Customer sez: I heart the cash If my customers make money, I make money Developers: “Gives me multi-tentant ecommerce!” Owait, say waht?
Part III: multi-tentant e-commerce? Conceptual breakdown: Every org can “enable e-commerce,” thereby making their content sellable Within every org, one should be able to model a supply chain of creator → syndicator → distributor → customer The distributor assigned by region and sets price: E.x. one dist for Scand, one for the UK Due to inter-org sharing (remember?), the same content can belong to several e-commerce “spheres”
QCon Org Syndicator Role Ecom sph SI Distributor Role Sub File 1 Folder DR Price List World Scand. U1
Finding the price So how do you actually figure out the price Throw the distributor Which is regionally bound Per content Per e-commerce sphere That's a shortest path algo!
QCon Org Syndicator Role Ecom sph SI Distributor Sub Finding the price: Role File 1 Folder Equivalent to finding the SI shortest path from U1 to File1 DR Price along “purple and black” List Price relationship types. List DR JAOO World Org Scand. Distributor Role U1
Admin Group U2 U1 Tree Trifork Woods Org EN EN +R Forest EN +W Plant Träd EN Skog SV Trifork SV Root -W QCon Folder Org Syndicator Role Ecom sph Sub v. 2 SI Distributor File 1 Role Sub File 1 Folder File 2 v. 1 Share SI DR Price List Price Sub List DR link File 1 JAOO World Org InfoQ Root File 2 Scand. Distributor +RW Role U1 U3
ECM conclusions One example of an evolving model Site had a lots of content, lots of users High read load Moderate write load Only backend: Neo4j “You go to a graph database for the performance... but you stay for the flexibility!”
How ego are you? (aka other impls?) Franz’ AllegroGraph (http://agraph.franz.com) Proprietary, Lisp, RDF-oriented but real graphdb Sones graphDB (http://sones.com) Proprietary, .NET, cloud-only, req invite for test Kloudshare (http://kloudshare.com) Graph database in the cloud, still stealth mode Google Pregel (http://bit.ly/dP9IP) We are oh-so-secret Some academic papers from ~10 years ago G = {V, E} #FAIL
Conclusion Graphs && Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if you’re open source, we’re open source” If you have proprietary software? Must buy a commercial license But the first one is free! Download http://neo4j.org Feedback http://lists.neo4j.org
Questions? Image credit: lost again! Sorry :(
http://neotechnology.com

More Related Content

PDF
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
byEmil Eifrem
 
PDF
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
byEmil Eifrem
 
PDF
Neo4j - The Benefits of Graph Databases (OSCON 2009)
byEmil Eifrem
 
PDF
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
byEmil Eifrem
 
PDF
Eifrem neo4j
byShridhar Joshi
 
PDF
An overview of NOSQL (JFokus 2011)
byEmil Eifrem
 
KEY
Intro to Neo4j presentation
byjexp
 
ODP
Grails goes Graph
bydarthvader42
 
NOSQL overview and intro to graph databases with Neo4j (Geeknight May 2010)
byEmil Eifrem
 
NOSQL Overview Lightning Talk (Scalability Geekcruise 2009)
byEmil Eifrem
 
Neo4j - The Benefits of Graph Databases (OSCON 2009)
byEmil Eifrem
 
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)
byEmil Eifrem
 
Eifrem neo4j
byShridhar Joshi
 
An overview of NOSQL (JFokus 2011)
byEmil Eifrem
 
Intro to Neo4j presentation
byjexp
 
Grails goes Graph
bydarthvader42
 

What's hot

PDF
Graph based data models
byMoumie Soulemane
 
ODP
Graph databases
byKarol Grzegorczyk
 
PDF
Gremlin: A Graph-Based Programming Language
byMarko Rodriguez
 
PDF
The Best of Both Worlds: Unlocking the Power of (big) Knowledge Graphs with S...
byGezim Sejdiu
 
PPTX
Performance of graph query languages
byAthiq Ahamed
 
PDF
Efficient Distributed In-Memory Processing of RDF Datasets - PhD Viva
byGezim Sejdiu
 
PDF
RDBMS to Graph
byNeo4j
 
PDF
Try NoSQL it doesn't hurts and is fun
byPere Urbón-Bayes
 
KEY
using Spring and MongoDB on Cloud Foundry
byJoshua Long
 
PPTX
Graph databases
byVinoth Kannan
 
PDF
mm-ADT: A Multi-Model Abstract Data Type
byMarko Rodriguez
 
PDF
Ivan Herman - Semantic Web Activities @ W3C
bysssw2012
 
PDF
Data Modeling with Neo4j
byNeo4j
 
PDF
STI Summit 2011 - Digital Worlds
bySemantic Technology Institute International
 
Graph based data models
byMoumie Soulemane
 
Graph databases
byKarol Grzegorczyk
 
Gremlin: A Graph-Based Programming Language
byMarko Rodriguez
 
The Best of Both Worlds: Unlocking the Power of (big) Knowledge Graphs with S...
byGezim Sejdiu
 
Performance of graph query languages
byAthiq Ahamed
 
Efficient Distributed In-Memory Processing of RDF Datasets - PhD Viva
byGezim Sejdiu
 
RDBMS to Graph
byNeo4j
 
Try NoSQL it doesn't hurts and is fun
byPere Urbón-Bayes
 
using Spring and MongoDB on Cloud Foundry
byJoshua Long
 
Graph databases
byVinoth Kannan
 
mm-ADT: A Multi-Model Abstract Data Type
byMarko Rodriguez
 
Ivan Herman - Semantic Web Activities @ W3C
bysssw2012
 
Data Modeling with Neo4j
byNeo4j
 
STI Summit 2011 - Digital Worlds
bySemantic Technology Institute International
 

Viewers also liked

PDF
Business Model Canvas Poster V.1.0
byAlexander Osterwalder
 
PDF
Web navigation systems for information seeking (updated in Feb 2015)
byJack Zheng
 
PDF
SDLC 101 Cartoon
byJack Zheng
 
PDF
KSU IT4983 Capstone Projects Report 2017 Update
byJack Zheng
 
PDF
Mobile Web Overview https://www.edocr.com/v/k52p5vj4/
byJack Zheng
 
PDF
Web Landscape - updated in Jan 2016
byJack Zheng
 
PDF
Information system a system view
byJack Zheng
 
PDF
Multidimensional Perceptual Map for Project Prioritization and Selection - 20...
byJack Zheng
 
PPTX
Semantic Web Landscape 2009
byLeeFeigenbaum
 
PPTX
Webinar: Back to Basics: Thinking in Documents
byMongoDB
 
KEY
Modeling Data in MongoDB
bylehresman
 
PPTX
Back to Basics Webinar 1: Introduction to NoSQL
byMongoDB
 
PDF
Hadoop and Graph Databases (Neo4j): Winning Combination for Bioanalytics - Jo...
byNeo4j
 
PDF
NoSQL Now! NoSQL Architecture Patterns
byDATAVERSITY
 
PPTX
MongoDB Schema Design: Four Real-World Examples
byMike Friedman
 
Business Model Canvas Poster V.1.0
byAlexander Osterwalder
 
Web navigation systems for information seeking (updated in Feb 2015)
byJack Zheng
 
SDLC 101 Cartoon
byJack Zheng
 
KSU IT4983 Capstone Projects Report 2017 Update
byJack Zheng
 
Mobile Web Overview https://www.edocr.com/v/k52p5vj4/
byJack Zheng
 
Web Landscape - updated in Jan 2016
byJack Zheng
 
Information system a system view
byJack Zheng
 
Multidimensional Perceptual Map for Project Prioritization and Selection - 20...
byJack Zheng
 
Semantic Web Landscape 2009
byLeeFeigenbaum
 
Webinar: Back to Basics: Thinking in Documents
byMongoDB
 
Modeling Data in MongoDB
bylehresman
 
Back to Basics Webinar 1: Introduction to NoSQL
byMongoDB
 
Hadoop and Graph Databases (Neo4j): Winning Combination for Bioanalytics - Jo...
byNeo4j
 
NoSQL Now! NoSQL Architecture Patterns
byDATAVERSITY
 
MongoDB Schema Design: Four Real-World Examples
byMike Friedman
 

Similar to NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)

PDF
Django and Neo4j - Domain modeling that kicks ass
byTobias Lindaaker
 
PDF
NOSQLEU - Graph Databases and Neo4j
byTobias Lindaaker
 
PDF
Neo4j -- or why graph dbs kick ass
byEmil Eifrem
 
PPTX
CSC 8101 Non Relational Databases
bysjwoodman
 
PPTX
No Sql Movement
byAjit Koti
 
PPTX
An Introduction to NOSQL, Graph Databases and Neo4j
byDebanjan Mahata
 
PDF
Graph Theory and Databases
byPere Urbón-Bayes
 
PDF
NoSQL intro for YaJUG / NoSQL UG Luxembourg
byNGDATA
 
PPTX
Anti-social Databases
byWilliam LaForest
 
PDF
NoSQL with Hadoop and HBase
byNGDATA
 
PDF
Graph Databases, The Web of Data Storage Engines
byPere Urbón-Bayes
 
KEY
Spring Data Neo4j Intro SpringOne 2011
byjexp
 
PDF
sones company presentation
bysones GmbH
 
PDF
Towards an Interlinked Semantic Wiki Farm
byAlexandre Passant
 
PDF
Neo4j Nosqllive
byPeter Neubauer
 
PDF
Gephi short introduction
bySébastien
 
PPTX
Neo4j Introduction at Imperial College London
byMichal Bachman
 
PDF
3/15 - Intro to Spring Data Neo4j
byNeo4j
 
PDF
Kickoff research project TU Ilmenau
byHenning Rauch
 
PDF
Non Relational Databases
byChris Baglieri
 
Django and Neo4j - Domain modeling that kicks ass
byTobias Lindaaker
 
NOSQLEU - Graph Databases and Neo4j
byTobias Lindaaker
 
Neo4j -- or why graph dbs kick ass
byEmil Eifrem
 
CSC 8101 Non Relational Databases
bysjwoodman
 
No Sql Movement
byAjit Koti
 
An Introduction to NOSQL, Graph Databases and Neo4j
byDebanjan Mahata
 
Graph Theory and Databases
byPere Urbón-Bayes
 
NoSQL intro for YaJUG / NoSQL UG Luxembourg
byNGDATA
 
Anti-social Databases
byWilliam LaForest
 
NoSQL with Hadoop and HBase
byNGDATA
 
Graph Databases, The Web of Data Storage Engines
byPere Urbón-Bayes
 
Spring Data Neo4j Intro SpringOne 2011
byjexp
 
sones company presentation
bysones GmbH
 
Towards an Interlinked Semantic Wiki Farm
byAlexandre Passant
 
Neo4j Nosqllive
byPeter Neubauer
 
Gephi short introduction
bySébastien
 
Neo4j Introduction at Imperial College London
byMichal Bachman
 
3/15 - Intro to Spring Data Neo4j
byNeo4j
 
Kickoff research project TU Ilmenau
byHenning Rauch
 
Non Relational Databases
byChris Baglieri
 

More from Emil Eifrem

PDF
Startups in Sweden vs Startups in Silicon Valley, 2015 edition
byEmil Eifrem
 
PDF
GraphConnect SF 2013 Keynote
byEmil Eifrem
 
PDF
An Overview of the Emerging Graph Landscape (Oct 2013)
byEmil Eifrem
 
PDF
Startups in Sweden vs Startups in Silicon Valley
byEmil Eifrem
 
PDF
An intro to Neo4j and some use cases (JFokus 2011)
byEmil Eifrem
 
PDF
NOSQL part of the SpringOne 2GX 2010 keynote
byEmil Eifrem
 
Startups in Sweden vs Startups in Silicon Valley, 2015 edition
byEmil Eifrem
 
GraphConnect SF 2013 Keynote
byEmil Eifrem
 
An Overview of the Emerging Graph Landscape (Oct 2013)
byEmil Eifrem
 
Startups in Sweden vs Startups in Silicon Valley
byEmil Eifrem
 
An intro to Neo4j and some use cases (JFokus 2011)
byEmil Eifrem
 
NOSQL part of the SpringOne 2GX 2010 keynote
byEmil Eifrem
 

Recently uploaded

PPTX
UiPath Data Fabric From Data Silos to Unified Flows.pptx
bysuhanisingh58689
 
PDF
Embedding sustainability: Tips for ebook and print production - Tech Forum 2025
byBookNet Canada
 
PPTX
RAPTOR_Intro_Exercises_Presentation.pptx
bySwatiMalik36
 
PDF
NPTEL Assignment Completed PDF FILE with ans
byadultme43
 
PDF
The Smol Training Playbook: The Secrets to Building World-Class LLMs
byRazin Mustafiz
 
PPTX
Assignment Review and Accessibility Audit Findings.pptx
byJulia Undeutsch
 
PPTX
Softaken SQLite DB Converter for Seamless SQLite Database file Conversion on ...
byDemetrio Parrilla
 
PDF
Ready, set, go: Pre-fall sales trends and data-driven insights - Tech Forum 2025
byBookNet Canada
 
PDF
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
PDF
Session 7 Specialized AI Associate Series: UiPath Communications Mining Overview
byDianaGray10
 
DOCX
Formula 1 - APIC general MACO calculation.docx
byMarkus Janssen
 
PDF
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
PDF
How does a computer vision development company help retailers decode consumer...
byNextbrain Technologies
 
DOCX
Formula 2 - APIC HBEL general calculation.docx
byMarkus Janssen
 
PDF
Build Agentic AI Applications with Oracle AI Database Private Agent Factory -...
bySandesh Rao
 
PDF
Igalia and WebKit: Status update and plans (2025)
byIgalia
 
PPTX
2025 - AI terms & Meanings for Marketers
byashishav29
 
PPTX
UiPath Automation Suite_Community Session_3/3
bysuhanisingh58689
 
PDF
2025 AI Adoption Report: Gen AI Fast-Tracks Into the Enterprise
byRazin Mustafiz
 
PPTX
OutSystems ONE 2025-recap presentation.pptx
bymostafaothman27
 
UiPath Data Fabric From Data Silos to Unified Flows.pptx
bysuhanisingh58689
 
Embedding sustainability: Tips for ebook and print production - Tech Forum 2025
byBookNet Canada
 
RAPTOR_Intro_Exercises_Presentation.pptx
bySwatiMalik36
 
NPTEL Assignment Completed PDF FILE with ans
byadultme43
 
The Smol Training Playbook: The Secrets to Building World-Class LLMs
byRazin Mustafiz
 
Assignment Review and Accessibility Audit Findings.pptx
byJulia Undeutsch
 
Softaken SQLite DB Converter for Seamless SQLite Database file Conversion on ...
byDemetrio Parrilla
 
Ready, set, go: Pre-fall sales trends and data-driven insights - Tech Forum 2025
byBookNet Canada
 
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
Session 7 Specialized AI Associate Series: UiPath Communications Mining Overview
byDianaGray10
 
Formula 1 - APIC general MACO calculation.docx
byMarkus Janssen
 
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
How does a computer vision development company help retailers decode consumer...
byNextbrain Technologies
 
Formula 2 - APIC HBEL general calculation.docx
byMarkus Janssen
 
Build Agentic AI Applications with Oracle AI Database Private Agent Factory -...
bySandesh Rao
 
Igalia and WebKit: Status update and plans (2025)
byIgalia
 
2025 - AI terms & Meanings for Marketers
byashishav29
 
UiPath Automation Suite_Community Session_3/3
bysuhanisingh58689
 
2025 AI Adoption Report: Gen AI Fast-Tracks Into the Enterprise
byRazin Mustafiz
 
OutSystems ONE 2025-recap presentation.pptx
bymostafaothman27
 

NOSQL Overview, Neo4j Intro And Production Example (QCon London 2010)

  • 2.
    No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
  • 3.
    No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
  • 4.
    No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
  • 5.
    Neo4j the benefits of graph databases + a NOSQL overview QCon London 2010 #neo4j Emil Eifrem @emileifrem CEO, Neo Technology [email protected]
  • 6.
    What's the plan? Why now? – Four trends NOSQL overview Graph databases && Neo4j A production example of Neo4j Conclusions
  • 7.
    Trend 1: data setsize 40 2007 Source: IDC 2007
  • 8.
    988 Trend 1: data setsize 40 2007 2010 Source: IDC 2007
  • 9.
    Trend 2: connectedness Giant Global Information connectivity Graph (GGG) Ontologies RDF Folksonomies Tagging User- Wikis generated content Blogs RSS Hypertext Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020
  • 10.
    Trend 3: semi-structure Individualization of content! In the salary lists of the 1970s, all elements had exactly one job In the salary lists of the 2000s, we need 5 job columns! Or 8? Or 15? Trend accelerated by the decentralization of content generation that is the hallmark of the age of participation (“web 2.0”)
  • 11.
    Aside: RDBMS performance Relational database Salary List Performance Majority of Webapps Social network Semantic } Trading custom Data complexity
  • 12.
    Trend 4: architecture 1990s: Database as integration hub
  • 13.
    Trend 4: architecture 2000s: (Slowly towards...) Decoupled services with own backend
  • 14.
    Why NOSQL 2009? Trend 1: Size. Trend 2: Connectivity. Trend 3: Semi-structure. Trend 4: Architecture.
  • 15.
  • 16.
    First off: thename NoSQL is NOT “Never SQL” NoSQL is NOT “No To SQL”
  • 17.
    NOSQL is simply Not Only SQL!
  • 18.
    Four (emerging) NOSQLcategories Key-value stores Based on Amazon's Dynamo paper Data model: (global) collection of K-V pairs Example: Dynomite, Voldemort, Tokyo* BigTable clones Based on Google's BigTable paper Data model: big table, column families Example: HBase, Hypertable, Cassandra
  • 19.
    Four (emerging) NOSQLcategories Document databases Inspired by Lotus Notes Data model: collections of K-V collections Example: CouchDB, MongoDB Graph databases Inspired by Euler & graph theory Data model: nodes, rels, K-V on both Example: AllegroGraph, Sones, Neo4j
  • 20.
    NOSQL data models Data size Key-value stores Bigtable clones Document databases Graph databases Data complexity
  • 21.
    NOSQL data models Data size Key-value stores Bigtable clones Document databases Graph databases (This is still billions of 90% nodes & relationships) of use cases Data complexity
  • 22.
  • 23.
    The Graph DBmodel: representation Core abstractions: name = “Emil” age = 29 Nodes sex = “yes” Relationships between nodes 1 2 Properties on both type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  • 24.
    Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  • 25.
    Code (1): Buildinga node space GraphDatabaseService graphDb = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
  • 26.
    Code (1): Buildinga node space GraphDatabaseService graphDb = ... // Get factory Transaction tx = neo.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
  • 27.
    Code (1b): DefiningRelationshipTypes // In package org.neo4j.graphdb public interface RelationshipType { String name(); } // In package org.yourdomain.yourapp // Example on how to roll dynamic RelationshipTypes class MyDynamicRelType implements RelationshipType { private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; } } // Example on how to kick it, static-RelationshipType-like enum MyStaticRelTypes implements RelationshipType { KNOWS, WORKS_FOR, }
  • 28.
    Whiteboard friendly owns Björn Big Car build drives DayCare
  • 30.
    The Graph DBmodel: traversal Traverser framework for name = “Emil” high-performance traversing age = 31 sex = “yes” across the node space 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  • 31.
    Example: Mr Anderson’sfriends name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  • 32.
    Code (2): Traversinga node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
  • 33.
    name = “TheArchitect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity” $ bin/start-neo-example Mr Anderson's friends: At depth 1 => Morpheus friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, At depth 1 => Trinity StopEvaluator.END_OF_GRAPH, At depth 2 => Cypher ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, At depth 3 => Agent Smith Direction.OUTGOING ); $
  • 34.
    Example: Friends inlove? name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 7 3 KN O WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity”
  • 35.
    Code (3a): Customtraverser // Create a traverser that returns all “friends in love” Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
  • 36.
    Code (3a): Customtraverser // Traverse the node space and print out the result System.out.println( "Who’s a lover?" ); for ( Node person : loveTraverser ) { System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) ); }
  • 37.
    name = “TheArchitect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS KN O CODED_BY 1 7 3 WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity” $ bin/start-neo-example new ReturnableEvaluator() Who’s a lover? { public boolean isReturnableNode( TraversalPosition pos) At depth 1 => Trinity { $ return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } },
  • 38.
    Bonus code: domainmodel How do you implement your domain model? Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive: // In package org.yourdomain.yourapp class PersonImpl implements Person { private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; } public String getName() { return (String) this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); } }
  • 39.
    Domain layer frameworks Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Defines Entities / Associations / Properties Sound familiar? Nodes / Rel’s / Properties! Neo4j is an “EntityStore” backend Jo4neo (http://code.google.com/p/jo4neo) Annotation driven Weaves Neo4j-backed persistence into domain objects at runtime
  • 40.
    Neo4j system characteristics Disk-based Native graph storage engine with custom binary on-disk format Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc Scales up Many billions of nodes/rels/props on single JVM Robust 6+ years in 24/7 production
  • 41.
    Social network pathExists() 12 ~1k persons 3 7 1 Avg 50 friends per person pathExists(a, b) limit 36 41 77 depth 4 5 Two backends Eliminate disk IO so warm up caches
  • 42.
    Social network pathExists() 2 Emil 1 5 7 Mike Kevin 3 John Marcus 9 4 Bruce Leigh # persons query time Relational database 1 000 2 000 ms Graph database (Neo4j) 1 000 2 ms Graph database (Neo4j) 1 000 000 2 ms
  • 45.
    Pros & Conscompared to RDBMS + No O/R impedance mismatch (whiteboard friendly) + Can easily evolve schemas + Can represent semi-structured info + Can represent graphs/networks (with performance) - Lacks in tool and framework support - Few other implementations => potential lock in - No support for ad-hoc queries +
  • 46.
    Query languages SPARQL– “SQL for linked data” Ex: ”SELECT ?person WHERE { ?person neo4j:KNOWS ?friend . ?friend neo4j:KNOWS ?foe . ?foe neo4j:name “Larry Ellison” . }” Gremlin – “perl for graphs” Ex: ”./outE[@label='KNOWS']/inV[@age > 30]/@name”
  • 47.
    The Neo4j ecosystem Neo4j is an embedded database Tiny teeny lil jar file Component ecosystem index meta-model graph-matching remote-graphdb sparql-engine ... See http://components.neo4j.org
  • 48.
    Example: Neo4j-RDF Neo4j-RDF triple/quad store OWL SPARQL RDF Metamodel Graph match Neo4j
  • 49.
    Language bindings Neo4j.py– bindings for Jython and CPython http://components.neo4j.org/neo4j.py Neo4jrb – bindings for JRuby (incl RESTful API) http://wiki.neo4j.org/content/Ruby Neo4jrb-simple http://github.com/mdeiters/neo4jr-simple Clojure http://wiki.neo4j.org/content/Clojure Scala (incl RESTful API) http://wiki.neo4j.org/content/Scala
  • 52.
  • 53.
    Scale out –replication Rolling out Neo4j HA... soon :) Master-slave replication, 1st configuration MySQL style... ish Except all instances can write, synchronously between writing slave & master (strong consistency) Updates are asynchronously propagated to the other slaves (eventual consistency) This can handle billions of entities... … but not 100B
  • 54.
    Scale out –partitioning Sharding possible today … but you have to do manual work … just as with MySQL Great option: shard on top of resilient, scalable OSS app server , see: www.codecauldron.org Transparent partitioning? Neo4j 2.0 100B? Easy to say. Sliiiiightly harder to do. Fundamentals: BASE & eventual consistency Generic clustering algorithm as base case, but give lots of knobs for developers
  • 55.
  • 56.
    Case: Enterprise ContentManagement Background: Enterprise Content Management (think: “CMS but also with non-web content,” or “big filesystem on the webotubes”) Thousands of users Various content types: PDFs, images, videos, doc files, organization-specific XML formats “Multi-tentant SaaS”
  • 57.
    Outline A sagain three parts Part I: we're a file system on the web Part II: sharing is caring Part III: profit
  • 58.
    Part I: We'rea file system on the web Let's get something out there We shall store files in folders Ya know, versions are kinda cool
  • 59.
    Part I: conceptmodel Root Folder Sub v. 2 File 1 File 2 v. 1
  • 60.
    Part I: SQL?NOSQL? So hmm, this whole relational database thingie... Modeling hierarchies? Doable but kinda painful. Sucky code. And hmm, quite a lot of joins. Activity feeds Wouldn't it be cool if you could subscribe to a folder and get changes fed to you. Whoa, massive amount of joins! Denormalization, write explosion, code complexity.
  • 61.
    Part I: conceptmodel Root Folder Sub v. 2 File 1 How do you represent this in a graph database? File 2 v. 1
  • 62.
    Tadaa! Root Folder How do you implement activity feeds? Sub v. 2 File 1 Easier when you do ~1M traversals per second. :) File 2 v. 1 No need to denormalize and aggregate events at each folder level.
  • 63.
    Part II: Sharingis caring We're oh-so SaaS and multi-tentant Would be useful if we could share content between organizations Since we're all kinda running on top of the same system (not just same software) anyway
  • 64.
    Part II: conceptmodel (a) Trifork Root Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2
  • 65.
    Part II: Security Whoa, guys, security? Customer sez: we need to model organizations And suborganizations And hierarchical user groups Customer sez: and add some security to all that So add ACLs And incremental security
  • 66.
    Admin Group U2 U1 Trifork Org +R +W Trifork Root -W Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2 +RW U3
  • 67.
    Part II: Keywordtranslations Customer sez: we need to cut costs Ouch But we spend a lot of time on manually translating keyword lists and things like that Let's model that in the graph! Also, this whole graph thing is really kinda flexible... so let's throw in some topologies while we're at it!
  • 68.
    Admin Group U2 U1 Tree Trifork Woods Org EN EN +R Forest EN +W Plant Träd EN Skog SV Trifork SV Root -W Folder Sub v. 2 File 1 File 2 v. 1 Share Sub link File 1 InfoQ Root File 2 +RW U3
  • 69.
    Part III: profit Customer sez: I heart the cash If my customers make money, I make money Developers: “Gives me multi-tentant ecommerce!” Owait, say waht?
  • 70.
    Part III: multi-tentante-commerce? Conceptual breakdown: Every org can “enable e-commerce,” thereby making their content sellable Within every org, one should be able to model a supply chain of creator → syndicator → distributor → customer The distributor assigned by region and sets price: E.x. one dist for Scand, one for the UK Due to inter-org sharing (remember?), the same content can belong to several e-commerce “spheres”
  • 71.
    QCon Org Syndicator Role Ecom sph SI Distributor Role Sub File 1 Folder DR Price List World Scand. U1
  • 72.
    Finding the price So how do you actually figure out the price Throw the distributor Which is regionally bound Per content Per e-commerce sphere That's a shortest path algo!
  • 73.
    QCon Org Syndicator Role Ecom sph SI Distributor Sub Finding the price: Role File 1 Folder Equivalent to finding the SI shortest path from U1 to File1 DR Price along “purple and black” List Price relationship types. List DR JAOO World Org Scand. Distributor Role U1
  • 74.
    Admin Group U2 U1 Tree Trifork Woods Org EN EN +R Forest EN +W Plant Träd EN Skog SV Trifork SV Root -W QCon Folder Org Syndicator Role Ecom sph Sub v. 2 SI Distributor File 1 Role Sub File 1 Folder File 2 v. 1 Share SI DR Price List Price Sub List DR link File 1 JAOO World Org InfoQ Root File 2 Scand. Distributor +RW Role U1 U3
  • 75.
    ECM conclusions Oneexample of an evolving model Site had a lots of content, lots of users High read load Moderate write load Only backend: Neo4j “You go to a graph database for the performance... but you stay for the flexibility!”
  • 76.
    How ego areyou? (aka other impls?) Franz’ AllegroGraph (http://agraph.franz.com) Proprietary, Lisp, RDF-oriented but real graphdb Sones graphDB (http://sones.com) Proprietary, .NET, cloud-only, req invite for test Kloudshare (http://kloudshare.com) Graph database in the cloud, still stealth mode Google Pregel (http://bit.ly/dP9IP) We are oh-so-secret Some academic papers from ~10 years ago G = {V, E} #FAIL
  • 77.
    Conclusion Graphs &&Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if you’re open source, we’re open source” If you have proprietary software? Must buy a commercial license But the first one is free! Download http://neo4j.org Feedback http://lists.neo4j.org
  • 79.
    Questions? Image credit: lost again! Sorry :(
  • 80.