MongoDB, Neo4j & Co. Einführung NoSQL-Datenbanken_

Tobias Trelle@tobiastrelle

Senior IT Consultant @codecentric

Software Architect

Java/JEE/RDBMS background

Committer Spring Data

@tobiastrelle

ttrelle

2

NoSQL

3

4

What‘s My Name Again?

It’s not about using no SQL any more but about thinking beyond SQL for certain problems!

NoSQL

5

Why NoSQL?

NoSQL databases help to solve the 3 problems of

BigData (Gartner, 2009):

• Volume (Menge an Daten)

• Variety (verschiedene Strukturen)

• Velocity (Geschwindigkeit)6

The Trouble with Scaling

Vertical vs.Horizontal

Scalability

7

Vertical Scalability

8

RAMCPU

Storage

Vertical Scalability

9

RAMCPU

Storage

Vertical Scalability

10

RAMCPU

Storage

Vertical Scalability

Size

Cost

„BIG“ DATABASES SCALE

VERTICALLY

BUT AT WHATPRICE?

11

Horizontal Scalability

RAMCPU

Storage

12

Horizontal Scalability

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

13

Horizontal Scalability

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

RAMCPU

Storage

14

Google Data Center

15

Vertical vs. Horizontal Scalability

NOSQL DATABASES SCALE HORIZONTALLY

NEED AN

EXAMPLE?

MESSAGING SYSTEM OF (POWERED BY )

- 100 NODE CLUSTER- 87 BILLION R/W-OPERATIONS / DAY- 9+ BILLION MESSAGES / DAY- 4 PB COMPRESSED DATA / DAY- 45% GROWTH / MONTH SOURCE: NICOLAS SPIEGELBERG @

BERLIN BUZZWORDS, JUNE 2012

QUITE IMPRESSIVE

NUMBERS, EH?

16

For a distributed computer system it is impossible to simultaneously provide all these three guarantees!

Partition

Tolerance

failure of single nodes doesn‘t effect

the overall system

CAP Theorem a.k.a Brewer’s Theorem (2000)

Consisteny all nodes

see the same data at the same time

Availability

a guarantee that every

request receives a response

PICK 2!

17

Partion

Tolerance

Visual Guide to (No)SQL systems

Consisten

cy

Availability

18

CAP – Netzwerk-Partition

19

ACID vs. BASE

Atomicity

Consistency

Isolation

Durability1983

RDBMS

20

ACID vs. BASE

ACID is a good idea, but it is not an unbreakable law! 2

1

ACID vs. BASE

Basically Available

Soft State

Eventually consistent

2008

NoSQL

22

Overview of NoSQL Categories

11 1 1

1 11 11

11

Column StoresKey-Value StoresK V

K V

K V

Graph Databases

23

Document Stores

KKK

V

V

K

V

V

Current trends in the (No)SQL World

http://www.db-engines.com

24

MongoDB

25

MongoDBhumongous, engl. = gigantisch, riesig

26

MongoDB > FaktenVersion 3.0.5Kategorie: DokumentenorientiertAbfragen: • Query-by-Example + reichhaltige

Operatoren• Aggregationen / MapReduce• Geodaten, Volltextsuche

Indizes: Primär-Index + frei wählbare, auch zusammengesetzte Sekundär-Indizes

Replikation: Master/Slave mit automatischen FailoverSharding: Auto-Sharding anhand wählbarem SchlüsselTX: Atomarität auf einem DatensatzAPI: • Binäres TCP/IP-basiertes Protokoll

• Treiber für viele Sprachen27

MongoDB > FaktenLizenz: APGL (DB), Apache V2 (Treiber)Hersteller: MongoDB Inc. (vormals 10gen)Web: http://www.mongodb.orgGithub: https://github.com/mongodb

28

MongoDB > Dokument

Dokument := Geordnete Menge von Key-Value-PaarenKey := UTF8-StringValue := Einfacher Datentyp | Array | Dokument

Einfacher Datentyp := UTF8-String| Integer32| Integer64| Double| Date| Boolean

…29

MongoDB > Dokument{"_id" : ObjectId("53e3663ccb3bd259f9252f67"),"typ" : ["gastro", "kultur"],"name" : "Unperfekthaus","tags" : "uph unperfekt perfekt haus essen","desc" : "Im Unperfekthaus bekommen Künstler & Gründer ... ","adresse" : { "str" : "Friedrich-Ebert-Straße 18",

"plz" : 45127,"ort" : "Essen"

},"location" : {"type" : "Point",

"coordinates" : [ 7.0075, 51.45902 ]}

}

30

MongoDB > Server

mongodServer

Database

Collection

local admin productstest

pois system.indexes system.profile

Doc 1Doc 2

Doc 3Index

...

...

Query Scope

31

MongoDB > APIs

MongoDB-Server

Browser REST-Client

Treiber

Anwendung

HTTP(optional)

REST(optional)

TCP/IP

verwendet

32

MongoDB > Java APIs

Object / DocumentMapper

Driver

MongoDB-Server

Java-Treiber

Spring Data MongoDB Morphia Jongo ...

TCP / IP

33

MongoDB > Tools > Mongo Shell

34

MongoDB > Tools > Robomongo

35

Neo4J

36

Neo4jGraph-Datenbank

37

Neo4j > FaktenVersion 2.2.5Kategorie: Graphen-DatenbankAbfragen: Query Languages: Cypher, Gremlin:

• Graph Traversal• Aggregation, Projektion, …

Indizes: Primär-Index + frei wählbare, auch zusammengesetzte Sekundär-Indizes

Replikation: Master/Slave mit automatischen FailoverSharding: Keine disjunkte Verteilung, aber sog. Cache

ShardingTX: ACID, mit JDBC auch JTA

38

Neo4j > FaktenAPI: • Java-API (Embedded)

• REST / HTTP• JDBC

Lizenz: GPL v3 (Community Edtion)Hersteller: Neo Technology Inc.Web: http://www.neo4j.org/Github: https://github.com/neo4j

39

Neo4j > Graph

Graph := ( V, E ) mit

V := Menge Knoten (Vertices)E := Teilmenge von (V x V) (Edges)

Neo4j-Graph := Gerichteter Graph,Knoten und Beziehungen können

Eigenschaftenhaben sog. Property Graph

40

Neo4j > Graph

name: "Joe"age: 35

name: "Neo4j" name: "MongoDB"

:Person

:Database :Database

:Graph :Document

:KNOWSskill: 10

since: 2007

:KNOWSskill: 5

41

Neo4j > Server

Neo4jServer

Database Graph

Doc 2Doc 3Index

KnotenDoc 2Doc 3Beziehungen

Doc 2Doc 3Labels

Constraints

42

Neo4j > APIs

Neo4j

(embedded)

JVM

Java API Anwendung

Neo4j

REST APITransactionalHTTP EndpointJDBC

43

Neo4j > Tools > Web-Interface

44

Redis

45

RedisKey-Value-Datenbank

46

Redis > FaktenVersion 3.0Kategorie: Key-Value-DatenbankAbfragen: Key-only, Pub/Sub

Indizes: Zugriff per se nur über Key

Replikation: Master/Slave mit autom. FailoverSharding: Hash-basierte, disjunkte Verteilung der

DatenTX: ACID mit opt. Locking auf Menge von KeysAPI: • Protokoll: ASCII over TCP/IP

• Treiber für viele Sprachen47

Redis > FaktenLizenz: Three clause BSD licenseHersteller: Salvatore Sanfilippo (Pivotal) and Pieter

NoordhuisWeb: http://www.redis.io/Github: https://github.com/antirez/redis

48

Redis > Key-ValueKey := StringValue := Simple Value | List | Set | Hash | ZSet

Simple Value := String | Integer

List := SV1, SV2, …

Set := SV1, SV2, … (eindeutig)

Hash := (K1, SV1, K2, SV2, …)

ZSet := (score_1, V1, score2, V2, …) (nach score sortierbar)

49

Redis >Beispiel

Simple a:1

l:12

"hello Redis"

"hello"List

"Redis" "hello"

Set s:1 "hello" "Redis"

Hash h:1 name "Redis"

typ "key-value"

version 2.8

ZSet z:1 42 "/"

23 "/inbox"

2 "/inbox/joe"

50

Redis > Server

redis-serverServer

Database

Keyspace

db0 db1 db15...

Key 1

Key 2

...

Value 1

Value 2

...

51

Redis > APIs

High LevelFrameworks

Low LevelJava Clients

Redis Server

JRedis

Spring Data Redis

Jedis Lettuce

ASCII over TCP / IP

52

Redis > Tools > redis-cli

53

Redis > Tools > Redis Desktop Manager

54