<Insert Picture Here>

Deutsche DWHs 2007 - Analyse und TrendsKai FischerStrategisch Technische Unterstützung (STU )

Terabyte Club 200713. Februar Oracle Frankfurt

Ziele: - Eine Oracle Community in D etablieren- Maßnahmen aus Trends ableiten

1. Terabyte Club Treffen 2005 10 von ( 21 ) Firmen, 20 TeilnehmerKeine Betreiber anwesendVorträge: O2, EDS Clearing House, DKRZ

2. Terabyte Club Treffen 200721 (von 38 ) Firmen, 37 TeilnehmerErste Betreiber sind dabei ( Atos Origin, Postbank Systems, T-Systems )Vorträge: Karstadt Quelle, Postbank Systems, JomoErste Analysen / Trends sind möglich

Deutscher Oracle Terabyte Club – Quo Vadis ?

- T-Mobile- E-Plus- Debitel- O2- T-Com- Talkline- Freenet / Mobilcom- Tengelmann- Bofrost- Jomo- Otto Versand- Karstadt Quelle- Globus SB Warenhaus- Deutsche Bank- Comdirect Bank- Postbank - HVB- Allianz

Deutscher Oracle Terabyte Club – Wer ?- Volkswagen- Ferdinand Bielstein- Continental Werke- KKH- AOK Berlin- Insight Health- Deutsche Post- Landesvermessungsamt BaWü- DKRZ- GfK Informatik- Deutsches Apotheken Prüfungsinstitut- Thyssen Krupp Stahl- ENBW- Schenker- Rhenus AG- BASF - ...

In Summe: 411 Terabytes Tablespace Used ( incl. Kompression )

50%

20%

13%

3%

7%7%

Kunde

T-Systems

ATOS

BASF IT

SBS

Postbank Systems

Betrieb des Data Warehouses

66%5%

29%

SMP

Loser Cluster

RAC

Aktuelle Architektur

37%

8%13%

21%

21%

SUN / FSC Sparc

HP PA-RISC

IBM Power

Itanium

x86/Opteron

CPU-Typ Verteilung

40%

18%

18%

13%

11%

Solaris

HP-UX

Linux

AIX

Windows

Betriebssystem Verteilung

34%

19%

19%

11%

11%3% 3%

EMC

HDS

HP

SUN

IBM

FSC

NEC

Storage Vendor Verteilung

85%

15%

SAN

Direct

Storage Anschluß SAN oder direkt ?

Oracle Release

66%

34%

Oracle 9.2

Oracle 10g

Oracle Datenhaltung

44%

28%

28%

Filesystem

RAW-Devices

ASM

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Du

rch

sa

tz M

B/S

ek

un

de

Durchsätze in MB/Sekunde

761 MB/Sek

0

100

200

300

400

500

600

Nu

be

r o

f p

hy

sic

al D

isk

s

Anzahl physikalischer Disks

170 Disks

Physikalische Disks/Terabyte

0

50

100

150

200

250

300

350

400

450

Ph

ys

ica

l D

isk

s/T

B

85 Disks / TB

45 Disks / TB

- Der Großteil der Kunden betreibt sein DWH selber- Betriebssysteme/CPUs gut verteilt, SPARC/Solaris

am stärksten, Linux bzw. x86 beginnt zu wachsen- RAC wird zu 29% verwendet !- Storage ist zu 85% über SAN angeschlossen- ASM und RAW Devices werden öfter als Filesystem

verwendet- Bei 10g DWH fast nur ASM - Sehr unterschiedliche DWH Leistungsfähigkeit im

IO-Bereich -> Disks/Terabyte- DWH „erben“ operationale Anforderungen ( Online

Backup, Mixed Workload, HA, Human Error etc. )

Status und Trends - Zusammenfassung

<Insert Picture Here>

Oracle Information Appliance InitiativeJoint solutions of Oracle with hardware vendors

<Insert Picture Here>

Agenda

• Data warehousing market trends• Oracle Information Appliance Initiative• Definition • Configuration methodology

• Q&A

Emerging Trends for DW

• Data warehouses have become ubiquitous parts of the IT infrastructure

• Problem: Data warehouse system configurations are easy to get wrong

• Goal: Make it easy to deliver• A scalable system• With reduced implementation times• Eliminated deployment risks• Lower cost

DW Configuration Problem: I/O

Database CPUs Memory Actuators LUNs Disks Raid

Database CPUs Memory Actuators LUNs Disks Raid

An unbalanced configuration

A balanced configuration

100%Possible

Efficiency

100%Possible

Efficiency

100%AchievedEfficiency

< 50%AchievedEfficiency

Sample Customer Configuration

FC-Switch1 FC-Switch2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

FC-Switch1 FC-Switch2

P69032 CPUs

24x 2Gb HBA

4x

4x

EMCDMX 2000P2 frames

6x

Max IO throughput: 25GB/sec

Max IO throughput: 18GB/sec72% capacity, 25%degradation

Max IO throughput: 5 – 6GB/sec20 -24% capacity, 76 – 80% degradation

<Insert Picture Here>

Agenda

• Data warehousing market trends• Oracle Information Appliance Initiative• Definition • Configuration methodology

• Q&A

Oracle Information Appliance Initiative

Appliance Foundations• Documented best-practice

configurations for data warehousing

• For customers requiring flexibility and choice

• Benefits:• High performance• Simple to scale: modular building

blocks• Built on Oracle database and

standard hardware

• Available today with HP, IBM, and EMC

Information Appliances• Scalable systems pre-installed and

pre-configured: ready to run out-of-the-box

• For customers looking for the simplest, fastest solutions

• Benefits:• High performance• Simple to buy• Fast to implement• Built on Oracle database and standard

hardware

• Available soon with Panta Systems• Further announcements in coming

months

<Insert Picture Here>

Oracle Information Appliance Foundations

Oracle Information Appliance FoundationWhat is it?

• Documented balanced system configurations for pre-defined DWBI environments

• Starting point for sizing a system• Balanced system consists of CPU, memory, I/O, and cabling• Sizing factors are raw data, avg. concurrent users and workload

complexity

• Leverages scalable, modular components• Enables incremental growth (scale-in, scale-out)• Mitigates implementation risks

• Different OS and HW configurations• Choice of Linux and proprietary Unix• Small, medium, and large nodes • “Classical” SMP and clustered solutions

• Different workload configurations• Price/Performance versus I/O intensive

• “Guided Choices”• Various alternative HW vendors and HW components• Sweet spot of price, performance, and fit into existing environment

Oracle Information Appliance FoundationWhat is it?

Var

ying

wor

kloa

d

Varying system size

• Spreadsheet-like guidance•First entry point, not all available configurations

• Choice of workload and data volume• Choice of hardware and operating system

Sizing and pricing starting point

(option 1)

Sizing and pricing starting point

(option 2)

Oracle Information Appliance FoundationSample OIA Foundation

• Balanced system configurations are built on modular balanced building blocks

• Add building blocks as you grow• Each building block is a balanced unit

DisksDisksDiskDisk DiskDiskDisksDisksDisksDisks DisksDisks DisksDisksDisksDisksDiskDisk DiskDisk DiskDisk DiskDiskDisksDisks

• Different building blocks provide “guided choices”• Many small nodes

versus few large nodes

Oracle Information Appliance FoundationWhat is it?

What is a balanced unit ?

•“The weakest link” defines the throughput

• Each building block is a balanced unit

•Components to consider:• CPU: Quantity and speed • HBA (Host Bus Adapter):

Quantity and speed• Switch speed• Controller: Quantity and

speed• Disk: Quantity and speed

FC-Switch1 FC-Switch2

DiskArray 1

DiskArray 2

DiskArray 3

DiskArray 4

DiskArray 5

DiskArray 6

DiskArray 7

DiskArray 8

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

Throughput Performance

Component theory (Bit/s) maximal Byte/s

HBA 1/2Gbit/s 100/200 Mbytes/s

16 Port Switch 8 x 2Gbit/s 1600 Mbytes/s

Fibre Channel 2Gbit/s 200 Mbytes/s

Disk Controller 2Gbit/s 200 Mbytes/s

GigE NIC 1Gbit/s 80 Mbytes/s

Infiniband 10Gbit/s 890 Mbytes/s

CPU 100-200MB/s

Grid Components Rough Sizing numbers

* 2Gbit based

Sample Balanced Unit

•“The weakest link” defines the throughput

• Each building block is a balanced unit

•Components to consider:• CPU: Quantity and speed • HBA (Host Bus Adapter):

Quantity and speed• Switch speed• Controller: Quantity and

speed• Disk: Quantity and speed

FC-Switch1 FC-Switch2

DiskArray 1

DiskArray 2

DiskArray 3

DiskArray 4

DiskArray 5

DiskArray 6

DiskArray 7

DiskArray 8

Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

Sample Balanced Unit

•“The weakest link” defines the throughput

• Each building block is a balanced unit

•Components to consider:• CPU: Quantity and speed • HBA (Host Bus Adapter):

Quantity and speed• Switch speed• Controller: Quantity and

speed• Disk: Quantity and speed

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

FC-Switch1 FC-Switch2

DiskArray 1

DiskArray 2

DiskArray 3

DiskArray 4

DiskArray 5

DiskArray 6

DiskArray 7

DiskArray 8

Each machine has 2 Gb HBAsAll 8 HBAs can sustain

8 * 200MB/s = 1600 MB/s

Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

Sample Balanced Unit

•“The weakest link” defines the throughput

• Each building block is a balanced unit

•Components to consider:• CPU: Quantity and speed • HBA (Host Bus Adapter):

Quantity and speed• Switch speed• Controller: Quantity and

speed• Disk: Quantity and speed

FC-Switch1 FC-Switch2

DiskArray 1

DiskArray 2

DiskArray 3

DiskArray 4

DiskArray 5

DiskArray 6

DiskArray 7

DiskArray 8

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

Each machine has 2 Gb HBAsAll 8 HBAs can sustain

8 * 200MB/s = 1600 MB/s

Each switch needs to support 800MB/s to guarantee a total system throughput of 1600 MB/s

Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

Sample Balanced Unit

•“The weakest link” defines the throughput

• Each building block is a balanced unit

•Components to consider:• CPU: Quantity and speed • HBA (Host Bus Adapter):

Quantity and speed• Switch speed• Controller: Quantity and

speed• Disk: Quantity and speed

FC-Switch1 FC-Switch2

DiskArray 1

DiskArray 2

DiskArray 3

DiskArray 4

DiskArray 5

DiskArray 6

DiskArray 7

DiskArray 8

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

HB

A1

HB

A2

Each machine has 2 Gb HBAsAll 8 HBAs can sustain

8 * 200MB/s = 1600 MB/s

Each switch needs to support 800MB/s to guarantee a total system throughput of 1600 MB/s

Each disk array has one 2Gbit controller All 8 disk arrays can sustain 8 * 200MB/s = 1600 MB/s

Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s

• High performance • Offers proven and well-known configurations

• Simple to scale: modular building blocks • Shorten the decision and implementation cycle

• Built on Oracle database and standard hardware• Optimal integration into existing hardware infrastructure

Oracle Information Appliance FoundationBenefits

Oracle Information Appliance Initiative

Appliance Foundations• Documented best-practice

configurations for data warehousing

• For customers requiring flexibility and choice

• Benefits:• High performance• Simple to scale: modular building

blocks• Built on Oracle database and

standard hardware

• Available today with HP, IBM, and EMC

Information Appliances• Scalable systems pre-installed

and pre-configured: ready to run out-of-the-box

• For customers looking for the simplest, fastest solutions

• Benefits:• High performance• Simple to buy• Fast to implement• Built on Oracle database and standard

hardware

• Available soon with Panta Systems• Further announcements in coming

months

Oracle Information Appliance Initiative

Information Appliance – An ExamplePanta 2700 Data Warehouse Appliance

• 8 Blade RAC cluster• 4 AMD 2.2Ghz dual core processors per blade• 8 GB memory per processor• Total of 64 cores and 256 GB memory

• 96 TB database storage• 532 disk drives• 250GB SATA 7200 RPM / drive

• 2 Silverstorm 9024 Infiniband Switches

• Up to 12 GB / sec I/O throughput observed in database processing

• Oracle Database 10g with RAC and Partitioning

• Red Hat Enterprise Linux Advanced Server 4 Update 3

Information Appliance:New 1TB TPC-H in October 2006

The system: • Example data warehouse appliance shown on the previous slide

• Pre-defined hardware and software stack

The results: • Performance: 59,353.9 QphH@1000GB

• Price-performance: $24.94 $/QphH@1000GB

Compelling combination of performance and price-performance: • #2 in absolute performance

• #2 in price-performance

As of October 23, 2006:  PANTA Systems PANTAmatrix, 59,353.9QphH@1000GB, $24.94/QphH@1000GB, available 4/15/07. 

Source:  Transaction Processing Performance Council (TPC), www.tpc.org

<Insert Picture Here>

Agenda

• Data warehousing market trends• Oracle Information Appliance Initiative• Definition • Configuration methodology

• Q&A

AQ&