Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Steinbuch...

Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH)

Steinbuch Centre for Computing (SCC)A large Multi-Functional HPC Centre

Rudolf Lohner, Frank Schmitz

2 | Steinbuch Centre for Computing | ZKI in Göttingen

University 11 faculties 120 institutes 4,000 employees 18,500 students250 mio € budget

University 11 faculties 120 institutes 4,000 employees 18,500 students250 mio € budget

Forschungszentrum 10 programs (research areas)

21 large institutes 4,000 employees

€ mio 310 budget

Forschungszentrum 10 programs (research areas)

21 large institutes 4,000 employees

310 mio € budget

Karlsruhe Institute of Technology (KIT)

10 km15 min

science

education

structures for the administration

institutes

services

infrastructure

KIT…


KIT / SCC

Karlsruhe Institute of Technology - two strong partners united

Universität Karlsruhe (TH) One of the most research oriented universities in Germany Nominated to one of three top universities in October 2006 Capitalized with over 100 million € over 5 years

Forschungzentrum Karlsruhe Member of the Helmholtz Association Internationally acknowledged non-university institution

Under the roof of KIT –Steinbuch Centre for Computing

Computing Center of the Universität Karlsruhe (TH) Institute of Scientific Computing of the Forschungszentrum Karlsruhe


IWR / FZK RZ Uni KA

Virtual Computing Centre since 1996

IT-Services for UKA High Performance Computing Numerical Methods on HPC Integrated Information Mgmt.

IT-Services for FZK Scientific Computing GridKa, CampusGrid, D-Grid Nat. & Intl. Grid Projects

Virtual Computing Centre Karlsruhe since 1996


SCC-BoardFour Professors and one

Technical Director

Networks andTelecommunication

Data Management and

Information-Services

Systemsand Server

Desktop- Workstation-

and Communi-cations Services

SOA and Virtualisation

Distributed Systems and Grid

Print-, Plot- und

Media Production

I/O, Operational

Services

and Help Desk

HPC -Systems,

-Applications, -Grid

IT-Security and Service-

Management

Project-Managemet and -Coordination

Quality- und Product-Mgmt.

Strategic Communiation & Relationship Mgmt.

Organisation Teaching, Advanced Education

Common Administration and Organisation

10 SCC Departments


KIT / SCC Roadmap

Karlsruhe Institute of Technology (KIT) Foundation Contract to be signed in 2007 Official start of KIT:

January 1st, 2008

Steinbuch Centre for Computing (SCC) Founded simultaneously with KIT Official start of SCC:

January 1st, 2008


SCC Portfolio

Infrastructure

Services

Projects


HPC platforms at SCC: HP XC6000 / XC4000

Q1 04

Q4 04

Q1 05

Q1 06

Q3 06

Phase 2:» Total of 3000 processor cores

» Total of 15.6 TFlop/s peak performance

» Total of 12 TB of main memory

» Total 110 TB of local disk space (scratch)

Phase 0 (Q1 2004)»12 2-way nodes (Itanium 2)»4 file server nodes

- 2 TB shared storage»Single rail Quadrics interconnect

Phase 1 (Q4 2004)»108 2-way nodes

- Intel Madison 6M, 1.5 Ghz, 12 GB»8 file server nodes

- Approx. 11 TB storage system»Single rail Quadrics interconnect

Phase 1 (Q1 2005)»6 16-way nodes

- Intel Madison 6M, 1.6 Ghz- 2 partions, each 8 CPUs, 64 GB

»Single rail Quadrics interconnect

Phase 2 (Q3 2006)»750 4-way nodes (two sockets)

- dual core AMD Opteron 2.6 Ghz- 16 GB main memory

»10 server nodes»10 file server nodes

- Approx. 56 TB storage system»Infiniband DDR interconnect


Globales Filesystem 56 TB

Infiniband 4x (DDR) Fast Interconnect

Ethernet Administration

Login nodesLustre

. . .

. . .

750 Compute nodes

Service nodes

Local Disksà 72 GB

. . .

Configuration of the HP XC4000


Parallel File System HP SFS on XC4000

Shared storage for all nodes of XC system

10 file server nodes 2 MDS / Admin 2 OSS for $HOME 6 OSS for $WORK

56 TB file space 8 TB $HOME 48 TB $WORK

Bandwidth Read / write from one node: 340 MB/s / 340 MB/s Total read / write bandwidth of $HOME: 600 MB/s / 360 MB/s Total read / write bandwidth of $WORK: 3600 MB/s / 2200

MB/s


GridKa, the Tier-1 computer center for LHCLHC=large hadron collider (CERN)

Grid based structure in a worldwide environment for high energy physics

• gLite

• storage resource manager (SRM) based on dCache

• Resource broker for world wide usage of LHC resources

• PBSpro for resource management

• Certification Authority (CA)


GridKa, usage in Germany

22 Research Centers

44 Working Groups

350 Scientists

Aachen (4)●Bielefeld (2)●Bochum (2)●Bonn (3)●Darmstadt (1)▲Dortmund (1)●Dresden (2)●Erlangen (1)●Frankfurt (1)●Freiburg (2)●Hamburg (1)▲Heidelberg (1)▲(6)●Karlsruhe (2)●Mainz (3)●Mannheim (1)●München (1)●(5)▲Münster (1)●Rostock (1)●Siegen (1)●Wuppertal (2)●

▲


GridKa, future plans

FZK has signed the WLCG MoU and is able to deliver resources for the LHC experiment up to 2022 as the Tier-1 center

resources for WLCG (Worldwide LHC Computing Grid Projekt


Motivation for the project CampusGrid

• heterogeneous IT-environment: vector-, SMP-, cluster-, blade- systems, SAN, NAS, Unix, Linux, Windows, Solaris, SuperUX, ….

• global view by the user

• only one user management (ADS from Microsoft)

• one job management

• metacomputing (MPI, ..), “real-time” applications

• access data for visualization at the local workstation

• global accounting

• seamless integration into different projects and middleware concepts like gLite, LCG, D-Grid, Unicore, GT4, ….


Infrastructure of the project CampusGrid

FSC Celsius

(MetaDataServer)

96 FSC RX 220

IBM eServer e325IBM p630 Power4

IBM xSeries 335

32 Sunfire V20z

Brocade Silkworm 48000

qLogic/Silverstorm IB directorIB-FC gateway

FibreChannel

InfiniBand10/20 Gbit/s

Brocade Switch and IBM Blade JS20/HS20/HS21

EMC Clariion CX700, S-ATA, FC disks

Intel Whitebox

16 Gbit/s

4 Gbit/s4 Gbit/s4 Gbit/s

4 Gbit/s

4 Gbit/s

FSC Celsius

(MetaDataServer)

4 Gbit/s

16 Gbit/s16 Gbit/s 32 Gbit/s

16 Gbit/s

NEC SX-8R environment


D-Grid (the German Grid initiative)

• starting September 2005 to build a sustainable Grid infrastructure

• six Community Grid projects an the D-Grid Integration Project (DGI)

• funded by BMBF, the Federal Ministry of Education and Research

• astronomy, climate research, high energy physics, engineering research, medical research, humanities

• gLite, GT4, UNICORE

• SRM/dCache for the data access

• heterogeneous CPU access (AIX, Linux, Solaris, SuperUX)


Shosholoza Project: America’s Cup

Form optimization/Model reduction/Weather forecast

Head of the Scientific Advisory Board

Metras Project: Weather Forecast

Cooperation with the University of Hamburg

Task force SCC (7 employees)

Simulation of Solid Oxid Fuel Cells

Detailed chemistry and 3D

United Airways: Simulation of the respiratory tract

Cooperation with the clinical Center (2 employees)

Numerical HPC

Numerics in HPC at the SCC - Projects


United Airways Project


Numerics in HPC at the SCC - Projects

Multicore and coprocessor technology

Cooperation with HP and ClearSpeed Dedicated Research Group

Junior Professor

Numerics on specific architectures Multicore Coprocessors

Visualization Project: on HP SVA

Instationary 3D flow

Software Development

HiFlow-Project: FEM, 3D, HPC (www.hiflow.de)

HiVision-Project: Visualization

OpenLB: Lattice-Boltzmann/HPC (www.openlb.org)


The influence of a Storage Lake in the Arid Elqui Valley in Chile on the Local Climate

out: Δx = 800 m, Δy = 800m

in(1): Δx = 400 m, Δy = 400 m

in(2): Δx = 200 m, Δy = 200 m

in(3): Δx = 100 m, Δy = 100 m

(out)

(in)

225 x 135 horizontal gridpoints 100 vertical levels


Thank you!

Questions?

Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Steinbuch...

Documents

Transcript of Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Steinbuch...