Forschungszentrum Jülich in der Helmholtz-Gemeinschaft UNICORE and Grid Computing in Europe Dietmar...

Forschungszentrum Jülichin der Helmholtz-Gemeinschaft

UNICORE UNICORE and Grid Computing in and Grid Computing in

EuropeEurope

Dietmar Erwin

Forschungszentrum Jü[email protected]

August 27, 2003

HPN Workshop via AG 2Forschungszentrum Jülichin der Helmholtz-Gemeinschaft

ContentsContents

• Introduction to FZJ

• Objectives and Status of UNICORE

• Grid Projects in Europe

• Eurogrid, GRIP and OGSA


“The Future is Our Mission”

Matter

Energy

Information

Life

Environment

Source FZJ


Forschungszentrum Jülich at a GlanceForschungszentrum Jülich at a Glance

Founded:Dec. 1956

Area:2.2 km2

Legal Form:PLC

Operator:•Federal Republic of Germany (90 %)•State of North Rhine-Westphalia (10 %)

Bodies:• Partners Meeting• Supervisory Board• Board of Directors• Scientific and Technical Council

Scientific/Technical Structure:•10 Departments (39 Institutes)• 7 Central Departments• 2 Programme Groups• 3 Project Management Groups

Budget: Institutional: € 236 Mio/yearNon-institutional: € 52 Mio /year

Staff: 4200Including: • 1000 Scientists

• 400 Postgraduate Students• 150 Undergraduate Students• 300 Trainees

Visiting Scientists: more than 700 annually from more than 30 countries

Source FZJ


UNICORE: Uniform Interface to UNICORE: Uniform Interface to Computing ResourcesComputing Resources

Developed software for a• seamless, • secure, and • intuitive access to distributed computing resources

http://www.unicore.orgFunded by BMBF, grant: 01 IR 001

ObjectivesObjectives


UNICORE HighlightsUNICORE Highlights

• Intuitive, system-independent creation and control of jobs

• Support for multi-system and multi-site jobs

• Dynamic flow control

• Integrated security through X.509 certificates

• Access to remote file stores and archives

• Extensible support for scientific & commercial applications

• Minimal intrusion into site autonomy


UNICORE UNICORE ScreenshotScreenshot


Batch Subsystem

AJO

User Certificate

Job Preparation Agent (JPA)

Job Monitor Controller (JMC)

Abstract status request

User authentication

List of SitesUNICORE Client

Target System Interface (TSI)

Incarnated job

Commands

User mapping,Resources info

Site-specific authorization

TSI TSI

Batch Subsystem

FZJ LRZ

...

...

TSI

Batch Subsystem

Preparation andControl of jobs

Network Job Superviser(NJS)

Unsafe Internet (SSL/https)

Safe Intranet (TCP/http)

IDB

Job incarnation

Sending jobs toother gateways,data transfer

Status request

SV1 T3E SR8000 files

UUDB IDBIDB UUDB IDB

NJS

UNICORE Gateway

optional firewall

optional firewall

UNICORE Gateway

NJS

UNICORE ArchitectureUNICORE Architecture


CPMD CPMD Application supportApplication support


Ad hoc Ad hoc weather predictionweather prediction

C: EUROGRID Projekt

Source DWD


UNICORE StatusUNICORE Status

• Project UNICORE Plus successfully completedReport at: http: //www.unicore.org

• UNICORE basis for a German HPC Grid(LRZ, HLRS, FZJ)

• Software available as Open Source for R&Dprojects http: //www.unicore.org/download

• UNICORE commercially supported and marketed by Pallas www.pallas.com

• UNICORE used in European Grid projects• UNICORE selected by Japanese NAREGI project


LRZ: SR 8000, VPP, LC*RUS: SX-5, 2xLC*FZJ: 2xT3E,SV1, LC*ZIB: T3E, SunDWD: IBM SP, SGIRUKA: IBM SPPC²: LC*TUD: T3E, SGIPallas: SunFujitsu: VPP, Sun

*LC = Linux Cluster

UNICORE DeploymentUNICORE Deployment

FZJFZJ

ZIBZIBPCPC22

TUDTUD

LRZLRZ

DWDDWD

RUKARUKA

RUSRUS


EGSOAVO

GRIA

CrossGridGridLab

GRIP

DataTAG

EuroGrid

DAMIEN

DataGrid

GEMSS

MammoGrid

BioGridSeLeNe

OpenMolGrid

COG

FlowGrid

GRACEMOSES

GRASP

Oct 2000 Oct 2001

Wave 1 Projects

Wave 2 Projects

Infrastructure

Applications

Middleware

Oct 2002

European ProjectsEuropean Projects

Source EC


• First actions launched in IST-FP5

• Grid research is a strategic objective in FP6

2000-2002 2002-2006

FP5

FP6

125M€

58M€From FP5 to FP6 Grid research

funding more than doubles

IST-FP6 commitment to Grid research

FP6 Grid FundingFP6 Grid Funding

Source EC


10

6

1 2

9

4

11

3

8

7

5

1

2

3

10

6

9

4

11

8

7

5

Forschungszentrum Jülich, D

Parallab - University of Bergen, N

CNRS - IDRIS, F

Pallas GmbH, D

debis Systemhaus,D

Warsaw University - ICM, P

Victoria University of Manchester, UK

Deutscher Wetterdienst, D

GIE EADS CCR, F

Fujitsu European Centre for Information Technology, UK

ETH Zürich, CH


Objectives Objectives

• Establish a European GRID network of leading European HPC centres

• Enhance UNICORE through:– Resource broker– Standard based File Transfer (gridFTP)

• Demonstrate Grid solution in – Bio molecular simulations,– Weather prediction,– Coupled CAE simulations,– Structural analysis.

Funded by EU, grant: IST-1999-20247


Globus

Application

Layer

Collective

Layer

Resource

Layer

Fabric

Layer

Grid ArchitectureGrid Architecture

Applications, Portals, Environments

Higher Services, APIs, Protocols

APIs, Protocols, SDKs

Computers, Networks, Devices

UNICORE


GRIP ObjectivesGRIP Objectives

1. To develop interoperability software between UNICORE and Globus based Grid systems

2. To develop and demonstrate techniques for inter-Grid applications in meteorology and bio-molecular science

3. To influence creation of international Grid standards through the Global Grid Forum (GGF)

1. To develop interoperability software between UNICORE and Globus based Grid systems

2. To develop and demonstrate techniques for inter-Grid applications in meteorology and bio-molecular science

3. To influence creation of international Grid standards through the Global Grid Forum (GGF)

Grid Interoperability: Creation of virtual organizations using different technologies based on common standardsGrid Interoperability: Creation of virtual organizations

using different technologies based on common standards

Seamless, secure, intuitive access to different systems and sites to solve complex problems

GRIP partners- lead GGF working/ research groups- work on the definition of OGSA, the Open Grid Service Architecture


GRIP PartnersGRIP Partners

Forschungszentrum Jülich (FZJ)

Pallas

University of Manchester (UoM)

ICM Warsaw (ICM)

Deutscher Wetterdienst (DWD)

University of Southampton (Soton)

Fujitsu European Laboratories

Argonne National Laboratories (ANL)


GRIP StatusGRIP Status

• UNICORE Interoperability Software implemented (FZJ)

• Interoperability UNICORE – Globus demonstrated

• Development of OGSA interfaces in 2003


Batch Subsystem

AJO

User Certificate

Job Preparation Agent (JPA)

Job Monitor Controller (JMC)

Abstract status request

User authentication

List of SitesUNICORE Client

Target System Interface (TSI)

Incarnated job

Commands

User mapping,Resources info

Site-specific authorization

TSI TSI

Batch Subsystem

FZJ

...

...

Globus TSI

Globus

Preparation andControl of jobs

Network Job Superviser(NJS)

Unsafe Internet (SSL/https)

Safe Intranet (TCP/http)

IDB

Job incarnation

Sending jobs toother gateways,data transfer

Status request

SV1 T3E files

UUDB IDBIDB UUDB IDB

NJS

UNICORE Gateway

optional firewall

optional firewall

UNICORE Gateway

NJS

UNICORE ArchitectureUNICORE Architecture


UNICORE & OGSI UNICORE & OGSI

UnicoreGateway

Unicore ClientNetwork

JobSupervisor

ResourceDatabase

UserDatabase

Organic Server

PrimePower

OGSA Service

ResourceBroker

• Work to be completed in 2003– Integrate Grid Service Hosting into Gateway – UPL portType

• Vsite Discovery• Consign Job• Retrieve Outcome

– Add Grid Service interface to Client– Broker Interfaces and Ontology

User certificatesecurity domain

User proxy certificatesecurity domain

Fujitsu Laboratories of Europe © 2003Fujitsu Laboratories of Europe © 2003


UnicoreGateway

Unicore ClientNetwork

JobSupervisor

ResourceDatabase

UserDatabase

Organic Server

PrimePower

OGSA Service

ResourceBroker

User certificatesecurity domain

User proxy certificatesecurity domain

Future WorkFuture Work

Full OGSA compliant UNICORE Implementation

Fujitsu Laboratories of Europe © 2003Fujitsu Laboratories of Europe © 2003


Further InformationFurther Information

• http://www.unicore.org

• http://www.pallas.com

• http://www.eurogrid.org

• http://www.grid-interoperability.org

• http://www.gridstart.org

• http://www.fz-juelich.de

Forschungszentrum Jülich in der Helmholtz-Gemeinschaft UNICORE and Grid Computing in Europe Dietmar...

Documents

Transcript of Forschungszentrum Jülich in der Helmholtz-Gemeinschaft UNICORE and Grid Computing in Europe Dietmar...