5 th EU DataGrid Conference, Budapest, September 2002 The European CrossGrid Project Marcel Kunze...

5th EU DataGrid Conference, Budapest, September 2002

The European CrossGrid Project

Marcel Kunze

Abteilung Grid-Computing und e-Science

Forschungszentrum Karlsruhe GmbH

On behalf of the CrossGrid Collaboration

Special thanks to Marian Bubak and Jesus Marco

www.eu-crossgrid.org

Forschungszentrum Karlsruhein der Helmholtz-Gemeinschaft


Outline

The CrossGrid Project

CrossGrid Aplications

Testbed Status

Deliverables and Plans


EU Funded Grid Project Space (Kyriakos Baxevanidis)

GRIDLAB

GRIA

EGSO

DATATAG

CROSSGRID

DATAGRID

Applications

GRIP EUROGRID

DAMIENMiddleware

& Tools

Underlying Infrastructures

ScienceIndustry / business

- Links with European National efforts

- Links with US projects (GriPhyN, PPDG, iVDGL,…)


CrossGrid Collaboration

Poland:Cyfronet & INP CracowPSNC PoznanICM & IPJ Warsaw

Portugal:LIP Lisbon

Spain:CSIC SantanderValencia & RedIrisUAB BarcelonaUSC Santiago & CESGA

Ireland:TCD Dublin

Italy:DATAMAT

Netherlands:UvA Amsterdam

Germany:FZK KarlsruheTUM MunichUSTU Stuttgart

Slovakia:II SAS Bratislava

Greece:AlgosystemsDemo AthensAuTh Thessaloniki

Cyprus:UCY Nikosia

Austria:U.Linz

21 21 institutinstitutees s

11 11 countriescountries


Main Objectives

EU Vth Framework Programme (IST) Project, started March 2002

21 partners from 11 countries

New category of Grid enabled applications Computing and data intensive Distributed Interactive, near real time response (a person in a loop) Layered

New programming tools

Grid more user friendly, secure and efficient

Interoperability with other Grids

Implementation of standards


Collaboration with other Grid Projects

Exchange of Information Software components

Partners DATAGRID DATATAG GRIDLAB EUROGRID and GRIP

GRIDSTART

Participation in GGF


Workpackages

WP1 – CrossGrid Application Development Biomedical simulation and visualization Flooding crisis support Interactive distributed data analysis in HEP Weather forecast and air pollution modeling

WP2 - Grid Application Programming Environments Tools for parallel programming and debugging on the Grid (MPI)

WP3 – New Grid Services and Tools Portals and roaming access Resource management Monitoring Optimisation of data access

WP4 - International Testbed Organisation Integration team, certification authority, support and test procedures Based on EDG 1.2 (see talk of Marcus Hardt in WP6 parallel session)

WP5 - Project Management Coordination, architecture, dissemination


Biomedical Application

CT / MRI scan

MedicalDB

Segmentation

MedicalDB

LB flowsimulation

VEWDPC

PDA

Visualization

Interaction

HDB

10 simulations/day60 GB/simulation> 20 MB/s


VR-Interaction


Cascade of Flood Simulations

Data sources

Meteorological simulations

Hydraulic simulations

Hydrological simulations

Users

Output visualization


Example of the Flood Simulation - Flow and Water Depth


Distributed Data Analysis in HEP

Objectives Distributed data access

Distributed data mining techniques with neural networks

Issues Typical interactive requests will run on o(TB) distributed data

Transfer/replication times for the whole data about one hour

Data transfers once and in advance of the interactive session

Allocation, installation and set-up of corresponding database servers before the interactive session

Integration of user-friendly interactive access (based on PROOF)


Parallel ROOT Facility: PROOF

Local

Remote

Selection

Parameters

Procedure

Proc.C

Proc.C

Proc.C

Proc.C

Proc.C

PROOF

CPU

CPU

CPU

CPU

CPU

CPU

TagDB

RDB

DB1

DB4

DB5

DB6

DB3

DB2

http://root.cern.ch


Distributed/parallel codes on the Grid Coupled Ocean/Atmosphere Mesoscale Prediction System

STEM-II Air Pollution Code

Integration of distributed databases

Data mining applied to downscaling weather forecast

Weather Forecast and Air Pollution Modeling


Key Features of CrossGrid Applications

Data Data sources and data bases geographically distributed

To be selected on demand

Processing Large processing capacity required; both HPC & HTC

Interactive

Presentation Complex data requires versatile 3D visualisation

Support for interaction and feedback to other components


Overview of the CrossGrid Architecture

Supporting Tools

1.4Meteo

Pollution

1.4Meteo

Pollution

3.1 Portal & Migrating Desktop

3.1 Portal & Migrating Desktop

ApplicationsDevelopment

Support

2.4Performance

Analysis

2.4Performance

Analysis

2.2 MPI Verification

2.2 MPI Verification

2.3 Metrics and Benchmarks

2.3 Metrics and Benchmarks

App. Spec Services

1.1 Grid Visualisation

Kernel

1.1 Grid Visualisation

Kernel

1.3 DataMining on Grid (NN)

1.3 DataMining on Grid (NN)

1.3 Interactive Distributed

Data Access

1.3 Interactive Distributed

Data Access

3.1Roaming Access

3.1Roaming Access

3.2Scheduling

Agents

3.2Scheduling

Agents

3.3Grid

Monitoring

3.3Grid

Monitoring

MPICH-GMPICH-G

Fabric

1.1, 1.2 HLA and others

1.1, 1.2 HLA and others

3.4Optimization of

Grid Data Access

3.4Optimization of

Grid Data Access

1.2Flooding

1.2Flooding

1.1BioMed

1.1BioMed

Applications

Generic Services

GRAMGRAM GSIGSIReplica CatalogReplica CatalogGIS / MDSGIS / MDSGridFTPGridFTP Globus-IOGlobus-IO

DataGridReplica

Manager

DataGridReplica

Manager

DataGrid Job Submission

Service

DataGrid Job Submission

Service

Resource Manager

(CE)

Resource Manager

(CE)

CPUCPU

ResourceManagerResourceManager

Resource Manager

(SE)

Resource Manager

(SE)Secondary

StorageSecondary

Storage

ResourceManagerResourceManager

Instruments ( Satelites,

Radars)

Instruments ( Satelites,

Radars)

3.4Optimization of

Local Data Access

3.4Optimization of

Local Data Access

Tertiary StorageTertiary Storage

Replica CatalogReplica Catalog

GlobusReplica

Manager

GlobusReplica

Manager

1.1User Interaction

Services

1.1User Interaction

Services


Status after M6++

Software Requirements Specifications together with use cases

CrossGrid Architecture defined

Detailed Design documents for tools and the new Grid services (OO approach, UML)

Analysis of security issues and the first proposal of solutions

Detailed description of the test and integration procedures

Testbed first experience Sites: LIP, FZK, CSIC+USC, PSNC, AuTH+Demo Basic: EDG release 1.2 Applications:

EDG HEP simulations (Atlas,CMS) first distributed prototypes using MPI:

NN distributed training Evolutionary Algorithms


CrossGrid Testbed Map

UCY NikosiaDEMO Athens

Auth Thessaloniki

CYFRONET Cracow

ICM & IPJ Warsaw

PSNC Poznan

CSIC IFIC Valencia

UAB Barcelona

CSIC-UC IFCA

Santander

CSIC RedIris Madrid

LIP Lisbon

USC Santiago

TCD Dublin

UvA Amsterdam

FZK Karlsruhe

II SAS Bratislava

Géant


Sample Testbed Sites

Valencia GoG farm, Santander (GridWall), FZK


Plans for the Future

Participation in production testbed with DataGrid All sites will be ready to join by end of September Common DEMO at IST 2002, Copenhagen, November 4th-6th

Collaboration with DataGrid in specific points (e.g. user support and helpdesk software)

Conference together with RI Forum and the “Across Grids” Santiago de Compostella, Spain, February 9th-14th,2003 With Proceedings (reviewed papers)

CrossGrid workshop, Linz (w/ EuroPVM/MPI 2002), September 28th-29th


Linz CrossGrid Workshop Sep.28th-29th

Evaluate the current status of all tasks

Contact partners who are developing software we are going to use: discuss interfaces and functionality

Understand what we may expect as first prototypes

Coordinate the operation of testbeds

Agree about common rules for software development:Standard Operational Procedures (SOP) document written

Start to organize the first CrossGrid EU review

Meet with EU DataGrid representatives

Discuss the technology for the future (OGSA)

Details at http://www.gup.uni-linz.ac.at/crossgrid/workshop/


CrossGrid Schedule and Deliverables

M6 (now) WPs 1-3 start to develop prototypes of the applications, tool environment, and the new grid services WP4 are working on launching the first testbed.

M9 (Dec.2002) WP4 internal status report WP5 CrossGrid website

M10 (Jan. 2003) WP4 First testbed prototype release

M11 (Feb. 2003) WP5 Project leaflet/brochure WP5 Dissemination and exploitation report CrossGrid conference in Santiago di Compostella, February 2002

M12 (March 2003) WP1 First software release WP2 First prototypes WP3 First prototypes WP5 Report on requirements on integration and interoperability with DataGrid WP5 Yearly report


1980s: Internet 1990s: Web 2000s: Grid

Where do we need to get to ?

Applications to support an “e-society” (“Cyber-Infrastructure”)

A Grid infrastructure which hides the complexities from the users (“Invisible Computing”)

A powerful and flexible network infrastructure

Where do we need to invest ?Applications targeted at realistic problems in “e-science”

Prototypes of Grid infrastructures

Maintain and improve the GEANT network

Expression of Interest for EU FP6 program:“Enabling Grids and e-Science in Europe (EGEE)”

Grid-enabled Applications

Prototype Grid Infrastructures

Gèant: World Class Networking

5 th EU DataGrid Conference, Budapest, September 2002 The European CrossGrid Project Marcel Kunze...

Documents

Transcript of 5 th EU DataGrid Conference, Budapest, September 2002 The European CrossGrid Project Marcel Kunze...