ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC...

23
ARM instrumentation of the PHOENIX ATC system for performance evaluation Dr. Kai Engels, Stefan Ruppert MyARM GbR Neue Straße 4, 63571 Gelnhausen-Rot Germany web: http://www.myarm.de eMail: [email protected] Ralf Heidger DFS Deutsche Flugsicherung GmbH Systemhaus, Dep. SH/T Am DFS Campus 7, 63225 Langen Germany eMail: [email protected] San Diego, CMG 2007, 6 December 2007

Transcript of ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC...

Page 1: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of thePHOENIX ATC system forperformance evaluation

Dr. Kai Engels, Stefan RuppertMyARM GbR

Neue Straße 4, 63571 Gelnhausen-Rot

Germanyweb: http://www.myarm.de

eMail: [email protected]

Ralf HeidgerDFS Deutsche Flugsicherung

GmbHSystemhaus, Dep. SH/T

Am DFS Campus 7, 63225 Langen Germany

eMail: [email protected]

San Diego, CMG 2007, 6 December 2007

Page 2: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 2

6 December 2007

Contents

Introduction PHOENIX ATC system Test environment and scenarios Response time types Measurement results MyARM environment Summary

Page 3: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 3

6 December 2007

Introduction

DFS develops their own ATC systems like PHOENIX

PHOENIX processes radar data and presents flights to a controller

ATC systems are subject to performance requirements (1 second requirement)

ARM measures distributed transactions (Correlators)

Performance evaluation for two different scenarios (live, artificial)

Page 4: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 4

6 December 2007

PHOENIX system architecture Client-Server based Runs on Linux OS Tracker (MRTS)

serves tracks by processing incoming radar data

Message Server (MSG) serves flight information data and links to tracks

Local Area Network

Controllerworkingposition(CWP)

Controllerworkingposition(CWP)

Controllerworkingposition(CWP)

TrackServer(MRTS)

MessageServer(MSG)

Controller Working Position (CWP) displays the tracks with all information

Radardata

Flightinfo

Page 5: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 5

6 December 2007

Page 6: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 6

6 December 2007

PHOENIX track label

Callsign

History

dots

Futurevector

Altitude

Speed

Page 7: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 7

6 December 2007

Scenarios (1) Live Scenario

− 30 minutes, 22. June 2006

− 700 tracks on average

Page 8: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 8

6 December 2007

Scenarios (2) Artificial

scenario, 10 minutes

− 100 tracks on 30 circles

− Exactly 3000 tracks on display

Page 9: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 9

6 December 2007

Standalone test environment

1 PC with Tracker (MRTS), Message Server (MSG) and CWP

Manager

Command line tools

WebInterface

Records

radar datagenerator

(d-gen)

Playback

radar datarecorder(d-rdr)

PHOENIX

Data Generation/Playback Test execution

Offline analysis

DB

MySQLServer

PHOENIX

ARMagentMRTSMSGCWP

Page 10: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 10

6 December 2007

Distributed test environment

1 MRTS/MSG Server, 110 CWPs PC, 1 DB Server

Manager

Command line tools

WebInterface

Records

radar datagenerator

(d-gen)

Playback

radar datarecorder(d-rdr)

PHOENIX

Data Generation/Playback Test execution

Offline analysis

DB

MySQLServer

PHOENIX

ARMagent

MRTSMSG

CWP

ARMagent

CWP

ARMagent

CWP

ARMagent

120CWPs

Page 11: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 11

6 December 2007

PHOENIX ARM instrumentation

ARM instrumentation points within PHOENIX processes:

MRTS MSG CWPPlayback Plots Tracks Tracks

decodePlots Conversion Correlate

TrackStore_updateTrack

MPA-ModeTrackStore_updateTrack TrackStore_updateTrackMPA TargetView_recalculate TargetView_redraw

Standard-ModeTrackStore_updateTrack TargetView_recalculate TargetView_redraw

Page 12: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 12

6 December 2007

Response time types (overview)

Synchronous response times (ARM 4.0)

Asynchronous response times (ARM 4.1)

− Asynchronous Parent-Child Duration (APCD)

− Asynchronous Child-Parent Duration (ACPD)

Page 13: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 13

6 December 2007

Response time types (APCD)

Transaction start times

− t1 MRTS decodePlots

− t2 MSG updateTrack

− t3 CWP redrawTarget

Asynchronous Parent-Child Duration of MRTS-CWP:

− APCDT1= t3 +∆t3 – t1

− only once per parent

Page 14: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 14

6 December 2007

Response time types (ACPD)

Transaction start times

− t1 MRTS decodePlots

− t2 MSG updateTrack

− t3 CWP redrawTarget Asynchronous Child-

Parent Duration of CWP-MRTS:

− ACPDT3/T1= t3 + ∆t3 – t1

− multiple times for a parent = once for each child

Page 15: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 15

6 December 2007

Results: artifical scenario, standalone test bed (1)

mean response times under heavy load around ~1 sec.

periodic outlier of ~4 sec.

bottleneck is CPU of CWP

Page 16: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 16

6 December 2007

Results: artifical scenario, standalone test bed (2)

MPA algorithm reduces response times by a factor of 4

MPA works as designed (reduce CPU usage)

MPA mode change decision needs to be checked

Page 17: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 17

6 December 2007

Results: live scenario, distributed test bed (1)

Mean response time ~137ms

~7 times better than 1 second

In MPA mode much higher mean ~373ms duration (factor 3)

Page 18: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 18

6 December 2007

Results: live scenario, distributed test bed (2)

less than 0,3% of all measurements are above 1 second

Fits into 3-sigma interval (99,7%)

Page 19: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 19

6 December 2007

MyARM environment (1)

Modular design of data sink and data sources for measurement data (DBs: MySQL, Oracle, Postgres)

Efficient measurement data transport (TCP/IP, Shared-Memory)

Tools for everyone

− Command line tools for batch processing

− Web interface for easy integration into existing platforms

− Powerful own management application

Page 20: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 20

6 December 2007

MyARM environment (2)

Transaction selection

Different views of ARM data

Supports any ARM concepts (Metrics, Properties)

Page 21: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 21

6 December 2007

Summary (Results) Operational environment (1 sec.

requirement)− mean response time is about 7 times

better− response time for presenting a track

fits into 3-sigma interval (99,7%). Under heavy load (3000 tracks)

− the mean response times go up to ~1048ms.

− High peaks (4 sec.) caused by CWP-MPA algorithm

Page 22: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 22

6 December 2007

Summary (ARM)

ARM measurements provide insight view of performance of main processing steps of PHOENIX

Constant usage of ARM is planned in the software development process of PHOENIX

Semi-automatic instrumentation of applications would be helpful (Qt® MOC Compiler using QArm)

ARM can provide more and better information about operational systems to the responsible system management

Page 23: ARM instrumentation of the PHOENIX ATC system for ... · ARM instrumentation of the PHOENIX ATC system for performance evaluation CMG 2007, San Diego, US Dr. K. Engels, R. Heidger,

ARM instrumentation of the PHOENIX ATC system for performance evaluation

CMG 2007, San Diego, US Dr. K. Engels, R. Heidger, St.

Slide 23

6 December 2007

The end

Thank you for listening

Any questions?