Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte...

124
Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik und Informatik Technische Universität Berlin

Transcript of Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte...

Page 1: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte

Publish/Subscribe-Systeme

Dr. Gero Mühl

Kommunikations- und BetriebssystemeFakultät für Elektrotechnik und InformatikTechnische Universität Berlin

Page 2: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 2

Agenda

> Introduction> Notification Selection> Data and Filter Models> Content-Based Routing Algorithms> Efficient Use of Multicast> Matching Algorithms> Structuring Publish/Subscribe Systems> Formal Specification of Publish/Subscribe Systems

Page 3: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Introduction

Page 4: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 4

Motivation

> Trend towards information-driven applications> Automated (or require at least reduced user interaction)> Depend on external data sources> Selection, filtering, and combination of data> React to changes

> Information no longer static but rather flowing from producers to consumers!> Mechanisms to access static data (queries)

different from those controlling the flow of information (subscriptions/filters)!

Page 5: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 5

Source: http://www.epm.ornl.gov/~geist/Icons/needinput.gif

Page 6: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 6

Motivation (contd.)

> Application examples> Personalized news services> Stock monitoring software> Distributed auction platforms> Monitoring and adaptation of distributed systems> Wireless sensor networks (data fusion etc.)> Zero latency enterprise (ZLE)

Page 7: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 7

Client/Server Systems

> Two Roles> Clients request data or functionality

from a server> Servers reply to clients‘ requests⇒ Request/Reply interaction

> Characteristics> Synchronous communication

(initiated by the client)> Direct Addressing⇒Tight coupling

Server(data source)

Clients(data sinks)

1. Request:Current quote of stock „DT“?

2. Reply:12,52€at 15:02:34

...

Page 8: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 8

Client/Server Systems (contd.)

> Well suited for accessing static information> Disadvantages regarding information-driven applications

> Data source must be queried periodically (polling)> Communication occurs regardless whether new data is available> Every query costs (network bandwidth, computing power)> Every client queries individually> Low latency implies high polling frequency> Inaccuracy chains along multiple information flow steps

⇒ Trade-off between accuracy and scalability⇒ Client/server systems do not suit

information-driven applications

Page 9: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 9

Publish/Subscribe Systems

> Two Roles> Producers publish notifications> Consumers subscribe to

notifications

> Notification service (NS)delivers notifications

> Characteristics of Pub/Sub> Asynchronous communication

(initiated by the producer)> Indirect Addressing (anonymous)⇒ Loose coupling

NS

n

n

producer 1.

2.

notification

Consumer subscribed to“Stock = DT”

(Stock, DT)(Price, 12,52€)(Time, 15:02:34)

Consumer subscribed to“Stock = DT ∧Price > 13€”

......

Page 10: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 10

Publish/Subscribe Systems (contd.)

> Advantages regarding information-driven applications> Communication only occurs when new data is available> Low latency because initiation done by producer> Distribution to many consumers enables optimization

(e.g., exploitation of multicast)

⇒ Publish/Subscribe better suits information-driven applications!

Page 11: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 11

Publish/Subscribe API

Page 12: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 12

Pub/Sub Industry Products

> TIBCO Rendezvous> CORBA Notification Service implementations

(e.g., by Iona)> Java message service (JMS) implementations

(e.g., SUN ONE Message Queue)> Oracle Advanced Queuing> IBM WebSphere MQ (formerly MQSeries)> MS .NET notifications> …

Page 13: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 13

Pub/Sub Research Prototypes

> SIENA (Univ. of Colorado at Boulder)> Jedi (Politecnico di Milano, Italy)> Elvin (DSTC, Australia)> REBECA (TU Darmstadt and TU Berlin, Germany)> Gryphon (IBM TJ Watson Research Center, USA)> X2TS (TU Darmstadt, Germany)> Hermes (Univ. of Cambridge, UK)> …

Page 14: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 14

Application Scenarios

> Securities data feed handler publish the latest stock prices to hundreds of traders on a trading floor simultaneously

> Materials movement systems distribute data to various materials handlers, controllers, and tracking systems on a factory floor

> Inventory levels flow continuously to accounting, purchasing, marketing, and other departments in a retail store

> Network components are automatically monitored for optimizing performance and detecting failures

> …

Page 15: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 15

Enterprise Application Integration (EAI)

> Integration of new and legacy applications> Heterogeneous environment

> Different protocols> Java RMI, Sockets, CORBA IIOP, SMTP, HTTP etc.

> Different message formats> XML, EDI, proprietary format etc.

> Different operating systems and hardware platforms> …

> Grown applications result in “Application Spaghetti”> At most n2 / 2 interfaces and network connections⇒ Point-to-point connections quickly become infeasible!

Page 16: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 16

Application Spaghetti

Page 17: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 17

EAI Architectures

> EAI main application and origin of message-oriented middleware in general and publish/subscribe in particular

> Reason is decoupling of senders and receivers

> Two main architectures> Hub-and-Spoke> Message Bus

Page 18: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 18

Hub-and-Spoke

MessageServer

Client

Client

Client

Client

> Centralized architecture> Reduces number of network

connections and interfaces> Decoupling of sender and

receiver> Most functionality embedded

into the server> Authentication and

authorization> Guaranteed message delivery> Message transformation and

routing

Hub

Spoke

Page 19: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 19

Message Busses

> Decentralized architecture> Messaging functionality

embedded into all nodes> Often rely on multicast

communication> More complete decoupling of

senders and receivers> No single point of failure or

bottleneck

Client Client

Client

Bus

Page 20: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 20

Zero Latency Enterprise

“Zero latency is the real-time, enterprise wide dissemination of new information distributed in such a way that allows businesses to react quickly to it, driving the competitive business advantage to its ultimate limits” (Paul Larson,Talarian Corporation)

“Instantaneous awareness and appropriate response to events across an entire enterprise”(Roy Schulte Vice President, World Services, Gartner)

According to Gartner, a real-time enterprise is event-driven [3].

Page 21: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 21

Information Dissemination

> Information is disseminated to large set of consumers> Optimal technique depends on

> Number of producers> Number of consumers> Types of information> Interests of consumers> …

⇒ Data Delivery Options

Page 22: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 22

Data Delivery Options

pull push

aperiodic periodic aperiodic periodic

1:1 1:n 1:1 1:n 1:1 1:n 1:1 1:n

Request/Reply

Request/Replywith snooping

Polling Pollingwith snooping

Callback,Email

Publish/Subscribe

Reminders,Email lists

Broadcastdiscs

Proposed by Franklin and Zdonik [1]

Page 23: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 23

Is it Push or Pull++?

> Web browser periodically requests web page ⇒ pull

> But this looks to the user like push!> Low latency implies high polling rate> Architecture used by Pointcast

WebBrowser

WebServer

User

Pull

Push

Page 24: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 24

Web Caching

Web Server

Internet

ClientCache

Cache

Cache

> Data is pushed from the web server to the caches

> Client is redirected to nearest cache

> E.g., text is retrieved from web server but images from nearest cache ( Akamai)

Page 25: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 25

Stock Monitoring / Automated Stock Trading

> Application requires low latency> E.g., stocks are bought or sold

> if prices rise above or fall under a given price> if prices rise or fall by at least a given amount in some time

interval (3% in 15 minutes)> Arbitrage trading exploits pricing differences at different

exchanges> Price movement of different stocks can be put into relation> Publish/subscribe prevalent in the financial industry!

> E.g., TIBCO Rendezvous used for the NASDAQ trading system

Page 26: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 26

Ubiquitous Computing

> Raises new requirements> Limited bandwidth, connectivity,

processing power, and battery capacity⇒ Information filtering becomes

more important⇒ Data aggregation on the

server-side> Context-aware services

> Location, speed etc.⇒ Spatial subscriptions

(photomontage)

Page 27: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 27

Related Work

> Active databases> GUI programming> Event-triggered real-time systems> Monitoring and debugging> Workflow systems> …

Page 28: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 28

Active Databases

> Allow to define triggers(aka. event condition action rules)

> Triggers can be executed if data changes (i.e., on update, insert, or delete of table rows) or if a transaction starts, commits, or aborts

> Active functionality today incorporated in most databases

> Trigger Syntax in Oracle> {BEFORE|AFTER} {INSERT|DELETE|UPDATE} ON <table_name>

[REFERENCING [NEW AS <new_row_name>] [OLD AS <old_row_name>]] [FOR EACH ROW [WHEN (<trigger_condition>)]] <trigger_body>

Page 29: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 29

Graphical User Interfaces (GUIs): Listener Pattern

> Register/notify used for a long time (e.g., Java AWT)> GUI components (e.g., buttons) emit events > Events (e.g., button pressed) correspond to state changes

of the respective component> Event listeners register event handlers directly at the

event source (i.e., the button)> Handlers are called if matching event is raised

Page 30: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 30

Event-Driven Real-Time Systems

> Real-time systems must obey timing conditions> E.g., earliest start time and deadlines for tasks

> Time-triggered real-time system> Predefined static schedule (task execution plan)> E.g., sensor is read every 50ms

> Event-driven real-time system> No predefined schedule> Tasks are executed when events occur> E.g., sensor issues interrupt if temperature changes> More difficult to analyze due to dynamic behavior

Page 31: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 31

Bibliography1. M. J. Franklin and S. B. Zdonik. "Data In Your Face": Push Technology in

Perspective. Proceedings ACM SIGMOD International Conference on Management of Data (SIGMOD'98), pages 516--519, Seattle, USA, 1998. ACM Press.

2. N. W. Paton and O. Diaz. Active Database Systems. ACM Computing Surveys, 31(1):63--103, 1999.

3. A. S. Tanenbaum and M. van Steen. Distributed Systems: Principles and Paradigms. Prentice Hall, 2002. pp. 702—716

4. D. C. Schmidt and S. Vinoski. Distributed Callbacks and Decoupled Communication in CORBA. C++ Report, 8(9):48--56, 77, Oct. 1996.

5. R. Schulte. A Real-Time Enterprise is Event-Driven, 2002.http://www.gartner.com/reprints/tibco/110277.html

Page 32: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Notification Selection

Page 33: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 33

Notification Selection

> Selecting the interesting notifications> Expressiveness of notification selection is crucial for a

notification service!> Trade-Off: Expressiveness vs. Scalability> Common approaches

> Channels> Subjects> Event Types> Content-based Filtering

Increasingexpressiveness

Page 34: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 34

Channels

> Channels categorize notifications (weather, news, stock quotes etc.)

> Producer publishes notification with respect to a channel

> Simple, little expressive model> Producers determines channel

⇒ Producers and consumers not fully decoupled

> Easily mappable to IP-multicast

> Used by the CORBA Event Service

1 2 3 4

5 6 7 8

8 9 10 11

Page 35: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 35

Subjects (also known as Topics)

> Subjects categorize notifications hierarchically> News.Politics.Germany, News.Stock Market.Quotes.Siemens> Used by many commercial systems (e.g., TIBCO Rendezvous)

> Wildcards> News.Stock Market.Quotes.* (select all child subjects)> News.Politics.> (select whole subtree)

News

Politics

Germany US

Stock Market

Telekom Siemens

Quotes

......

...

...

Page 36: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 36

Subjects (contd.)

News

PoliticsGermany

US Stock MarketQuotes

Telekom Siemens

Page 37: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 37

Subjects (contd.)

> More expressive than channels> Subject determined by producer

⇒ Still no full decoupling between producers and consumers

> Problem: How to divide the notification space into subjects?> Division has implications on the possible selections!

> News.Stock Market.Quotes.Siemens orNews.Stock Market.Siemens.Quotes ?

> Subjects only suited for categorizing along a single dimension> Changes to the subject tree have implications on consumers!

Page 38: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 38

Subjects (contd.)

> Mapping to multicast more complicated> Implementations heavily depend on wildcard semantics

> Potential multicast-based implementations> For every subject and every subtree a multicast group

> Notification published multiple times> Consumer subscribes to one group

> Multicast group for every subject> Notification published only once> Consumer subscribes to several groups

(e.g., to all of a subtree)> Delivery to all nodes and apply local filtering> …

Page 39: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 39

Type-Based Filtering

> Allows for type-based categorization of notifications> Usually common abstract superclass Event> Event is subclassed to generate specific classes of events

(e.g., StockQuote)> Producers publish event objects> Consumers subscribe to classes of events

⇒ Subclasses are also delivered(this offers some limited kind of extensibility)

Page 40: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 40

Type-Based Filtering (contd.)

> Single inheritance is similar to subjects⇒ Several possibilities to structure the class hierarchy> Multi inheritance might be a solution but has its own

drawbacks

Event

HardwareEvent SoftwareEvent

Event

PerformanceEvent FaultEvent

vs.

Page 41: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 41

Content-Based Filtering

> Based on a data model with corresponding filter model> E.g., name/value pairs and conjunctive attribute filters

> Filters evaluate the whole content of notifications using logical expressions comprised of filter predicates > E.g., {(Stock = Telekom ∧ Price > 15€)}

> Hence, a filter F is a function that maps notifications to the Boolean values true and false

> Set of matching notifications N(F) = {n | F(n) = true}

Page 42: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 42

Content-Based Filtering (contd.)

> Used by, e.g., CORBA Notification Service and Java Message Service (JMS)

> Most expressive but complex> Cannot be easily mapped to IP-Multicast

(#groups may grow exponentially in #filters)> Centralized implementation not scalable to wide-area

scenarios

⇒ Powerful distributed infrastructure needed

Page 43: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 43

Notification Selection Summary

EasyModerateModerateEasyAdministration

HighModerateModerateLowComplexity of Infrastructure

HighModerateModerateLowDecoupling / Flexibility

Moderate

Moderate

Types

ComplexModerateEasyMapping to Multicast

HighModerateLowExpressiveness

ContentSubjectsChannels

Page 44: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 44

Bibliography1. P. Eugster, R. Guerraoui, and J. Sventek. Type-based publish/subscribe.

Technical Report DSC ID:200029, EPFL Lausanne, 2000.2. Object Management Group. CORBA Event Service Specification, version 1.0. OMG

Document formal/2000-06-15, 2000.3. Object Management Group. CORBA Notification Service. OMG Document

telecom/99-07-01, 1999.4. D. Schmidt and S. Vinoski. Overcoming Drawbacks in the OMG Events Service.

C++ Report, June 1997.5. D. Schmidt and S. Vinoski. The OMG Events Service. C++ Report, Feb. 1997.6. Sun Microsystems, Inc. Java Message Service Specification 1.1, 2002.

Page 45: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Data and Filter Models

Page 46: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 46

Data and Filter Models

> Determine the expressiveness of content-based notification selection

> Examples> Tuples/Templates> Name-value pairs/Conjunctive Attribute Filters> XML/XPath> Event Objects/Filter Objects

Page 47: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 47

Tuples

> Notifications are tuples (ordered sequences of attributes)> (“Stock Quote”, ”Telekom”, “15€”)

> Templates used for matching> (“Stock Quote”, “Telekom”, *)> Similar to a query-by-example mask

> Template matches notification iff1. Template has same number of attributes as notification2. Every attribute in the template matches the attribute at the

same position in the notification> Most common operators: equality, prefix, and don’t care (*)

Page 48: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 48

Tuples (contd.)

> Known from LINDA Tuplespaces (JavaSpaces, TSpaces, etc.)

> Used by JEDI (Java Event-based Distributed Infrastructure)

> Simple but not very expressive model> Model does not allow for optional attributes

Page 49: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 49

Name/Value Pairs

> Notifications consist of set of attributes> Attributes are name/value pairs

{(Type, Quote), (Name, “Telekom”), (Price, 23.24)}

Name Value

Page 50: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 50

Name/Value Pairs (contd.)

> Filters are conjunctions of attribute filters> F = f1 ∧ … ∧ fn⇒ Filter is satisfied if and only if all attribute filters are satisfied

> Attribute filter applies a constraint to a named value> E.g., (Price > 20€)> Set of satisfying values n(fi)> Usually at most one attribute filter per attribute allowed

> E.g., {(Type = Quote) ∧ (Name = “Telekom”) ∧(Price > 20€)} is satisfied by {(Type, Quote), (Name, “Telekom”), (Price, 23.24)}

> Model used by many pub/sub systems, e.g., SIENA

Page 51: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 51

XML/XPath

> Notifications are modeled as XML documents

<notification><auction endtime=“05/18/02 22:17:42” minprice=“$50”><seller name=“Smith” id=“1234”/><item>

<cpu manufacturer=“AMD” type=“Athlon”socket=“Socket A” clock=“800”/>

</item></auction>

</notification>

> Filters are modeled as XPath queriese.g., /notification[auction/item/cpu[@manufacturer=“AMD”]]

Page 52: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 52

XML/XPath (contd.)

> XPath expressions are very expressive> Can also be used to select only parts of notifications> Expressiveness must be limited to make routing

optimizations feasible

Page 53: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 53

Objects

> Notifications and filters are modeled as objects(e.g., as JAVA classes)

> Common superclasses> Event Baseclass of all events> Filter Baseclass of all filters> Subscription and Advertisement

are subclasses of Filter> Good integration with object-

oriented programming languages

Page 54: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 54

Implementation of StockQuote Class

> StockQuote subclasses Event

public class StockQuote implements Event {String symbol;double price;long time;public String getSymbol() {

return symbol;};...

}

Page 55: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 55

Implementation of EventClassFilter

> EventClassFilter subclasses Filter

public class EventClassFilter() implements Filter {Class eventClass;

public EventClassFilter (Class eventClass) {this.eventClass = eventClass;

}public boolean match(Event e) {

return eventClass.isInstance(e);}

}

EventClassFilter filter = new EventClassFilter(StockQuote.class);

Page 56: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 56

Filter and Event Implementation (contd.)public class StockQuoteFilter extends EventClassFilter {

public StockQuoteFilter() {super(StockQuote.class);

}

boolean matchStockQuote(StockQuote sq) {return true;

}

public boolean match(Event e) {if (super.match(e))

return matchStockQuote((StockQuote)e);else

return false;}

}

Page 57: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 57

Filter and Event Implementation (contd.)public static void main(String[] args) {

...StockQuoteFilter filter = new StockQuoteFilter() {

boolean matchStockQuote(StockQuote sq) {return sq.getSymbol().equals(“DT”) &&

sq.getPrice() > 20;}...

};

Page 58: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 58

Objects (contd.)

> Necessary serialization and deserialization of events and filters might lead to degraded performance!

> Routers and consumers must have access to class implementations

> Routing optimizations difficult if filter objects contain arbitrary program code

> Solution: Tests are implemented by the objects

public boolean identical(Filter f) {if (f instanceof StockQuoteFilter)

return ((StockQuoteFilter)f). getSymbol().equals(this.symbol);

elsereturn false;

}

Page 59: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 59

Bibliography1. G. Mühl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis,

Darmstadt University of Technology, 2002. Chapter 42. P. T. Eugster, R. Guerraoui, and C. H. Damm. On Objects and Events. Proceedings

of the OOPSLA '01 Conference on Object Oriented Programming Systems Languages and Applications, pages 254--269, Tampa Bay, FL, USA, 2001. ACM Press.

Page 60: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Content-Based Routing

Page 61: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 61

The Need for a Distributed Implementation

> Disadvantages of a centralized notification service> Single point of failure> Bottleneck wrt. bandwidth and computing power> No exploitation of locality although most interactions will be

local⇒ Neither reliable nor scalable to wide-area scenarios

> Service must be distributed and replicated⇒ Multiple cooperating service instances necessary!

Page 62: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 62

Broker Network

> Service instances called brokers> Each broker manages a set of

exclusive local clients to which it forwards matching notifications it processes

> Notifications are stepwise forwarded through broker network

> Cyclic forwarding and delivery of duplicates must be avoided!> Easy in acyclic topologies but these

provide no fault tolerance> Complex in topologies with cycles

B1

Localclients

B2B3

B4

Page 63: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 63

Flooding

> Each notification is delivered to all brokers> A broker forwards an incoming notification to all neighbors if

it was received from a local client and to all other neighbors if it was received from a neighbor

> Advantages> No routing tables for neighbor brokers necessary> Subscriptions become active immediately> Implementations can easily exploit IP-Multicast

> Disadvantage> Many notifications may be processed and forwarded

unnecessarily wasting computing power and network bandwidth

Page 64: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 64

Efficiency of Flooding

> Efficient if > most notifications need to be delivered to a majority of

brokers or> subscriptions are highly transient

> Not efficient if > filters are highly selective or> locality among subscriptions exists

Page 65: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 65

Content-Based Routing

> Each broker manages a filter-based routing table

> A routing table TB consistsof a set of routing entries

> A routing entry is a pair (F, D) consisting of a filter Fand a destination D which can be a local client or a neighbor broker⇒ Local (not shown in Fig.) and

remote routing entries

B1

B2 B3

B4

nn

nsub(F)

sub(G)

(F,B2)(G,B3)

(G,B4)

Routingtables

nn

n n ∈ N(F)n ∈ N(G)

Page 66: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 66

Content-Based Routing (contd.)

Broker B1

Forwarding EngineRouting Table

(F, B3)(G, B4)(H, B3)

n

n ∈ N(F)n ∉ N(G)n ∉ N(H)

B3 B4B

2

1.n2.

Page 67: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 67

Notification Forwarding in Acyclic Topologies

> Each broker forwards an incoming notification to all neighbor brokers and local clients for which there is a matching routing entry in the routing table.

> If the notification was received from a neighbor broker, it is not forwarded to that neighbor.

Page 68: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 68

Routing Algorithms

> Routing tables must be updated if subscriptions are issued or revoked> Update processes triggered> Many update processes might be in progress currently⇒ Routing algorithm

⇒ Subscriptions become active gradually⇒ Race conditions and latency⇒ Fully active after corresponding update process terminated

> Forwarding of unsubscriptions only done for efficiency> Delivery of non-matching notifications can be prevented

locally

Page 69: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 69

Routing Algorithms (contd.)

> A correct routing algorithm ensures that 1. Each update process terminates within finite time2. After an update process terminated, all published

notifications matching this subscription are delivered to the subscribing client

3. No duplicates and no non-matching notifications are delivered

Page 70: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 70

Routing Algorithms (contd.)

> Trade-Off> Flooding notifications vs.

filtering at intermediate brokers and updating of routing tables

> Optimal approach heavily depends on current setting!⇒ In dynamic environments adaptive solutions necessary

Page 71: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 71

Simple Routing

> Only assumption: Filters can be uniquely identified> Each subscription stored in every routing table

> Size of routing tables grows linearly in the number of active subscriptions and the number of brokers

> Each routing table affected by a new/cancelled subscription> #brokers – 1 update messages necessary if update information is

not batched

⇒ Not scalable!

B4

B1

B2

B3

(F,B3)

(F,B1)1. sub(F)

2. sub(G)

(F,B3)

G

(F,B1)(G,B2)

(G,B3)

(F,B3)(G,B3)

GG

Page 72: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 72

Simple Routing (contd.)

> New subscription flooded into the broker network> If a broker receives a subscription F from a neighbor B,

> it adds an entry (F, B) to its routing table and > forwards the subscription F to all its other neighbors

> Unsubscription flooded into the broker network> If a broker receives an unsubscription F from a neighbor B,

> it removes the entry (F, B) from its routing table and> forwards the unsubscription F to all its other neighbors

Page 73: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 73

Advanced Content-Based Routing

> Goals> Smaller routing tables> Reduced filter forwarding overhead

> Possible Solutions> Routing algorithms that exploit similarities and merge subscriptions> Integration of advertisements (announcements of producers)> Trade accuracy vs. efficiency

> perfect routing: Notifications are forwarded only if a matching subscription exists in respective subnet

> imperfect routing: Notifications may be forwarded without any matching subscription

Page 74: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 74

Identity-Based Routing

> Uses identity tests for routing decisionsF≡G ⇔ N(F) = N(G)

> Avoids> Forwarding of identical filters> Routing entries with identical filters for the

same neighbor broker

F≡G

geometricalvisualization

n1

n2

B4

B1

B2

B3

(F,B3)

(F,B1)sub(F)

(F,B3)

(F,B3)

F

F

(F,B1)(F,B2)

sub(F)

Page 75: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 75

Covering-Based Routing

> Uses covering tests for routing decisionsF covers G ⇔ N(F) ⊇ N(G)

> Avoids> Forwarding of covered filters> Routing entries with covered filters

for the same neighbor broker

> Forwarding of subscriptions similar to identity-based routing> Forwarding of unsubscriptions more complex due to

uncovered subscriptions

FG

Page 76: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 76

Covering-based Routing (contd.)

B4

B1

B2B3

(F,B3)

(F,B1)sub(F)

(G,B3)

(F,B3)

G

G

(F,B1)(G,B2)

sub(G)

> Processing of a new subscription F from a neighbor B> Routing entries regarding B whose filter is covered by F are

dropped> F is forwarded only to those other neighbors to which no

covering subscription was forwarded before which is still active

F G

Page 77: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 77

Covering-Based Routing (contd.)

> Processing of an unsubscription F from a neighbor B> F is forwarded only to those other neighbors to which no

covering subscription was forwarded before which is still active

> Along with F, uncovered subscriptions must be forwarded

B4

B1

B2B3

(F,B3)

(F,B1)(G,B2)(H,B2)sub(F)

sub(G)sub(H)

F GH

F

(F,B3)

FG

HF(G,B3)(H,B3)

(G,B3)(H,B3)

Page 78: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 78

Merging-Based Routing

> Filters can be merged> perfectly N(F) = N(G) ∪ N(H)> imperfectly N(F) ⊃ N(G) ∪ N(H)

> Merging generates new covers

F GH

GHF

B1

B2

B31. sub(G) B4

(G,B3)

(G,B1)H

2. sub(H)

(G,B1)(H,B2)

(G,B3)

(H,B3)

H F

(F,B3)

Page 79: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 79

Merging-Based Routing (contd.)

> Merging-based routing is perfect iff> only perfect mergers are generated and> mergers are forwarded and cancelled in a way such that no

unnecessary notifications are received later on> … otherwise it is imperfect

Page 80: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 80

Perfect Merging-Based Routing

> Possible implementation on top of covering-based routing> Brokers merge suitable routing entries (F1, D),…, (Fn, D) with

the same destination D to a single entry (F, D) such that F is a perfect merger of {F1, …, Fn}

> Merged entries are removed from routing table> Merger is added to routing table and forwarded like a normal

subscription received from D

Page 81: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 81

Hierarchical Routing Algorithms

> One broker is labeled as root broker R

> Any notification and filter is forwarded “upwards” to R

> Notifications are onlyforwarded “downwards” if a matching filter exists

> Leads to smaller routing tables but increases the load imposed on brokers on higher levels such as the root broker

R

B1 B2

B3 B4 B5 B6

1. sub(F)

(F,B3)

(F,B1)

n

n

n

n

2. pub(n)n ∈ N(F)

Page 82: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 82

2. pub(n)n ∈ N(F)

Hybrid Routing Algorithms

> Combine hierarchical with peer-to-peer routing

B0

B1 B2

B3 B4 B5 B6

B7

Peer-to-PeerRouting

HierarchicalRouting

1. sub(F)

(F,B1)(F,B1)

(F,B0)

n

nn

n n

(F,B3)

n

Page 83: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 83

Advertisements

> Producers specify the notifications they will publish> Advertisements are filters that are forwarded like

subscriptions by the help of a second routing table> Subscriptions are only forwarded to those destinations for

which there is an overlapping advertisement

> Test for overlapping: N(F) ∩ N(G) ≠ ∅

G

F

Page 84: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 84

Advertisements (contd.)

G

F F

(G,B4)(H,B5)

Advertisements Routing Table

B4

B1

B3

B5

FH

... ...

1. 2.

Subscription

Advertisements

Page 85: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 85

Advertisements (contd.)

> New advertisement> If B receives a new advertisement from B’, B forwards those

overlapping subscriptions to B’ which can newly be served

> Cancelled advertisement> If B receives a cancelled advertisement from B’, B drops

those overlapping subscriptions which can no longer be served

Page 86: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 86

Advertisements (contd.)

> Race condition> Published notifications may be dropped if they match only a

newly servable subscription that has not yet arrived!

> Potential solutions> Keep notifications that match only the new advertisement

until all newly servable subscriptions have arrivedcorrupts FIFO producer ordering

> Block a producer that publishes a notification that matches only the new advertisement until all newly servable subscriptions have arrived introduces some synchronicity

Page 87: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 87

Topology Changes

> Easy approach preserving an acyclic topology> A new broker B is connected as a leaf

(i.e., to exactly one broker B’)> B’ sends all interesting subscriptions to B

> Simple Routing: all subscriptions> Identity-based Routing: identical subscriptions are suppressed> Covering-based Routing: covered subscriptions are suppressed

> If advertisements are used, they are exchanged instead of subscriptions

> Only leaf brokers can be removed

Page 88: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 88

Supporting Routing Optimizations

> Complexity of routing optimizations depends on the underlying data/filter model> In the most general case not computable> E.g., covering test among relational expressions is NP-

complete> Scalability vs. Expressiveness

⇒ Expressiveness must be limited to a sensible level

> We assume conjunctive filters consisting of attribute filters with at most on attribute filter per attribute in the following

Page 89: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 89

Identity among Filters

> Two filters F1 and F2 are identical, written F1 ≡ F2, iff (a) they contain the same number of attribute filters and (b) for each attribute filter in F1 there is an attribute filter in F2 such that these attribute filters are identical

> Example of identical filtersF1 = {x ≥ 2} ∧ {y > 5}

F2 = {x ≥ 2} ∧ {y > 5}≡ ≡≡

Page 90: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 90

Overlapping among Filters

> Two filters F1 and F2 are disjoint iff for an attribute filter in F1 there exists an attribute filter in F2 that constrains the same attribute such that these attribute filters are disjoint

> Two filters are overlapping iff they are not disjoint

> Example for disjoint filtersF1 = {x ≥ 2} ∧ {y > 5}

F2 = {x < 1} ∧ {y = 7}⊓ ⊓⊓

Page 91: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 91

Covering among Filters

> A filter F1 covers a filter F2, written F1 ⊒ F2, iff for each attribute filter in F1 there is a covered attribute filter in F2

> Example of covering filtersF1 = {x ≥ 2} ∧ {y > 5}

F2 = {x = 4} ∧ {y = 7} ∧ {z ∈ [3,5]}⊒ ⊒⊒

Page 92: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 92

Filter Lattice

> Filters are arranged in a lattice according to covering relation

> Lattice is updated according to new and cancelled subscriptions

> Lattice is used by covering-based routing algorithm

F1:={x>5}

F2:={x>8}

F3:={x=9}

F4:={x=6} F5:={x>6 ∧ y>7}

F6:={x>7 ∧ y=9}

F7:={x=9 ∧ y=9}

Page 93: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 93

Merging of Filters

> Two filters F1 and F2 can be merged perfectly iff they are identical in all but a single attribute filter

> Merging ExampleF1 = {x ≥ 2} ∧ {y < 4}

F2 = {x ≥ 2} ∧ {y > 6}

F3 = {x ≥ 2} ∧ {y ∉ [4,6]} (Merger)

Page 94: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 94

Example: Geographic Information Systems (GIS)

> F = {(Type = TrafficInformation) ∧(Location around(Frankfurt,50km))}

> G = {(Type = TrafficJam) ∧(Length ≥ 5km?) ∧(Location around(Darmstadt,20km))}

⇒ F covers G

> H = {(Type = TrafficJam) ∧(Location around(Frankfurt,40km))}

> I = {(Type = TrafficJam) ∧(Location around(Wiesbaden,40km))}

⇒ H and I can be merged imperfectly

XX

FrankfurtDarmstadt

X XX

Frankfurt

Wiesbaden

Page 95: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 95

Bibliography1. A. Carzaniga. Architectures for an Event Notification Service Scalable to Wide-area

Networks. PhD thesis, Politecnico di Milano, Milano, Italy, Dec. 1998.2. G. Mühl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis, Darmstadt

University of Technology, 2002. Chapter 33. A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Design and Evaluation of a Wide-Area

Event Notification Service. ACM Transactions on Computer Systems, 19(3):332--383, 2001.

4. G. Mühl, L. Fiege, F. C. Gärtner, and A. P. Buchmann. Evaluating advanced routing algorithms for content-based publish/subscribe systems. In A. Boukerche, S. K. Das, and S. Majumdar, editors, The Tenth IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2002), pages 167-176, Fort Worth, TX, USA, October 2002. IEEE Press.

Page 96: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Efficient Use of Multicast

Page 97: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 97

Use of Multicast Mechanisms

> Seems naturally to use IP-multicast for publish/subscribe but> IP-Multicast not reliable> Number of groups limited

(in theory 224, in practice much smaller)> Tunnels necessary to connect not multicast-enabled areas

> Easy solution for channels> Use for each channel a separate multicast group

Page 98: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 98

Use of Multicast Mechanisms (contd.)

> Solution for subjects more complex> Depends on the possible filtering expressions

(e.g., wildcards)> E.g., one multicast group for each subject and subtree

> Content-based filtering most complex> The only simple solution is flooding

Page 99: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 99

Ideal Multicast

> Direct send a notification to all matching brokers> May require 2N groups (N number of all brokers)

> Number of all subsets of all brokers> Not practical for large numbers of brokers

> Alternatives> Reduce group precision (i.e., use overly broad groups)> Do multiple sends (i.e., split set of destinations)> Send using multiple hops (i.e., send to neighbors only)

Page 100: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 100

Clustered Group Multicast (CGM)

> Divides brokers into C mutually exclusive subsets (clusters) with K = N / C brokers

> Requires C sends and n = C · 2K = C · 2N/C groups> Reduces number of groups by a factor of 2C/C> Reduces efficiency of multicast> Still not feasible for larger N

(e.g., N = 100, C = 10 ⇒ n = 10240 groups necessary)> Clustering assignment important

> Subscriptions (clients’ interests)> Geographic location of brokers

Page 101: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 101

Threshold Clustered Group Multicast (TCGM)

> Sends to whole cluster if number of matching brokers in the respective cluster exceeds a certain threshold T

> Requires only groups for all subsets of the cluster with cardinality less than T, i.e., groups1

T

i

KC

i=

⎛ ⎞⋅ ⎜ ⎟

⎝ ⎠∑

Page 102: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 102

Neighbor Matching Algorithm

> Each broker only sends to matching neighbors> Requires multiple hops> Multicast group for all combinations of neighbors> Advantages

> More scalable for larger N(only N · 2k groups necessary, k number of neighbors)

> Knowledge about groups can be local> Disadvantages

> Requires extra bandwidth on links between brokers and network

> Matching must be executed at every broker introducing additional delays

Page 103: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 103

Comparison of Multicast Strategies

ModerateModerateModerateTrivialHard++Manageability

k · 2k-12K-112N-1Groups/Broker

N · 2kC · 2K12NGroups

1 per hop1…C1…C11Multicasts

1[(T+1)/K,1]1PN1Precision

NeighborTCGMCGMFloodingIdeal

1

T

i

KC

i=

⎛ ⎞⋅ ⎜ ⎟

⎝ ⎠∑

∑ −

= ⎟⎟⎠

⎞⎜⎜⎝

⎛ −1

1

1T

i iK

K = N / C Number of brokers in a cluster1 ≤ T ≤ K Threshold

N Total number of brokersk Number of neighborsC>1 Number of clusters

Page 104: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 104

Bibliography1. L. Opyrchal, M. Astley, J. Auerbach, G. Banavar, R. Strom, and D. Sturman.

Exploiting IP Multicast in Content-Based Publish-Subscribe Systems. In J. Sventek and G. Coulson, editors, IFIP/ACM International Conference on Distributed Systems Platforms (Middleware 2000), volume 1795 of LNCS, pages 185-207. Springer-Verlag, 2000.

2. G. Banavar, T. Chandra, B. Mukherjee, J. Nagarajarao, R. E. Strom, and D. C. Sturman. An Efficient Multicast Protocol for Content-Based Publish-Subscribe Systems. In Proceedings of the 19th IEEE International Conference on Distributed Computing Systems, pages 262-272, 1999.

Page 105: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Matching Algorithms

Page 106: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 106

Matching Problem / Forwarding Problem

> Matching Problem> How can one efficiently determine all matching subscriptions

given a notification?

> Forwarding Problem> How can one efficiently determine the set of destinations

(neighbor brokers and clients) which have a matching subscription given a notification?

> Note that it is not necessary to find all matching subscriptions!

Page 107: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 107

Matching Algorithms

> Many known algorithms> Brute Force> Predicate counting> Decision trees> Decision diagrams> …

Page 108: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 108

Brute Force

> Matches a notification against all subscriptions separately> Most simple implementation> No complicated data structures necessary> Relations among predicates not exploited

> E.g., the same predicate may be evaluated multiple times> Can be applied to any kind of filter

> E.g., not restricted to conjunctive filters

Page 109: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 109

Predicate Counting Algorithm

> A conjunctive filter is matched iff all its predicates are satisfied

> The algorithm counts the number of predicates of a filter that are satisfied by a given notification

> Requires for each predicate a list of filters it occurs in> Avoids testing a predicate multiple times> Evaluates every predicate at most once> Works best for conjunctive filters

Page 110: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 110

Predicate Counting Algorithm (contd.)

<for each filter a counter is initialized to zero>FOR EACH <predicate matched by given notification> DO

<increment the counter of all filters containing this predicate>

END<all filters whose counter is equal to their number of

predicates are matched by the given notification>

Page 111: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 111

Predicate Counting Algorithm (contd.)

> If attribute filters are used, a multi-level index structurewith specialized indexes can be used to efficiently determine the satisfied predicates> Attribute name index used to find all predicates

constraining a given attribute> Operator index used to find all predicates with a certain

operator> Value index

> Allows to find all satisfied predicates > Depends on the characteristics of the operator

(e.g., hashing for equality tests)

Page 112: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 112

Predicate Counting (contd.)

(Stock, “DT”)(Price, 15) Price

>

<

…121417…

Stock = DTMSFTTIBX…

F1

F2

F3

DT

DTDT

15

…181610…

15

15

15

15 …F1 := {Stock = “DT” ∧ Price > 12}

F2 := {Stock = “MSFT” ∧ Price > 14}

F3 := {Stock = “TIBX” ∧ Price < 18}

I1

I2

I3

I4

AND OR

Page 113: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 113

Decision Trees

> Predicates and subscriptions are arranged in a graph> Non-leaf nodes are tests> Leaf nodes are

subscriptions> Edges are test results

> Graph is traversed in depth-first order

> Reached subscriptions are matched by notification

Stock=

Price< Price> Volume>

TIBX *DT

F1 F2 F3 F4

15 12 16 1,000,000

(Stock, “DT”)(Price, 14)(Volume, 2,000,000)

Tested notification

F1 := {Stock = “DT” ∧ Price < 15}

F3 := {Stock = “TIBX” ∧ Price > 16}

F4 := {Volume > 1,000,000}

F2 := {Stock = “DT” ∧ Price < 12}

Page 114: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 114

Decision Trees (contd.)

> Efficient for equality tests because> test results are exclusive and> max. 2 matching outgoing edges exist

> Commonalities of subscriptions are not adequately exploited (subtree repetition)

> Limited expressiveness> Restricted to conjunctive filters> Expensive updates> Used by Gryphon [1]

Page 115: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 115

Binary Decision Trees

Stock = DT

Price < 12 Stock = TIBX

*

F1 F2 F3 F4

0 Price > 16 Volume > 1,000,000

*

1

1

1

1 1

F1 := {Stock = “DT” ∧ Price < 12}

F2 := {Stock = “DT” ∧ Price ≥ 12}

F3 := {Stock = “TIBX” ∧ Price > 16}

F4 := {Volume > 1,000,000}

Stock, ”DT”Price, 16Volume, 2,500,000

Page 116: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 116

Binary Decision Diagrams

> BDDs are directed acyclic graphs (DAGs)> Each non-terminal node corresponds

to a filtering predicate and has two outgoing edges> Low edge (predicate not satisfied)> High edge (predicate satisfied)

> 2 terminal nodes represent constants true and false

> Can be constructed from binary decision tree

Stock = “TIBX”

Price > 15$

01

Volume > 1Mio.$

F := {Stock=“TIBX” ∧ [Price>15$ ∨ Volume>1Mio.$]}

start node

terminal nodes

Page 117: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 117

Corresponding If-then-else Program

1. If (Stock=“TIBX”) then 2 else 4;2. If (Price>15) then 5 else 3;3. If (Volume>1Mio.$) then 5 else 4;4. Output 0;5. Output 1; Stock=“TIBX”

Price>15$

01

Volume>1Mio.$

then

then

then

elseelse

else

Page 118: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 118

Evaluation of Binary Decision Diagrams

Stock=“TIBX”

Price>15$

01

Volume>1Mio.$

Stock, ”TIBX”Price, 16Volume, 500,000

match

Page 119: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 119

Evaluation of Binary Decision Diagrams

Stock=“TIBX”

Price>15$

01

Volume>1Mio.$

Stock, ”TIBX”Price, 14Volume, 2,000,000

match

Page 120: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 120

Evaluation of Binary Decision Diagrams

Stock=“TIBX”

Price>15$

01

Volume>1Mio.$

Stock, ”TIBX”Price, 14Volume, 500,000

no match

Page 121: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 121

Binary Decision Diagrams (contd.)

> Ordered BDDs (OBDDs)> Nodes are number from 1 to n such that for every path the

number assigned to the nodes is monotonically increasing> For every filter the node with the lowest number is called the

output node ; it represents the outcome of the filter > Reduced BDDs (RBDDs)

> No redundant nodes> No isomorphic subgraphs

> Size (#nodes) of BDDs may be exponential in the number of predicates> Ordering has great effect on size> Finding optimal ordering is NP-hard

Page 122: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 122

Binary Decision Diagrams (contd.)

> Ordered BDDs can be evaluated bottom up

FOR v:=n DOWNTO 1 DOIF v is terminal node

THEN value[v]:=label[v];ELSE BEGIN

a:=eval[v]; //evaluate filter predicatevalue[v]:= a AND value[high[v]] OR

NOT a AND value[low[v]];END

FIEND

Page 123: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 123

Binary Decision Diagrams (contd.)

Stock=“TIBX” Stock=“TE”

Price>15$

0 1

Volume>1Mio.$

F1:={Stock = “TIBX” ⋀ Price > 15 ∨Stock ≠ “TIBX” ⋀ Volume > 1,000,000}

F2:={Stock = “TE” ⋀ Price > 15}

ITE(a,h,l)=(⌐a⋀l) ∨ (a⋀h)

10Stock, ”TIBX”Price, 16Volume, 500,000

0

1

1 0

Order: Stock = “TIBX”, Stock = “TE”,Price > 15, Volume > 1,000,000, 1, 0

If-then-else

output nodes

Page 124: Middleware-Konzepte Publish/Subscribe-Systeme · 2009-01-15 · Middleware-Konzepte Publish/Subscribe-Systeme Dr. Gero Mühl Kommunikations- und Betriebssysteme Fakultät für Elektrotechnik

Middleware-Konzepte © Gero Mühl 124

Bibliography1. M. Aguilera, R. Strom, D. Sturman, M. Astley, and T. Chandra. Matching events

in a content-based subscription system. In Proceedings of the 18th ACM Symposium on Principles of Distributed Computing (PODC 1999), pages 53-61, Atlanta, Georgia, USA, 1999.

2. A. Campailla, S. Chaki, E. Clarke, S. Jha, and H. Veith. Efficient filtering in publish-subscribe systems using binary decision diagrams. In Proc. of the 19th Conference on Software Engineering, Toronto, Canada, May 2001.

3. A. Carzaniga, J. Deng, and A. L. Wolf. Fast forwarding for content-based networking. Technical Report CU-CS-922-0, Department of Computer Science, University of Colorado, Nov. 2001.

4. T. W. Yan and H. Garcia-Molina. Index structures for selective dissemination of information under the Boolean model. ACM Transactions on Database Systems, 19(2):332--364, 1994.

5. G. Mühl. Generic constraints for content-based publish/subscribe systems. In C. Batini, F. Giunchiglia, P. Giorgini, and M. Mecella, editors, Proceedings of the 6th International Conference on Cooperative Information Systems (CoopIS '01), volume 2172 of LNCS, pages 211--225, Trento, Italy, 2001. Springer-Verlag.