Event-Driven Business Process Management 2nd edBPM Workshop 2nd ServiceWave November 23-27, 2009 Stockholm Rainer v. Ammon 1 Workshop Agenda Introduction edBPM: What it means, Basic concepts, Reference Model, Reference Architecture, Taxonomy, Dissemination and Teaching (CITT) (45 min) Hands-on: Implementing a use case live (Starview, jCOM1/Valial) (60 min) Grand Challenges of the EASSy-project (IBM/FZI/NSN) (15 min) SmartHealthcare Maccabi (Maccabi/FZI/NSN) (15 min) SmartCity/SmartTransportation Genova/Shanghai (Thales/TXT) (15 min) SmartPlant/SmartSCM Siemens (Siemens Brasov) (15 min) SmartEmergencyManagement Cologne (SAP) (15 min) Q&A Rainer v. Ammon 2 Agenda – Introduction edBPM The forecast of edBPM for the next decades edBPM – a combination of two disciplines: Business Process Management (BPM) and Complex Event Processing (CEP) A reference model for edBPM – how its components work together edBPM-enhancements of the NEXOF-Reference Architecture edBPM-enhancements of modeling and execution standards Instrumenting of Business Processes and Services - the „Event Tornado“ Domain-specific standards for Notification Event Architectures (NEAx) Standardising Event Processing Languages? Domain-specific reference models for use cases and event patterns Taxonomy of Event-driven Adaptivity of Service-based Systems Dissemination and teaching edBPM Rainer v. Ammon 3 The forecast of ED-BPM for the next decades The forecast of Prof. David Luckham… … we need skilled people at least up to 2050… … we are only at the end of the period of Simple CEP Rainer v. Ammon 4 The forecast of ED-BPM for the next decades The warning of Roy Schulte (VP of Gartner) since 2006… …we won‘t have enough skilled people who would be able to do all the jobs and projects The statement of Prof. Mani Chandy/California Techical University at the Gartner Event Processing Summit 2007… …The work of IT during the next twenty years will be to complete the evolution of business processes from sequences of slow-moving, disjointed applications to more responsive end-to-end, event-based straight-through flows of action. Rainer v. Ammon 5 Managing and monitoring of processes and services mean ... monitoring of time-critical bottleneck factors and transaction control representation of the effects of system availabilities and -disturbances Workflow Management process and systemlinked emergency and Business Activity Monitoring transparency over integrated represented processes disturbance management integration of external processes and services integration of external processes and services monitoring of SLA-compliance Rainer v. Ammon 6 Processes and services communicate via a global event cloud event cloud with thousands of events per sec… passwdchange new auto pay deposit transfer enquiry deposit new auto pay logout account login enquiry account login account login withdrawal deposit new auto pay activity history logout passwdchange event patterns and complex event processing… account login withdrawal account balance deposit transfer enquiry account login enquiry new auto pay account login logout activity history logout …e.g. above a bank The important steps: 1. Redesign the business processes for SOA and BPM 2. Make a SOA, identify services, build WSDL-interfaces… 3. Precise description of patterns of events 4. Detecting patterns in the event cloud 5. Abstraction of complex event pattern instances to higher level events Rainer v. Ammon 7 The Pain Point: The Event Cloud, the IT-Blindness and the „Event Tornado“ Often even additional events are needed for BAM and a better Business Insight startedService_7 startedService_6 Today‘s existing event cloud and thestartedService_5 IT-blindness startedService_4 exitService_4 exitService_5 exitService_6 startedService_8 exitService_8 Low level events without semantics exitService_7 startedService_1 exitService_1 startedService_2 exitService_2 … BusinessProcess 1 startedService_3 exitService_3 … Visualization of the processed/correlated events via Business Activity Monitoring BusinessProcess n … Rainer v. Ammon 8 The Challenge and the Principle of ED-BPM – Reference Model Rainer v. Ammon Event-Driven Business Process Management in the Encyclopedia of Database Systems, Ling Liu and M. Tamer Özsu (Eds.), Springer-Verlag, 2009. Enterprise cockpit Workflow Modeler Event Modeler realize scenario process instances Domain specific reference models for event patterns set parameters Monitor / Analyze / Act Workflow Engine workflows „unus mundus“ Internet services and their events Low Level Event Streams Model ^= Scenario analyse history… Event Store based on BPEL CEP Engine „special“ SQL resp. other languages AppServer Normalized events, build business level events Adapters e.g. RFID, topics of Pub/Sub, … e.g. JMS pub/sub e.g. GPS-signal e.g. Traffic Message Controls e.g. Weather Forecast e.g. RFID … e.g. payments IF … AND … FOLLOWED BY… WITHIN… ACTION show BAMview, trigger a BP, change BP9 Rainer v. Ammonflow… NEXOF-RA enhanced by ED-BPM Rainer v. Ammon 10 Vision – event generation via middleware (diploma thesis Brandl/Guschakovski 2007, study for NextGeneration easyCredit / Teambank) Service Consumers Service Orchestration (OpenSource or BEA WLI 9.x or IBM Process Server or Oracle …) Service Bus Layer (SOPware or AquaLogic Service Bus or WebSphere …) Service Enablement (WLI 9.x or WebSphere …) Resources 10.11.09 Page 11 Initial position and first CEP enhancements (diploma thesis Brandl/Guschakovski 2007, study for NextGeneration easyCredit / Teambank) Interceptor 10.11.09 Page 12 The stream and its events (diploma thesis Brandl/Guschakovski 2007, study for NextGeneration easyCredit / Teambank) Interceptor 10.11.09 Page 13 Technical realization of CBEs via Servlet Filters and Interceptors (diploma thesis Brandl/Guschakovski 2007, study for NextGeneration easyCredit / Teambank) http RMI Servlet Filter Bean Interceptor CBE CBE EJB Interceptor CBE EJB Interceptor CBE Categorizing of ED-BPM use cases First sketch of a taxonomy of Event-driven adaptivity Process type Process instance Action type {list of domain specific processes} {new, running, all} {instantiate, stop, continue, terminate, change, new define} Example: Use Case „Fraud-Management“ in the Banking-Domain According to the reference model of edBPM-based Fraud-Management, we describe a simple example of the process “Withdraw” in connection with a potential event pattern of fraud and related processes in order to exemplify the edBPM principle: 1. An instance of a transaction process is started in the case of withdrawing at a certain ATM. 2. A lot of process instances of the same type are instantiated in a more or less short/certain timeframe at different ATM’s. 3. Each process step generates an event, if so of different event types (JMS publish/subscribe, special ATM-banking event type according to the banking standard “<…> ” 4. The global event cloud is analyzed in real-time by the CEP-system and optionally by some “intelligent” components like discriminant analysis and neural networks. A suspicious event pattern is detected because the login-data respectively the card is used more than once and at different locations in a timeframe whereas a service is called in order to check the probability that the same customer could use the same card at the different locations. 5. …< see Mona+ 09 paper > Rainer v. Ammon 15 A Reference Model of ED-BPM-based Fraud Management – non-deterministic approach fraud management processes ... ... alert alert alerts Real-time BAM, statistics… output evaluation neural network feed forward step input: current and historic discriminant values not fraud fraud suspicious suspicious needed to filter unkown suspicious event patterns preclassification needed to reduce the amount of suspicious event patterns discriminant analysis decision tree needed to process thousands of events per sec trigger fraud management process event filtering, enriching, correlating ... adapter for event type-1 CEPengine known suspicious event patterns: adapter for event type-n suspend transaction process event cloud transaction processes instance-1 instance-n ... ... ... e.g. ATM‘s in Tokyo, Rome, Munich e.g. Internet banking Rainer v. Ammon 16 Implementing of the use cases Typical workpackages before implementing use cases: • SOA-appropriate BP-modeling of the use cases per domain • Event-modeling of the use cases and instrumenting process steps and services for an appropriate event generating • Definition of BAM-views per use case • Description of event patterns per use case according to the edBPM/DoReMoPat-pattern framework – use of reference models Rainer v. Ammon, Christian Silberbauer, Christian Wolff Domain Specific Reference Models for Event Patterns – for Faster Developing of Business Activity Monitoring Applications VIPSI 2007 Lake Bled, Slovenia, 8-11 October 2007 Rainer v. Ammon 17 Domain-specific Notification Event Architectures – Existing and future standards DeliveryCommonData ThorougfareCommonData - DeliveryServiceQualifier [0..1] - DeliveryServiceIndicator [0..1] - DeliveryServiceType [0..1] - SupplementaryDeliveryPointData [0..1] - DefiningAuthority [0..1] - BuildingConstructionType [0..1] - BuildingConstruction [0..1] - ExtensionIdentifier [0..1] - SecondaryThoroughfareType [0..1] - SecondaryThoroughfareName [0..1] - ThoroughfareType [0..1] - ThoroughfareQualifier [0..1] - ThoroughfareName [0..1] DateTimeCommonData -@TypeCode [1] EventCommonData AddressCommonData NameCommonData - GivenName [1] - SurnamePrefix [0..1] - Surname [1] - NameQualifier [0..1] - Qualification [0..*] - FormOfAddress [0..1] - Function [0..1] - SupplementaryDispatchData [0..1] - AddresseRoleDescriptor [0..1] - Door [0..1] - Floor [0..1] - Wing [0..1] - +DeliveryData [0..1] - StreetNumberOrPlot [0..1] - +Thorougfare [0..1] - District [0..1] - Town [0..1] - ProximateTown [0..1] - Region [0..1] - Country [0..1] - Postcode [1] CustomerCommonData OrganizationalCommonData - OrganizationUnit [0..1] - OrganizationName [1] - LegalStatus [0..1] - MaileeRoleDescriptor [0..1] -@Gender [0..1] -<choice>[0..1] -CustomerID[1] -+CustomerDemographic[1] -+IDDemographicDataPair[1] -</choice>[1] -+Name[0..1] -+Address[0..1] -+TelephoneNumber[0..1] -+Email[0..1] -PrivacyOptOut[0..1] -BirthDayMonth[0..1] -BirthYear[0..1] - @Severity [0..1] - @Priority [0..1] - @Mode [0..1] - SequenceNumber [1] - +EventDateTime [1] - EventDescription [0..1] - SourceName [0..1] - <choice> - +SourceURI [1] - SensorID [1] - </choice> - Instance [0..1] - BusinessUnit [0..1] - +OrganizationalHirarchy [0..1] BusinessUnitCommonData -@Name[1] -@TypeCode[1] SourceURIType -@TypeCode[1] OperatorCommonData -@OperatorName[0..1] -@WorkerID[0..1] -@OperatorType[0..1] CustomerEventType -@TypeCode[1] -+Customer[1] -+Operator[0..1] -+Workstation[0..1] @TypeCode - CustomIdentifiedEvent -CustomerAddedEvent PostalAddressCommonData - <choice> - +Name [0..1] - +Organization [0..1] - </choice> - +Address [1] CustomerEvent - @MajorVersion [1] - @MinorVersion [0..1] - @FixVersion [0..1] “Common Postal Address” Architecture of NEALogistics EventCommonData PostalAddressCommonData - <choice> - +Name [0..1] - +Organization [0..1] - </choice> - +Address [1] @ Gender - Male - Female CustomerCommonData PackageCommonData - PackageID [1] - +DeliveryPoint [1] - +PostalAddress [1] - @Severity [0..1] - @Priority [0..1] - @Mode [0..1] - SequenceNumber [1] - +EventDateTime [1] - EventDescription [0..1] - SourceName [0..1] - <choice> - +SourceURI [1] - SensorID [1] - </choice> - Instance [0..1] - BusinessUnit [0..1] - +OrganizationalHirarchy [0..1] - @Gender [0..1] - CustomerID [1] - +PostalAdress [1..*] - +TelephoneNumber [1 ..*] - Email [0..*] - +BirthDay [0..1] Customer Event Domain Model of NEARetail EventCommonData - @Severity [0..1] - @Priority [0..1] - @Mode [0..1] - SequenceNumber [1] - +EventDateTime [1] - EventDescription [0..1] - SourceName [0..1] - <choice> - +SourceURI [1] - SensorID [1] - </choice> - Instance [0..1] - BusinessUnit [0..1] - +OrganizationalHirarchy [0..1] PostalAddressCommonData - <choice> - +Name [0..1] - +Organization [0..1] - </choice> - +Address [1] PackageCommonData - PackageID [1] - +DeliveryPoint [1] - +PostalAdress [1] InternalAccount CommonData - AccountNo [1] @AccountType - @AccountType [1] - Checking Account - Interest Checking Account - BankCode [1] - IBAN [1] - Money Market Account - BIC [1] - Savings Account - Certificate of Deposit ... ExternalAccount CommonData + Name [1] <option> <optionNational> - AccountNo [1] - BankCode [1] </optionNational> <optionInternational> - IBAN [1] - BIC [1] </optionInternational> </option> EventCommonData - @Severity [0..1] - @Priority [0..1] - @Mode [0..1] - SequenceNumber [1] -+EventDateTime [1] - EventDescription [0..1] - SourceName [0..1] - <choice> -+SourceURI [1] - SensorID [1] -</choice> - Instance [0..1] - BusinessUnit [0..1] -+OrganizationalHirarchy[0..1] AccountCommonData ShipmentCommonData - @ShipmentType [1] - @Package [1..*] - ShipmentID [1] - +TrackingEvent [1..*] @ ShipmentTypeData - Shipment - Transmittal - Reconsignment - Forewarding - ... OrderEventType - OrderID [1] - +Shipment [0..*] - +Service [0..*] - +Customer [1] - +Regulator [1..*] ServiceData - ServiceID [1] - SeviceName [1] - +Partner [0..*] @ ShipmentTypeData @TrackingTypeData - Shipment - PickupEvent - Transmittal - CommitalEvent - Reconsignment - DepartureEvent - Forewarding - EntryEvent - DeliveryLineEvent - DeliveryEvent TrackingEventType - EscapeEvent - @TrackingType [1] - TrackingID [1] - @ShipmentType [1] - @Package [1..*] - ShipmentID [1] - +TrackingEvent [1..*] @TransactionType - Incoming Payment - Withdrawal - Withdrawal ATM - Withdrawal Check - Check Payment - Savings Transfer - Debit Interest - Credit Interest - ... - TransactionID [1] - @TransactionType [1] -+Name [1] - Description [1..4] - Amount [1] - @PostingType [1] - PostingDate [1] - ValutaDate [0..1] - @TransactionState[1] -+AccountSource [1] -+AccountDestination [1] -+Customer [1] - AccountBalance[0..1] -+Card [0..1] @PostingType - Debit Posting - Credit Posing TrackingEvent - @MajorVersion [1] - @MinorVersion [0..1] - @FixVersion [0..1] CreditCard CommonData - CardNo [1] - @CardType [1] - Vendor [1] - Limit [1] - ValidFrom [1] - ValidTo [1] - CVC2 [1] @CardType - Maestro - Visa - Mastercard - ... @CardType - American Express - Diners Club - Visa - Mastercard - ... CardCommonData -+Name [1] AddressCommonData AccountTransactionType ShipmentCommonData DebitCardCommonData - CardNo [1] - @CardType [1] - ValidTo [1] <choice> +Name [0..1] +Organization [0..1] </choice> +Address [1] CustomerCommonData - CustomerID [1] - @Gender [1] -+Name [1] -+Address [1..2] -+TelephoneNumber [1..2] - Email [0..1] - Birthday [1] AccountTransactionEvent @TransactionState - Posted - Declined - Booked - @MajorVersion [1] - @MinorVersion [0..1] - @FixVersion [0..1] OrderEvent - @MajorVersion [1] - @MinorVersion [0..1] - @FixVersion [0..1] Order Event Architecture Tracking Event Architecture Model of the “AccountTransactionEvent” of NEAFinance Rainer v. Ammon 18 Standardizing Event Processing Languages? Different types of users for different BAM-views and needed skills (from 2007, a lot has changed in the meantime) EPL-Approaches CEP-Platform Usability / User Type Pseudo-SQL - Coral8 (CCL) - StreamBase (StreamSQL) - Esper (EQL) - Aleri - ?Oracle (CQL)? - skilled EPL-programmers - not the community of SQL-programmers?! Special Rules Languages - AMiT - ?Apama? - Tibco - ?Oracle? - skilled proprietary EPL-programmers - will never be a community Java- or other 3GL generated Code - Tibco - Apama - Esper? - Aleri Studio - ?Oracle? - community of Java-programmers Proprietary 4GL-based approaches - ?Aleri XML? -? - skilled proprietary 4GL-programmers - will never be a community GUI-based approaches (graphical editors) and code generation - ?AMIT? - ?Apama? - AptSoft - StreamBase - C-level managers? - marketing employees? - appropriate for all requirements of applications? - only GUI or additionally to a EPL? Rainer v. Ammon 19 Existing standards and current research projects related to edBPM Rainer v. Ammon 20 edBPM-enhancements of standards, e.g. WS-BPEL Pattern 1: Stop one or more running processes upon event This pattern solves the case 5 of para. 3.3. This could be achieved by using the fault handling mechanism available in WS-BPEL In order to activate the corresponding fault handler we can the use event handler that is activated by a complex event occurrence. The event handler throws a fault that stops all the currently running activities. A strong benefit of using loosely coupled eventdriven solution is the ability to stop any number of running processes upon a single event: all the processes subscribed to this event will be affected. This is opposed to a ‘conventional’ peer-to-peer approach that would require explicit sending a request to each one of the processes that should be stopped. Pattern 2: Start a new process instance upon event This pattern solves the cases 6, 7 and 9 of para. 3.3. Standard WS-BPEL allows starting a new process instance upon an incoming message (via the receive element). We extend the receive element with the ability to specify an event name or an event pattern instead of a message. In order to activate a new process instance the enhanced receive version could be used. If there are several possible events or messages that should start the same process the pick element can be used in a similar manner. Pattern 3: Activate a task or a process in the absence of the expected event within some time period This pattern solves the case 8 of para. 3.3. The absence of some event is an event pattern by itself. It can be specified in an event processing language and detected by an event processing system. Therefore, this pattern is a special case of the above one (start a new process upon event). Pattern 4: Suspend a process and resume upon event This pattern solves the cases 7 and 8 of para. 3.3. This pattern can be implemented using one of the synchronous BPEL constructs, i.e. receive or pick. For this specific use case (‘fraud’) pick is preferable because it allows handling time out situations, that is what happens if the ‘false positive’ decision is never made. Pattern 5: Start a process modification/adaptation upon event Master thesis Alex Kofman Technion University Haifa supervised by Opher Etzion This pattern solves case 10 of para 3.3. The pattern can be implemented using the enhanced version of the event handler mechanism. The event handler is activated upon an instance of the specified trigger event (e.g. ‘Too Many False Positives’). The enclosed activity can modify the BPEL variable that participates in ‘fraud detection’ pattern calculation, or submits an update request to the event processing system. If the case requires modifying the workflow, the workflow adjustment could be considered as a separate process that is started upon this event. As soon as the new process version is available, it will be activated. The process and possible issues of activating a modified process in presence of an older version of the process is out the scope of this paper. ... Rainer v. Ammon 21 Domains of edBPM and their interdependencies Rainer v. Ammon 22 Definition of basic ED-BPM concepts – needed for a systematic setting up of use cases with customers Rainer v. Ammon 23 Definition of basic ED-BPM concepts – needed for a systematic setting up of use cases with customers Domain Sub-Domain SmartNavigation Use Cases Services/Processes Calculate Route defineStartDest definePoI ... Find Offers SmartNaviVolvo ... SmartFleetManagement SmartCity SmartCityTourism ... SmartTourism SmartTourismGenoa Rainer v. Ammon 24 Proposal for contributing to NESSI Software+Services Platform NESSI platform consists of nine working groups: Open Source Services Sciences Security, Trust and Dependability Software Engineering User Services Interactions Business Process Management Semantic Technologies Service Engineering Service Oriented Infrastructure NESSI also founded a SME-working group. NESSI has 3 "Committees" for the main foci "Future Internet", "Standardisation" und "Strategic Research Agenda (SRA)". We already contribute EASSy is a NESSI Strategic Project Rainer v. Ammon 25 Dissemination and Teaching ED-BPM Rainer v. Ammon 26 The business modeler and the event modeler – different qualifications not in personal union A proposal for a curriculum of a new international Master course of study “Event-Driven Business Process Management” 1. Description: Curriculum together with required credits and examinations Certificate: Master of Science Programme Duration: Four Semesters (120 credits/cr) Mastercourse-EDBPM-v02.doc Rainer v. Ammon 27 The business modeler and the event modeler – different qualifications not in personal union Concept: The course consists of the fields of study Business Process Management, Complex Event Processing, Business Activity Montoring included Business Intelligence and Data Warehousing, Computer Networks, Messaging as well as several application disciplines like Algorithmic Trading, Supply Chain Management in the retail domain, fraud detection in the banking and insurance domain etc. All courses are completed with course-related tests and Credits (cr) according to European Credit Transfer System (ECTS) are awarded. All courses are given in English and are designed as distance learning/eLearning courses. → course will be based on a new Technology-EnhancedLearning approach, developed in the project “CloudBox-edBPM“ Rainer v. Ammon 28 Dissemination and Previous Work Examples of last contributions to conferences edBPM/DoReMoPat at FP7-ICT Proposers' Day 2009, January 22, Budapest http://ec.europa.eu/information_society/events/cf/stream-items.cfm?id=20 edBPM exhibition SSOKU 2009 1st European Conference on Software Services and SOKU technologies, Brussels, January 13 – 14, 2009 edBPM-Workshop ServiceWave 2008, Madrid, December 10 – 13, 2008 9th edBPM Expert Meeting Regensburg, Germany, December 7 – 8, 2009 Rainer v. Ammon, Andreas Hehmann Mainz, Germany, October 7 - 9, 2008 http://www.bpm-event.com/ "Event-Driven Business Process Management Taking the Example of Hamburger Sparkasse" EDBPM-Haspa.ppt Rainer v. Ammon, Christoph Emmersberger, Florian Springer, Christian Wolff Vienna, September 28 - 30, 2008 FIS 2008 / 1st International Workshop on Complex Event Processing for Future Internet - Realizing Reactive Future Internet "Event-Driven Business Process Management and its Practical Application Taking the Example of DHL" FIS08_AmmonSpringer.pps Rainer v. Ammon New York/Stamford, September 17 - 19, 2008 4th EPTS symposium "Proposal for a new Master course of study Event-Driven Business Process Management" EDBPM-mastercourse.ppt Adrian Paschke, Rainer v. Ammon Irsee Monastery, Bavaria, Germany, July 9 - 13, 2008 Focus Group for EuroPLoP 2008 Domain-specific Complex Event and Rule Patterns http://hillside.net/europlop/ Agenda for the focus group... Rainer v. Ammon 29 29 EASSy team – Event-driven Adaptivity of Servicebased Systems Rainer v. Ammon 30 CloudBox team – Technology-Enhanced-Learning based on edBPM Rainer v. Ammon 31 Thanks for your attention! Rainer v. Ammon 32