Event Driven Business Intelligence


By Ali BELCAID – Managing Consultant
Context : Traditional BI



       New BI Architecture



         Business Event Approach to Project



      Business Event Implementation



Event Management - Where it is Applied
Context
                                                                Traditional BI Architectures




                                                                      Business Intelligence (BI)
ERP
                                                                      •   Reports
                                                                      •   Olap
                                                                      •   Dashboards


CRM                       ETL

                    ETL

FIN                         ETL
                                    Datawarehouse
                                        or ODS




 HR




 Files                                                                Real-time BI
         Operational sources                        Datamarts
                                                                      •   Real-time Operational
         •   Relational Databases                                         Dashboards
         •   Mainframe                                                •   Alerts and escalation
         •   Flat files                                               •   Collaboration
Context
                                                     Traditional BI Architectures



As matter of fact :

 Business users are relying on inflexible, heavy and costly BI platform :
  each time they ask for new request, they have to wait for

 Business users are relying on past figures, so that they are delaying their
  decision every time waiting for the right information until they get it

 Their BI platform is not synchronized with their business processes : there
  is a clear GAP between their BI and their processes

 There is no way to have insights from what Business User is doing at the
  moment : He has to wait for having it !

 Many companies are struggling to get people having real-time data but
  they are doing it in the wrong way
Context
                                                              Continuous Intelligence – Avoid Latency



Data Latency is what is causing
Traditional BI not to reach the                           Business Event
goal of having the right data at                        Trigger every Data
the right time.
                                                             Handling
                                                                                   Do not wait for a long
                                                                                           Time:
                                 Business Event
                                                                                    quicker response to
                                                                                      detected issues
      Potential Business Value




                                          Data Stored

                                                                                                    Avoid :
                                                   Information Delivered                        Data Latency
                                                                                               Analysis Latency
                                                                                               Decision Latency
                                                                   Information Analyzed



                                                                                                   Action Taken


                                                         Action time
Context
                                                                         Real World Event



In real world (and
real business also)                             Customer
                                                 Logon
we are surrounded                                                            New
by Events. We have                                                           Order
to respond to these                                       Customer
events in such way                                         Checks
                                       Fed             “Close Account”
there will be no                    Base Rate             Web Page
interruption in what                 Increase
we are doing : This                                                          Contract
is where Business      Production                             Mobile Call   Submitted
Event comes to play       Item                                 from CT
                                             New               @11.13
a new role in the BI   Arrives at          Liability
universe as support       Store             Added                           Contract
to our Business                                                             Returned
decisions.                                                     Rental       thru EDI
                                Rental
                                                                 Car
                                 Car
                                                              Returned
                               Crashed
Context
                                                          Where is the information ?




These Events when                              Risk of
there identified they         Fraud           Customer                  Change in
                               Risk!          Defection               Product Sales
could mitigate many
risks that companies                                                      Trend
are facing in day to               Rental
day operations                    Contract          Customer               Cell phone
                                  Complete          CrossSell              fraud alert
                                                   Opportunity

                           Employee
                           Over hours                                     Contract
                                         Compliance                       Validated
                                            Limit
                                         Approached       Customer
                                                          now rated
                                                            Gold          Contract
                                                                           Valid
So, applying Business Event Processing to BI
makes a real sense to bring real-time support
decision to Business Users
Context : Traditional BI



       New BI Architecture



         Business Event Approach to Project



      Business Event Implementation



Event Management - Where it is Applied
New BI Architecture
                                                                                           Streaming Analytics


                                Store Data for :
                                   Trend Analysis                                        Business Intelligence (BI)
                                   Comparative analysis                                     Reports
                                   Analysis on historical data                              Olap
  ERP
                            ETL
                                                                                             Dashboards

                      ETL
  CRM
                                 ETL
                                                       Datawarehouse
                                                           or ODS


  FIN
                ESB

                A
   HR           D
                A
                                       Filter /
                                     Aggregate
                                                                          Datamarts

                P
   Files        T     Capture
                                     Business
                                      Event             Correlate

                E                   Processing

    X
                R                                                       Event
                                                                                          Real-time BI
                                                                                             Operational Dashboards
                S
  Events
                                        Act                            Database              Alerts and escalation
                                                                                             Collaboration

External Data                                     Continuous data flows
New BI Architecture
                                                   Example – Action from Insight




This is an illustration
of the way a
Marketing Director
can manage his
                                                     Execute Action
campaigns according                                  Provide values for the following action parameters:

to info's brought into                                Begin Marketing Campaign


his dashboard. This is                                              Region    West
                                                              Campaign ID     42684
all about bringing
intelligence into
processes.                                            Help                                                 Cancel   OK




     Bringing real-time data will turn this into
     Continuous Process Intelligence
Context : Traditional BI



       New BI Architecture



         Business Event Approach to Project



      Business Event Implementation



Event Management - Where it is Applied
Business Event Approach
                                                               How do we tackle a Business Management
                                                                                         Event Project?


 Bringing Event Processing Management into BI world requires the use of new approach to
 tackle projects. You can use a phased approach containing 3 steps : Sense, Analyze and
 Respond. This approach is well known in the CEP (Complex Event Processing) arena.
     Sense                          Analyze                               Respond

  Event 1

  Event 2                  Correlation with
                           external Events                                                          - Alert in Dashboard
  Event 3                                                                  Act by
                                                                                                    - Activate a Process
                            Business Rules                                Making                    - Activate another application
  Event 4
                                                                          Decision                  -…
  Event 5                        Analysis

     …                    The analysis part correlates or          The responders execute the
                          fuses data to determine an               then clauses. This part
  Event n                 estimate of the current state, and       responds when reality
                          to compare the real state with           deviates from expectation by
                          the plan. In effect, the analysis        modifying expectation (the
The sensing part          part continuously evaluates the          plan) and executing
obtains information       when clauses of when-then                responses such as sending
from within and outside   rules by fusing information from         alerts, invoking applications,
the enterprise. This      all the sensors and determines if        and initiating operations by
part obtains the data     the then clauses should be               actuators.
that defines “reality.”   executed.
Business Event Approach
                                                     Project Phases - Sense



           Functional Requirements

  Events   1.   Determine Business Objectives that you are looking at

           2.   Determine the kind of events you want to manage and if your
                system allows you to get them internally or externally.
Group 1
           3.   Classify and group events that could be threats or opportunities
                depending on the expected analysis.
Group 2
           Technical Requirements
Group 3
           1.   Determine Event Sources
                 •    Do you have access to the events ?
Group 4          •    What are the security Requirements

           2.   Determine Event Transport Services
                 •    Publish and subscribe messaging
                 •    Feed backed transport such as RSS or ATOM
                 •    Request / Reply (Pooling) and SOA

           3.   Determine your adaptation requirement
                 •    SNMP
                 •    File adapters
                 •    Database
Business Event Approach
                                                                       Project Phases - Analyze




                                                                                                true
                                                                true
                       Group 1



                                                                                        false

                                                                false
1. What are the          2. What is the
   sources of               temporal or spatial   3. What rules & calculations should be applied?
   data?                    link between the
                            data sequences?
                                                                                          Complex Event

                                                                                       match

            Stream1
                                                                                       match
            Stream 2

            Stream 3
                                                                                      4. How to correlate ?
            Stream n



                       Stream Adapters                      Event Correlation
Business Event Approach
                                            Project Phases - Respond




                     Decision-Centric
                       Automation
      Investigate                                  Manage
      to Respond     Routes work among           Exceptions
                      people
                     Keeps track of work
                      status
                       Task-oriented
                        Collaborative
                        Informational
Automated                                           Transaction-
Detection                                        Centric Automation

                                                  Straight-through-
                         Automatic
                                                   processing
                         Response                 Optimized to avoid
                                                   manual steps and
                                                   redundancy
                                                  Event-driven
Context : Traditional BI



       New BI Architecture



         Business Event Approach to Project



      Business Event Implementation



Event Management - Where it is Applied
Business Event Implementation
                TIBCO Business Event Server – The platform




                           Complex Event Processing
                                 Event Filtering, Aggregation,
                                  Correlation
                                 Missing Event Detection
                                 Causal & Timing Relationships
                                 Multiple Event Patterns & their
                                  relationships
                                 Reactive Rules Triggered By
                                  Event Patterns
                           End-to-end visibility into
                            corporate performance
                                 SLA violation
                                 Fraud detection
                                 Up-sell opportunity
                           Offers Real-Time Analysis
Source Tibco®               of Events
Business Event Implementation
                   TIBCO Business Event Server – State
                     Model – Visualize Object Life Cycle




Source Tibco®
Business Event Implementation
                                                    TIBCO Business Event Server – BE
                                                           Features – Rules Analyzer



                                      • Visualize dependencies of Rules.
                                      • Eliminate conflicts among Rules/Conditions.
                                      • Visualize Concept and Event relationships through rules.
                                      • Drill down to a concept, and see its dependencies.
                                      • Visualize what gets fired when something changes, and its
                                      ripple effect.




  • Profile runtime metrics/statistics of the engine for each rule, and optimize the rule.
  • Combined with the debugger, shows when a rule is fired.

Source Tibco®
Business Event Implementation
                     TIBCO Business Event Server – BE
                          Features – Decision Manager




Source Tibco®
Business Event Implementation
                     TIBCO Business Event Server – BE
                               Features – Rules Tester




Source Tibco®
Context : Traditional BI



       New BI Architecture



         Business Event Approach to Project



      Business Event Implementation



Event Management - Where it is Applied
Business Event Management
                                                                              Fields where it is applied



Financial Services                      Health Care                           Telecommunications

   Algorithmic Trading in many asset      Patient monitoring                   Network management
    classes                                Fraud detection                      SLA management
   MIFID compliance,                                                            OSS/BSS systems
   Surveillance                                                                 Location-based services
                                        Public Sector / Government
   Fraud detection
   SWIFT money transfers                  Electronic battlefield            Retail / Supply Chain
   Real-time risk management              Surveillance
   Smart order routing                    Emergency response                   Automated supply chain
   Market making                          War fighter monitoring               Smart shelves
   Market aggregation                     Security                             Real-time point-of-sales analysis
   Treasury centralization                Net centric applications              and one-to-one marketing
   International funds transfers
   ATM monitoring and control                                                Web
   Clearing                            Travel
   Cash and securities payments                                                 Click-stream analysis
    with interbanking compensation         Flight logistics and operations      Real-time web page analysis
                                           Automated customer service           On-line fraud detection
Energy                                      (kiosks, etc.)
                                           Automated and computer            Entertainment
   Energy trading                          assisted baggage handling
   Pipeline monitoring                     (e.g., with RFID)                    Casino fraud detection
   Power grid monitoring and              Real-time fare pricing               Dynamic customer loyalty programs
    control                                                                      On-line gambling and gaming.
http://intelligenteenterprise.blogspot.com/
http://www.linkedin.com/in/albel

Albel Pres Continuous Intelligence Overview

  • 1.
    Event Driven BusinessIntelligence By Ali BELCAID – Managing Consultant
  • 2.
    Context : TraditionalBI New BI Architecture Business Event Approach to Project Business Event Implementation Event Management - Where it is Applied
  • 3.
    Context Traditional BI Architectures Business Intelligence (BI) ERP • Reports • Olap • Dashboards CRM ETL ETL FIN ETL Datawarehouse or ODS HR Files Real-time BI Operational sources Datamarts • Real-time Operational • Relational Databases Dashboards • Mainframe • Alerts and escalation • Flat files • Collaboration
  • 4.
    Context Traditional BI Architectures As matter of fact :  Business users are relying on inflexible, heavy and costly BI platform : each time they ask for new request, they have to wait for  Business users are relying on past figures, so that they are delaying their decision every time waiting for the right information until they get it  Their BI platform is not synchronized with their business processes : there is a clear GAP between their BI and their processes  There is no way to have insights from what Business User is doing at the moment : He has to wait for having it !  Many companies are struggling to get people having real-time data but they are doing it in the wrong way
  • 5.
    Context Continuous Intelligence – Avoid Latency Data Latency is what is causing Traditional BI not to reach the Business Event goal of having the right data at Trigger every Data the right time. Handling Do not wait for a long Time: Business Event quicker response to detected issues Potential Business Value Data Stored Avoid : Information Delivered Data Latency Analysis Latency Decision Latency Information Analyzed Action Taken Action time
  • 6.
    Context Real World Event In real world (and real business also) Customer Logon we are surrounded New by Events. We have Order to respond to these Customer events in such way Checks Fed “Close Account” there will be no Base Rate Web Page interruption in what Increase we are doing : This Contract is where Business Production Mobile Call Submitted Event comes to play Item from CT New @11.13 a new role in the BI Arrives at Liability universe as support Store Added Contract to our Business Returned decisions. Rental thru EDI Rental Car Car Returned Crashed
  • 7.
    Context Where is the information ? These Events when Risk of there identified they Fraud Customer Change in Risk! Defection Product Sales could mitigate many risks that companies Trend are facing in day to Rental day operations Contract Customer Cell phone Complete CrossSell fraud alert Opportunity Employee Over hours Contract Compliance Validated Limit Approached Customer now rated Gold Contract Valid So, applying Business Event Processing to BI makes a real sense to bring real-time support decision to Business Users
  • 8.
    Context : TraditionalBI New BI Architecture Business Event Approach to Project Business Event Implementation Event Management - Where it is Applied
  • 9.
    New BI Architecture Streaming Analytics Store Data for :  Trend Analysis Business Intelligence (BI)  Comparative analysis  Reports  Analysis on historical data  Olap ERP ETL  Dashboards ETL CRM ETL Datawarehouse or ODS FIN ESB A HR D A Filter / Aggregate Datamarts P Files T Capture Business Event Correlate E Processing X R Event Real-time BI  Operational Dashboards S Events Act Database  Alerts and escalation  Collaboration External Data Continuous data flows
  • 10.
    New BI Architecture Example – Action from Insight This is an illustration of the way a Marketing Director can manage his Execute Action campaigns according Provide values for the following action parameters: to info's brought into Begin Marketing Campaign his dashboard. This is Region West Campaign ID 42684 all about bringing intelligence into processes. Help Cancel OK Bringing real-time data will turn this into Continuous Process Intelligence
  • 11.
    Context : TraditionalBI New BI Architecture Business Event Approach to Project Business Event Implementation Event Management - Where it is Applied
  • 12.
    Business Event Approach How do we tackle a Business Management Event Project? Bringing Event Processing Management into BI world requires the use of new approach to tackle projects. You can use a phased approach containing 3 steps : Sense, Analyze and Respond. This approach is well known in the CEP (Complex Event Processing) arena. Sense Analyze Respond Event 1 Event 2 Correlation with external Events - Alert in Dashboard Event 3 Act by - Activate a Process Business Rules Making - Activate another application Event 4 Decision -… Event 5 Analysis … The analysis part correlates or The responders execute the fuses data to determine an then clauses. This part Event n estimate of the current state, and responds when reality to compare the real state with deviates from expectation by the plan. In effect, the analysis modifying expectation (the The sensing part part continuously evaluates the plan) and executing obtains information when clauses of when-then responses such as sending from within and outside rules by fusing information from alerts, invoking applications, the enterprise. This all the sensors and determines if and initiating operations by part obtains the data the then clauses should be actuators. that defines “reality.” executed.
  • 13.
    Business Event Approach Project Phases - Sense Functional Requirements Events 1. Determine Business Objectives that you are looking at 2. Determine the kind of events you want to manage and if your system allows you to get them internally or externally. Group 1 3. Classify and group events that could be threats or opportunities depending on the expected analysis. Group 2 Technical Requirements Group 3 1. Determine Event Sources • Do you have access to the events ? Group 4 • What are the security Requirements 2. Determine Event Transport Services • Publish and subscribe messaging • Feed backed transport such as RSS or ATOM • Request / Reply (Pooling) and SOA 3. Determine your adaptation requirement • SNMP • File adapters • Database
  • 14.
    Business Event Approach Project Phases - Analyze true true Group 1 false false 1. What are the 2. What is the sources of temporal or spatial 3. What rules & calculations should be applied? data? link between the data sequences? Complex Event match Stream1 match Stream 2 Stream 3 4. How to correlate ? Stream n Stream Adapters Event Correlation
  • 15.
    Business Event Approach Project Phases - Respond Decision-Centric Automation Investigate Manage to Respond  Routes work among Exceptions people  Keeps track of work status Task-oriented  Collaborative  Informational Automated Transaction- Detection Centric Automation  Straight-through- Automatic processing Response  Optimized to avoid manual steps and redundancy  Event-driven
  • 16.
    Context : TraditionalBI New BI Architecture Business Event Approach to Project Business Event Implementation Event Management - Where it is Applied
  • 17.
    Business Event Implementation TIBCO Business Event Server – The platform  Complex Event Processing  Event Filtering, Aggregation, Correlation  Missing Event Detection  Causal & Timing Relationships  Multiple Event Patterns & their relationships  Reactive Rules Triggered By Event Patterns  End-to-end visibility into corporate performance  SLA violation  Fraud detection  Up-sell opportunity  Offers Real-Time Analysis Source Tibco® of Events
  • 18.
    Business Event Implementation TIBCO Business Event Server – State Model – Visualize Object Life Cycle Source Tibco®
  • 19.
    Business Event Implementation TIBCO Business Event Server – BE Features – Rules Analyzer • Visualize dependencies of Rules. • Eliminate conflicts among Rules/Conditions. • Visualize Concept and Event relationships through rules. • Drill down to a concept, and see its dependencies. • Visualize what gets fired when something changes, and its ripple effect. • Profile runtime metrics/statistics of the engine for each rule, and optimize the rule. • Combined with the debugger, shows when a rule is fired. Source Tibco®
  • 20.
    Business Event Implementation TIBCO Business Event Server – BE Features – Decision Manager Source Tibco®
  • 21.
    Business Event Implementation TIBCO Business Event Server – BE Features – Rules Tester Source Tibco®
  • 22.
    Context : TraditionalBI New BI Architecture Business Event Approach to Project Business Event Implementation Event Management - Where it is Applied
  • 23.
    Business Event Management Fields where it is applied Financial Services Health Care Telecommunications  Algorithmic Trading in many asset  Patient monitoring  Network management classes  Fraud detection  SLA management  MIFID compliance,  OSS/BSS systems  Surveillance  Location-based services Public Sector / Government  Fraud detection  SWIFT money transfers  Electronic battlefield Retail / Supply Chain  Real-time risk management  Surveillance  Smart order routing  Emergency response  Automated supply chain  Market making  War fighter monitoring  Smart shelves  Market aggregation  Security  Real-time point-of-sales analysis  Treasury centralization  Net centric applications and one-to-one marketing  International funds transfers  ATM monitoring and control Web  Clearing Travel  Cash and securities payments  Click-stream analysis with interbanking compensation  Flight logistics and operations  Real-time web page analysis  Automated customer service  On-line fraud detection Energy (kiosks, etc.)  Automated and computer Entertainment  Energy trading assisted baggage handling  Pipeline monitoring (e.g., with RFID)  Casino fraud detection  Power grid monitoring and  Real-time fare pricing  Dynamic customer loyalty programs control  On-line gambling and gaming.
  • 24.