Loras College 2014 Business Analytics Symposium | Aaron Lanzen: Creating Business Levers with Smart Analytics

Editor's Notes

• #2: Audience intro up:My name is Aaron Lanzen and I work for Cisco in Technical Services.What is that? 3000+ Highly specialized Engineers working 24 X7 -> 365 in 50 + sites around the world to support Customers having an issue with any Cisco productContext:If any of you have a support contract with Cisco and have called in over the last 3 years… The engine I’m going to talk about handled the assignment of your case to an engineer. What is a business Lever…. To me, its what Execs always ask for. It’s the slightly fuzzy concepts that everyone would like to steer the company with. This preso is how Cisco makes those real for our decision makers.
• #3: We still joke about giving the hamster a rest.I’m bringing this up because we don’t spend 10’s of millions of dollars on Analytics. We start small and pick off the value as we go.Before this. All we could do was count things.THE MESSAGE: Don’t think you need millions of dollars to start. You need:Unfettered Access to Data Place to store it People to do analysis on it Most importantly : PEOPLE WHO CAN MAKE A DECISION( When I came in the door )No direct access to data. Only business objects.Everything was spreadsheets based off of SQL dumps then macros & VB took care of the calculations.Debugging was a disaster. No Hierarchy management. No IDE, No SQL Client toolsTook 1 year to get it cleaned up bus still had the same infra. : Custom hierarchies : Capacity model + optimization DWH to source from New data feeds Decision trees to explain
• #4: New leadership had a 1 week long executive offsite meeting trying among other things to solve acustomer satisfaction problem.Legendary stories: No breaks, no fun, 12-14 hour daysProblem 298 focused on Customer surveys and the text comments drove the conversation. -- BTN Now, our executives are not silly. They knew this was not the final answer. It was the level of agreement they could reach on just ONE of the 500 things they covered for the week.Project Manager took down the notes literally and a project landed on my boss’s desk.This is what it looked like:
• #5: This is almost what the meeting looked like. The BIG IDEA landed and everyone was thinking : BtnWhat is the world did he just say?PRESS BtnThis conflicted with every portion of our existing resource allocation strategy. We had an escalation based model where you started near the bottom of the pyramid of staff and IF you needed more help you were escalated to another higher resource.First Question I asked was: So say you assign your best 2 engineers to the first 2 cases of the day and then … the City of Detroit calls in with a 911 system that is down.What do you want to do then? Your top talent is engaged with random cases. Time of day cant impact service.NO ANSWERS The rest of this story how we used Analytics to transform this SINGLE IDEA into a patent pending resource allocation model
• #6: 7 min to here 3-4 hereLeadership has a different set of needs than the folks who have to implement and manage the solutionGetting people on the same page is critical. Keeping them on the same page is the difference between success and failureThis is what you can count on: The Idea is not a solution Different stakeholders will have objections. ( Do you know what this is?) The IDEA will need to morph or it will die / become irrelevantI In my experience even a small VALID GAP can derail an entire implementation. If Smart Analytics are done well…. The IDEA will evolve AND still make sense to execs plus be actionable “Bridges power points to bus requirements”THIS IS THE BALLGAME!Make sureyou keep the original idea relevant to the people who created it while you implement it Get their feedback ( review clarification to their IDEA not asking them to read a 50 page specification) Guarantee their support ( Because its still their IDEA) Get invited back to the table ( because you respected their ideas .. Future meetings == More insight) Create partnership ( Turning ideas into better ideas )Keep things Real ( does not matter how bad your model is or if the data is erratic.. A real definition is worth more that a PowerPoint plan) Leadership can help overcome shortcomings IF you show the value Example -> (data quality never funded till $$)Smart Analytics are the foundation we use to create Business Levers. On to Level 1
• #7: Key is Historical perspectiveThese are what good BI teams do. Make sense of what happened.A Tonof value is generated by Level 1 ( WE WERE HERE FOR A FEW YEARS)Pre-cursor to “business levers”-----------------------------POST CLICK------------------------------They don't solve ->? “Find the highest skilled Engineer ASAP” but they simply and help you define what you should/could DO.BENEFITS on next slide
• #8: 12 min in15 outWe completely expected to some of the benefits.Same models on Reports, KPI’s, DashboardsSame capacity models in KPI, and PlanningPeople giving us their feedback and suggestions to make a better modelPeople threw up objections and we simply adapted the model or created a new metricNow we are on the same sheet of music, people understand its going to change and get better over time.All of these ideas were adopted by Ops and planningAllowing our bright people to look deeply at a process or decision and teas out the valuable informationWe did not expect:Deep engagement. When you make a business concept real… people want to participate ( this is how you KNOW you are working on the right stuff ) ( Watch for people who count on churn to not be held accountable .. They expect a 9-12 month turn around )Stakeholders showing up with great ideas combining concepts and identifying tradeoffs. They were taking ideas from other projects and acting like it was a foundational componentNo one expected the “fact based conversation” to speed up the process so much. ( My Boss believed a metric was THE answer…. 1 decision tree broke that idea and he influenced everyone for us )(Complicated 5 way unwind -> )Funding flowed: Fix this data, add that button, track that data, extend this idea… YOUR ROADMAP is criticalMost interesting benefit: The conversations this creates is impressive. Its not just a word on a slide. People start to understand and accept its real. Changes the game-- dup on next slide delete belowObservation 1: Business Levers can be used to identify the situation and/or the action(s) leadership may want to take. This intuitive “When -> Then” pattern is natural for business people and it fits the rule engine paradigm perfectly.Observation 2: Real world business decisions:Seldom “hard” left or right turns based on a True/False attributes. we live in a world with shades of greyUsually they involve other decision components and they are used together to create “tradeoffs”Complex because many: many shades of grey lead to difficulty in explaining how the decision will be made to governance boards, the sales team or any stakeholder in generalEven MORE a pattern for rule engine--- SHOW A DECISION MAP with easy fixed components and a few fuzzy business levers 
• #9: 16 – 19Motivation to get past level 1How do youmake them business levers vs reporting concepts?Your entire year you make: 1 visionary decision: Spend a lot of time on it. It may have a huge impact 3-4 Strategic decisions: Spend a fair amount of time on it: They should have big impactsAre you ignoring your tactical decisions?Why not invest time in automating the millions of tactical decisions so they work with your visionary and strategic plans. Cumulatively they can have as much impact as the othersAs I said before: YOU invest in Analytics to turn your ideas into business levers! THINK AUTOMATED DECISIONS
• #10: 20 - 22When people start saying : “When something… Then we want to do something” IN REAL TIMEThen you have to start thinking about the next step in your program.Till now you have just been describing history. BUT how do you take action? (SHOW DECISION DIAGRAM ?) == BTNDoes NOT have to be a rule engine. But its really low risk especially if you have an ESB. (WHY)Safe and secure was our direction (EXAMPLE) Insert in flow but take no action , Then ………Look for Lifecycle of an Analytic slide== btnCore Capabilities:The people who know own the rules , Data, governance, logging , and decisions. IT owns the plumbing: Messages to / from our decision serviceSimulation is CORE to our success. Sim Drive Development.2 BIG Bus reasons why Rule EngineWhen -> ThenComplex Decisions can only really be answered by simulation. Technical:Partial inferenceLinear scalabilityPeep hole development in version 3NEXT IS WHY DO YOU WANT TO GET TO LEVEL 2
• #11: 23-25Level 2 analytics are simply when you ADD the capability of maintaining real time level 1 analyticsExample: Creating live classifications of availability for EVERY engineer.Could use live data compared to historical data to make the live choice. (Important Concept)Use 1: logging and using the data to find useful patterns, hot spots, etc ( try before you buy)You could end up with models or just valuable insight that creates a more concrete and rich definition of availability “Smart Data”Use 2: If you have real time information… leverage this data in your live decisions BUT for logging only CALL THIS: “Log and Learn” from your new ANALYICUse 3: Once you have compared what you would have done to what you actually did.. You can start making real time decisions Real time execution + “Log and Learn”THIS IS WHAT WE DIDNEXT lifecycle of an analytic
• #13: 28-31It’s a bit like a full circle. Back to the high level ideas again BUT they are anchored by FACTSThis is where Cisco is today. Now we get to LEARN all the time: Example: Analysis suggests that there are 3 models that seem to work across several regions. BUT NO AGREEMENT How do you decide which of the 3 ideas wins?DO YOU HAVE TO?Why not use all 3 and “see” how they perfomr? Since we have all the data we can TUNE or eliminate the lowest performer. This allows us to have true AB testing. Even is ALL 3 turn out to perform lower than the existing strategues WE STILL LEARNWE have a platform to resolve hard problems like this. Plus, once resolved they are in produciton. THIS IS WHY YOU INVEST IN ANALYTICS AND DECISION MANAGEMENTYOU CAN LEARN, ADJUST, Experiment , and blend strategies in real time because you have a solid PLATFORM from which to work  If you remember that original idea. Thru this process we evolved that nugget into a patent pending resource allocation model
• #14: 31-32Just a quick glimpse at the ecosystem.-Our engine sits between SalesForce.com and our processGovernance is the funnel , funding , and is integrated into our process. It is our INNOVATION ENGINE ( NOT A Quagmire of paperwork ) ( TRUST has been built here. 1:1 relationships thrive here )-- We started SMALL AND SAFE -> “DO NO HARM”You can reap huge value even IF your decision engine simply logs information and your analytics program uses it-- Gave a presentation in Las Vegas last year: Governance as a strategic tool and innovation engine-- Your developers are valuable. If there is a place to invest in having people in the same room.. This is it
• #15: 33-35Why not put Nobel Prize and National Medal of Science winners in charge vs Excel spreadsheets?We are not here yet. I’d like to get us here in 18-24 months. It’s a cultural change so its slower than technologySee how this jives with the original intent?Leonid Vitaliyevich Kantorovich and Tjalling C. Koopmans 1975 Nobel Prize “for their contributions to the theory of optimum allocation of resources“George Dantzig -> Simplex MethodYou have made the idea real, improved it, and related it back to the people who “invented it”
• #16: 35-36Stakeholders: Usual behavior is to object to new things -> Address Objections on roadmaprule engine allows customization ( Objections are not show stoppers)Trust : Simulation = Production …. Planning = Execution Supportability: Since every decision is fully documented. Ideas multiply and platform scales since you have a solid foundation. ( Rule engines love this kind of pattern matching)
• #17: 36-37Projects : Metrics and data are priorities not features that fall below the budget line: (Part of our dev/governance process so you cant do it w/o data) Extremely low bug rates: EXAMPLE: 40% change in actionable rules for an object redesign. 1 bug on a disabled feature Impact is known, tracking exists day 1, adjustments are easy 
• #18: 37-38Data: The commitment to data allows people to log ( correlated logging ) to capture data that does not exist anywhere else in the system Remind them of the story to an analytic Every developer, analyst, and requestor is responsible for developing the data to prove their idea worked ( Governance)
• #19: 38-40Operations Tying planning and operations to live execution opens a world of opportunities Impact assessment can be linked to Headcount planning Changes in staffing availability can dynamically adjust live decisions Governed business processes can simply feed their real ( OR WHAT IF ) data into the decision engine and you get concrete answers Bottom line: WE have a saying: “Differentiate for Consistency” CORE: Giving people the choice to try things and be held accountable. The good ideas become better The BAD ideas get used less.If you set up your program correctly: Allows projects to harness the full power of the organization because MANY people nowhave skin in the game and will participate Ideas can be shared across domainsYour PLATFORM allows you to finally have synergy across the company NOT roadblocks  
• #20: 40 -42You need an Ecosystem and time to create awareness and build a reputation.Remember its business people who have the ideas. Technology just makes it easier to make them realAnalytics should evolve as your business changes. They also evolve to accommodate business realities even if they are regional.Plan for this and leverage analytics to keep the solution REALDon’t try and boil the ocean. Pick off a project that easily, inexpensively does several things:Creates great data , provides insights, helps people make sense of multiple problems… AND lays the foundation to become an actionable BUSINESS LEVERThe more successful you are the more Communications and Governance will become critical. Getting your leadership to buy in and create a “front line” of defense will make sure you have the right priorities…. This will be hard but will help you succeedThe future will probably lead down automated decisions. Start that conversation early because it will blur the lines between IT and the Business and that scares many people
• #21: Step 1: Make sureyou keep the original idea relevant to the people who created it while you implement it Get their feedback ( not asking them to read a specification that resembles nothing of their idea ) Guarantee their support over time ( Because its still their IDEA not a convoluted mess ) Get invited back to the table ( respect for the idea gets you the feedback and the insight) Create partnership ( Turning ideas into better ideas )Step 2: Keep things Real ( don’t care how bad your model is, don't care if the data is erratic.. A real definition is worth more that a PowerPoint plan) As you share your models leadership will help overcome some of these issues if you can show the value (data quality never funded till $$) Adjust and blend ideas and reuse other foundational components ( people will start to expect and reference your collection of ideas ) Find your data ( make it if you have to )We face opposing forces Execs don’t get bogged down by details and they don’t provide solutions. They provide direction and desired outcomesVsStaff needs to make it work and bump these ideas into reality. ( map slide here ? Along with “It’s the material your employees use to build the rudder, map, throttle, and even the GPS” )We face opposing forces Execs don’t get bogged down by details and they don’t provide solutions. They provide direction and desired outcomesVsStaff needs to make it work and bump these ideas into reality. Here is one thing you can count on:ANY idea will get objections from stakeholders so the idea will need to grow and morph or it will dieSmart Analytics and raw data are how you create these items in a way that will not lose meaning to execs and be actionable.This slide is meant to remind everyone that people working together is the most valuable thing:Analytics don’t change that. They make it more predictable This preso is the approach Cisco takes to align how we vision , plan, and executeWhat does not seem to work: Relying on PPT or a staff meeting to “execute” strategic initiatives.Buying tools to … but unless that tool modeled your business perfectly you have gaps.
• #23: Critical notions: Decision Engine allows state management on a huge number of items. Rules allow the efficient rendering of this data into statefull analytics aka streaming analytics. (change only, truth mgt)We also leverage the traditional “historical” analytics in conjunction with this live state. (Leveraged in other business process, human oversight + more computationally efficient, percentiles)With this state at our fingertips and a lightning fast access it encourages not just making the live decisions but “logging” the correct snapshot or correlation states (that the engine leveraged, modified, or transformed)DC says tell this story: “the lifecycle of an analytic” We are even requesting events from systems that we plan no direct action. Correlated Logging only. Then state mgt, then referencing this data in other decisions, then reacting to the arrival of these events or the changes in state.Cisco Response: Data lifecycle ( send to log, store a bit and log, reference it to decide , react when it changes 1:M and Criteria based thresholds for actions. “State watchers drive actions not necessarily events” Allowing the engine to respond at the right time to the cumulative impact of all the changes in state. (Reshuffle via degradation of previous answer)How to make data valuableLet me describe our situation at Cisco Services.We have 5000 transactions per day and half of them will define the total yearly experience with the Customer? Let’s add a few more complexities:If the Customer is calling, they are having a terrible day and are looking to you to make the difference. Success for Cisco is similar to an EHarmony moment. We need to create chemistry and value between the Customer and the engineerThe most important factor in creating success is motivating our engineer to consistently do the right thing after we make the automated decision.Each situation we are deciding is incredibly different. (AB testing is terrifying. Verifying Results Massively different technologies, install bases, customer types, skill levels, expectations, etc.We have multiple strategies, sometimes overlapping that reflect regional perspectives. Keeping the impact of each separate for continuous improvement and tuning is criticalCost is not a consideration at the transactional levelSo if changes in 6 weesk are you staffed to hanle itClearly making a couple million automated decisions is very useful at consistently bringing Cisco’s strategy to bear in a vast decision space but its not the total picture. If people are your decision or your decision leads to personal interaction then some of the current analytics success stories may not apply.IF YOU HAVE PEOPLE in your decision… you have a complex problemI’m going to talk about how Cisco is creating a culture of success that believes in and leverages data to make both business and personal decisions in support of our Customers.Focus on simple but not simplistic models that can be communicated to people-- In the end people create the relationship not your decisionThis is philosophy and focus rather than a technologyThe cornerstone to managing a modern business is leveraging and creating accurate data People will find a way around your decision IF they don’t buy into itHow to tap into the vast power of your employees who may have the best ideasCreate awesome data. Don’t just rely on what is currently available.– our capabilities and vendor suggestions Long term success for Cisco Services depends on our ability to do three things simultaneously:Align strategy to accurate live facts : Situation-> Customized ServiceCommunicating the nuance and expectation of “Customized Service” to our engineer Profitably managing multiple, potentially overlapping strategies, in our business processes   CSE realities:The same CSE can face a state’s 911 system being down @ 9am and by lunchtime be handling a case where the : The Customer, a 16 year old high school student, bought a 10 year old router off of EBay last week and wonders why a light is blinking.You can’t script a response for them. You have to explain expectations differently. They have to know that they are not a machine, that a case is not a case, and they are not judged by tic marks.Trust the data…. How do you get there? w/o this… slow for validation. Manual manipulation piecemeal= inconsistency Its often both more affordable and impactful to simply track the right things in the regular system than trying to interpret what happens to be available.
• #24: Ops Data: It’s the path for the business to create/source any missing data. Refreshes data into memory as objects that make writing rules EASY These can be business process outputs and allow the system to adapt without rule changesLogging: We require every rule to audit itself. It’s a complete record of what was used in the decision and what decision was made memory state, raw event data, the decision made, and even rules just to log critical and interesting perspectivesBus owns the authoring rules and release schedule.Simulation: Sim driven development ( lower level , more technical, BUT stakeholders can use it. Removes translation from stakeholder to real system) -- testing data , rule dev, logging, validation each day not after many daysBusiness simulation ( larger scenarios like 1 full year, compare different “runs” and determine impacts ) Stakeholders already familiar with data and reports so they “know” how to ask questions note: the reports are ready for production data as soon as you release for validationGovernance:Right projects, Right impacts, Right solutions ( support/understanding)Engaged Stakeholders and developers that focus on supportable solutions that add value -- Other capability in Drools is a stateful knowledge session. -> previous decisions matter to the next decision