More Related Content
Similar to Big Data, Analytics and Data Science (20)
Big Data, Analytics and Data Science
- 1. Data Big Data and Data Science David Lambert
- 2. Agenda • Introduction to Big Data • The V’s • What Businesses Need to Know • Market Research • Analytics • Modeling • Data Science
- 3. Introduction Census 2013: Internet and Smartphone use
- 4. Introduction 140,000-190,000 job shortfall by 2018 4.4 million jobs in Big Data related by 2015 in US Gartner 2012
- 5. Introduction
- 7. What Is Data Two Basic Types of Computer Data • Structured • Variable or Metric information that is easily accessible and shared between computers and databases. (time, date, location, user ID, file.pathway, sensor) • Unstructured • Information that is difficult to quantify such as text, emails, pictures, videos and other socially generated content and is beyond typical processing power.
- 9. What Is Data
- 10. Share of Global Internet Searches 1.2 Trillion searches per year 40,000 searches per second How Much Data http://www.internetlivestats.com
- 12. How Much Data 64kb for the 1969 moon launch
- 13. Why Now Digital Storage Has Become Very Cheap Price for CPU Performance is Cheap
- 14. • Information Connectivity • Databases • Machines • Employees • Customers • Products 1 Basic Corporate Interaction with Clients Experiences are ‘Pushed’ on Consumers by companies. Companies advertise and market to what they want the consumer to think with little or no feedback. 2 Back and forth communication between ISOLATED users and producers. Push-Pull marketing efforts evolve to gauge consumer experience. Exclusive Company/Customer Dialogue, costly Surveys, and limited Focus Groups were typical of this type of corporate relationship. 3 Consumer community connections and corporate relationships. User-to-user connections grew exponentially with the rise of social media platforms, like Facebook and the internet. Consumer experience shifted away from getting information from companies but rather gaining insight through advanced social webs. The general thought being that consumers-to-consumer interactions are less bias and convey more knowledge about an experience than an organizations. There Is an increased feeling of trust in dealing with a non-partisan opinion. 4 Cloud Server Cloud and Mobile system integration and infrastructure Through the development of advanced computer infrastructures, information growth was so rapid that it is referred to as an explosion of Big Data. The transition to smartphone use over standard computer use caused greater behavioral data to be captured by cloud services. Smartphones act as the most common gateway or remote to this advanced network computers. Cloud Service 5 Internet of things and User Connectivity The cost and benefit of connecting a product to a cloud or internet service captures so much value that companies everywhere are integrating systems to utilize this capability. Interactions between consumers and their products and product-to- product interactions will become increasingly more frequent and will boast a wave of new services to better integrate these systems into the consumer’s life. Progression Cloud Cloud ServiceService Service Service PullPush Web Cloud Internet of Things Server
- 15. Big Data People are creating more Data People, Products and Companies are creating ENORMOUS amounts of Data Companies are creating more Data Products are creating more Data
- 16. Big Data The 3 Vs Chart Innovation must be done more quickly and effectively due to this increase in competition, availability of servicesand dynamically changing technology. Old business issues are still prevalent in BD except the speed, scope &tempo of business offerings have dramatically increased. Businesses MUST take more control over their service offerings. The importance of maintaining a consistent and authentic experience throughout all operations is much more significant with the influence of BD. Organizations must choose or transition to Attractive channels for their objectives, customers, and culture. Greatest Challenge: Simply put, Big Data is an increase in the relationships between Hardware & Software, Products & Services, Customers & Employees within an organization or business.
- 17. The V’s
- 18. The Infinite V’s 1. Volume 2. Velocity 3. Variety 4. Variability 5. Veracity 6. Visualisation 7. Value Data Capture Data Cleaning Data Preparation and Reporting Data MarketingProcess
- 19. Data Capture Data Cleaning Data Preparation Data Marketing Process What Businesses Should Know
- 20. What Businesses Should Know https://infocus.emc.com/william_schmarzo/big-data-business-model-maturity-chart/ • Consultant at EMC Global Services • Former Vice President of Advertising Analytics at Yahoo Bill Schmarzo Big Data Author and Blogger at EMC *** Text is excerpted or paraphrased from several articles by Bill Schmarzo at his EMC blog. 53 41 2
- 21. What Businesses Should Know 1.Integrate (structured) Meta-Data with detailed (unstructured) Behavioral Data to provide new metrics and new dimensions against which to monitor and optimize key business processes. Initial Big Data Focus: Optimize Internal Business Process https://infocus.emc.com/william_schmarzo/the-4-ms-of-big-data/ There are three big data capabilities that organizations canleverage to expand their business intelligence and data warehouse investments to optimize versus just monitor. 2.Deploy predictive analytics to uncover insights buried in the massive volumes of detailed structured and unstructured data. Having business users slice-and-dice the data to uncover insights does not work very well when dealing with terabytes and petabytes of data. 3. Leverage real-time(or low-latency) data feedsto accelerate organizational processes to identify and act upon business and market opportunities in a timely manner.
- 22. What Businesses Should Know Ultimate Big Data Opportunity: Monetize External Customer Insights As organizations advance along the maturity index, three organizational transformations take place to create new monetization based upon the customer, product and market insights gleaned from the first three phases of the maturity index 1. Organizations start to treat data as an asset, not a cost of business. 2. Organizations place formal processesto capture, inventory,refine,and protect their analytics as intellectual property (IP). Analytics, models, processes, etc. 3. Organizational confidence in making decisions using dataand analyticswill grow. Organizational investments in data, analytics, people, processes, and technology will be used to justify decision making.
- 23. What Businesses Should Know
- 24. A SHIFT FROM SURVEYS TO INTERACTION • Surveys are difficult to… • Scale • Incentivize customers • Assess validity • Ask appropriate questions for meaningful insight • Interaction monitoring is more successful to… • Scale • Incentivize customers • Assess Validity • Monitor Appropriate behavior for meaningful insight Interaction Collection = Data Mining Market Research
- 25. Market Research The platform or “skeleton” service initially starts out with very little information about that particular user and thus the experience for that user is typically unexciting or ambiguous. User Fundamental service strategy: solutions selling, system integration Interaction Behavior Knowledge Consumer Interaction Platform
- 26. As the User begins to interact with the platform, the user provides user-defined inputs to basic profile characteristics. User defined variables should be limited to varaibles that are not easily gathered from general usage information. Age, Gender, income or other basic characteristics may be identified. Interaction Behavior Knowledge Market Research User Fundamental service strategy: solutions selling, system integration Consumer Interaction Platform
- 27. General activity and interaction with the Service Platform gives enough behavioral data to give an extrapolated rough sketch of a consumer’s behavioral profile. General usage data can include activity hours/per month, activity duration per use, typical activities performed during use. Interaction Behavior Knowledge Market Research User Fundamental service strategy: solutions selling, system integration Consumer Interaction Platform
- 28. Adoption of the service platform into the clients lifestyle gives a caricature-like view of a consumer, with exaggerated likes and dislikes. The consumer engages with the platform regularly and has customized it to their preferences and application. Interaction Behavior Knowledge Market Research User Fundamental service strategy: solutions selling, system integration Consumer Interaction Platform
- 29. Immersion into the platform by the user creates a sophisticated profile that can be leveraged to gain unprecedented insight and predictability into consumer behavior. The dedicated use of a service platform by a single user details an almost life-like portrait of the user. If the user is engaged in multiple channels the data becomes even more relevant as it is analyzed across different markets segments and applications.Interaction Behavior Knowledge Market Research User Fundamental service strategy: solutions selling, system integration Consumer Interaction Platform
- 30. Low Customization Profile Activation Low Account Activity: A picture, couple friends Active Member: Moderate Photos, Postings, Friendship Prolonged Use: Lots of Engagement Interaction Behavior Knowledge Consolidated User Profile with Behavior Generate Value by analyzing behavior Compile Similar Users into Segment Data Packages Mobile Advertising and Proximity Analytics Service Innovation and IT-‐Enablement through Behavioral Analytics. ExactTarget Market Research Consumer Interaction Platform
- 31. Interaction Behavior Knowledge Consolidated User Profile with Behavior Leverage Data (Generate Value) Compile Similar Users into Segment Data Packages Many Companies reflect only a rough sketch of their customers Prolonged Use: Lots of Engagement Market Research Consumer Interaction Platform
- 32. Interaction Behavior Consolidated User Profile with Behavior Leverage Data Compile Similar Users into Segment Data Packages Many Companies reflect only a rough sketch of their customers Prolonged Use: Lots of Engagement Consumer Interaction Platform Market Research Marketing based on Behavior It is important to note that this also applies to process.
- 33. Market Research
- 34. Analytics
- 35. Defining a system of metrics or variables to be monitored, compared and contrasted within that system to determine the PREDICTIVE power of specific variables to specific outcomes. Analytics Key Performance Indicator (KPI) • ROI • Net Profit/Profit Margin • Customer Acquisition Cost
- 36. Analytics Advertising/Sales/Marketing Interaction/Usage Sentiment Brand Culture Customer Acquisition ROI Conversion Rate Social Media Engagement Distinguishing Attributes or Characteristics Behavioral Rituals and Norms Attitudes Lexicon Positioning User Experience Where is the interaction Usage Time
- 37. Analytics Product/Corporate Relationship Advertising/Sales/Marketing Interaction/Usage Sentiment Brand Culture Customer Acquisition ROM Conversion Rate Social Media Followers/Mentions Distinguishing Attributes or Characteristics Behavioral Rituals and Norms Attitudes Lexicon Positioning User Experience Where is the interaction Usage Time • ROI • Net Profit/Profit Margin • Customer Acquisition Cost • Lifetime Value of Customer
- 38. Analytics
- 39. Analytics
- 40. Analytics
- 41. Analytics
- 42. Analytics
- 43. Analytics
- 44. Analytics
- 45. Analytics
- 46. Analytics How Google uses Analytics to drive Technology
- 47. Analytics How Google uses Analytics to drive Technology • Purchased by Google in 2006 • All Music videos except #8 • Music has a lot of replay value. Sharable • Psy and Katy Perry Appear twice on the top 15 http://en.wikipedia.org/wiki/List_of_most_viewed_YouTube_videos
- 48. How Google has Used Analytics to drive Technology Analytics
- 49. Google Adwords • Sunshine Dairy Analytic Modeling • Media Buy • Geography • Click Through-Rate • Impressions • Conversion Rate • Cost Per Click • Ad Position • Size • Media (Vid/Pic/Audio) • Lifetime Value of Customer • Creative • Style (Color/Mood/Message) • Short/Long Term Branding Metrics
- 50. Analytic Modeling Analytic Modeling is using the identified variables to forecast possible outcomes for the future. These models will be compared to actual results to build better models that more accurately predict consumer influences and outcomes on the market. A & B TestingForecast Method
- 51. Analytic Modeling A & B TestingForecast Method Exponential Smoothing Simple Regression Multiple Regression Moving Averages Substitution Forecasting Hybrid Forecasting
- 52. Data Science A Data Scientist is a generic term for someone who possess the ability to do a combination of jobs which include; Data Development Large Data Statistics Data Analyst
- 53. Data Science T - Shaped Skills
- 54. Data Science
- 55. What To Learn Learn to Code Software Engineering Algorithms & Data Structures Visualization Data Munging Distributed Computing Machine Learning Supervised (SVM, Random Forest) Unsupervised (K-means, LDA) Validation, Model Comparison
- 56. What To Learn Linear Algebra (Matrix Factorization) Calculus (Integrals, Derivatives) Distribution (Binomial, Poisson) Summary Statistics (Mean, Variance, Std Dev.) Multivariate Analysis Mathematics Statistics Learn Math and Stats
- 57. Where To Learn
- 60. Questions ?