Data in Action
1 like498 views
The document presents a comprehensive overview of Natalino Busa's professional background, focusing on data science, machine learning, and big data applications across various industries like music, banking, and healthcare. It highlights the evolution of data analytics, from early exploratory data analysis to modern techniques such as deep learning, emphasizing the importance of effective data utilization for business value and predictions. Additionally, it covers technological advancements including distributed computing, Spark, and operational analytics, underscoring the impact of data-driven decision making.
Data in Action
- 1. Data in Action Natalino Busa
- 2. Pleased to meet you! Italian by birth (43) In the Netherlands since 1998 Married, 3 kids (Ryan, Nemo, Maya) Likes: Movies, Music, Photography, Design, Fashion, Windsurfing, Architecture, Traveling, Salsa, Sailing. On the web: https://www.google.nl/search? q=natalino+busa
- 3. 1/3 slide history of me: Researcher: Compilers, VLIW processors, Multimedia MPEG4, Assisted driving, Audio compression
- 4. 2/3 slide history of me: Product Development: VLSI layout optimization, DRC rule checking & correction Lots of math: Simplex_algorithm Computational_geometry Lots of traveling: Japan, South Korea, US, Germany
- 5. 3/3 slide history of me: Big Data and Data Science: Two verticals: - Music, Video industry (broadcasters) - Banking and Finance Distributed computing Microservices Anomaly detection, Time serie predictions
- 6. about: how to grok data with machines and generate business value
- 7. data is the fabric of our lives all our relationships are based on data
- 8. Data is the new ...
- 9. Data as a Relationship
- 10. Data as a Relationship Trust Transparency of Use Customer First Regulations and Laws Respect and Protect Providing a Service
- 11. Help the customer Propose, Advise, Select, Filter, Connect, Simplify Protect the customer Detect, Prevent, Alert, Block, Defend, Identify, Authorize Actionable Data, Ethical approach
- 12. Techs
- 13. ● Massive multi-rack systems ● 100’s of Computing Cores ● 100’s Terabytes of Storage ● Distributed computing ● Share nothing Architecture ● Advanced Query Plans ● Columnar Data Models ● Re-programmable hardware The Advent of MPP ALAPs (Early 00s)
- 14. ● Simpler programming paradigm ● Distributed, Replicated File System Map-Reduce and Hadoop (Early 00s)
- 15. Streaming and Real-Time Analytics
- 16. 16 Real Time APIs Streaming Data Data Sources, Files, DB extracts Batched Data Training, Scoring and Exposing models
- 17. in-memory computing is winning! Spark is emerging as an improved, faster, better, “new” hadoop. https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html The RAM is the new Disk
- 18. Unified Distributed Computing paradigm: SQL, Statistics Machine Learning Graph Analytics Polyglot Programming: R Python Scala Java Integrated Data Science
- 19. https://spark.apache.org/ Spark Streaming SQL MLlib Graphx Analytics, Statistics, Data Science, Model Training HDFS NoSQL SQL Data Sources Map-Reduce HDFS KAFKA Spark: Hadoop evolved
- 20. Kafka + Spark + Cassandra + Akka (noSQL stack, Fast Data) MPP + HDFS + Spark (“new” Hadoop / Data Lake) Popular Operational Analytics Stacks (10s)
- 21. Science
- 22. Exploratory Data Analysis In 1977, Tukey published Exploratory Data Analysis, arguing that more emphasis needed to be placed on using data to suggest hypotheses to test and that Exploratory Data Analysis and Confirmatory Data Analysis “can—and should—proceed side by side.” Analytics goes mainstream (70s, 80s)
- 23. Knowledge Data in Databases (1996)
- 24. http://flowingdata.com/2009/06/04/rise-of-the-data-scientist/ http://medriscoll.com/post/4740157098/the-three-sexy-skills-of-data-geeks The Rise of the Data Scientist (00s)
- 26. Applications: Kickstarter you’re not selling analytics, you’re selling biz development !!! Advices on how to setup a better kickstarter page Projections for investors and B-series angels
- 27. Prediction based on Big Data PredPol: Los Angeles police department and University of California 13 Million recorded crimes Prediction based on historical data, geo location and recency
- 28. Prediction based on geolocated events Clustering geolocated data using Spark and DBSCAN How to group users’ events using machine learning and distributed computing By Natalino Busa January 28, 2016
- 29. Data Science: Interpretability vs Accuracy Traditional Regression ARIMA ANOVA Naive Bayes Decision Trees Splines Modern Interpretability prevails Random Forests Cons: feature engineering Modern Accuracy prevails Neural Networks Cons: hard to explain why Modern Clustering SVMs Gaussian Processes Bagging Boosting
- 30. - Deep Learning ConvNets and the rebirth of neural networks
- 31. Deep Learning to assist doctors treating and classifying cancer http://www.enlitic.com/ Cancer Classification and Treating
- 33. T-SNE and dimensionality reduction https://www.oreilly.com/learning/an-illustrated-introduction-to-the-t-sne-algorithm
- 34. - Topological Data Analysis Analyze high-dimensional data, visually http://datarefiner.com/ Analysis of NetFlix Prize Dataset. Data sets statistics: ● 100,480,507 ratings ● 480,189 users ● 17,770 movies ● 2.8 GB CSV file size Topological Data Analysis
- 35. Fraud Detection and Graph Analysis
- 36. Fraud Detection and Graph Analysis “ Find those cases where the doctor or the examiner is also a participant in another case, and phone numbers have been reused ” Sub Graph isomorphism Problem
- 37. Takeaways Think Business and ROI
- 38. Takeaways Think Business and ROI Better Interaction Streaming Computing: Data in Action!
- 39. Takeaways Think Business and ROI Better Interaction Streaming Computing: Data in Action! Better predictions and models: Machine Learning + SQL
- 40. Takeaways Think Business and ROI Better Interaction Streaming Computing: Data in Action! Better predictions and models: Machine Learning + SQL Better summarization and Feature Extraction Deep Learning
- 41. Takeaways Think Business and ROI Better Interaction Streaming Computing: Data in Action! Better predictions and models: Machine Learning + SQL Better summarization and Feature Extraction Deep Learning Better Analysis of Relations Graph Databases and Algorithms