Download as PDF, PPTX
Uploaded byAndrei Savu
Cloud as a Data Platform
The document discusses the concept and characteristics of big data, focusing on its volume, velocity, and variety. It evaluates the Amazon data platform's tools and their applications in data processing and analysis. Additionally, it highlights the importance of having a skilled data science team and emphasizes the need for businesses to define the problems they want to solve with big data.
More Related Content
Similar to Cloud as a Data Platform
Cloud as a Data Platform
- 1. Cloud as aData Platform What is (Big) Data? Amazon Data Services
- 2. Andrei Savu Founder ofAxemblr.com Co-organizer of Bucharest JUG Lead of Apache Provisionr Passion for Automation & Data Analysis Connect with me on LinkedIn
- 3. @ Axemblr Data ProcessingInfrastructure Deployment Automation on IaaS platforms Product: Hadoop On-Demand Appliance Apache Provisionr (Open Source) Consulting & Professional Services
- 4. Topics Introduction on (Big)Data ●Characteristics ● In Practice ● Value Amazon Data Platform ● Tools ● How they fit
- 5. What is (Big)Data? Beyondthe Hype (Source)
- 6. ... size &speed are relative
- 7. Characteristics #1 Too big,Too fast, Unstructured
- 8. 1. Volume "Simple modelswork better with more data" The Unreasonable Effectiveness of Data Alon Halevy, Peter Norvig, and Fernando Pereira, Google Challenging from a technical perspective Needs scalable storage Distributed query engines (massively parallel)
- 9. 2. Velocity Nothing newfor financial traders Tight feedback loop as competitive advantage Complex event processing (CEPs) Online stream summarization (estimation) Online aggregation (key-value stores) Long term storage for batch processing
- 10. 3. Variety The realityof data is messy and the format evolves over time Entity Resolution, Language Detection etc. Mantra: Detect Schema, Annotate, Enrich
- 11.
- 12. (Big) data ismessy 80% efforts go into identifying sources, integration and cleaning Messy and disconnected: different systems, different networks, different departments Consider data-markets
- 13. (Big) data hasgravity Tends to attract processing services The cost of moving may be large
- 14. Cloud or in-house? Cloud: ●for development & exploration ● low usage or variable capacity needs In-house: ● due to strict regulations ● for performance and cost efficiency
- 15. People & DataScience You need a team that combines: math, programming and scientific instinct Building data-science teams http://radar.oreilly.com/2011/09/building-data-science-teams.html
- 16.
- 17. ... answer themw/ Data
- 18. Enables New Products Recommendationengines (think Amazon, Netflix, Facebook, LinkedIn) Advanced advertising (more later) Advanced search & spelling suggestions (and many more)
- 19. Rule of thumb "Adviceto businesses starting out with big data: first, decide what problem you want to solve." * Christer Johnson, IBM’s leader for advanced analytics in North America * create data-driven business processes (more)
- 20.
- 21. Based on mywork at Magnolia Labs Inc. http://magnolialabs.com/ San Francisco, CA based company with R&D in Romania Various products: RTB (real-time bidding), Secure Browsing etc. They are hiring! [email protected]
- 22.
- 23.
- 24.
- 25. Amazon EMR (ElasticMapReduce)
- 26.
- 27.
- 28.
- 29.
- 30. Thanks! Questions? Andrei Savu- asavu @ axemblr.con