Big Data on AWS
3 likes1,156 views
Big Data is growing rapidly in terms of volume, variety, and velocity. The cloud is well-suited to handle Big Data challenges by providing elastic and scalable infrastructure, which optimizes resources and reduces costs compared to traditional IT. In the cloud, users can collect, store, analyze and share large amounts of data without upfront investment, and scale easily as needs change. Real-world examples show how companies in industries like banking, retail, and advertising are using the cloud's Big Data services to gain insights from large datasets.
More Related Content
Similar to Big Data on AWS (20)
More from Amazon Web Services LATAM (20)
Big Data on AWS
- 1. BIG Data on AWS Paul Duffy
- 2. Characteristics of Big Data How the Cloud Is Big Data’s Best Friend Big Data on the Cloud In the Real World
- 3. Characteristics of Big Data
- 4. The cost of data generation is falling rapidly Dramatic increase in volume, velocity and variety of data
- 5. BIG DATA A collection of tools, techniques and technologies that allow you to work productively with data at any scale.
- 6. Big Data is Getting Bigger 2.7 Zetabytes in 2012 Over 90% will be unstructured Data spread across a wide array of silos
- 7. Features driven by MapReduce
- 8. Variable data structures and sources Computer Generated Human Generated • Application server logs • Twitter “Fire Hose” 50m (web sites, games) tweets/day 1,400% • Sensor data (weather, growth per year water, smart grids) • Blogs/Reviews/Emails/P • Images/videos (traffic, ictures security cameras) • Social Graphs: Facebook, Linked-in, Contacts
- 9. The Role of Data is Changing
- 10. Traditional analytics required a fixed data model, based on pre-known questions Big Data promotes data exploration and experimentation which leads to innovation
- 11. Collection & Computation Collaboration Generation storage & analytics & sharing
- 12. Lower costs, faster throughput Collection & Computation Collaboration Generation storage & analytics & sharing Increased pressure on traditional IT and tools
- 13. Require tools designed for data collection and computation at any volume, velocity or format.
- 14. Software • Designed for distribution • Easy programming models • Flexible language choice • Platform for abstraction and ecosystem • Good example: Hadoop
- 15. Infrastructure • Designed for distribution • Easy programming models • Flexible language choice • Platform for abstraction and ecosystem • Good example: Cloud computing
- 16. Software Infrastructure
- 17. How the Cloud Is Big Data’s Best Friend
- 18. How do we define the cloud? By Benefits!
- 19. No Cap Ex Pay Per Elasticity Use Cloud Fast Time to Market Focus on core competency
- 20. Why is the Cloud Big Data’s Best Friend?
- 21. We know we want collect, store, organize, analyze and share it. But we have limited resources.
- 22. The Cloud Optimizes Precious IT Resources i.e. Skilled People
- 23. “Over the next decade, the number of files or containers that encapsulate the information in the digital universe will grow by 75x. While the pool of IT staff available to manage them will grow only slightly. At 1.5x” - 2011 IDC Digital Universe Study
- 24. Deploying a Hadoop cluster is hard
- 25. Cloud computing 30% 70% The Old Managing All of the IT World Using Big Data “Undifferentiated Heavy Lifting”
- 26. Cloud computing 30% 70% The Old Managing All of the IT World Using Big Data “Undifferentiated Heavy Lifting” Cloud-Based Configuring Infrastructure Analyzing and Using Big Data Cloud Assets 70% 30%
- 27. Managed Reusability Services Scale Innovation
- 28. Managed Reusability Services Scale Innovation
- 29. Managed Reusability Services Scale Innovation
- 30. Managed Reusability Services Scale Innovation
- 31. Managed Reusability Services Scale Innovation
- 32. The Cloud Optimizes Capacity Resources
- 33. Elastic Compute Capacity On and Off Fast Growth Variable peaks Predictable peaks
- 34. Elastic Compute Capacity WASTE On and Off Fast Growth Variable peaks Predictable peaks CUSTOMER DISSATISFACTION
- 35. Elastic Compute Capacity Capacity Traditional IT capacity Elastic cloud capacity Time Your IT needs
- 36. Elastic Compute Capacity On and Off Fast Growth Variable peaks Predictable peaks
- 37. The Cloud Empowers Users to Balance Cost and Time
- 38. 1 instance for 500 hours = 500 instances for 1 hour I like this! I scale
- 39. The Cloud Reduces Cost For Experimentation
- 40. The Cloud Enables Collection and Storage of Big Data
- 41. Storage Costs are Declining
- 42. Simple Storage Service 1 Trillion 1000,000 750,000 500,000 250,000 0,000 750k+ peak transactions per second
- 43. Global Accessibility Region US-WEST (N. California) EU-WEST (Ireland) GOV CLOUD ASIA PAC (Tokyo) US-EAST (Virginia) US-WEST (Oregon) ASIA PAC (Singapore) SOUTH AMERICA (Sao Paulo)
- 44. Amazon DynamoDB Managed NoSQL database service Unlimited size Unlimited scale Flexible key/value store Consistent, low latencies (single digit milliseconds, SSD) Robust, durable data storage Integrated analytics with Elastic MapReduce
- 45. Amazon Elastic MapReduce On-demand, managed analytics platform Powered by Hadoop Integrated with Spot instances to lower costs Vibrant ecosystem of tools Elastic clusters Flexible programming model (Java, Python, Ruby etc)
- 46. Big Data on the Cloud In the Real World
- 47. Big Data Verticals Social Media/Advertisi Financial Oil & Gas Retail Life Sciences Security Network/Gamin ng Services g User Anti-virus Targeted Monte Carlo Demographics Recommend Advertising Simulations Seismic Genome Fraud Usage analysis Analysis Analysis Detection Image and Transactions Video Risk Analysis Processing Analysis Image In-game Recognition metrics
- 48. Visualizations
- 49. Bank – Monte Carlo Simulations “The AWS platform was a good fit for its unlimited and flexible computational power to 23 Hours to our risk-simulation process requirements. With AWS, we now have the power to decide 20 Minutes how fast we want to obtain simulation results, and, more importantly, we have the ability to run simulations not possible before due to the large amount of infrastructure required.” – Castillo, Director, Bankinter
- 50. Recommendations The Taste Test http://www.etsy.com/tastetest
- 51. Recommendations Gift Ideas for Facebook Friends etsy.com/gifts
- 53. Click Stream Analysis User recently purchased a sports movie and Targeted Ad is searching for (1.7 Million per day) video games
- 54. Characteristics of Big Data How the Cloud Is Big Data’s Best Friend Big Data on the Cloud In the Real World
- 55. Thank you…