Downloaded 126 times
More Related Content
Similar to Presentation by TachyonNexus & Baidu at Strata Singapore 2015
Presentation by TachyonNexus & Baidu at Strata Singapore 2015
- 1. Interactive Data Analytics withSpark on Tachyon in Baidu Bin Fan (Tachyon Nexus) [email protected] Xiang Wen (Baidu) [email protected] Dec 02 2015 @ Strata + Hadoop World, Singapore 1
- 2. Who Are We? •Bin Fan • Tachyon Project Contributor • Software Engineer at Tachyon Nexus • Xiang Wen • From Baidu Big Data Department • Senior Software Engineer at Baidu 2
- 3. • Team consistsof Tachyon creators, top contributors • Series A ($7.5 million) from Andreessen Horowitz • Committed to Tachyon Open Source Project • www.tachyonnexus.com 3
- 4. Agenda • Tachyon Basics& New Features • Motivation • Building an interactive data service –Spark + Tachyon • Future Works 4
- 5. History of Tachyon •Started at UC Berkeley AMPLab – From summer 2012 • Open sourced – April 2013 (two and half years ago) – Apache License 2.0 – Latest Release: Version 0.8.2 (November 2015) 5
- 6. One of theFastest Growing Big Data Open Source Project > 170 Contributors (v0.8) 3x increment over the last year http://tachyon-project.org/community/ 6
- 7. > 50 Organizations Oneof the Fastest Growing Big Data Open Source Project 7
- 8. An Open Source Memory-Centric DistributedStorage System What is 8
- 9.
- 10. Memory-speed data sharing acrossjobs and frameworks Spark Job Spark mem Hadoop MR Job YARN HDFS / Amazon S3 HDFS disk block 1 block 3 block 2 block 4 Tachyon in-memoryData Why Use Tachyon? Data survive in memory after computation crashes crash Off-heap storage, no GC 10
- 11. Enable Faster Innovationin Storage Layer 11
- 12. What if data sizeexceeds memory capacity? 12
- 13. Tiered Storage: TachyonManages More Than DRAM MEM SSD HDD Faster Higher Capacity 13
- 14. Configurable Storage Tiers MEMonly MEM + HDD SSD only 14
- 15. Pluggable Data ManagementPolicy Evict stale data to lower tier Promote hot data to upper tier 15
- 16. Pin Data inMemory 16
- 17.
- 18. Unified Namespace AcrossUnder Storage Systems 18
- 19. More Features • RemoteWrite Support • Easy deployment with Mesos and YARN • Initial Security Support • One Command Cluster Deployment • Metrics Reporting for Clients, Workers, and Master 19
- 20. Rich Choice ofUnder Storage Supports 20
- 21. How Easy toUse Tachyon in scala> val file = sc.textFile(“hdfs://foo”) scala> val file = sc.textFile(“tachyon://foo”) 21
- 22. Use Case: aSAAS Company • Framework: Impala • Under Storage: S3 • Storage Media: MEM + SSD • 15x Performance Improvement 22
- 23. Use Case: aBiotechnology Company • Framework: Spark & MapReduce • Under Storage: GlusterFS • Storage Media: MEM and SSD 23
- 24. When Tachyon MeetsBaidu ~ 100 nodes in deployment, > 1 PB storage space 30X Acceleration of our Big Data Analytics Workload 24
- 25. Agenda • Tachyon Basics& New Features • Motivation • Building an interactive data service –Spark + Tachyon • Future Works 25
- 26. Background Logs Data Warehouse Online DataServices26 Hours or Days Hours
- 27. Frustrated data explorers •Example: – John is a PM and he needs to keep track of the top user actions for a new feature – Based on the top actions of the day, he will perform additional analysis – But John is very frustrated that each query takes tens of minutes to finish 27
- 28. A dedicated servicefor data exploring • Manages PBs of data • Most queries within one minute 28
- 29. User Scenario Web SiteClient Structured Logs Data Warehouse Data Marts Service Gate Query Engine select some_action, count(*) from event_table where event_day=‘20151123’ group by user_action Have first try Try another way select some_action, event_hour, count(*) from event_table where event_day=‘20151123’ group by user_action, event_hour Not as expected! See what happens in original log select * from event_log where event_day=‘20151123’ and event_hour=’01’ limit 10 29
- 30. Agenda • Tachyon Basics& New Features • Motivation • Building an interactive data service –Spark + Tachyon • Future Works 30
- 31. Choose Spark ascompute solution Data Warehouse BFS Service Gate Service Gate Hive Map Reduce 4X Improvement but not good enough! Compute Center Data Center 31
- 32. Baidu File System(BFS) Data Center Choose Tachyon as storage solution Spark Task Spark mem Spark Task Spark mem HDFS disk block 1 block 3 block 2 block 4 Tachyon in-memory block 1 block 3 block 4 Compute Center • Read from remote data center: ~ 100 ~ 150 seconds • Read from Tachyon cluster local node: 10 ~ 15 sec • Read from Tachyon machine local node: ~ 5 sec Tachyon Brings 30X Speed-up ! 32
- 33. Overall Performance 0 200 400 600 800 1000 1200 MR (sec)Spark (sec) Spark + Tachyon (sec) Setup: 1. Use MR to query 6 TB of data 2. Use Spark to query 6 TB of data 3. Use Spark + Tachyon to query 6 TB of data Results: 1. Spark + Tachyon achieves 50- fold speedup compared to MR 33
- 34. Architecture SparkContext Operation Manager View Manager Run Query Build Cache DataWarehouse Cache Scheduler Ask & Profile 34
- 35. Catalyst helps tobe ‘transparent’ lookupRelation CacheableRelation Tachyon HDFS Union HiveTableScan withUncachedPartitions HiveTableScan withCachedPartitions 35
- 36. Cache Policy • Prefetch –Fetch the views daily in advance when system is idle – The views fetched are based on the pattern of the past query history profiling, e.g. 3 months query logs • On Demand caches – Fetch the views at runtime when system is serving regular queries. – Using machine learning to generate policy file monthly for views/tables – When a query is accessing some views, and parts of views match our pre-generated policy, those views will be cached at that time. 36
- 37. Hot Query: CacheHit 37 Spark Tachyon HDFS Operation Manager Query UI View Manager Cache Meta
- 38. Cold Query: CacheMiss 38 Spark Tachyon HDFS Operation Manager Query UI View Manager Cache Meta
- 39. Daily Stats withCache • Daily – Table • Queries: 100 – 300 • Hit Rate: ~40% – Partition • Queries: 80K – 120K • Hit Rate: ~40 – 50% • Performance with Cache – avg 2 - 3 time faster than without Cache 39
- 40. Agenda • Tachyon Basics& New Features • Motivation • Building an interactive data service –Spark + Tachyon • Future Works 40
- 41. Improve Caching System •As Extended Meta Service – Improve legacy schema/input-format – Load block meta into cache layer – Index / Materialized View • Cost Based Caching/Optimizing – Better performance, hit rate & execution – Lower storage needs for cache layer 41
- 42. More User Scenario •If John is data scientist – Need a way to construct dataset conveniently – Usually have many tries with same dataset • An interactive system should help a lot – Spark is an ideal solution 42
- 43. Hardware assisted big datainfrastructure • Hardware – GPU – FPGA • Applications – Accelerate common SQL and ML operators – TableScan && InputFormat && Serde • Lower down the cost – 10 dollars for big data – 1 more dollar for interactive big data 43
- 44.
Editor's Notes
- #27 Logs -> Data Warehouse Transfer from online services to HDFS ETL, By Hive or MapReduce Take hours or even days Data Warehouse -> Data Mart Usually several steps Typically Hive Take hours on each step Data Mart -> Online Data Services End of the long journey Water drop from ocean
- #42 http:/// http://calcite.apache.org/
- #44 exploit