Parallel Tuning of Machine Learning Algorithms, Thesis Proposal
1 like1,287 views
This document proposes an intelligent system for automatically tuning machine learning algorithms in a scalable way. The system would take in a dataset and application type, then use genetic algorithms or stochastic kriging to select the optimal algorithm and configuration. It would parallelize the tuning work across an Hadoop cluster to efficiently scale to large datasets and workloads. The goal is to help cloud providers maximize revenue by keeping costs linear as income grows exponentially through improved scalability and automation of the tuning process.
Parallel Tuning of Machine Learning Algorithms, Thesis Proposal
- 1. Parallel auto-tuning of machine learning algorithms Gianmario Spacagna [email protected] 16 October 2012 AgilOne, Inc. (877) 769-3047 1091 N Shoreline Blvd. #250 (408) 404-0152 fax Mountain View, CA 94043 [email protected]
- 2. Motivation • Increase revenue of cloud service providers à Keep cost curve linear w.r.t. the expected exponential income growth. Income Cost • Technically achievable through Scalability: • Scalability in terms of resources à Distributed Parallel Computing (Hadoop). • Scalability in terms of multi-tenancy à Same system running for several customers. • Scalability in terms of auto-configuration à Avoiding manual tuning up operations. 2
- 3. Good Work Flow Good ML Good data Algorithm results! Tuning (Adjusting configuration) 3
- 4. General Tuning diagram Test Data Run algorithm with conf. X Are no Change results configuration good? X yes Tuned 4
- 5. Tuning of Machine Learning Algorithms • We need tuning when: • New algorithm or version is released. • We want to improve accuracy and/or performance. • New customer comes and the system must be customized for the new dataset and requirements. We need to make it smart, automatic and scalable! 5
- 6. Vision Request: • Data set Response: • Best algorithm • Application (prediction, Magic • Optimal clustering, classification…) • Algorithm Box configuration (ANN, LR, K-means…) • Metrics • Fitness metrics evaluation (Std. dev, Prob. of false true, clustering coeff., randomness…) • Goal constraints (x> 0.9 & 0.3<y<0.5) 6
- 7. Architecture Design Upper Applications API Initializer Controller Scheduler Executor Executor Executor ANN LR K-Means Evaluator Evaluator Evaluator Data Data Data Sampler Sampler Sampler Cloud Local Hadoop Service 7
- 8. Upper Applications API Tasks: Possible data format: • Interfaces the communication • JSON between the system and the • STDIN/OUT upper applications layer. • Parse requests and results and generates the related output domain object. 8
- 9. Initializer Tasks: Possible implementations: • Generates the initial set of • Random points configuration. • Latin Hyper Cube • Dataset similarity 9
- 10. Controller Tasks: Possible implementations: • Compares and generates • Random search configurations. • Grid search • Decides the convergence of the tuning. • Stochastic Kriging • Genetic Algorithms • Adapt the data sampling request. 10
- 11. Scheduler Tasks: Possible implementations: • Checks if the requests are • First available covered by the available services. • Oldest idle • Schedules and parallelizes requests executions. • Load balanced • Serialized (single node) • Optimizes resources. • Collects evaluated results. 11
- 12. Executor Tasks: Possible implementations: • Executes the providing algorithm • Local execution with the specified configuration. • Hadoop cluster • Cloud service Sub components: • Evaluator: Evaluates results standing to the specified fitness metrics. • Data Sampler: Down and Up sampling of data. 12
- 13. Tuning diagram Test Data Test execution Test control Scheduler, Run algorithm Executor with conf. X Initializer, Controller Are no Change results configuration good? X yes Tuned 13
- 14. SUNS: Simple, Unclever and Not Scalable STDIN/OUT Random Points Random Search – Grid Search Serialized Executor K-Means Evaluator Local 14
- 15. SNS: Smart but Not Scalable STDIN/OUT or JSON Latin Hyper Cube Genetic Algorithm / Stochastic Kriging Serialized Executor K-Means Evaluator Local 15
- 16. VSNS: Very Smart but Not Scalable STDIN/OUT or JSON Dataset Similarity Genetic Algorithm / Stochastic Kriging Serialized Executor K-Means Evaluator Local 16
- 17. VSS: Very Smart and Scalable STDIN/OUT or JSON Dataset Similarity Genetic Algorithm or Stochastic Kriging First Available Executor K-Means Evaluator Hadoop 17
- 18. VSVSO: Very Smart, Very Scalable and Optimized STDIN/OUT or JSON Dataset Similarity Genetic Algorithm or Stochastic Kriging Load Balanced Executor K-Means Data Evaluator Sampler Hadoop 18
- 19. Thesis It is possible to build an intelligent system based on Genetic Algorithm/Stochastic Kriging that automatically selects and tunes machine learning algorithms, such as K-Means and LR, parallelizing the work on an Hadoop cluster to scale in a cost-efficient manner. 19
- 20. Project Plan Order of priorities: 1. Design the entire application in Scala in a testable and expandable way. 2. Implement the Genetic Algorithm or the Stochastic Kriging controller. 3. Implement the Latin Hyper Cube initializer. 4. Test with local instance algorithms (K-Means and/or LR). 5. Develop and test at least one algorithm in MapReduce fashion using Hadoop. 6. Test with real AgilOne cluster of servers. 7. Implement the Dataset Similarity initializer. 8. Implement the Dataset Sampler. 20
- 21. Questions, feedbacks, suggestions? 21
- 22. Thank you! 22