Erlang as a cloud citizen, a fractal approach to throughput
0 likes1,417 views
The document discusses the challenges and lessons learned from a cloud-hosted gaming project, emphasizing the need for efficient scaling of infrastructure. It highlights the transition to using Erlang and AWS for better handling of computing capacity and throughput. Key points include the effectiveness of stateless application servers and the importance of understanding data state in cloud environments.
![He can tell you!
• Rich Hickey
• Clojure author
“...If not in Erlang which I
think has a complete story
for how they do state”[1]
[1] Value Identity State @0.27
http://goo.gl/Zdjv0
http://www.flickr.com/photos/ghoseb/5120173586](https://image.slidesharecdn.com/efsf2012-120330141730-phpapp01/85/Erlang-as-a-cloud-citizen-a-fractal-approach-to-throughput-35-320.jpg)
Erlang as a cloud citizen, a fractal approach to throughput
- 1. Erlang as a cloud citizen Paolo Negri @hungryblank
- 2. 1 day at • 10 millions players • 2 billions game server requests (http) • 20 devops people
- 3. Cloud “A cloud is made of billows upon billows upon billows that look like clouds. As you come closer to a cloud you don't get something smooth, but irregularities at a smaller scale.” Benoît B. Mandelbrot http://www.flickr.com/photos/nirak/644336486
- 4. AWS Cloud
- 5. This talk will answer • Why building a system targeting the cloud? • How many EC2 instances do you need to respond to 0.25 billion uncacheable game reqs/day?
- 6. 15 months ago... http://www.flickr.com/photos/wheatfields/515068701
- 7. 1st cloud hosted project, lessons learned 1 Pushing live the 60th application server? not different from adding the 6th! push a button
- 8. 1st cloud hosted project, lessons learned 2 local network/local disk are low performance general purpose tools (nothing to do with ad hoc data center solutions)
- 9. 1st cloud hosted project, lessons learned 3 Complete automation is cool Ease of adding hosts/automation can lead to bloated infrastructure
- 10. 1st cloud hosted project, points of pain • A lot of inefficient app servers (as per tweets) • Much effort to scale up/maintain databases (mySQL & Redis) • Expensive, not crazy expensive, but expensive
- 11. Why trying again? http://www.flickr.com/photos/kky/704056791/
- 12. Uncertainity • will we reach100K or 3 millions users? • 3 million users in 2 weeks or 12 months? • cheat tool released 1h ago => single game call up 5000% • weekly releases, new feature performance impact?
- 13. the cloud • standard units (instances) of computing capacity • a network connecting all instances • an API to provision/dismiss instances
- 14. the cloud Sounds like a good framework to compose computing capacity Why didn’t work as a framework to compose throughput?
- 15. Scaling in the cloud the recipe CLOUD: composable units of computing capacity + DEVELOPER: turn a unit of computing capacity in a unit of throughput = composable throughput, a plan for scaling BIG
- 16. turn a unit of computing capacity in a unit of http://www.flickr.com/photos/pasukaru76 throughput
- 17. Unit of throughput, where? App server Database
- 18. Unit of throughput, joke? App server App server App server App server Cache Database Database
- 19. Unit of throughput? App server Database No unit!
- 20. Unit of throughput? App server Database Tightly coupled throughput?
- 21. Unit of throughput? App server Database Monolithic throughput?
- 23. Monolithic throughput • likes monolithic infrastructure! • scales well vertically • wants screaming fast stack (network, disks...) • any performance glitch impacts the whole system
- 24. Tightly coupled throughput + loosely coupled hardware (like cloud) = frustration
- 25. Who leads the tightly coupled dance? App server Database
- 26. Who leads the tightly coupled dance? App server Database The stateless application server!
- 27. Stateless application servers guarantee one thing... which?
- 28. Data is never where you need it
- 30. If you can feed them data fast enough... they’ll choke on garbage collection
- 31. We measure memcache HIT / MISS why app servers need to be 100% MISS?
- 32. Where’s the best knowledge about hot/ cold data?
- 33. Even the reverse makes more sense Database 1. pick your data up 2. go in the stateless App server app server
- 34. What Went Wrong?
- 35. He can tell you! • Rich Hickey • Clojure author “...If not in Erlang which I think has a complete story for how they do state”[1] [1] Value Identity State @0.27 http://goo.gl/Zdjv0 http://www.flickr.com/photos/ghoseb/5120173586
- 36. Most languages and runtimes don’t have a safe solution for concurrent, long lived state Erlang stands out as an exception in this panorama
- 37. Erlang... Processes are the primary means to structure an Erlang application. wikipedia
- 38. Erlang + OTP Generic Server Behaviour A generic server process (gen_server) implemented using this module... otp documentation
- 39. Erlang + OTP Generic Server Behaviour handle_call(_Request, _From, State) -> {reply, ignored, State}. handle_cast(_Msg, State) -> {noreply, State}. handle_info(_Info, State) -> {noreply, State}. terminate(_Reason, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}.
- 40. gen_server • An erlang process • With LOCAL state • responding to requests from clients A unit of throughput!
- 41. Composing throughput with erlang gen_server + EC2 instance
- 42. Composing throughput with erlang EC2 instance 1 EC2 instance + 1 erlang VM = N kilo gen_servers (N kilo units of throughput)
- 43. Composing throughput with erlang EC2 instance 1 EC2 instance + 1 erlang VM = N kilo gen_servers (N kilo units of throughput)
- 44. Composing throughput with erlang EC2 instance 1 EC2 instance + 1 erlang VM = N kilo gen_servers (N kilo units of throughput)
- 45. Composing throughput with erlang EC2 instance 1 EC2 instance + 1 erlang VM = N kilo gen_servers (N kilo units of throughput)
- 46. Composing throughput with erlang EC2 instance 1 EC2 instance + 1 erlang VM = N kilo gen_servers (N kilo units of throughput)
- 47. Scale by adding units instances
- 48. Scale up adding units instances
- 50. Throughput complexity VS. • Losely coupled peers • Tightly coupled roles • Independent throughput • Dependent throughput
- 51. Where does the state come from? Start Database gen_server Stop
- 52. Now with database AWS S3
- 53. Database scalability DB is (almost) never on latency critical path AWS S3 No need for low latency DB
- 54. Database scalability Throughput required is low AWS S3 We can approximate S3 capacity as infinite
- 55. Database scalability Ubiquitous and uniform from application servers point of view. AWS S3
- 56. Remember? AWS S3
- 57. How it actually works Data from the S3 is uniformly available to any ec2 instance
- 58. And as you zoom in...
- 59. And zoom in...
- 60. And zoom in you see... EC2 instance
- 61. Always the same kind of structure A fractal approach to throughput
- 62. Fractal “A cauliflower shows how an object can be made of many parts, each of which is like a whole, but smaller.” Benoît B. Mandelbrot http://www.flickr.com/photos/paulobrabo/3588387063
- 63. Homework The exact same solution might not work for you... but look for that unit of throughput
- 64. Answer time You need XXXX Smallish instances to serve 0.25 billions uncacheable reqs/day
- 65. Answer time You need 0XXX Smallish instances to serve 0.25 billions uncacheable reqs/day
- 66. Answer time You need 00XX Smallish instances to serve 0.25 billions uncacheable reqs/day
- 67. Answer time You need 0012 Smallish instances to serve 0.25 billions uncacheable reqs/day
- 68. What’s 1200 ?