More Related Content
What's hot (20)
Similar to Clustered PHP - DC PHP 2009 (20)
Recently uploaded (20)
Clustered PHP - DC PHP 2009
- 1. Clustered PHPDC PHP 2009by Marcel Esser – [email protected] Developer & Systems Analyst, CROSCON
- 2. what is clustering?“A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer.”- Wikipedia
- 3. what is clustering?“A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer.”- Wikipedia…in other words, it’s really hard.
- 4. what is clustering not?Clustering is not high availability.… but you often achieve high availability with clustering.
- 5. what is clustering not?Clustering is not high availability.… but you often achieve high availability with clustering.Clustering is not high volume.… but you often achieve high volume with clustering.
- 6. what is clustering not?Clustering is not high availability.… but you often achieve high availability with clustering.Clustering is not high volume.… but you often achieve high volume with clustering.We want both.
- 7. also…clustering is really,really hard
- 8. some do it well
- 9. some … ehwell, nevermind.
- 10. why cluster?service more users
- 11. why cluster?service more usersservice users faster
- 12. why cluster?service more usersservice users fasterincrease reliability
- 13. why cluster?service more usersservice users fasterincrease reliabilityget rich
- 15. 1 server - 100 lolcats per second
- 16. 2 servers - 200 lolcats per second
- 17. 3 servers - 300 lolcats per second
- 18. objectiveslinear capacity increaselinear cost increase
- 19. 100 lolcats - $100
- 20. 200 lolcats - $200
- 21. 300 lolcats - $300
- 22. objectiveslinear capacity increaselinear cost increaseexponential reliability increase
- 23. common topics in clustering php Load Balancing
- 24. Database Scaling
- 26. Backups
- 27. Data Caches
- 30. Debugging
- 31. Background Servicesload balancingLoadServersClientBalancer
- 32. load balancingYour load balancer may or may not… remove bad nodes from the pool
- 33. balance by performance
- 34. balance by weight
- 35. route by geolocation
- 36. support sticky sessions
- 37. have 1 million other featuresload balancing toolssome among thousands… DNS servers
- 38. Big IP
- 39. Perlbal
- 40. nginx
- 41. Varnishdatabase scalingcommon things you can do:partitioningreplicationsharding
- 42. database partitioning every user is assigned to a database server
- 43. users don’t share data between each other (between servers)
- 44. when you need more capacity, add another database server
- 45. works for some apps, doesn’t work for othersimplementation example: invoice and timesheet management app
- 46. database replication (mysql)master - mastermaster - slavemaster - many slave
- 47. database replication (mysql)master - masterserver 1 replicates (as master) to server 2 (acting as slave)server 2 replicates (as master) to server 1 (acting as slave) works well to a point
- 48. complete nightmare when replication gets desynchronized
- 49. doesn’t actually improve write performance
- 50. good for basic high availabilitydatabase replication (mysql)master - slaveserver 1 replicates (as master) to server 2 (acting as slave) good first step
- 51. makes you re-write your application to consider slave queries
- 52. doesn’t increase write performance
- 53. de-synchronization is relatively painless
- 54. replication lagdatabase replication (mysql)master – many slaveserver 1 replicates (as master) to many servers (acting as slaves) thundering read performance
- 55. makes you re-write your application to consider slave queries
- 56. doesn’t increase write performance
- 57. de-synchronization is relatively painless
- 58. replication lagdatabase sharding data is split between multiple database servers
- 59. logical index is kept of what data is where (for example, a mathematical index or a lookup chart)
- 60. you have to grab, parse, and correlate data across servers
- 61. theoretically limitless scalability
- 62. complicatedimplementation example: digg, facebook, etc
- 63. replicated storagecommon things you can do:replicated file systemlookup tablesstorage serviceshuge NAS arrays
- 64. replicated file system (glusterfs) very affordable
- 65. various replication modes
- 66. nothing to keep track of in your app
- 67. easy to implement
- 68. can cause massive failures if poorly configuredlookup tables very affordable
- 69. limitless mode; entirely up to you
- 70. entirely dependent on your application logic
- 71. can cause massive failures if poorly implementedstorage services very expensive
- 72. theoretically limit-less capacity
- 73. easy to use
- 74. data must be pulled back first if used locally
- 75. costs and bandwidth usage can be mitigated (for example, by putting a proxy in front of it)large NAS arrays insanely expensive
- 78. bullet-proof fault tolerance… at a price
- 79. easy to use… for a pricebackupscommon methods:all-RAID (doesn’t work)snapshotscopying from slaves
- 80. backupsall-RAID doesn’t workWhy?RAID won’t keep your application fromdeleting data everywhere
- 81. snapshotsuse a mechanism to take a snapshot of the partitioni.e. LVM partitions works really well
- 82. easy if you do it from the beginning
- 83. requires some planning
- 84. should be used with RAID drivescopying from slavestake a slave out of rotation and copy from iti.e. MySQL databases works really well
- 85. easy if you do it from the beginning
- 86. requires some planning
- 87. backups can be out of datedata caches (memcached)PHP doesn’t have cross-request persistence, sosomeone added it: memcached in-memory
- 88. fast
- 89. scalable
- 90. proven
- 91. use itGot configuration data? Small, high-TTL data sets?Use APC.Large, high-TTL data sets? Use files.Mind the race condition.
- 92. replicated sessionspick your poison: memcache w. redundancy
- 93. database
- 94. shared file system (don’t actually do this)staging strategyIf you value your free time:DevStageLiveTest
- 95. staging strategy (dev) do use source control systems (subversion, etc)
- 96. do profile your to loop for obvious performance issues
- 97. do use phpdoc tags
- 98. do make your dev environment as similar to live as practical (i.e., don’t develop on Windows and run live on UNIX)
- 99. do document all your changes
- 100. do use TDD (test-driven development)staging strategy (test) do make test functionally identical to live, except for data
- 101. do create data fixtures that are representative of real-life data
- 102. do create functional tests for the user interface (Selenium)
- 103. do not push anything to stage that did not pass unit testsstaging strategy (stage) do make stage identical to a live node
- 104. do connect to the live database
- 105. do have test ‘users’ to perform destructive operations against
- 106. do have a mechanism to automate pushing stage to livestaging strategy (live) do not ever make changes by hand on live
- 107. do automate pushing updates
- 108. do take nodes out rotation when you push updates
- 109. do not allow ssh access to live except when really neededdebugging do use xdebug on dev, test, and stage
- 110. do prepare an automated action that can turn xdebug and profiling on/off on 1 of the live nodes. you can and will run into errors that only exist on live.
- 111. do write a test case to replicate the the bug first and then fix the bug, whenever possible
- 112. do first look if bugs are explainable by platform differences between development and production systems (i.e., don’t develop on Windows and deploy on UNIX)
- 113. do go to my talk at ZendCon in October, “It Works on Dev”background services do void launching background processes from the web app
- 114. PHP doesn’t have a native message queue, so (many) people wrote some. example, gearmand. do use a message queue.
- 115. do check for memory leaks in background tasks! many php libraries and also many php versions themselves still leak memory. try to write a loop in bash for a background task rather than in php. recycle the process often.
- 116. do plan your message format carefully
- 117. do persist important messagesquestions…anybody?
- 118. shameless plugCROSCONBespoke Application DevelopmentConsultingSecurityMarcel [email protected](202) 730-9728Personal? [email protected]