Downloaded 21 times
Uploaded byPhil Sarin
Queues: An Invisible Money Drain
Queues introduce delays and economic costs in product development. Common queues include features waiting to be designed/developed/tested/released and bugs waiting for triage/fixing/verification. Large queues increase cycle times, variability, and require management. They also delay feedback and hurt morale. The presenter advocates reducing queues by using swarming, generalizing work across teams rather than specializing, obliterating unnecessary queues, and reducing testing batch sizes from months to weeks or even days through continuous delivery. Case studies show how these techniques helped the presenter's company ship features more quickly while maintaining quality.
More Related Content
Similar to Queues: An Invisible Money Drain
Queues: An Invisible Money Drain
- 1. Queues: An Invisible MoneyDrain Phil Sarin VP Engineering, GameChanger Media @philsarin http://216ways.net
- 2. “It ain’t whatyou don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. — Mark Twain
- 3. Things I oncebelieved You shouldn’t release often because releases have high overhead. You need to track all of your bugs. Slack is bad. Your team should be busy. Specialists will always be more productive than generalists.
- 4. The Biggest Misunderstanding: Wethink queues are free.
- 5. What we’ll covertonight What a queue is Identifying the queues in your organization Tactics for handling queues Some war stories
- 6. Part 1: Queuesand why they matter Flickr: Jeff Turner
- 7. “Queues are theroot cause of the majority of economic waste in product development.” — Donald G. Reinertsen
- 8. Examples of queues FeatureQueues Waiting to get designed Waiting to get developed Waiting for testing Waiting to get released Waiting to get validated Bug Queues Waiting for triage Waiting to get fixed Waiting to get verified Waiting for release
- 9. What’s wrong withbig queues? Big queues increase cycle time and delay cost Big queues increase variability Big queues require management Big queues delay feedback Big queues hurt morale
- 10. The conservative team vs Theaggressive team
- 11.
- 12.
- 13.
- 14.
- 15.
- 16. 1 year 3 months vs 3months 3 months 3 months vs
- 17. Why small batchesare (usually) better Realize benefits sooner Faster feedback Higher motivation Less schedule slippage
- 18. Old School: Releasesare expensive so let’s batch up work for release. New School: Let’s drive down the cost of releases so that we can release more often.
- 19. Queue takeaways Queues introducedelay. Queues have economic and psychological costs. High capacity utilization drives up queue size. Small batches can reduce the cost of queues.
- 20. Part 2: Howwe apply this theory at GameChanger Flickr: Jeff Turner
- 22. How we’ve managedqueues 1. Swarm 2. Generalize queue handling 3. Obliterate queues 4. Reduce testing batch sizes
- 23.
- 24.
- 25. The stages ofswarming 1. Skepticism/suspension of disbelief 2. Guilt and busywork 3. Scope blows up 4. The “surge capacity effect”
- 26. Announcements Invite Tools Alerts Announcements InviteTools Alerts vs
- 27.
- 28. 2. Generalize Flickr: KalebFulgham
- 29. Multiple queues, 1 servereach Single queue, Single server Single queue, pool of servers
- 30. Multiple queues, 1 servereach Single queue, Single server Single queue, pool of servers Sustained huge queues are least likely
- 31. Pressures to specialize Aspecialty differentiates your business “I don’t want to touch that code!” Initial builder owns it forever. Early adopters become de-facto specialists. Some people want specialist careers.
- 32. Resisting specialist queues Outsourcenon-core specialties Despecialize Challenge someone Staff up
- 33. Generalist teams andmorale
- 34.
- 35. 4. Reduce testingbatch sizes
- 36. That time wemissed our deadline by four months
- 37. 2 months development 2weeks testing 2 months development 2 weeks testing 4 months bug-driven development
- 38. 2 weeks dev test 2 weeks dev test 2weeks dev test 2 weeks dev test 2 weeks dev test Can we reduce this batch size further? Our first batch size reduction
- 39. 2 weeks dev test 2 weeks dev test 2weeks dev test 2 weeks dev test 2 weeks dev test Can we reduce this batch size further?
- 40. 2 weeks dev+test T 2 weeks dev+test T 2weeks dev+test T 2 weeks dev+test T 2 weeks dev+test T 2 weeks dev+test T Continuous pre-release delivery?
- 42.
- 43.
- 44. How we fixedit More automated test coverage Instituted small batch manual testing process Hired more testers
- 46. Release Days 5.15 21 5.1414 5.13 26 5.12 21 5.11 23 5.10 4 5.9 66 5.8 7 5.6 21 It worked, mostly, eventually
- 47.
- 48. Things I oncebelieved You shouldn’t release often because releases have high overhead. You need to track all of your bugs. Slack is bad. Your team should be busy. Specialists will always be more productive than generalists.
- 49. Mindset shifts Being busyvs being productive Individual vs team Rituals vs principles
- 50. Takeaways Queues are realand expensive You should know where your queues are There are a bunch of ways to reduce the cost of queues
- 51. Further Reading The Principlesof Product Development Flow, Donald G. Reinertsen “Software Inventory” by Joel Spolsky: http://www.joelonsoftware.com/ items/2012/07/09.html “How we fixed more bugs by deleting our bug DB” and “Building around generalists” on my blog (216ways.net)