Uploaded byMike Burns
Three baseline metrics & what they can tell you about your team.
The document discusses the importance of metrics in evaluating team performance, distinguishing between raw measures and processed metrics that provide context. It outlines three key metrics: cycle time, throughput, and work item sizes, explaining their significance in understanding work processes. Additionally, it emphasizes the necessity of measuring and analyzing to implement effective changes in team management and performance.
More Related Content
Similar to Three baseline metrics & what they can tell you about your team.
![Devfest Harare 2025 Slides [GDG Harare] - Combined Slide Deck](https://cdn.slidesharecdn.com/ss_thumbnails/ilovepdfmerged12-251029161756-3b829161-thumbnail.jpg?width=640&height=640&fit=bounds)
Three baseline metrics & what they can tell you about your team.
- 1. Three baseline metrics andwhat they can tell you about your team @maddogmikeb maddogmikeb
- 2. Why did Istart thinking about metrics and using them help my teams?
- 3. What’s the differencebetween a metric and a measure?
- 4. Measure Raw data, unorganized factsthat need to be processed. The numbers we capture. “I am 178cm tall”
- 5. Metric Processed, organized, structuredor presented data in a given context so as to make it useful. Metrics are the information we can generate based on the measures. “I’ve lost 5kgs in the last 2 months”
- 6. Where did Istart… • Number of defects found • Total number of stories (or tasks or defects) completed • Number of lines of code • Number of workshops v. gamed To manipulate dishonestly for personal gain. To be messed up or screwed over http://www.urbandictionary.com/define.php?term=gamed#3866612 & http://www.thefreedictionary.com/gamed Gamed
- 7. Vanity metrics arethe numbers you want to publish on TechCrunch to make your competitors feel bad. Eric Ries Number of: • Downloads – “Over 1 million downloads!” • Registered users – “10 million registered users” • Tweets a day – “1 million tweets per day” • Daily/Monthly Active Users • Daily/Monthly sessions per user
- 8. Metrics are a mirror “Are these goingto affect my monthly one- on-one?” “Are you going to compare us to another team now?” “What about this other metric (usually a vanity metric) isn’t one that better to show the boss?”
- 9. To make asuit that fits right, The tailor must measure.
- 10.
- 12. Planned Value Actual Cost EarnedValue Return On Investment Cost Variance Cost Performance Index Cost Of Managing Processes Planned Hours Of Work Vs Actual Situation Overdue Project Tasks / Crossed Deadlines Schedule Variance Schedule Performance Index Missed Milestones Percentage Of Tasks Completed Resource Utilization Percentage Of Projects Completed On Time Percentage Of Cancelled Projects #AgileIsForEveryone http://agilemanifesto.org/
- 13. The twelve principlesof agile development include: • Customer satisfaction through early and continuous software delivery • Accommodate changing requirements throughout the development process • Frequent delivery of working software • Collaboration between the business stakeholders and developers throughout the project • Support, trust, and motivate the people involved • Enable face-to-face interactions • Working software is the primary measure of progress • Agile processes to support a consistent development pace • Attention to technical detail and design enhances agility • Simplicity • Self-organizing teams encourage great architectures, requirements, and designs • Regular reflections on how to become more effective http://agilemanifesto.org/
- 14. What metrics didI use and what did they tell me about my teams?
- 15. Metric 1: CycleTime From cut to fit
- 16. Cycle Time: Time takenfrom starting to finishing a piece of work.
- 17. Time taken to“do” the work – from being defined to being done.
- 23. The “Surfs Up” The“Bobsled” The “Flow” Cycle Time Patterns The “Heartbeat” The “Valley” The “Camel”
- 24. Metric 2: Throughput Thecuts to make the pant
- 25. Throughput: The number ofwork items completed within a period.
- 27. The “Stairway toHeaven” The “Devils Staircase” Throughput Patterns The “Yo-Yo” The “Goldilocks” The “Odd One Out” The “Out One Odd” The “Suburbs” The “Cityscape”
- 28. Metric 3: WorkItem Sizes The size of the suits
- 29. Work Item Sizes: Thenumeric representation of the estimated size of work items within a period.
- 30.
- 32. The “Mixed Bag” The“Night & Day” The “Liquorice Allsorts” Work Sizes Patterns The “Upsize” The “Stripped Socks” The “Easy Does It” The “Sliced Loaf”
- 33.
- 34. Tip – CombineThroughput & Work Item Sizes
- 35.
- 36. The “Heartbeat” The “Camel” The“Flow” Cycle Time Throughput The “Suburbs” The “Yo-Yo” The “Goldilocks” The “Sliced Loaf” The “Odd One Out” The “Night & Day” Work Sizes The “Upsize” The “Flow” The “Surfs Up”
- 37. Benefits as aCoach Know the signs Know the hooks Data is King Better coaching plans, better interactions An easy to use tool
- 38. What gets measured,gets done Tom Peters
- 39. @maddogmikeb www.linkedin.com/in/maddogmikeb/ “If you don’tmeasure something, you can’t change it” Mitt Romney If you don’t measure something and use it in a metric, you can’t know if the change you make is working
Editor's Notes
- #2 Welcome. I’m a part-time dev, a recovering project-manager, and full-time agilist but in my current role as Agile Coach at Tatts Group metrics are key to good planning and allowing me to coach multiple teams effectively. As a coach, I soon realized that the teams metrics are my metrics, and that I could use metrics to help drive continuous improvement. So they became as important to me as I knew they would be for my teams.
- #3 Metrics are important to every business, and therefore important to agile teams. Geoffrey Mika in his book, "Kaizen Event Implementation Manual" identified waste associated with working to the wrong metrics or no metrics. So, in order to reduce waste and continually improve we need metrics. Because, if you’re not measuring and using those measures in a metric, you can’t know if the change you make is working. Whether that change is adding a new button to a web page, or a new marketing campaign, or coaching an agile team through splitting a story to help reduce pushing unfinished work from iteration to iteration, also known as snow-ploughing – metrics help teams and coaches alike. I thought about jobs that need to measure. I thought of a tailor making a suit for someone. No customer is the same, no suit is the same.
- #4 First thing was to understand what’s the difference between a measure and a metric was?
- #5 Measure are the numbers – the raw data. By themselves they aren’t all that useful. They give a point in time or a snap-shot of what is happening.
- #6 When we convert the data from the measures into something informative we get a metric. A metric requires two or measures to be meaningful – to compare something to something. Teams are made up of people so there is always qualitative data required to produce real information. Capturing the ‘right’ metrics will lead to “informed” and proactive dialogue. Metrics must have the story behind them as well.
- #7 I started at the same place you probably did or would. The easiest metrics I could find, more importantly, the ‘simplest’ ones to capture. These ones… / Who has used one of these before? Show of hands. They sounded impressive - “We have 1 million lines of code” – wow, however, they weren’t useful. **** And worst of all they were easily gamed. So to game these we could, for example, “Number of defects found – easy just find more defects – so let’s not test as thoroughly while the code is being developed and only test when its in UAT – that’s sure to raise our defect count”. When you game metrics, you are gaming yourself, which is gaming your team. There needs to be a story behind the metrics so they are not used as a weapon.
- #8 These metrics are known as Vanity Metrics. Vanity metrics are the numbers which look good on paper but don't help you make decisions. Vanity metrics usually lead to bad behaviour of some kind – knowingly or not. Ever heard of a bot? I can get you 10 million registered users – none of them will be real but they will have an email address and be registered with you. Want more twitter follows – you can buy them! None of these necessarily lead to more value just more numbers, and numbers are ripe for gaming.
- #9 When I first introduce metrics to my teams – they started freaking out. They didn’t believe me, that metrics were going to be used for good not evil and that they would help take the guess work out of decision making. I found that you must make sure that the team feels safe and understands how the metrics will be used. Not as a stick but as a way to help them improve. Metrics can also encourage bad behaviour. I was once in a PM role where every month we needed to supply a report to the CEO. Every month we painstakingly calculated measures and progress traffic lights only to show to the Product Manager before the meeting and have them say “we can’t show those numbers, you need to change them – the traffic light can’t be red it should be green – next month we’ll be in a better position and they’ll forgotten this months numbers by then”. Needless to say, I didn’t stay in that role very long. But, it was then that I realised that metrics should be based on raw measures and be reproducible by anyone at anytime. They are a mirror – and throwing dirt on the mirror is not an option. Don’t like the information that the metric is showing - change what you are doing.
- #10 How does the tailor know the right length of the sleeve? They measure the arm of the person. How does the tailor know the number of suits per length of material, so that he can make the most of the material and get the best return on his investment? What is the suits per material metric? You measure number of suits per roll of material and calculate the ROI – you create the right metric to get the information. What information did I need? What metrics are right for my team? What metric will get me the information? What measures will get me the metric?
- #11 I already had some metrics. I was using a Kanban board. As things progress along – the time dimension – it showed how much work is in progress at each stage of the lifecycle. **** A time vs amount of work metric. **** As a column filled up I could see that there may be a problem with our WIP and could have a directed and purposeful conversation with the team. “Why is there so much work going on here?” “Is there a blocker I can help with?” “Do you have enough team members / gear / money / food / lollies / red-bull?” I could ask the right questions at the right time.
- #12 I also saw other commonly used tools were actually metrics. Burn-ups and burn-downs – A scope over time metric. They are great and powerful – but I wanted more!
- #13 Hang on. Isn’t the primary measure of progress working software? Isn’t that all that I should care about? Firstly, yes it is, but is only part of the story. **** And for those not in software development…. #AgileIsForEveryone.
- #14 Secondly, people work in processes, so I need something that I can use to help continuously improve both the people and the process. The metrics need to help us follow the Agile principles. The metrics needed to show the team GOOD behaviour. The agile principles describe attitude, mindsets and behaviours rather than mechanics – therefore metrics need to encourage the right attitude, mindsets and behaviours. The metrics are more than a report they are an improvement tool.
- #15 I needed measures that could be repeated, that were simplistic, that actually meant something – that is were not easily gamed, and could show that we were getting better – that showed a trend of behaviour. What I needed was a series of metrics that when looked at altogether they showed something meaningful. Having multiple metrics, or a multi-dimensional metric, means that gaming is pretty difficult, cause a increase here can have a detrimental impact over there. A multi-dimensional metric is a metric that takes into account more than 2 measures usually at least one qualitative measure as well. Just like the tailor, we measure not only the length of the arm but the circumference too.
- #16 Do a Google search for agile metrics and from the millions of results, you’ll probably find cycle-time pretty close to the top of the list. The tailor also uses cycle time. How long does it take to create a suit from the time they start cutting the material to the time it is fitted influences how often can they accept a new measuring-up without the customer waiting too long.
- #17 Time taken from starting to finishing a piece of work.
- #18 If we look at a typical Kanban board…. ….this is the time from item entering the work cycle to being delivered.
- #19 In the “simplest” version of a Kanban board, it’s the doing column. If we look at one iteration, for example, we can mark the date on the card when it moved into the doing column, **** then when it moves to the "done" column we can mark on it the total number of days it took.
- #20 At the end of the iteration we can calculate the average days it takes to complete a work item. We have measured that, in this iteration, on average it takes 6 days to start and finish a piece of work – ok but not really that useful. To make this a metric we need to track the average cycle time over more iterations. This is the simplest way to turn a measure into a metric - data over time. **** So, now we can table the results. Number of iterations and the average cycle time. We can also create the average over all the iterations. In this case, we get an average cycle time of 8.5 days. That means, on average, it takes 8 and a half days to complete a piece of work. They were working in a two-week, or 10 day cycle, so on average they were completing their work within the iteration.
- #21 We can then chart this data to come up with something like this.
- #22 We can then, with a few mathematical calculations, create… …a trend line.
- #23 This is an example of one of the charts from a team that I coach. They were pretty lumpy – sort of like a heart beat. And here – wow – they had a real blow out. When talking with the team it was due to half their team being sick – and the data showed it. What I learnt was that the data nearly always showed it, but it was important to have the conversation along with the metric. As the metric alone was only an indicator. Looking at both the average and trend is important too. The average of 8.5 was ok – it was inside the 10 days for the iteration, but the trend is showing that it is increasing, so if things don’t change it will probably be higher, meaning that they will start to not be able to finish work within the iteration. Generally, if the work is completed within half an iteration this will mean that the team is getting a good flow of work moving through the process. A trend up means that the cycle time is getting longer – it may be because of a large hump – like this – or it may be that the team is struggling. A trend down is generally better, but again, only an indicator not the full story. Who is using cycle time as a metric for their team? Keep your hand up if you have average and trend as well?
- #24 By coaching a number of teams I’ve seen some interesting patterns of cycle time. Knowing what they are can mean asking the right questions and helping the team see problems before they happen. I tend to have a time scale of weeks rather than iterations – this allows multiple teams to use the same charts without having to align iteration start and end dates. Also, it reduces the feedback loop to a week rather than iterations – allowing teams to make quicker decisions. For example, the first one is called… The heart beat This is where most of the teams I’ve worked with start out. They back-end everything to the end of the iteration usually creating a high test/UAT load. This usually leads to not finishing everything so it’s snow-ploughed to next iteration. Potential fixes this pattern are to split stories, commit to less, finish what you start, and most importantly work as a team. But, first – talk to the team – find the qualitative data behind the chart too. The Camel Unplanned leave / sickness Waiting for an external team ** Might be ok - look for trends – find out what the story behind the data is – but seeing the hump means that something has happened. The Valley Planned leave - holidays, xmas break etc. Something has happened ** Might be ok - look for trends – hidden work at the end of a release cycle The Surfs Up Team size changed / new team members therefore taking longer due to handovers, learning new things Might be paying down technical debt which is taking longer than expected Have given up on agile practices and are no longer creating small work increasing the cycle time ** Might actually be getting better – maybe they have identified that the process was missing something important - plateau soon required The Bobsled Team ramping down in size - end of a project / Hidden work increasing ** Might actually be getting better The Flow Work is flowing through at a nice constant rate / Are they gaming it? ** Toyota way says "No problem is a problem" Knowing that patters and knowing what good looks like allows to more readily see divergent behaviour. In this case “The Flow” is a good pattern.
- #25 The next metric I used was throughput.
- #26 The number of work items completed within a period. Throughput is based on real data of team’s capability to deliver. The tailor sees throughput as the number of suits completed in a time period. The more suits the more potential sales. But, there is a catch – a way that you could try and game this one – quality. If the suits aren’t of good quality then they won’t sell. Often teams use velocity as a way to capture how much work they are doing. Velocity is a great tool, but it does rely on a few things. The team have a shared understanding of story points. Because it is based on Story Points it represents load, not the amount of work completed. Trends up or down reflect the impact of factors such as story size, complexity, urgency, and skills within the team. To level up our throughput metric we can incorporate “little’s law” to help predict a target throughput. But that’s a topic for another talk.
- #27 This is a real example from one of my teams. They started off pretty strong, about 10 items completed. But then something happened, but it was recovered and you can see the spike here – probably did some snowploughing from the past iteration. But lately something has been happening and they aren’t doing as much. This is also shown in the trend line. Generally an upward trend is what you are looking for here – the hope is that as teams learn more and continuously improve that they can complete more work – or at least not complete less. You also don’t want really high stacks then low stacks, not too high and not too low. Of course it does depend on the team, the work, etc. but generally consistently sized stacks represent a good flow. Good flow means that we can maintain the right level of quality. Hands up if you are using a Throughput metric?
- #28 These are a few of the throughput patterns I have seen. The suburbs Low output / Small team? / Hidden work? / ** Understand the story behind the metric – is there a blocker stopping them from producing more? Or is it ok? The city scape Very high output / Tasks as stories / Large team? / ** Trend here is important also - Is it a sustainable pace? Is the trend heading up higher and higher leading to the inevitable crash? The Odd One Out What happened? It could mean they had a really good iteration, but what happened to their good fortune? Look for trends The Out One Odd Not as common – but usually some explanation – e.g. leave. / Look for trends The Yo-Yo No flow – makes it hard to predict Two week iterations but back-ending work? The Stairway to Heaven Increasing the amount of work - getting better? Or have they found a way to game it. The Devils Staircase Decreasing the amount of work - getting better? Or have they stopped recording work. The "Goldilocks" Gaming it - no problem is a problem
- #29 The final metric is work item sizes.
- #30 The numeric representation of the estimated size of work items within a period. The tailor can use a work item size metric too. Two large men walk in asking for new suits – the total number of suits that can be made that month or from that roll of material may go down. A young boy comes in looking for his first suit, maybe the number increased.
- #31 Typically agile teams use relative estimation to size work. Something like Fibonacci numbers. But t-shirt sizes or any other estimate will work just as good. Unfortunately, our #NoEstimates friends wont be able to use this one. Who estimates their work using relative sizing? Who is capturing that data in a metric?
- #32 I’ve found the best way to show this is through a stacked bar chart. With each block pattern representing the different sizes. And the bar representing the total number. Count = Height , Size = Pattern.
- #33 The Upsize Work you like fries with that? / Similar sized work every iteration Might be a good thing - consistency helps prediction The Stripped Socks One large and many small / Might be a good thing - consistency helps prediction – can the large be split so that we have more small – this will probably lead to better cycle times. The “Liquorice Allsorts” All sizes, random / Difficult to create flow and probably showing problems in other areas too. The Mixed Bag Not as random as the allsorts but has a mixture of sizes Can also be harder to create flow – and the larger items will probably affect cycle times. The Easy Does It Doing the easy stuff first - Saving the hard stuff to the end Have they stopped doing agile practices and just creating big work items cause its too hard to split the work. This is probably showing up in other areas too. The “Night & Day” Snow ploughing is probably happening, most likely the large items were much large than estimated causing no smaller items to be completed. Or maybe one or two big things were created, but then lots of defects to fix it – quality might be a problem also. The Sliced Loaf All of one type - too perfect?
- #34 The fitting sessions are where the tailor shows the progress – we can do the same through dashboards.
- #35 You can combine these two into one chart.
- #36 Dashboards are necessary to show to everyone how the team is progressing – keeps everyone honest. Doing our work in public. Allows for questioning and help to be given.
- #37 How do these metrics combine to create a multi-dimensional metric – one that is increasingly difficult to game? What does good look like? Understanding what different patterns can be indicating is great – but knowing that a pattern can mean asking a certain question to understand the story behind the hump or dip or increasing trend is where metrics become really powerful to a teams continuous improvement.
- #38 Metrics become the levers that as a coach I can pull. When teams are looking for answers to improve they are the first port of call. When teams start using them in their retros they can see the correlation between a change they have made and the data that is produced. Teams always slow down before they get better. **** But if you know the signs you’ll know the right hooks to latch on to and help teams more effectively, and allows you to more easily coach multiple teams. **** Data is king – the more data that you have the better your coaching plans are going to be and the better your interactions are going to be through data-driven decisions. **** Metrics are easy to implement, talk about, and use both as an improvement and coaching tool.
- #39 Metrics have to be a conversation starter – not the final word. While creating the suit, a good tailor, needs to ask the right questions based on the measurements they’ve made. Agile metrics indicate something happening in the team – good or bad – they invite us to ask the right questions.
- #40 That brings me to the end, I ask, if you can take a moment to think about what was most useful for you from this – was it cycle time, throughput, work item sizes or how to work better with you teams. Remember, always tailor your metrics to suit you and if the suit fits wear it. Thank-you.