In almost all managerial interviews, I was asked about team metrics: How do I know the team is working well? I usually didn’t answer cheerfully. So, questions about which metrics are less harmful to examine and which should not be used as targets often occupy my mind.
Naturally, there will be the classic Goodhart’s law here, ‘When a measure becomes a goal, it ceases to be a good measure.’ And Deming’s quote, simultaneously, ‘You can’t manage what you can’t measure.’ And, oops, he didn’t say it; he meant something else. In a complex system, if something can be measured, we eventually look only at what is being measured and change it.
However, there are some internal teams that may want ambitious numbers about what they’ll achieve in Q2. The teams are about process improvement: introducing research where there wasn’t any, avoiding launching what was bad research into development, and so on.
Maybe a couple of principles would be good to start with: 1) let’s make things as simple as possible, 2) let’s make the feedback loop from information from the customer to their feedback on that feature as short as possible, or similarly the product hypothesis testing loop. Principles are not like North Star metrics for product development. They are principles based on which we choose whether we do something in favor of something or not.
The first is more a matter of competence and trying not to overengineer the system (we developers sometimes like to do that). The second can be hard to measure. How do you track the time from insight from cast dev to implementation? In general, if there is no way, it’s okay. Keep this principle in mind to understand that even if we can optimise something locally, globally, it may become no better or even worse (this is from the Theory of Constraints).
From Lean, we can borrow Cycle Time metrics for all kinds of different stages. By measuring how tickets travel from the moment of creation to the end of development, you can see Development Cycle Time. That is, how long a feature was developed before production. Surprise-surprise, this Development Cycle Time is usually the fattest.
This means that we need to optimize it for development. It also means that we need to put into development the best possible stories in terms of profit or Cost of Delay (how much money we will lose if we don’t make the feature). The metric is the number of development hours saved not implementing not best stories, and the profit (or loss reduction) gained with the new chips. We can retrospectively calculate: for example, sit down with a development lead and estimate at the top level how much it would have cost to make unusable features and how many hours of support and cutting them out would have been required. That is, in the end, how much money you would have lost minus a small possible profit.
But the problem is that we can’t set a goal of how much money you want to save the company or how long the feedback loop will be because we build the system, but we don’t directly influence what the sales, analysts, or development teams do. However, painting this picture for executives could help build trust.
And since I’m an expert at recommending good books based on the recommendations of good people that I haven’t read myself, I want to read a book that discusses this in more detail. Lean Enterprise: Jez Humble, Joanne Molesky, Barry O’Reilly https://www.goodreads.com/book/show/18167218-lean-enterprise
Comments & email subscription
wholesystems.substack.com
I’m Alex, tech lead in Prestatech with experience from startups to middle-sized and big-tech companies. Loves to make effective and friendly team processes. The registry of talks and articles I gave is on the Published page.
You can freely drop a message to [email protected] or linkedin.com/in/aptakhin to greet and ask questions about any topic.