Metrics are one of those topics where everyone agrees they matter and almost no one agrees on which ones to use. Engineering teams end up with dashboards full of numbers reviewed in sprint ceremonies and then ignored, or with a single velocity number that becomes a proxy for everything and measures nothing useful. The purpose of a metric is to answer a question you care about, not to have a number. Think of them like the instruments on a car dashboard: a compass tells you direction, a speedometer tells you how fast, a fuel gauge tells you how much runway you have left. You need all three to drive well, and none of them by itself is the whole picture.

For an engineering team, the two questions I want to be able to answer at any point are: are we a high-performing team that is shipping customer value with a low failure rate, and how are our customers actually experiencing the service? Everything else is secondary.

What makes a metric good

A good metric is:

  • Relevant to the questions you are trying to answer, not just easy to collect.
  • Actionable in the sense that if it moved in the wrong direction, you would know what to do about it. If the answer is no, it is probably measuring the wrong thing.
  • Quantifiable and comparable over time, so you can tell whether things are getting better or worse.
  • Measuring outcomes, not output. Output is what the team produced. Outcomes are the effect that production had on customers and the business. This is the criterion teams most often get wrong.

The traps to avoid

  • Vanity metrics look good but tell you nothing actionable. Story point velocity is a classic example. If the team completed 1,385 points in a quarter, up from 991 the quarter before, that looks like progress. But it bundles customer-facing features, bug fixes, and internal refactors into a single number. You cannot tell from it whether customers got more value or whether the team just worked harder.
  • Watermelon metrics are green on the outside and red on the inside. A team reports 97% support SLA compliance, which sounds healthy until you look at which bugs are actually being fixed on time. If the P1s are handled but the P3s are quietly accumulating into a backlog that degrades the product experience, the headline number is misleading you.
  • Too many metrics is its own failure mode. When everything is measured, nothing is prioritized. Teams spend time maintaining dashboards instead of acting on them. Pick a small number that between them answer your most important questions, and be ruthless about adding new ones.
  • Metrics not aligned to business outcomes create perverse incentives. If engineers are measured on tickets closed, they will close tickets. If they are measured on features adopted by customers, they will think harder about whether what they are building actually solves the problem.

A worked example: change failure rate

Change failure rate is the percentage of deployments that cause a failure in production, measured by rollbacks, patches, and hotfixes. It satisfies almost every criterion for a good metric: it is quantifiable, comparable week over week, directly tied to customer experience, and highly actionable.

When we first started tracking it on one of my teams, the number came back at 27%. The benchmark for a healthy engineering team is under 1%. That gap was uncomfortable, but discomfort is exactly what a good metric should produce when something is wrong. We could look week by week at which deployments failed, trace them back to specific changes, and course-correct on code review practices, test coverage, and deployment process. Within two quarters we were under 1%, and we would not have known where to start without that number in front of us.

An example metric set

Here is how I structured engineering metrics for a 25-30 person engineering org, but the broad principles apply to larger and smaller orgs. Essentially, there were two questions that I was trying to answer: (a) Are we a high-performing engineering org? (b) How do customers experience our product?

Are we a high-performing engineering org?

A high-performing engineering org consistently ships value to customers over time, with a low change failure rate. The metrics that answer this question are:

  • Value shipped (story points of completed customer-facing feature work). The benchmark we used was 1 point per developer per day, giving a quarterly target of roughly 1,040 points for a 16-person team working across a 65-day quarter. If this number drops, the next question is why, which leads to the diagnostic metrics below.
  • Value vs. KTLO ratio. KTLO ("keep the lights on") covers bug fixes, reliability work, onboarding, and tech debt. A team spending most of its time on KTLO is not shipping much customer value, and that ratio is worth tracking explicitly. We started with total story point velocity and then broke it into these two buckets from Q2 onwards, which made the number considerably more useful.
  • Planned vs. unplanned work. A high proportion of unplanned work is a signal that the team is being pulled off roadmap commitments by fires, and that the system is less stable than it should be.
  • Change failure rate. The share of deployments that result in an incident, rollback, or failure, calculated as (number of failed changes / total number of changes) * 100. Benchmark is under 1%. When we started tracking this we were at 27%, which was the clearest possible signal that something needed to change in how we were testing and reviewing code.

How do customers experience the product?

  • Bug impact score. Rather than counting raw bug numbers, this is a weighted sum by severity, so a P0 bug counts for much more than a P3. It is a proxy for the impact product quality has on customers, and more honest than a simple bug count that treats all issues the same. Our target was a weekly bug impact score below 10.
  • Support SLIs. Timeliness in fixing customer-reported bugs, by criticality. The targets we used: P0 bugs mitigated within 0 to 1 business days, P1 bugs within 3 business days. Without severity-based targets, teams naturally optimize for closing the easy tickets, which inflates overall SLA compliance while the high-severity issues sit.
  • Uptime. Measured as (total available time / total time) * 100, with availability defined as the ability to log in and have the application load within baseline latency. For a SaaS product with contractual uptime commitments, this is non-negotiable to track continuously rather than in periodic reports.

Leading vs. lagging indicators

Leading indicators are input-oriented and measure things that predict future outcomes, like deployment frequency or test coverage. Lagging indicators are output-oriented and measure results, like uptime or customer retention. You need both. Lagging indicators tell you how you did; leading indicators tell you where you are headed. A team that only tracks lagging metrics is always looking in the rearview mirror, finding out about problems only after they have already landed on customers.

Cadence

A metric reviewed quarterly is not a management tool, it is a retrospective artifact. For metrics to actually change behavior, they need to be reviewed at a cadence short enough to act on. Change failure rate is worth looking at weekly. Uptime warrants continuous monitoring. Velocity makes sense at the sprint level. The right cadence depends on how quickly the underlying thing can change, but if you cannot do anything with a metric between reviews, you are reviewing it too infrequently.

The practical approach: start by asking what questions you most need to answer, then work backwards to the metrics that would answer them. Three to five metrics that between them address shipping velocity, quality, and reliability is usually enough for an engineering team. Add a metric only when you can articulate what decision it will help you make, and be willing to retire the ones that no longer drive action. The goal is not a comprehensive picture of everything the team is doing. It is enough signal to run the team well.