CONTINUOUS TESTINGFrom Vanity to Value: Redefining Agile Metrics

Business agility isn’t easy. And for that matter, anyone who tells you that it is has clearly never tried to implement lean-agile practices at the enterprise level. We need to measure metrics that give true business value in an Agile environment.
Bryn Saunders Bryn SaundersNovember 18, 20194 min

Business agility isn’t easy. And for that matter, anyone who tells you that it is has clearly never tried to implement lean-agile practices at the enterprise level. The reality is that organizations seeking to scale agile will have to continually work on all aspects of their culture – that includes not only development organizations, but also business, finance, and the PMO.

What Metrics Should We Measure In An Agile Business Culture?

Building an agile culture that spans the enterprise means rethinking traditional metrics. Simply put, the way things have always been done will no longer cut it. Metrics also need to be redefined and reimagined to fit the new culture of the business (and iterated on, for that matter). Otherwise, how can you truly gauge the impact your efforts have on both the customer and the business as a whole?

Sadly, traditional KPIs breed bad behavior. Previously, teams have been measured on metrics including how many lines of code they’ve produced, total number of bugs/defects that have been addressed, and the ability to hit a strict project deadline. While many of these metrics have been phased out during the shift from Waterfall to Agile practices, vanity metrics like this still exist at the team, program, and portfolio level, and even worse – can cost you money, time, and resources. 

Take velocity as one example. Across many teams, velocity is still a primary metric that’s tracked. Now, within the context of a single, self-organized, and self-managed team, velocity still has its place in terms of tracking and estimating the number of work items that can be delivered during a sprint/iteration. But outside of the team-level, in a larger business context, it has no place. And here’s why.

Just because a team is closing points at an increasingly high rate, doesn’t mean that they’re delivering business value. In fact, what we’ve seen here at Rally Software is that many teams use other agile tools to hide or alter their work in order for it to show well to others. That’s not to say that velocity has no place in the world of agile metrics – it just has a tendency to be abused, and often results in negative team behaviors when measured in isolation.

The same concept applies to individuals or teams that are measured on items like the number of unit tests written. While this may sound like a good idea in the realm of test driven development, what it really leads to is developers writing tests with no regard to quality. This behavior is often described as the Hawthorne Effect, where individuals modify their behavior based on the fact that they’re being observed.

So instead of measuring the following types of metrics in isolation:

  • Velocity
  • Lines of code written
  • Defects resolved per person
  • Number of tests written

We need to measure metrics that give true business value in an Agile environment.

Value Metrics for Agile Business Success

Agile businesses should make the shift to measuring these types of value-based metrics: 

  • Value Delivered: Rather than measuring team velocity, which not only varies per team, but also promotes negative team behaviors when used comparatively, start by measuring the total number of accepted features, or revenue received per feature. That way, you’re measuring the impact your teams have on customers, rather than their performance.


  • Retention/Churn: This metric applies to both customers and employees. The health of your business depends on both. Once again, you’re shifting the focus from performance/productivity to that of customer/employee impact.


  • NPS: A tried-and-true metric, your Net Promoter Score still holds weight. It not only evaluates customer impact and provides a great feedback mechanism, but also helps provide a leading indicator for customer churn. 


The idea is to “level-up” the metrics that matter most. In the world of Agile, that not only means delivering working software, but also delivering value to customers. The key is to make sure you’re not measuring in isolation – velocity has its merits, but not as an individual KPI. Measure the team, not the individual. Better yet – measure the business, not the team. Outcomes over output. Collaboration over competition.

And don’t forget happiness. That’s the true value metric worth measuring.

Looking for more agile best practices? Join us at our upcoming webinar, Collaboration Over Competition in an Agile Environment.

Bryn Saunders

Bryn Saunders

Bryn Saunders is a product marketer for Rally Software. With more than 7 years experience in the technology industry, he has a passion for great storytelling, great products, and great customers.