How do you Measure the Effectiveness of Agile Approaches?

I meet with a lot of organizations who want to be Agile. When I ask why, most will respond with mechanical and tactical reasons:

  • Improve On Time Delivery
  • Align Business and Technology
  • Flexibility to Respond to Change 

Are these the right things to measure?  If you achieve this, will you create a competitive advantage, or even parity?  Do these represent business agility?  

Lean thinking would say to evaluate your value stream.  That is, look at the activities and the timeline from when the customer places an order to when the customer receives the goods and pays. This is called the order to cash cycle. Lean organizations work ruthlessly to shorten this cycle. 

We could apply this thinking to the work of developing technology solutions in organizations. The customer order is the need or request from the external customer, or an internal business stakeholder. And the satisfaction of that need is when the technology team delivers the solution and it is tested and accepted by the requester. 

Based on discussions with organizations, I don't think many give much thought to the actual steps in the process or the timeline from request to satisfaction. They often think about "on time" delivery and look at Scrum or other agile methods to speed the delivery part of the cycle.  But they often ignore the wasted effort and time associated with the processes they use to identify, prioritize, and approve technology. And they rarely have any idea how long it takes to go from idea or request to delivered solution. 

We can use lean tools like value stream mapping to visualize and analyze our order to cash cycle. We recently mapped the value stream for a client and found that requests were typically satisfied in 14 to 26 months. 

The picture at the top of this post is a generalized example of what most organizations probably follow.  In that cycle you will find that:

  • 50% of the cycle is just spent gathering and analyzing these requests, and meeting to discuss and approve them. This includes the annual budgeting process which consumes lots of time and energy and produces zero value in the eyes of the customer. 
  • The next 25% of the time is spent on initiation activities. This includes identifying deliverables, developing resource plans, and creating schedules. There is likely also another approval step. Like the previous step, these activities consume time and energy and produce no value in the eyes of the customer. 
  • The last 25% of the time is when a team is given the work, they develop and test the solution, and they work with the requester to test and get acceptance. These activities actually produce value in the eyes of the customer. 

Scrum and other agile techniques can help us to improve that last 25%.   But it sort of misses the point. We can deliver fast and efficiently for the last 3-4 months but we are still working on items that were identified 14-15 months earlier and working their way through slow and ineffective review processes. 

Forget on time success measures.  Measuring "on time" delivery misses the point. You can deliver on time, based on your project schedule, but that is still 14 to 26 months after the need was identified.  

Instead, start measuring cycle time, that is, the time from when an item is requested until it is delivered. True business agility is meeting business needs, as quickly as possible. 

 

By Anthony Mersino | Tuesday, March 28, 2017

 

About Anthony Mersino

Anthony is passionate about helping technology teams THRIVE and organizations TRANSFORM.  He loves partnering with organizations to help teams with Agile thinking and the Scrum Framework.  He teaches Agile and Scrum as well as the cultural elements that are necessary for an organization to gain true business agility. Anthony has  authored numerous articles and two books: Agile Project Management, and Emotional Intelligence for Project Managers.

 

RELATED POSTS

Agile Teams - Deciding What to Measure

We Are Using Agile - Why Are Things Getting Worse?

Improve Team Productivity - Stop Measuring Hours Worked

Comments

Hello, I read all of them, very good. Quick question in regards to this last article. How do you improve the "cycle time". Yes, we can measure it but it seems 75% of the time is outside scrum, how you speed that process? We can chat more later in the week if you have some time but this is a very timely question at Northern Trust. Thank you Anthony, and hope you had a good weekend. Carlos