Agile development promises faster, more responsive development, that better aligns with the transformation, digital or otherwise, of organizations, as they face heightened, more competitive environments. Driven by market and technology changes, organizations are re-structuring themselves and their products and services to be more Agile and opportunistic to market changes. Agile should be suited to delivering this responsiveness when building and supplying technology capabilities to transforming organizations.
But, frequently, it isn’t.
By its nature, Agile can and should be a major enabler supporting these changes, but many organizations find it difficult to manage and extract this value, due to challenges in measuring productivity, quality, performance, and forecasting delivery. It’s hard to manage what can’t easily be measured.
Why is Agile hard to measure and harvest value from?
Waterfall and other goal- or milestone-focused development methodologies are structured with clear definitions of project phases (requirements gathering, sequential development, codified dev-test-QA-production flows) and milestones. Agile is more fluid. Agile measures productivity in terms of qualitative measurement, Story Points, that make cross-team productivity comparisons difficult. Agile value is based on individuals and interactions getting it done over process. It drives to create working code (moving quickly) while back-seating documentation. By working closely with the customer in the development process, it is more responsive and adaptable at the risk of increasing backlog, and expanding scope and requirements.
While Agile is well suited for delivering capabilities in a modern, competitive landscape, getting that value is hard, but not impossible. Typically, organizations struggle in three areas of Agile value:
- Predictable delivery of capabilities (reliable productivity)
- Quality
- Cost to performance, including with service providers
IDC Metri’s Agile Value Management product addresses these management challenges by assessing agile development efforts across team and product performance categories. Key team factors assessed are productivity, cost efficiency, delivery speed, and quality. For product quality, we evaluate robustness, efficiency, security, changeability, transferability, and technical debt. Future, it allows for benchmarking team performance against other teams within an organization and against market peers. These assessments filter up into management dashboards, to help identify trends, and engineering dashboards that drill into specific recommendations and remediation.
Predictable delivery of capabilities
With the Agile framework being structured around sprints (typically two-week cycles of refactoring, back-log attack, development, and just-in-time requirements gathering), Story Points for goals, and velocity (Story Point clearing) for progress, it’s hard for organizations to translate these measures to more traditional measures of progress. A lot of motion and momentum is demonstrated, but how this leads to predictable delivery of capabilities is elusive. To address this, IDC Metri uses a proven methodology for assessing progress and ensuring predictability—automated and enhanced function point analysis (FPA).
The IDC Metri Agile Value Management (AVM) solution assess a development team’s progress using both enhance and automated function point analysis. FPA delivers a concrete assessment of size delivered (value) and enables comparison of productivity across teams and benchmarking against industry peers. AVM provides management with the progress measurement dashboards for productivity and delivery speed. To measure and assess these, IDC Metri uses the functional output a team has delivered in a certain timeframe, leveraging the NESMA standard of functional size added + changed + deleted. In the case of automatic measurement of functional size, IDC Metri measures according to the ISO 19515 standards of Automated Function Points (AFP), and Enhancement Function Points (EFP). This data is presented in a fashion that allows managers to understand progress to goals and transparency to understand and predict capability delivery.
Quality
Ensuring predictable, or efficient development, only matters if the product being produced is of the quality (stability, security, efficiency, etc.) necessary to meet the business goals. For this reason, it is important to balance performance measures of the team with quality measures of the code. We don’t want measures and goals for performance to have the unintended consequence of driving down quality.
AVM provides source code analysis. This analysis provides ongoing assessment and trends in team quality over time and highlighting key areas of deficit. With this analysis, an Engineering dashboard is created showing the (critical) violations found, why these are violations, where they are found and how to solve them. The most critical ones are put on an action plan. This data is also presented in easily digestible fashion for managers responsible for managing and ensuring product quality.
The Engineering dashboards clearly identifies poor code and critical violations (CVEs) allowing the development team to better, and more rapidly, address quality issues. When adopting the guidance from the Engineering dashboard, overall development team practices improve. Quality and performance enhances, due to lower testing efforts, resulting from enhanced coding practices. Also, improving practices and identifying better practices reduces team stress and enables recently onboarded team members to become more rapidly productive.
Cost to performance
Sourced Agile development projects are typically time and materials (T&M), which shifts budget risk from the sourcing vendor to the buyer. Previously, development projects were typically fixed prices where risk (especially financial) was weighted towards the sourcing vendor. Similarly, even with internal projects, budgeting and cost were more predictable, due to the structure and predictable nature of methodologies like Waterfall.
AVM, by putting measurable, traceable and consistent metrics around development, helps make cost management and cost efficiency easier and transparent. Also, by providing benchmarking within an organization and against peers, a client has the context to understand the competitive meaning of these assessments (i.e., is my team underperforming in my industry in the cost/performance ratio for development?). Further, by assessing sourcing vendor current performance versus cost, goals can be set and measured consistently, over time, for assessment. AVM, with its combination of market benchmarking for services and concrete performance metrics, benchmarks the sourcing vendor performance against market peers. This enables buyers to determine whether the service capabilities they procured are delivered competitively to other vendors in the market. Furthermore, it gives leverage to the buyer in ensuring that a T&M development contract is performing at a minimum to market peers, i.e., that the buyer is not over-paying for the quality and productivity of the development they receive.
A client example illustrates this. The client company nearshored application development and maintenance. They were concerned they were paying more than the value they received. IDC Metri performed an AVM assessment demonstrating gaps in value based on the hours (cost) put into the sprints. Productivity was 30% lower and cost 22% higher than market average. Maintenance cost four times the market average. This assessment culminated in supplier improvement actions to comply with performance and product health metrics (with ongoing verification by IDC Metri).
To rephrase an earlier observation: if you can’t easily measure something, you can’t easily manage it. AVM allows organizations to clearly understand how their Agile development teams (staff or sourced or hybrid) perform and deliver value. It cleanly addresses three key organization struggles around Agile: predictability, quality, and cost. It makes it easy to measure and assess Agile development, which means it enables easier and effective management of Agile.
Want a deeper example of an organization that overcame challenges in quantifying Agile value? Read “A Management Primer: How Agile Development Teams Can Deliver Value”
