Metrics

The key to successful measurement programs is to make the metrics meaningful and tailor them to the organization—however small it might be.

Management love their metrics. The thinking goes something like this, 'We need a number to measure how we’re doing. Numbers focus people and help us measure success.'

Whilst well intentioned, management by numbers unintuitively leads to problematic behavior and ultimately detracts from broader project and organizational goals. Metrics inherently aren’t a bad thing; just often, inappropriately used.

Metrics have a purpose and a place in organizations and teams. They cannot be used as a substitute for thinking. Nor can organizations think management by numbers is enough for effective software delivery.

Organizations must be vigilant against the undesirable behaviors that emerge due to the inappropriate use of metrics. Double loop learning helps us understand focusing on the individual to behave differently cannot exist until the organization learns a more appropriate use for metrics.

Forty years ago, Thomas McCabe introduced his famous cyclomatic complexity (CC) metric. Today, it’s still one of the most popular and meaningful measurements for analyzing code.

A proposed data-driven software quality improvement method has three elements. First, the downstream Customer Quality Metric (CQM) quantifies quality as customers perceive it. On the basis of data collected after systems are deployed, it measures how serious defects affect customers. Second, the upstream Implementation Quality Index (IQI) measures the effectiveness of error removal during development. IQI predicts future customer quality; it has a positive correlation with CQM. Finally, prioritization tools and techniques help focus limited development resources on the riskiest files in the code.

Technical debt refers to maintenance obligations that software teams accumulate as a result of their actions. Empirical research has led researchers to suggest three dimensions along which software development teams should map their technical-debt metrics: customer satisfaction needs, reliability needs, and the probability of technology disruption.

It's difficult to express a software architecture's sustainability in a single metric: relevant information is spread across requirements, architecture design documents, technology choices, source code, system context, and software architects' implicit knowledge. Many aspects influence economic sustainability, including design decisions facilitating evolutionary changes, adherence to good modularization practices, and technology choices.

Developers should factor rework into sizing and productivity calculations when estimating software effort. Reworked code is software created during development that doesn't exist in the final build. Using lines of code as a sizing metric is helpful when estimating projects with similar domains, platforms, processes, development teams, and coding constraints.

Process metrics can ignite strong opinions because they represent an area where technical considerations bump up against human aspects of software development.

The recommendations for analyzing productivity in the appendix to the ISO/IEC 15939 standard are inappropriate. … [P]roblems with the ISO/IEC advice can be compounded if software engineers attempt to apply statistical process-control techniques to software productivity metrics.

Basili's contributions cover three broad areas: research in the 1970s and early 1980s on software measurement and the Goal Question Metric (GQM) model, research in the 1980s and 1990s on these measurement ideas' maturation into a software engineering model of empirical studies, including the development of the Quality Improvement Paradigm (QIP) and the influence of the NASA Goddard Space Flight Center Software Engineering Laboratory, and research since 1990 in the Experience Factory as a model for creating learning organizations for continuous software process improvement.

Traditional performance metrics no longer suffice to measure results and guide organizations in today's fast-changing economies. Firms need to link performance metrics to strategic objectives that will promote positive future results and accurately capture past performance.

Software measurement has the potential to play an important role in risk management during product development. Metrics incorporated into predictive models can give advanced warning of potential risks.

An organized, comprehensive metrics program can bring order to the chaos of small-project management and form the foundation for a concerted process improvement effort.

Despite rapid changes in computing and software development, some fundamental ideas have remained constant. This article describes eight such concepts that together constitute a viable foundation for a software engineering discipline: abstraction, analysis and design methods and notations, user interface prototyping, modularity and architecture, software life cycle and process, reuse, metrics, and automated support.

In 1991, Philips' CEO named a Software Process Improvement task force to focus on the increasing importance of software … In addition to improving its processes, the organization improved its requirements-and-design engineering architecture and its inspections, and it introduced metrics.

[F]ive problems … can contribute to a software project's failure … : inadequate system engineering during the proposal and during front-end development; inadequate tracing, tracking, and management of system and software requirements; improper sizing of the target hardware; selection of design, production, and integration and test methodologies that are inappropriate to a large software development; and failure to provide a metrics program that would let managers track the progress of software production and test.

You can't control what you can't measure. That fundamental reality underlies the importance of software metrics, despite the controversy that has surrounded them since Maurice Halstead put forth his idea of software science. Sketpics claim metrics are useless and expensive exercise in pointless data collection, while proponents argue they are valuable management and engineering tools.

Empirically comparing structural test coverage metrics reveals that test sets that satisfy one metric are likely to satisfy another metric as well.