Using standard indicators – opportunities, challenges and risks
Standard indicators can be characterised by uniform definition, data collection methods and interpretation. They produce data that can be aggregated (and compared) across interventions, countries or regions, for example. Standard indicators can be distinguished from “custom indicators”, which are formulated to describe specific phenomena or to measure certain changes under unique conditions.
Standard indicators can be formulated at different levels of the results chain. Examples are:
- Output: Number of farmers trained
- Short-term outcome: Land under improved management practices
- Medium-term outcome or impact: Individual Dietary Diversity Score
Standard indicators are used for three main purposes:
- To align projects towards common goals (planning tool): standard indicators are usually chosen to represent donor priorities. Through mandatory standard indicators, donors aim to ensure that supported projects focus on strategic goals.
- To compare results across projects (management tool): by using the same standard indicators across projects, donors aim to assess projects’ value for money.
- To report on results (accountability and public relations tool): standard indicators provide a snapshot of aggregate results achieved across countries and interventions.
Unambiguous definitions as well as clear guidelines on data collection methods and instruments are important to ensure that results are comparable and aggregatable. Differences in definitions, methods and instruments can lead to high differences in reported results, which are unrelated to project performance. It is especially challenging to develop guidelines applicable to various project contexts that specify how to measure who benefits from a project. While a project which trains farmers in good agricultural practices can count those who participate in trainings as beneficiaries, it is less clear whom to count as a beneficiary when a project supports the implementation of a national food security strategy through policy advisory work.
Limitations and Risks
Standard indicators at output and short-term outcome level tend to be very broad in scope in order to be applicable to as many projects as possible. As a result, even if common definitions, methods and instruments are used, results are often not comparable. The IFAD indicator “land under improved management practices”, for example, captures any type of initiatives aimed at promoting sustainable management of natural resources, such as integrated natural resource management practices, agroforestry practices and improved water management practices.
Standard indicators at medium-term outcome or impact level, such as the SDG indicators, are not suitable to measure results of one donor, because they capture changes that are products of the joint efforts of partner countries, donors and other influencing factors.
Standard indicators are a suitable tool to report on aggregate results achieved across interventions and countries. There are risks, however, when using them to align projects towards common goals and to compare results across projects.
Alignment: According to aid effectiveness principles, donors should base their overall support on partner countries’ national development strategies. To assess performance of interventions, indicators drawn from country-level results frameworks should be used. If donors use standard indicators to enforce donor priorities, conflicts can arise with the principle of alignment.
Comparison of results: Since most standard indicators do not account for qualitative differences in results and context conditions are not considered, their use can lead to two adverse effects. First, standard indicators encourage a focus on quantity instead of quality. For example, an indicator measuring the number of farmers trained does not distinguish between farmers who took part in a two-hour workshop and farmers who participated in a year-long training programme. Second, standard indicators may lead to a focus on low-hanging fruits and a neglect of areas where the per capita costs of delivering results are comparably high. The per capita costs of providing nutrition education to communities in a densely populated part of Cambodia, for example, can be expected to be much lower than in a remote part of the country. Thus, when comparing results across projects based on standard indicators, the context conditions should always be taken into account.
Sarah Holzapfel, PhD, is a research fellow at the German Development Institute in Bonn. In her research she focuses on how to increase the results orientation of agricultural, rural development and food security interventions in line with the aid effectiveness agenda.