As work moves forward to complete the proposed Sustainable Development Goals, a working list of preliminary indicators for each of the goals recently circulated online. This list of indicators was developed by the UN Statistics Division and was shared with National Statistics Offices. Rather than talk about each of the goals and their indicators in detail, I will build on my previous discussions of the Governance SDG, which is Goal 16. As with the previous work, I will focus more on the expressly political elements of Goal 16, and put aside the peace and violence components of the goal. There are four points worth noting:
1) Official Statistics Have Their Limits.
Some of the public commentary on SDG 16 recognized an important issue at the outset. Many of the things that Goal 16 is intended to measure, namely measures of a state’s degree of transparency and accountability, do not have their basis in official state statistics. It is not hard to understand why this is the case. Even if national level measures of these concepts existed, some states would most certainly not release them. One of the findings of academic research on variations in levels of country transparency is that democratic regimes are more transparent, which is a finding that is robust across studies, measures, and model specifications.
As a result, some of the proposed indicators for some of the targets under Goal 16 (namely 16.3, 16.6, and 16.7) rely on public surveys that do not exist in many countries. These will need to be constructed, and there are financial implications for this, of course. But the larger point remains that our measures for these targets (Promoting rule of law and ensuring equal access to justice; Developing effective, accountable and transparent institutions at all levels; and Ensuring responsive, inclusive, participatory and representative decision-making at all levels) are based not on actual measures of these phenomena, but on perceptual measures of citizens of these phenomena. Responsiveness, transparency, and accountability will become what citizens think they are. Given that we live in a world in which state power is used to oppress as well as create, the danger of this becoming a largely semantic exercise is a real one.
2) Paths not taken in indicator development.
It is worth contrasting this approach with one articulated by Mary Hilderbrand in a paper developed for the Copenhagen Consensus Project. Many of the indicators proposed in this paper are based on existing measures developed elsewhere. The World Bank’s Worldwide Governance Indicators, Transparency International’s Corruption Perceptions Index, Freedom House country ratings, and data from the CIRI Human Rights Data Project are all included as possible indicators in addition to country surveys. As I note elsewhere, many additional indicators in this area exist, and many have been developed by academics in their own empirical research.
While relying on indices developed elsewhere may produce political problems (in that the UN Statistics Division would be largely endorsing already existing measures not developed by countries themselves), they do bring with them several important correctives. Not only are they are more likely to be impartial, but they might also bring us closer to actually measuring the concepts we’re trying to study. This is important in light of my next point.
3) The dangers of conceptual slippage.
One of the things that was genuinely exciting about the High Level Panel Report was its insistence on the importance of good governance as a development necessity:
Responsive and legitimate institutions should encourage the rule of law, property rights, freedom of speech and the media, open political choice, access to justice, and accountable government and public institutions.
This theme was restated in the Secretary General’s Synthesis Report:
We also know that participatory democracy, free, safe, and peaceful societies are both enablers and outcomes of development.
And herein lies a problem. The proposed indicators are not measures of political freedom or participatory democracy. In their stead are measures of state capacity. This is not necessarily a bad thing, since we know that stronger states are less prone to state failure. But from the standpoint of those advocating greater transparency and accountability, this is a case of overpromising and underdelivering. We will not know from the surveys for the indicators noted above whether countries are more or less free or whether there are actual choices at the ballot box.
4) The Coming Challenge.
The inherent limits of these proposed indicators invite both reflection and strategy. Our goal should be to get the indicators we need for the world we want. Even if we are left in a world of “second best” indicators for Goal 16, the challenge is to use other pieces of information from sources not referenced in the proposed list of indicators to call attention to their shortcomings. This is where many of the organizations that develop these indicators can play an important role to speak truth to measures.