The World Economic Forum estimates the cost of corruption to be more than 5% of global GDP (US $2.6 trillion), and the World Bank believes over $1 trillion is paid in bribes each year. Of course, given the secretive nature of corrupt exchanges, we cannot know the true value of how much is actually lost, but there can be little doubt that corruption represents a major cost to the public. Given such staggering numbers, it is understandable that both academics and policymakers would want to develop measures of corruption. These measures aim to show how much corruption exists in the world and where it occurs, and ultimately provide guidance about how to stop it. Unfortunately, currently available measures of corruption are beset by conceptual, methodological, or political problems (or a combination of all three) that constrain their utility as a guide to developing effective anti-corruption policies.
Probably the most widely used measure of corruption today is Transparency International’s Corruption Perceptions Index (CPI). Published yearly, usually to extensive media coverage, this measure attempts to quantify perceptions about corruption held by experts, business leaders, and the public. The CPI has been extremely helpful in raising awareness about the extent of corruption in different countries and provides a reasonably useful heuristic about the overall extent of corruption within countries. Nonetheless, despite how the measure is often treated, it offers a highly imperfect reflection of actual corruption within countries. In practice, the relationship between perceptions and experiences of corruption is both inconsistent across countries and non-linear. Also of concern in the CPI is the implausible level of consistency in the individual country scores. We analyzed this consistency across an eleven-year period in CPI scores. The results are shown in the image below. As can be seen from this analysis, there is a very high degree of consistency between the scores across a large time period. Indeed, the consistency is so strong that we believe it does not make sense to treat individual years scores are separate evaluations. In fact, given such consistency, we believe that such perceptual evaluations could be done on a five- or ten-year basis, rather than yearly, with very little loss in precision.
In our manuscript, we repeated this analysis for another widely-cited perceptual measure of corruption, the World Bank’s Control of Corruption Index, and found essentially identical results: extreme consistency even across an eleven year time period.
Given such findings, the use of perceptual measures as a way to measure corruption may have serious – and still underappreciated – limitations. It is true, though, that concerns about perception-based measures have grown over recent years, and in response, a number of attempts have been made to create non-perceptual measures that rely on (seemingly) objective data. Such approaches have often focused on the use of crime statistics or accountancy data. It is hoped that these data provide information about the objective rate of corruption, since they highlight where money is being spent with no return, or the number of people actually convicted of a corruption-related crime. Whilst such measures are promising, they suffer from serious conceptual limitations. In the case of crime statistics, the data are particularly susceptible to (ultimately subjective) judgments about what ‘counts’ as corruption. Is bribery always corruption? What about fraud? When is theft an example of corruption? In the case of accountancy data, judgments have to be made about the classification of ‘missing’ resources. Is all money that has disappeared the result of corruption? Could it be a result of theft? And if it was theft, is that theft an example of corruption? Does chronic inefficiency in some areas have to be a result of corruption?
One area where such conceptual concerns about the use of ‘objective’ data are minimized (although not eliminated) is in the area of electoral fraud. Analyses of electoral fraud attempt to exploit the mathematical properties of electoral returns to determine when the vote tallies have been tampered with. To take one example, an effort has been made to evaluate the final digit of vote counts in various elections. Given the mathematical properties of such figures, the final digit of the true results should have a uniform distribution – that is to say that a 0 should be as likely as a 1, 2, 3, 4, and so on. However, it seems that people have inherent biases towards certain numbers and against others: in effect, a desire to make the number not ‘feel’ suspicious undoes the objective properties of such distributions and highlights that the results have been tampered with. There have been well-founded critiques of some applications of such measures, but the fundamental idea is a good one. Indeed, this method has relatively few requirements in terms of the data, and moreover exploits a trait in human psychology that is difficult for corrupt officials to escape.
Nonetheless, such ‘objective’ indicators are rarely available to those who seek to measure corruption. Ultimately, most measurements face the problem that they implicitly see corruption as ‘one thing’, an indivisible property of political systems that can be summarized through a single number or score, applicable to the whole of a territory. In practice, though, actual instances of corruption take place in concrete settings and specific places that do not easily map onto the nation state – either because there may be significant variance at local level when dealing with particular sectorial types of corruption or because of corruption that involves transnational or cross-border networks. Every country has regions with a (deserved or undeserved) reputation for significantly higher levels of corruption than average for their country. Even within particular sectors, there can be significant variation in levels of corruption according to location. The lesson we need to draw is that, when we seek to measure corruption, we need to be very specific about what kind, where, and for what purpose.