Environmental Ranking Wars

By Arin de Hoog

In any good boxing match there is the defending Champion, the Contender, and the new Upstart waiting to take on the winner of the headline bout.

The Champion has usually held on to the title for a while, but his prowess has come into question. The Contender is strong, and although maybe not as good an all-around fighter, has honed specific punches and dodges which may prove to unseat the Champion. The Upstart has trained hard under a modernized exercise regiment and is confident that he would be a challenge to both. In the world of climate change a contest is emerging between the lists which rank countries against each other according to their national environmental activities.

Although all fighting the same fight — identifying those countries which are laggards and leaders in the battle against climate change — these lists all wish to be considered the definitive list when policy-makers and the media compare countries in international ranking. This is a worthy goal when you consider that these lists are designed to positively affect national environmental policy.

Clearing the existing fogginess surrounding climate change necessitates as much consistency as possible, whenever possible, which means one decisive ranking. Also, not only are these lists waved at each other in the halls of government, and conferences, they tend to be heavily cited by the media to convey praise, and far more often, disappointment.

The current Champion is the Environmental Performance Index (EPI) which remains the most widely recognized of the three. It is the product of a collaboration between Yale and Columbia Universities and the European Commission, at the behest of the U.N. — all heavyweights themselves, which gives this list a certain amount of gravitas.

The worthy Contender is the Climate Change Performance Index (CCPI) created by the NGO Germanwatch. This list comprises the collaboration of over 190 climate experts, and has been covered by the media in over 100 countries.

The new Challenger on the fight card unfortunately does not have name which can be whittled down to a sleek acronym. It is the product of hard scientific and mathematic cooperation around the world which shies away from politics. Instead it strips down the data to a proportional representation of each country’s standings based on absolute values, and then crunches the numbers. As it centers around the efforts of Professor Corey Bradshaw at Adelaide University, it will be called the Adelaide List (AL).

Ostensibly the three lists are going for different results — the EPI list incorporates the most variables in sustainability and environment for an overall perspective, while the CCPI looks solely at climate, and the AL ignores policy in favour of empirical data — but the way that the lists are used and perceived is going to be the same: Where does a country rank against other countries in terms of environmental best practice?

The difference, however, comes into sharp focus when you take two countries and see where they rank on the three lists from 2010. Canada and The Netherlands, for example, find themselves in widely different places throughout. EPI ranks them a close 46th and 47th respectively. CCPI ranks The Netherlands 27th but has Canada 59th on a 60 country list (CCPI ranks 163 countries, while AL ranks 171). The AL has them at 12th and 64th respectively.

These numbers, in the halls and rooms where climate change is heatedly debated, drastically effect the direction that policy can go, when it doesn’t stymie it all together. This is indicative of a general lack of consistency that seems to continually trip-up any cohesive climate policy action.

When asked about the amount of recognition the AL has received, Professor Bradshaw was cautiously optimistic, saying, “Perhaps this is where we have been less successful – no one I know of seems to be using it, but it’s still early days.” Germanwatch’s CCPI, on the other hand, proudly displays many of the instances when their list has been cited in the international media on their website. What is unspoken is that both these lists are striving to be recognized with various degrees of success.

The similarities, however, end there. Jan Burck heads the team that annually produces the increasingly popular CCPI. His team cooperates with environmental NGOs and think-tanks throughout the world, relying on feedback from no less than three — usually many more —organizations per country before they consider ranking. Strikingly, they tend to avoid talking to scientists, because, as Burck said, “They refuse to judge.” This is in stark contrast to the methodology behind the AL which exclusively uses quantifiable data provided by organizations which keep a tally on such things.

For example, in the abstract to their list, they state “We obtained plantation forest area and total forest area from 1990 and 2005 from the Food and Agriculture Organization (FAO) Global Forest Resources Assessment 2005 (www.fao.org). Area of natural forest of each country was calculated by subtracting plantation forest area from total forest area.” These are absolute numbers that duck away from those particular environmental policy initiatives and failures which tend to involve a more intangible calculation.

In fact, the language that each list uses in the description of their respective methodologies speaks volumes about their confidence and differences. The AL abstract points to other lists to indicate their shortcomings, stating that the EPI “confound[s] environmental performance with indicators of human health.” The CCPI Contender, while not mentioning other lists, proudly asserts that “The astounding press echo to the CCPI shows its relevance…Both at the national as well as international level, numerous media reported on the outcomes and on how well their country did. Awareness was also raised in politics.”

The people behind the EPI seem to have no need for that kind of talk, instead explaining purely their methodology and humbly claiming to be the basis for further review while openly citing its shortcomings on the outset: “The EPI provides a framework for greater analytical rigor in the environmental domain but, at the same time, reveals severe data gaps, weaknesses in methodological consistency, and the lack of any systematic process for verifying the numbers reported by national government”.

On the other hand, the stated goals of each list sound remarkably similar. The EPI abstract says their list “can assist in refining policy choices, understanding the determinants of environmental progress, and maximizing the return on governmental investments.” The CCPI aims to “raise the pressure… on decision makers and move them to consequently protect the climate…” while the AL is designed to “improve policy and practice in the regions identified as having the poorest environmental performance.” The aim here, across the board, is to be heard by the people in power. Unspoken, is that some media attention wouldn’t hurt in that respect at all.

When you set their similar goals of policy and self-promotion against the wide disparity in rankings in terms of countries like Canada and the Netherlands, the underlying question is: Who’s right?

The obvious answer is: It depends on what you’re looking for. From a purely scientific perspective the AL solely relies on the raw data made available from the accumulation of hard numbers. In this way they can avoid the amorphous comparison of national climate policies versus actual environmental activity. By the removal of all politics they have effectively created purely mathematical matrices on which to compare the countries.

The CCPI, although 80%  quantitative (the AL is 100% quantitative), also ads 20% which is qualitative. This is the data that they have accumulated though rigorous discussion with the various worldwide NGOs they have within their network. This raises the question of certain NGOs being overly critical or positive when judging their own country, and therefore negatively affecting the ranking.

CCPI Head, Jan Burck, openly acknowledges the haziness when incorporating this data into the sum total, but sees no better way of showing the effectiveness of environmental policy in a certain country. So, the information gathered from the questionnaires his team sends out is cross-checked, and re-cross-checked, by experts associated with the Germanwatch network.

Burck also says that what distinguishes the CCPI from the EPI is that Germanwatch solely looks at climate concerns. Which, when looking at the data that goes into the EPI, is quite true. Where the AL and the CCPI shun the “human element” a good portion of the EPI data concerns “Environmental Burden of Disease”, “Access to Drinking Water”, “Access to Sanitation”, “Urban Particulates”, and “Indoor Air pollution”. This, and many of the climate elements used in the AL and CCPI, is included in “Policy Categories” and compared against the “Environmental Health” and “Ecosystem Vitality Objectives” of each country for the final tally of where they stand. Essentially, policy is factored in 100% in the EPI, as opposed to 0% in the AL.

The EPI is quick to acknowledge the inherent problems associated with this kind of analysis, however: “The 2010 EPI uses the best environmental data available, but complete country coverage is precluded by limits in both quality and quantity in data sources.” This statement neatly summarizes one of the inherent problems which affects both the Champion and The Contender; people unable to give accurate information — or lying.

Recently Canada has been slammed by the international community for providing missing and conflicting information in an Environment Canada submission to the U.N. This says two things: One, is that Canada is concerned about their perception, and this can be directly attributed to someone’s list. And two: If a so-called developed nation can brazenly lie, who else is lying too? Along those lines the people behind the CCPI “hope[s] that the data basis, in Russia for example, improves, so that this source category can be included in future CCPI.”

Another problem with the lists is a longitudinal one. That is, it’s very difficult to measure particular policy effectiveness over a short period of time. The EPI notes that, “One of the biggest weaknesses in the current framework is the lack of ability to track changes in performance over time.” The CCPI clarifies this by saying that “the absolute amounts of  CO2 that a country emits can only be changed in long time periods…” as a result “Climate Policy” and “Emission Trends” are only given 40% more credence than actual “Emission Levels” on the CCPI.

What is interesting to note is the various general conclusions that the list-makers come to when looking at their own score card. Germanwatch doesn’t give an overall perspective on commonalities between the worst climate offenders, but rather looks at each country individually on a performance basis. The EPI people hesitantly offers that respective wealth does something to their ranking, but that differs, saying finally that “in many cases good governance contributes to better environmental outcomes.” The AL, however, unequivocally states “Richer countries generally exploit more resources for the same population size as the relationship between human population and proportional environmental impact…” Essentially as resources go down the toll on the environment goes up. Or as Professor Bradshaw, the man behind the AL, said, “Wealth accumulation and population growth will lead to higher and higher environmental degradation.”

So, on the next few fight cards, who would you place your bet on? Well, the lists only have the authority granted to them by the people that cite and use them. The crux of the dilemma is a matter of who is using which list to further what agenda. Underlying this is a matter of arbitrary interpretation by policy-makers and the media about what the lists really say about their country and others.

What occurs is a kind of Catch-22 because it’s those very policy-makers and media which are necessary to make the list legitimate in the first place. And this comes down to money, as it is the person with the Champion list that will get the most funding. This leaves climate change, and its negative environmental effects, the true World Title holder.

[twitter style=”vertical” float=”left”] [fblike style=”standard” showfaces=”false” verb=”like” font=”arial”]