Environmental, social and governance (ESG) ratings and ratings agencies have faced substantial criticism lately for a fair amount of mismeasurement and a lack of transparency in their procedures. Some are calling for a complete overhaul or abandonment of the ratings altogether.
In a new journal article – “The Signal in the Noise” – we argue that abandoning or standardising ESG ratings is not the answer. The ethical behaviour of firms remains essential to the health of economies, societies and the natural environment. And ESG ratings, though flawed, are currently still the best option to measure the ethical behaviour of firms. They can empower investors and other stakeholders to hold firms accountable.
Our research suggests that ESG ratings can be an important source of information for investors, and can be distilled into four key messages:
- the information that ESG raters produce is valuable;
- assessing ESG performance is conceptually challenging because we need to measure contextuality, intentionality and preferences;
- ESG raters, specialised ESG data providers and aggregators can harness economies of scale; and
- regulators should enforce transparency of measurement and aggregation practices to increase competition between ESG raters to incentivise improvement.
The information that ESG raters produce is valuable. There is still a ‘signal’ in ESG rating scores, especially for the relationship between stock returns and ESG scores. In our recent research, we think of the score of a particular ESG rating agency as the combination of some noise and an underlying, true ESG performance.
If we correct for the noise, we find that the relationship between ESG scores and stock returns is positive and highly significant economically, as well as statistically.
The reason this relationship can be hard to detect in the data is precisely because the data is ‘noisy’. Imagine trying to listen to an academic lecture with noisy construction work in the background. The noise will drown out the lecturer and make the lecture harder to understand. Knowledge, however, is still being imparted.
Assessing ESG performance is conceptually challenging because we need to measure contextuality, intentionality and preferences. ESG ratings agencies need to assess many complex issues, such as CO2 emissions, water, discrimination, product safety, supply chain, taxes and many others. They need to produce regularly updated assessments for thousands of companies.
Some argue that ESG ratings should take on just one key issue, such as CO2 emissions, but we argue that this is not a good idea. For CO2 emissions, for example, components are measured with different degrees of precision.
Even if firms provide data about CO2 measurements, this tells us only about the firm’s emissions in the past. We do not yet know the firm’s future emissions – which are based on the decisions that the firm is making today – nor do we gain an understanding of the emissions in the firm’s supply chain. The CO2 data needs to be put in context to truly understand a firm’s impact on society and the natural environment. We also need to know a company’s intention to reduce CO2 emissions in the future.
Likewise, complex problems such as discrimination and mistreatment of historically disadvantaged groups in the labour force cannot be summarised by simply looking at the proportion of individuals in these groups who are in management roles. If regulators focus only on this statistic, firms might comply and achieve the right proportion in management, but continue to mistreat and limit them in various other ways. Even if firms hit the target, they could still miss the entire point.
ESG raters, specialised ESG data providers and aggregators can harness economies of scale. Competition among ratings agencies helps to drive down costs – if the market is set up the right way. For regulators, the key is to create a competitive market, where competition is centred around the quality of measurement.
We believe there are three useful steps regulators should take: (1) standardise ESG disclosure (not the ratings) and make it mandatory; (2) enhance transparency about methodologies; and (3) encourage compatibility between ratings systems. With regard to ESG ratings, we believe standardisation of how and what ESG ratings measure, with the aim of making them diverge less, would ultimately result in less reliable information.
However, regulators should increase transparency about measurement practices and aggregation rules. Without transparency, there cannot be any competition between the best measurement practices or aggregation rules.
Florian Berg, Jason Jay, Julian Kölbel and Roberto Rigobon are academics and co-founders of the MIT Sloan Aggregate Confusion Project.
