CSRHub Blog Research on ESG metrics and comments on sustainability best practice

The following is part 3 of a 3-part series on “Big Data.”

By Bahar Gidwani

In the past several posts, we have defined Big Data, shown the problems we hope it will address, and described how CSRHub has implemented a Big Data approach to creating corporate social responsibility (CSR) and sustainability ratings.  It is time now to discuss the benefits and drawbacks of the “Big Data” approach.

The assumption is that this approach offers many benefits which are not available under traditional analyst-based ratings methods:

• A broad measure of perceived performance.  Input from most of the “stakeholders” who evaluate a company’s sustainability performance is captured.  Investor input from the ESG/SRI sources, community input from NGOs and government groups, and input from suppliers, employees, and customers via supply chain tools, employee surveys, and product ratings are included.  While no one can claim to measure true company performance—no external system can do this, it is possible to give an accurate overall multi-stakeholder-based estimate of how a company is perceived.
• Increased transparency and accountability.  The system described automatically reveals to users which sources have reported on each company rated.  Via subscriber-accessible tables and custom reports, users can inspect the details of the data gathered.  This allows companies and their stakeholders to identify the data elements that affect how they are perceived (transparency) and to respond to or correct data that may not be accurate (accountability).
• Reduced impact from errors and bias.  If a source contains a lot of factual errors or an undisclosed bias, this system automatically reduces the weight given to the source.  In this way, the effect on our results of poor quality sources is minimized and corrected for systemic biases.  Because sources generate their information independently, there is good statistical accuracy for our aggregated scores.
• Regular update and trend tracking.  Some sources update their information daily, some quarterly, some only once per year.  However, because there are so many sources, our ratings are updated each month.  This allows the system to show trend charts that connect actions and outcomes with perception.
• Broad coverage of industries, geographies, and company types.  An aggregation system is dependent on its sources for coverage.  We do not yet have full data on small companies or on those in remote geographies or unusual industries.  But, the system allows us to use whatever is available.  We may not be able to rate all aspects of each new company we add to our system, but any ratings we can generate should be consistent across our system.

We Don’t Yet Measure Thousands of Smaller Corporate, Not-for-profit, and Government Entities

• This approach supports a fourth Big Data “V”—“Veracity.”  There is free access to basic ratings information to everyone.  As a result, any stakeholder can check scores and audit results.
• Users can adjust ratings to fit their own personal views.  There is sufficient data from a wide enough range of sources that we can present alternate sides of many contentious issues.  Users can record a profile that adjusts our ratings to match their own view.  Users can emphasize the priority of environment, employee, community or governance issues, be in favor of nuclear power or against it, or focus on the risks from mercury in fish.  They can then share their personal overall ratings of company sustainability with the other users.

This approach to ratings has a few drawbacks that are common to Big Data systems:

• Perception is not reality.  The data that companies self-report is focused mostly on its policies and intent.  A company that is good at communicating and “spinning” its story could raise its ratings on the system to a level they do not deserve.  Of course, as more data is secured—especially from bottoms up “crowd” sources—this type of behavior will likely eventually be detected.  A GovernanceMetric tool called Audit Integrity does this type of sleuthing on corporate financial reports.
• Best practices are not immediately obvious.  It is fairly easy to discover that certain activities seem connected with better ratings.  For instance, companies who use the Global Reporting Initiative (GRI) guidelines or who participate in the UN Global Compact have statistically better ratings than those who do not.  However, it is hard to tell if a program at one company has more effect on its perceived CSR performance than a different program at another company.  The system described can only provide a base of data—the study and explanation of ratings differences must be done by CSR professionals.
• We cannot correct individual company errors that are found.  There are conflicts in the views of disparate sources, on a regular basis.  These discrepancies can’t be “resolved” even when we suspect that some are caused by a source’s data collection or analysis error.  The best that can be done is to report a suspected error to the source and allow it to research and correct the error, in its own way.

We believe the benefits of using a Big Data approach to measure corporate social responsibility and sustainability performance far outweigh the drawbacks.  A Big Data system can be extended to include thousands of smaller companies and organizations.  We hope to expand our universe of coverage while we keep narrowing down the “error bars” in our ratings and to see if we can discover some of the drivers that change them.  For instance, as we build up a tail of detailed historic data, we may be able to prove that certain actions lead to ratings changes and others do not.  As more data become available the approach outlined above can be applied to virtually any company regardless of size and location.

See part 1, Using “Big Data” to Rate Corporate Social Responsibility: One Company’s Approach.

See part 2, A Big Data Approach to Gathering CSR Data.

Bahar Gidwani is CEO and Co-founder of CSRHub.  He has built and run large technology-based businesses for many years. Bahar holds a CFA, worked on Wall Street with Kidder, Peabody, and with McKinsey & Co. Bahar has consulted to a number of major companies and currently serves on the board of several software and Web companies. He has an MBA from Harvard Business School and an undergraduate degree in physics and astronomy. Bahar is a member of the SASB Advisory Board.  He plays bridge, races sailboats, and is based in New York City.

CSRHub provides access to corporate social responsibility and sustainability ratings and information on 9,200+ companies from 135 industries in 106 countries. By aggregating and normalizing the information from 348 data sources, CSRHub has created a broad, consistent rating system and a searchable database that links millions of rating elements back to their source. Managers, researchers and activists use CSRHub to benchmark company performance, learn how stakeholders evaluate company CSR practices and seek ways to change the world.

The following is part 2 of a 3-part series on “Big Data.”

By Bahar Gidwani

We have previously defined “Big Data” and shown how we feel a Big Data system built by CSRHub could help address some problems that exist in collecting corporate social responsibility (CSR) and sustainability data on companies.  We have also further described the problems with the currently dominant method of gathering this data—an analyst-based method.

CSRHub uses input from investor-driven sources (known as “ESG” for Environment, Social, and Governance or “SRI” for Socially Responsible Investment), non-governmental organizations, government organizations, and “crowd sources” to construct a 360 degree view of a company’s sustainability performance.

The illustration below shows the steps in our process.

The steps are:

•  Convert measurement from each data source into a 0 (low) to 100 (high) scales.  This  requires understanding how each source evaluates company performance.
•  We next connect each rating element with one or more of our twelve subcategory  ratings.  (Some elements may also map partially or exclusively to special issues such as  animal testing, fracking, or nuclear power.)
•  We compare each source’s ratings with those for all other sources.  Each company we  study gives us more opportunities to compare one source’s ratings with another.  The  total number of comparisons possible is very large and growing, exponentially.  We  use the results of our comparison to adjust the distribution of scores for each rating  source so that they fall into a “beta” distribution that has a central peak around 50.
•  Some sources match up well with all of our other data.  Some sources don’t line up.  We add weight to those who match well but continue to “count” those who don’t.

We then repeat steps A to D as many times until we have found a “best fit” for the available data.  Each time we add a new source, we go through an initial mapping, normalization, and weighting process.

An Example

It may help explain our data analysis process by using a specific example.  Hewlett Packard is a heavily tracked company.  We have 154 sources of data for this company that together provide 17,571 individual data elements.  Only 62 of these data sources provided data for our July 1, 2014 rating—the rest of the data sources provided data for previous periods (our data set goes back to 2008).  The 62 current data sources provided 575 different types of rating elements and a total of 610 different ratings values that do not affect/apply to special issues.

After their conversion to our 0 to 100 scale, we map the rating elements into our twelve subcategories.  We now have 1,403 ratings factors.  We selected our subcategories to allow an even spread of data across them. You can see that we have a reasonably even spread for Hewlett Packard:

Before we can present a rating, we need to check first that we have enough sources and enough “weight” from the sources we have, to generate a good score.  In general, we require at least two sources that have good strength or three or four weaker ones, before we offer a rating.  As you can see, we have plenty of sources to rate a big company such as HP.

Even after normalization, the curve of ratings for any one subcategory may have a lot of irregularities.  However, we have enough data to provide a good estimate of the midpoint of the available data, for those ratings we report.  Below you can see that some sources have a high opinion of HP’s board while others have a less favorable view.  The result is a blended score that averages to less than the more uniform Leadership Ethics rating.

The overall effect of our process is to smooth out the ratings input and make them more consistent.  As you can see in the illustration below, the final ratings distribution is organized well around a central peak.  The average overall rating of 64 is below the peak, which is around 80.  The original average rating was 61.

By making a few assumptions about how the errors in data are distributed, one can assess the accuracy of ratings.  In a previous post, we showed that CSRHub’s overall rating accurately represents the values that underlie it to within 1.8 points at a 95% confidence interval.

In our next post, we will discuss the benefits and drawback of using this complex and data intensive approach to measuring company CSR performance.

See part 1, Using “Big Data” to Rate Corporate Social Responsibility: One Company’s Approach.

Bahar Gidwani is CEO and Co-founder of CSRHub.  He has built and run large technology-based businesses for many years. Bahar holds a CFA, worked on Wall Street with Kidder, Peabody, and with McKinsey & Co. Bahar has consulted to a number of major companies and currently serves on the board of several software and Web companies. He has an MBA from Harvard Business School and an undergraduate degree in physics and astronomy. Bahar is a member of the SASB Advisory Board.  He plays bridge, races sailboats, and is based in New York City.

CSRHub provides access to corporate social responsibility and sustainability ratings and information on 9,200+ companies from 135 industries in 106 countries. By aggregating and normalizing the information from 348 data sources, CSRHub has created a broad, consistent rating system and a searchable database that links millions of rating elements back to their source. Managers, researchers and activists use CSRHub to benchmark company performance, learn how stakeholders evaluate company CSR practices and seek ways to change the world.

The following is part 3 of a 3-part series on “Big Data."

By Bahar Gidwani

In the past several posts, we have defined Big Data, shown the problems we hope it will address, and described how CSRHub has implemented a Big Data approach to creating corporate social responsibility (CSR) and sustainability ratings.  It is time now to discuss the benefits and drawbacks of the “Big Data” approach.

The assumption is that this approach offers many benefits which are not available under traditional analyst-based ratings methods:

• A broad measure of perceived performance.  Input from most of the “stakeholders” who evaluate a company’s sustainability performance is captured.  Investor input from the SRI sources, community input from NGOs and government groups, and input from suppliers, employees, and customers via supply chain tools, employee surveys, and product ratings are included.  While no one can claim to measure true company performance—no external system can do this, it is possible to give an accurate overall multi-stakeholder-based estimate of how a company is perceived.
• Increased transparency and accountability.  The system described automatically reveals to users which sources have reported on each company rated.  Via subscriber-accessible tables and custom reports, users can inspect the details of the data gathered.  This allows companies and their stakeholders to identify the data elements that affect how they are perceived (transparency) and to respond to or correct data that may not be accurate (accountability).
• Reduced impact from errors and bias.  If a source contains a lot of factual errors or an undisclosed bias, this system automatically reduces the weight given to the source.  In this way, the effect on our results of poor quality sources is minimized and corrected for systemic biases.  Because sources generate their information independently, there is good statistical accuracy for our aggregated scores.
• Regular update and trend tracking.  Some sources update their information daily, some quarterly, some only once per year.  However, because there are so many sources, our ratings are updated each month.  This allows the system to show trend charts that connect actions and outcomes with perception.
• Broad coverage of industries, geographies, and company types.  An aggregation system is dependent on its sources for coverage.  We do not yet have full data on small companies or on those in remote geographies or unusual industries.  But, the system allows us to use whatever is available.  We may not be able to rate all aspects of each new company we add to our system, but any ratings we can generate should be consistent across our system.
• This approach supports a fourth Big Data “V”—“Veracity.”  There is free access to basic ratings information to everyone.  As a result, any stakeholder can check scores and audit results.
• Users can adjust ratings to fit their own personal views.  There is sufficient data from a wide enough range of sources that we can present alternate sides of many contentious issues.  Users can record a profile that adjusts our ratings to match their own view.  Users can emphasize the priority of environment, employee, community or governance issues, be in favor of nuclear power or against it, or focus on the risks from mercury in fish.  They can then share their personal overall ratings of company sustainability with the other users.

This approach to ratings has a few drawbacks that are common to Big Data systems:

• Perception is not reality.  Primarily policies and intent are measured.  A company that is good at communicating and “spinning” its story could raise its ratings on the system to a level they do not deserve.  Of course, as more data is secured—especially from bottoms up “crowd” sources—this type of behavior will likely eventually be detected.  A Corporate Library tool called Audit Integrity does this type of sleuthing on corporate financial reports.
• Best practices are not immediately obvious.  It is fairly easy to discover that certain activities seem connected with better ratings.  For instance, companies who use the Global Reporting Initiative (GRI) guidelines or who participate in the UN Global Compact have statistically better ratings than those who do not.  However, it is hard to tell if a program at one company has more effect on its perceived CSR performance than a different program at another company.  The system described can only provide a base of data—the study and explanation of ratings differences must be done by CSR professionals.
• We cannot correct individual company errors that are found.  There are conflicts in the views of disparate sources, on a regular basis.  These discrepancies can’t be “resolved” even when we suspect that some are caused by a source’s data collection or analysis error.  The best that can be done is to report a suspected error to the source and allow it to research and correct the error, in its own way.

We believe the benefits of using a Big Data approach to measure corporate social responsibility and sustainability performance far outweigh the drawbacks.  A Big Data system can be extended to include thousands of smaller companies and organizations.  We hope to expand our universe of coverage while we keep narrowing down the “error bars” in our ratings and to see if we can discover some of the drivers that change them.  For instance, as we build up a tail of detailed historic data, we may be able to prove that certain actions lead to ratings changes and others do not.  As more data become available the approach outlined above can be applied to virtually any company regardless of size and location.

Bahar Gidwani is a Cofounder and CEO of CSRHub. Formerly, he was the CEO of New York-based Index Stock Imagery, Inc, from 1991 through its sale in 2006. He has built and run large technology-based businesses and has experience building a multi-million visitor Web site. Bahar holds a CFA, was a partner at Kidder, Peabody & Co., and worked at McKinsey & Co. Bahar has consulted to both large companies such as Citibank, GE, and Acxiom and a number of smaller software and Web-based companies. He has an MBA (Baker Scholar) from Harvard Business School and a BS in Astronomy and Physics (magna cum laude) from Amherst College. Bahar races sailboats, plays competitive bridge, and is based in New York City.

CSRHub provides access to corporate social responsibility and sustainability ratings and information on nearly 5,000 companies from 135 industries in 65 countries. By aggregating and normalizing the information from over 170 data sources, CSRHub has created a broad, consistent rating system and a searchable database that links millions of rating elements back to their source. Managers, researchers and activists use CSRHub to benchmark company performance, learn how stakeholders evaluate company CSR practices and seek ways to change the world.

The following is part 2 of a 3-part series on “Big Data.”

By Bahar Gidwani

We have previously defined “Big Data” and shown how we feel it could help address some problems that exist in collecting corporate social responsibility (CSR) and sustainability data on companies.  We have also further described the problems with the currently dominant method of gathering this data—an analyst-based method.

CSRHub uses input from investor-driven sources (known as “ESG” for Environment, Social, and Governance or “SRI” for Socially Responsible Investment), non-governmental organizations, government organizations, and “crowd sources” to construct a 360 degree view of a company’s sustainability performance.  To better understand this process, let’s consider an example.

Hewlett Packard is a heavily tracked company.  We have 56 sources of data for this company that together contribute 494 different rating elements.  We map each of these elements into one of twelve different CSR subcategories.  For instance, here are mappings for 20 of the elements that contribute to the Hewlett Packard rating:

Some of these data elements could map to more than one subcategory.  For instance, a company that is on the list of “Best Workplaces for Commuters” would get credit both for its energy saving effort (in “Energy & Climate Change”) for the benefit its programs bring to its employees (in “Compensation & Benefits).

The list above includes examples of each of the three main contributors to the system: Investment-related sources (Asset4/Thomson Reuters, Carbon Disclosure Project, GovernanceMetrics International/Corporate Library, IW Financial, MSCI, Trucost, Vigeo); Activists and NGOs (Accountability, BSR, CorporateRegister, CR 100, EIO, FCPA, Top 50 Socially Responsible); and Government & Consumer (Better World, Black Engineer, EICC, EPEAT, UN Global Compact, UNODC, Working Mother).  The completed mapping process connects the 494 data elements from the 56 sources for HP into our twelve subcategories in 971 different ways.

While investment-related sources contribute more data elements than the other types, there are at least some of each type present in each subcategory.  Another way to look at this is to see that many sources contribute to each subcategory:

Each value from each data element is converted into a zero to 100 rating (zero = lowest, 100 = highest).  These scores are then adjusted by comparing them to each other.  In the example above, there are 11 sources for HP’s board performance.  Suppose three of them gave it a great rating, six a medium rating, and two a poor one.  Computer analytics would guess that the six scores that agree are correct and that HP’s board rating is in the medium range.  The assumption is that three sources tended to be biased towards high scores and two towards low scores.  This chart shows the actual distribution of scores at the subcategory level, along with a calculation of the “normal” error curve that results.

When the analysis is repeated across thousands of companies, a picture emerges as to which sources tend to be overly positive or negative and which tend to predict the “mean” of the other sources.  All sources can be adjusted, based on this feedback—moving them up or down so they more accurately match the opinion of all other sources.  After a large number of iterations in this process, there is a consensus score for each subcategory for each company analyzed.

By making a few assumptions about how the errors in data are distributed, one can assess the accuracy of ratings.  In a previous post, we showed that CSRHub’s overall rating accurately represents the values that underlie it to within 1.8 points at a 95% confidence interval.

In our next post, we will discuss the benefits and drawback of using this complex and data intensive approach to measuring company CSR performance.

Bahar Gidwani is a Cofounder and CEO of CSRHub. Formerly, he was the CEO of New York-based Index Stock Imagery, Inc, from 1991 through its sale in 2006. He has built and run large technology-based businesses and has experience building a multi-million visitor Web site. Bahar holds a CFA, was a partner at Kidder, Peabody & Co., and worked at McKinsey & Co. Bahar has consulted to both large companies such as Citibank, GE, and Acxiom and a number of smaller software and Web-based companies. He has an MBA (Baker Scholar) from Harvard Business School and a BS in Astronomy and Physics (magna cum laude) from Amherst College. Bahar races sailboats, plays competitive bridge, and is based in New York City.

CSRHub provides access to corporate social responsibility and sustainability ratings and information on nearly 5,000 companies from 135 industries in 65 countries. By aggregating and normalizing the information from over 170 data sources, CSRHub has created a broad, consistent rating system and a searchable database that links millions of rating elements back to their source. Managers, researchers and activists use CSRHub to benchmark company performance, learn how stakeholders evaluate company CSR practices and seek ways to change the world.