Thoughts on ERM Dashboards

One of the natural consequences of evolving enterprise risk management programs is that they are generating more and more information. So it follows that in the midst of accumulating this mountain of data, (all of which is informative to some degree), a key question has to be asked: “What is the basic, most essential information that we need to understand our risk profile?”  Once we determine this we can then render this information in an executive level report, or dashboard.

The metaphor in this case is actually surprisingly accurate. Consider for a moment how you drive a car, keeping in mind that this is a device that, improperly managed, can have fatal consequences. Today’s modern vehicles are largely managed by very complex computer systems that make thousands of decisions every few seconds and also generate a massive amount of information, 99.999% of which you and I will never see.  But what do we see?

• Standard metrics (constant display): speed, engine RPM, fuel level, engine temperature, etc.
  • These are key pieces of data that need to be monitored at all times because they provide routine risk metrics that can change very quickly but that must be maintained within a certain acceptable levels
• Optional metrics (display on request): miles remaining, gas efficiency (mpg), time to next oil change, etc.
  • These are informative metrics, but are either not critical to the vehicle’s operation and safety or are secondary to one of the standard metrics.
• Elevated risk metrics (display on threshold): indicators for low oil, low tire pressure, engine trouble, etc.
  • These are indicators that are specifically designed to display only when the metric has reached a cautionary state outside of a normal tolerance level.

All in all, we routinely operate our vehicle, often at very high speed, with conservatively about 10 pieces of information (combined with our own instincts and other subject measures).  Whether these metrics are monitored or not is a different story (after all, no one ever runs out of gas for a lack of information) but the data that we need to manage risk is there for us. All of the other detailed data can be evaluated by a trained mechanic on periodic intervals.

When designing enterprise-risk dashboards we really should keep the above design schema in mind, because it actually provides a pretty sound approach. There are some metrics that should always be shown, some that should be on request and some that should automatically appear once they reach a certain threshold.

The rest of this article is a series of design considerations based on research, field experience and observation. Again, just to reiterate, this is an emerging topic and best practices are still evolving.

So what should be included in an ERM dashboard? This is a complex question and there is no one-size-fits-all answer. It really depends on the nature of the organization and its moving parts. However, some metrics that should be at least considered include:

• Key risk ratings or profiles (e.g., if possible, aggregate risk ratings for credit, market, liquidity, operational, etc.)
• Top level key risk indicators (e.g., policy exceptions, loan delinquency rates, staff turnover, etc.)
• Key performance indicators (e.g., efficiency ratios, new originations, budget to actuals, etc.)
• Significant activity (e.g., major events, projects under management, significant changes, etc.)
• Concentration rates (e.g., by product, geography, industry, etc.)
• Testing and validation results (e.g., stress testing, control testing, etc.)

Talking in Pictures

It is highly commonplace to use graphical representations in executive dashboards, which is a highly effective technique, but only when done correctly. When using pictures to represent data, consider the following.

• Singular data points that need a short description are often best represented by just indicating the value and including a color code, such as:

In this case showing the description, value, context (average), color (relative to tolerance) and direction (arrow).

• Heat maps (grids) are obviously the most common method for expressing data with two key attributes where there are multiple data points. For example:
• Heat matrices, on the other hand, are very good for expressing data with two key attributes where there is one discrete value per cell. In this particular case the “heat” is largely attributed to one of the dimensions. The callout shows an additional dimension (data quantity) for one of the cells.
• Bar, line and area graphs are best for single attribute sets of data (e.g., policy exceptions by department.)

• Speedometers can be good for single metrics that need to be shown within context, showing both the scale and the “heat” value (just like the car’s speedometer where you see both the range of speeds possible as well as where “red” begins.) However, while they look cool, they do consume a lot of real estate and, quite honestly, can just as easily be represented with a single point bar.Obviously there are an infinite number of derivations of these different types, as well as other graphical types that can be just as informative. So when considering the most effective graphical method possible, you have to ask yourself a couple of questions to understand what sort of context is needed for each data point, including:

• How many attributes are associated with the data point (1, 2, 3, ?)
• What is the range of possible values and is that informative?
• Do you need historical data to show both the prior values and the trend?
• Do you need a relative risk indication (the “heat” factor)?
• Is the data point relative to other values that also must be shown?
• Does the data fall logically into “buckets” or is it all over the place?

Answering these should at least give you some direction in choosing the correct format for rendering the data.  But the ultimate question for any piece of data is, what is this information telling me? Is it informative and actionable? If not, don’t include it.

Remember, pictures are good because they allow you to digest information very, very quickly (most people learn visually.)  But they can also be deceptive because they are so visually appealing. Simply put, it looks good, therefore, I assume it is right.  In addition, visual images don’t tell us the whole story, they only tell us the state of the story. (i.e., the condition of this metric is “yellow” or is a 6.5 on a scale of 1 to 10.)  What is meaningful is how that state is interpreted.  So in addition to any context necessary to give a metric clarity, the presenter must always be prepared to back up any data point (visual or otherwise) if it is challenged.

Narrative, on the other hand, is better for telling the story, but it is not easily digested, particularly if there is a lot of it. It can be very easy to become lost in the proverbial trees and loose site of the forest. A good scorecard must balance between these two methods using a combination of both visuals and narrative.  Consider using cover sheets of graphs, summary metrics and limited narrative, backed up by pages of more detailed narrative (which is then available on demand.)

Ultimately, a well designed ERM dashboard should accomplish a couple of things.  It should allow the reader to absorb a good bit of high level information quickly, giving them a general indication of the current state (the forest), while serving as a foundation for further discussion around areas of concern. The good news is that they don’t have to be perfect to be practical. While you certainly don’t want them changing dramatically every month, there’s nothing wrong with starting with a couple of pages of good data and evolving as you go. It’s better to start simple and grow then to start with 30 pages of pictures and numbers, in which case you’re right back to the same information overload, but with pictures this time.

One last thought.  Paper is good, electronic is better. I do believe that the natural evolution of ERM dashboards is by using a tablet computer which allows the user to quickly navigate information and including some selective drill-down capability. If you aren’t already using this technology you should begin exploring it.

The ERM Advantage: By creating smart, efficient risk management dashboards, risk managers can communicate a great deal of information to executives and the Board of Directors about risk profiles and areas of concern. This leads to more engaged management, more informed decision making and greater confidence in the enterprise-wide risk management program.

For assistance in building efficient, effective ERM dashboards contact Eric Holmquist at Accume Partners at (856) 793-1581 or eholmquist@accumepartners.com. Visit accumepartners.com

Disclaimer: The material presented here is in no way represented as definitive. The concept of ERM dashboards is still evolving with tremendous room for further innovation. Nevertheless, presented here are some emerging best practices to consider.