Because energy savings is the difference between the energy that was used after an intervention and a baseline estimate of what would have been used without an intervention (known as a counterfactual), efficiency is never actually measured--it is calculated. To put it another way, once you change a building, there's nothing left to measure against. To figure out how much energy was saved, you have to compare energy consumption with what you think would have happened if no action were taken.

Because of this, “measuring” energy efficiency is challenging on a few fronts. While transparent and open source platforms such as the OpenEEmeter can provide consistent and replicable calculations of savings that are adjusted for routine changes such as weather or occupancy, engineering adjustments to account for events that can’t be predicted often have the effect of inflating savings rather than increasing accuracy.

Calculating savings at the meter starts with making routine adjustments for things that are relatively predictable, such as weather or occupancy. Non-routine adjustments may be necessary in the baseline and may also come after the intervention in response to events such as new loads or changes in operating hours. Non-routine adjustments generally come down to someone noticing that something changed, and making a one time correction to account for it. (The basics around how to do these adjustments is covered in IPMVP Option C.)

In order to isolate the impacts of a specific project or intervention on savings for individual buildings, it's important to take non-routine events into account. But the reality is, non-routine adjustments generally go only one way--which is up.

Simply put, it’s rare to see a non-routine adjustment that reduces savings. Efficiency providers generally don’t spend money to perform M&V unless they have something to gain, and building owners are rarely paying close attention. It’s much more likely that an aggregator or project developer will roll a truck or open a model when they stand to lose money than when their savings are looking good.

This bias is one of the reasons that OpenEE is focused on portfolio-level savings, and prefers to use the law of large numbers to wash out non-routine events. A portfolio of metered site-level savings without non-routine adjustments will actually deliver a more accurate savings yield than a portfolio of site adjusted assets, because non-routine events will be evenly distributed rather than concentrated on the side of addressing poor performance. At the portfolio level, you win some, you lose some, and it averages out.

With a sufficient number of assets, both utilities and ratepayers can have confidence that the grid and climate benefits of efficiency are being measured without requiring that adjustments on each building be engineered, tracked, and regulated.

Taking a portfolio approach dramatically lowers costs, while reducing brain damage for all parties. This is the key to scale.

Like many things however, the real solution is more nuanced. In reality, we often don’t have huge portfolios, which means that one customer who installs an olympic pool behind his or her house can really skew the savings. Never mind the impact of a new data center on an aggregator who has only five commercial projects.

In cases like these, non-routine adjustments may be needed to account for the uncertainty of major events on small portfolios in order to manage customer expectations and reduce adverse cash-flow impacts on aggregators in pay-for-performance programs. The goal of managing adjustments is to find a sweet spot that minimizes transaction costs while bounding risk so that aggregators, insurers, and investors have the confidence to participate.

In sum, whenever possible we should rely on the law of large numbers, rather than complex non-routine adjustments, as a lower cost and ultimately more reliable measure of savings. When circumstances require non-routine adjustments, we should scale the approach carefully based on clear upfront criteria to align with both business models and customer expectations.

To learn more about how to implement non-routine adjustments for customer contracts and for individual buildings, check out our blog Bankable M&V for Commercial Buildings.