After six months working with many of the top minds in energy efficiency measurement and verification, CalTRACK 2.0 methods have been tested, approved, and published.

From January through August 2018, Open Energy Efficiency led a stakeholder group with over fifty participants, anchored by commitments from NYSERDA, the Energy Trust of Oregon, and the California Energy Commission, alongside participation from utilities, businesses, program implementers, and M&V professionals. This all-star group of contributors provided a wealth of perspectives, experience, and input into the methods process and selection of those approaches that worked both empirically and in practice. A complete history of the process can be reviewed on the CalTRACK working group blog.

These important updates and new methods expand CalTRACK capabilities to include the generation of hourly savings estimates using AMI data, use in commercial buildings, as well as a host of other improvements to handle edge cases.

The links below provide details on the CalTRACK method updates for:

1. Calculation of hourly savings
2. Handling of extreme climate zones
3. Support for non-residential buildings
4. Generation of portfolio uncertainty
   

In August, we presented findings from the CalTRACK process at the ACEEE Summer Study conference. The full paper can be found on the ACEEE site, however, the key takeaways are summarized below.

The major learning from two rounds of CalTRACK methods development and testing is that implementation variance is a big deal. It’s really easy to assume that CalTRACK methods have been implemented correctly when in fact they haven’t.

The empirical testing process used during the development of the CalTRACK methods exposed significant implementation variance amongst participating testers. This implementation variance can be due to a variety of different factors, including bugs in software development, incorrect ordering of operations, inadequate or incorrect data cleaning, or incomplete understanding of methods. But the inescapable conclusion is that merely “following the CalTRACK methods” only gets you in the ballpark of replicability.

This is somewhat surprising because CalTRACK is specified in substantially greater detail than IPMVP, UMP, or ASHRAE. We expected to be able to define a set of methods with enough specificity that would allow for little or no implementation variance. However, the process demonstrated that any “compliant” implementation would need to be fully inspected and tested to ensure replicability.

We’ve attempted to solve for some of this implementation variance by providing an open-source implementation of the CalTRACK methods in the OpenEEmeter codebase.

While the OpenEEmeter is a verified representation of the CalTRACK methods for calculating savings, it is only as good as the data that goes into the process. The CalTRACK methods go further and provide a clear set of steps and standard that also include the extracting, transforming, and loading (ETL) process for preparing data for analysis -- there are at least 50 steps in this process that need to be verified for full compliance.

The second major takeaway from the process is that an open-source, collaborative community results in a much better set of methods and methods implementations than those trying to go it alone. Input from M&V experts, stakeholders who are getting paid in a P4P program, utilities responsible to regulators, evaluators responsible to ratepayers, and others involved in the process, proved to be the difference in the success of CalTRACK 2.0.

To this end, we are working on establishing a more permanent open-source framework for the development of the CalTRACK methods. We were excited to see at ACEEE Summer Study the variety of pay-for-performance programs that are basing payments on CalTRACK savings calculations and to learn more about the different implementations of the OpenEEmeter that extend the core savings calculation functionality to solve new challenges.

We look forward to continued growth in the open-source community as stakeholders embrace the value of a common framework for “weights and measures” in the energy efficiency sector.

We are thankful to all those who contributed to the CalTRACK 2.0 Methods and believe this body of work represents a major step forward in our industry’s ability to reliably measure the impact of energy efficiency.