Case Study
CAISO Measures the True Impact of Demand Response with Recurve
With Recurve developing advanced comparison group methodology, CAISO gained a comprehensive, auditable view of real-world performance
Company Overview
The California Independent System Operator (CAISO) has run the state's wholesale electricity market since 1998. A non-profit public benefit corporation operating under a FERC-approved tariff, CAISO manages the high-voltage grid and energy markets that serve roughly 32 million consumers across 26,000 circuit miles of transmission lines.
When a recent extreme heatwave drove record demand, CAISO called on demand response resources to help manage manage rotating outages. Afterward, it found that the industry measured demand response performance in at least four distinct ways. Without a common method, no one could tell whether a load impact figure reflected true program performance or just the measurement approach behind it.
CAISO commissioned Recurve to test and refine an open-source comparison group methodology, now part of OpenDSM, and determine whether it could serve as a tariff-compliant option for settling demand response dispatches statewide.
Quick Facts
Program Type(s)
Demand Response (Supply-Side, IOU & CCA)
Location
California
Study Scope
38 DRP/LSE/event combinations, 11 climate zones, 24,000+ customers
Results
Positive savings in all 38 events, with 15 cutting event-period load by more than 20%
Challenge
Existing Baseline Methods Broke Down During Extreme Heat
CAISO's tariff relied on "X of Y" baseline methods like 10 of 10, which average pre-event day usage and a capped “same-day adjustment.” During the August heatwave, that cap proved too restrictive and regulators subsequently lifted it temporarily. Capped or uncapped, these methods carry structural problems: they don't account for rebound effects, which may occur throughout the event day, they inherently assume the same composition of end uses during the midday and evening periods, and they can penalize results when customers adopt permanent load management strategies.
Variation in Measurement Methods Created Uncertainty
When results depend on methodological choices that differ by program, party, or use case, it is difficult to compare actual performance. Increasingly, for demand response to become a scalable, reliable peak energy resource, measurement methods need to be consistent and transparent, and the results need to be accurate. CAISO turned to Recurve to help establish methods that could meet those objectives for the California market.
Limited Access to Non-Participant Data
OpenDSM-style comparison group methods depend on individual-meter, non-participant data to build matched comparison groups. Third parties, including most demand response providers, typically lack the infrastructure or data access to build this kind of comparison group on their own, and data privacy requirements add another layer of complexity to any centralized approach.
Solution
Recurve's platform securely ingests and structures meter-level data for both participating and non-participating customers, applying OpenDSM methodology consistently to each event. Because OpenDSM is open source and fully transparent, CAISO and demand response providers can inspect each step and see how the results were produced. Recurve applied this approach to 38 distinct demand response events called during the August heatwave, spanning Investor-Owned Utility and Community Choice Aggregator territories, 11 climate zones, and more than 24,000 residential and commercial customers across multiple demand response providers.
Meter-Level Baseline Modeling
OpenDSM starts with a weather-normalized hourly model that generates a baseline prediction, or counterfactual, for every meter in both the treatment and comparison groups.
Comparison Group Matching
Event-day load impacts are then calculated as the difference between the treatment group's observed and predicted usage, adjusted for the same difference measured in the matched comparison group. This adjustment removes both model error and the effects of factors unrelated to the program, like public conservation messaging during power outages.
Fig. 1: The average weekday (left) and weekend (right) load shape of a meter in the treatment group (green), comparison pool (blue), and comparison group (grey).
A Complete Picture, Hour by Hour
OpenDSM measures savings in every hour, both during events and around them. That gave CAISO visibility into what happened before and after each dispatch, which settlement baselines had never shown.
After reviewing the results, CAISO confirmed its tariff-compliance in an official letter accompanying the report.
Results
Consistent Positive Savings Across a Volatile Grid Event
All 38 DRP/LSE/event combinations studied produced event-period savings during the heatwave. Of those, 35 delivered load reductions greater than 5%, 25 delivered more than 10%, and 15 delivered more than 20%. Three of the five program groups reduced total electricity consumption on event days by more than 2.5%.
What Standard Baselines Miss
In a residential case study of non-solar customers, OpenDSM measured nearly 8% savings in the first event hour. Savings then tapered as the event continued, and by the fourth event hour they had dropped to roughly half of the first. The analysis also revealed a rebound effect. Usage ran slightly above baseline (less than 1%) in hours more than two hours from an event, then rose to about 2% above baseline in the two hours before and after it. “X of Y” baselines typically estimate savings only during the event, so they don't measure this pattern. And when a same-day adjustment draws on hours affected by rebound, event-period savings can be biased high.
Fig. 2: Group A3, Non-Solar: Average participant hourly savings during the indicated hour types (x-axis). Participant counts for each category are shown as dots and refer to the right-hand axis.
Different Sectors, Different Response Patterns
A separate non-residential case study of 138 large commercial and industrial customers showed a different shape entirely: savings held up for the full duration of five-hour events with no rebound, and some hours immediately around the event even showed small positive savings, raising a question CAISO could now investigate directly, whether load was being shed or shifted to another day.
A Tariff-Compliant Path Forward
Because OpenDSM is built on open-source code, CAISO gained a measurement approach it could inspect and audit, not a black-box result. With the tariff-compliance question resolved, CAISO can extend consistent, revenue-grade measurement to future demand response settlement, and other state agencies, including the CEC and CPUC, can draw on the same methodology as they take up resource adequacy and summer reliability planning.
About the Report
This case study is adapted from “Demand Response Advanced Measurement Methodology: Analysis of Open-Source Baseline and Comparison Group Methods to Enable CAISO Demand Response Resource Performance Evaluation,” the report prepared by Recurve for CAISO in February 2022. The report details the full methodology and results from the August 2020 heatwave. It refers to the methodology by its previous name, FLEXmeter, now known as OpenDSM. You can learn more about Recurve's transparent approach on our Open Source page.