Settlement Cycle Mapping

A preliminary charge posts to a July fiscal period, the final run arrives 30 days later carrying a revised loss factor, and unless the two are bound to the same operating-day key the true-up lands in the wrong month and the close breaks. That is the failure mode this component prevents: intervals that drift between accounting periods because external settlement timelines were never deterministically mapped to internal ones. Within the Core Architecture & Market Taxonomy for Energy Settlements framework, settlement cycle mapping is the temporal-alignment layer sitting between raw feed ingestion and the reconciliation engine — it takes each ISO/RTO, pipeline, and exchange settlement run and assigns every MWh and $/MWh to a canonical (operating_day, interval_index) bucket before any downstream posting occurs. Get the mapping wrong and traders see unexplained P&L drift, settlement analysts inherit unreconciled variance buckets, and utility operations miss FERC-mandated reporting deadlines. Get it right and preliminary, final, and long-horizon true-up runs all collapse onto the same key, so a revision is a delta against a known accrual rather than a mystery re-post.

Temporal Alignment Architecture #

The diagram below shows how a single operating day flows through the staggered settlement windows — from preliminary cash flows to final settlement and long-horizon true-ups — with variances routed to an exception queue for resolution before each statement is finalized.

Energy markets operate on overlapping, non-aligned settlement cycles. Day-Ahead (DA) markets clear roughly 24 hours before physical delivery, Real-Time (RT) markets settle on 5-minute or 15-minute intervals, and final settlements publish 30 to 55 days post-delivery following meter validation, loss-factor application, and regulatory adjustments. Preliminary settlements act as provisional cash flows subject to true-up. Mapping these external cycles onto internal fiscal periods requires a deterministic time-windowing engine that normalizes UTC timestamps, applies daylight-saving (DST) transition rules, and aligns interval boundaries to each market’s operating-day definition.

For instance, CAISO defines its operating day from 00:00 to 24:00 Pacific Time, while NYISO applies Eastern Time. Both must handle the daylight-saving transitions that occur at 02:00 local time: the spring-forward day has 23 hours (the 2 a.m. hour is skipped) and the fall-back day has 25 hours (the 1 a.m. hour repeats), so interval indexing cannot assume a fixed 24 hours per operating day. Expressed as an interval count for a market with fixed sub-hourly granularity, the number of settlement intervals per operating day is

$$N_{intervals} = H_{day} \times \frac{60}{g}$$

where \( H_{day} \in {23, 24, 25} \) is the number of clock hours in the local operating day and \( g \) is the interval granularity in minutes (5 for RT, 60 for DA/hourly-integrated). A mapping engine that hard-codes \( H_{day} = 24 \) will silently truncate or double-count the DST boundary hour, and that error propagates straight into imbalance and settlement totals.

This alignment layer feeds the ETRM System Architecture downstream: once normalized, mapped cycles synchronize with physical delivery schedules, financial contracts, and hedge positions, translating external settlement intervals into internal position identifiers of the form contract_id | node_id | interval_start. That translation is what enables automated P&L attribution, margin-call calculation, and counterparty-exposure tracking.

Specification & Standards Reference #

Cycle mapping is not a free design choice — the run cadence, revision windows, and interval granularity are fixed by tariff and business-practice manuals, and the mapping engine must be versioned against them. Each RTO publishes its settlement timeline and charge-code cadence in a business practice manual (BPM): PJM Manual 28 (Operating Agreement Accounting), CAISO’s Business Practice Manual for Settlements and Billing, ERCOT’s Nodal Protocols Section 9, and SPP’s Market Protocols. Gas nomination cycles follow NAESB WGQ Standard 1.3 (the timely, evening, and intraday nomination windows anchored to the 09:00 gas day). FERC Form 1 and the Electric Quarterly Report (EQR) define what settled data a participant must retain and report, and those retention obligations dictate that every mapped interval carry an auditable lineage back to its source run.

The practical consequence is that a mapping table must be effective-dated against a jurisdiction: the same charge code can change cadence across a tariff revision, so the engine resolves the applicable timeline by (iso, tariff_version, effective_date) rather than assuming today’s semantics hold for a back-dated interval. Adherence to the ISO/RTO Data Format Standards is what makes the raw run parseable in the first place; cycle mapping then layers the temporal contract on top of that structural contract.

The table below is the canonical cycle-window reference the engine encodes — the publication offsets and ledger treatment that every mapped interval is classified against.

Step-by-Step Cycle Mapping Implementation #

The mapping engine runs as a deterministic sequence: normalize the timestamp, resolve the operating-day window, validate the record against the schema, assign the internal position key, then post idempotently. Financial amounts use the decimal module throughout — never float — because charges are quantized to the cent and summed across thousands of intervals, where binary floating-point drift eventually flips a rounding boundary and breaks reconciliation.

1. Normalize the timestamp to an operating-day boundary #

Every record is anchored to a UTC-aware timestamp at ingest, then snapped to the market’s local operating-day window. zoneinfo handles the DST arithmetic; explicit boundary clipping guards the transition days.

import logging
from zoneinfo import ZoneInfo
from datetime import datetime
from typing import Literal

logger = logging.getLogger(__name__)

CycleType = Literal["DA", "RT", "PRELIM", "FINAL"]

def normalize_settlement_window(
    timestamp_utc: datetime,
    market_timezone: str,
    cycle_type: CycleType,
) -> datetime:
    """Snap a UTC timestamp to the settlement-window boundary for a market and run type."""
    try:
        tz = ZoneInfo(market_timezone)
    except Exception as exc:
        logger.error("Invalid timezone '%s': %s", market_timezone, exc)
        raise ValueError("Unsupported market timezone") from exc

    local_dt = timestamp_utc.astimezone(tz)

    if cycle_type == "RT":
        interval_minutes = 5
        snapped_minute = (local_dt.minute // interval_minutes) * interval_minutes
        return local_dt.replace(minute=snapped_minute, second=0, microsecond=0)

    # DA, PRELIM and FINAL all key to the operating-day boundary (local midnight).
    return local_dt.replace(hour=0, minute=0, second=0, microsecond=0)

2. Enumerate the operating day, DST-aware #

Rather than assume 24 hours, build the interval index by walking real local time so the skipped and repeated hours are handled correctly.

from datetime import timedelta

def operating_day_intervals(operating_day: datetime, granularity_min: int) -> list[datetime]:
    """Return every interval start in a local operating day, honouring 23/24/25-hour DST days."""
    tz = operating_day.tzinfo
    start = operating_day.replace(hour=0, minute=0, second=0, microsecond=0)
    next_day = (start + timedelta(days=1)).astimezone(tz)  # true local next-midnight
    intervals, cursor = [], start
    while cursor < next_day:
        intervals.append(cursor)
        cursor = (cursor + timedelta(minutes=granularity_min)).astimezone(tz)
    return intervals

3. Validate the record before it enters the mapping layer #

Raw settlement files arrive as CSV, XML, EDI 867, and proprietary binaries. Strict schema validation at ingestion prevents silent corruption; malformed records are quarantined rather than propagated. The field contract mirrors the one enforced by the Schema Validation Frameworks on the trade side, so the model is shared, not duplicated.

from decimal import Decimal
from pydantic import BaseModel, field_validator

class SettlementRecord(BaseModel):
    node_id: str
    interval_start: datetime          # tz-aware, UTC-anchored at ingest
    settled_mwh: Decimal
    lmp_usd_mwh: Decimal              # may be negative — congestion/oversupply
    run_type: CycleType

    @field_validator("settled_mwh")
    @classmethod
    def volume_non_negative(cls, v: Decimal) -> Decimal:
        if v < 0:
            raise ValueError("settled_mwh cannot be negative")
        return v

4. Assign the internal position key and compute the amount #

The mapping engine translates the external interval into an internal position identifier and computes the settlement amount with quantized Decimal arithmetic.

from decimal import ROUND_HALF_UP

CENTS = Decimal("0.01")

def map_to_position(rec: SettlementRecord, market_timezone: str) -> dict:
    interval = normalize_settlement_window(rec.interval_start, market_timezone, rec.run_type)
    amount = (rec.settled_mwh * rec.lmp_usd_mwh).quantize(CENTS, rounding=ROUND_HALF_UP)
    return {
        "position_key": f"{rec.node_id}|{interval.isoformat()}|{rec.run_type}",
        "operating_day": interval.date().isoformat(),
        "amount_usd": amount,
    }

5. Post idempotently against the run key #

Reprocessing a settlement file must never duplicate postings or overwrite a finalized true-up without an explicit audit trail. Posting is keyed on the run-specific idempotency key from the reference table, and a later run posts as a delta against the prior accrual.

def post_delta(store, position_key: str, run_seq: int, new_amount: Decimal) -> Decimal:
    """Append-only, idempotent delta posting. Re-delivery of the same run is a no-op."""
    prior = store.latest_amount(position_key)          # None on first post
    if store.already_posted(position_key, run_seq):
        return Decimal("0.00")                         # exactly-once: redelivery is a no-op
    delta = new_amount - (prior or Decimal("0.00"))
    store.append(position_key, run_seq, delta=delta, absolute=new_amount)
    return delta

Edge Cases & Failure Modes #

The mapping engine earns its keep on the pathological days, not the clean ones. Each of the following must have explicit handling code rather than an implicit assumption.

• DST spring-forward (23-hour day): the 02:00 local hour does not exist. operating_day_intervals skips it because zoneinfo never materializes it, so the index runs one hour short. Any code that pre-allocates a 24-slot array must instead consume the generated list length.
• DST fall-back (25-hour day): the 01:00 local hour repeats. Two distinct UTC instants map to the same wall-clock hour, so the position key must retain the UTC-anchored interval_start — keying on wall-clock time alone collapses the two hours and double-books one of them.
• Negative LMPs: congestion and oversupply routinely drive nodal prices below zero, so a negative lmp_usd_mwh is valid market data, not a malformed record. Only volumes are non-negative — which is why the validator in step 3 constrains settled_mwh and never lmp_usd_mwh. This is the same discipline the Pricing Logic Implementation enforces when decomposing nodal price into energy, congestion, and loss components.
• Zero-volume intervals: a curtailed or offline node reports settled_mwh = 0. The interval is still real and must map to a bucket (amount 0.00), because a missing interval and a zero interval mean different things at reconciliation — the former is a gap to chase, the latter is a settled fact.
• Stale telemetry / late publication: feeds are subject to latency and outages. Automated retry with exponential backoff, provisional posting from an RT-to-DA proxy, and manual override prevent bottlenecks. When a fallback route activates, affected records are flagged, downstream financial approvals are restricted, and the provisional values reconcile automatically once authoritative data arrives.
• Schema drift: a tariff revision renames a charge code or shifts a run’s cadence. Because the engine resolves the timeline by (iso, tariff_version, effective_date), a drifted record fails the effective-dated lookup and quarantines rather than mis-mapping to the current semantics.
• Multi-ISO and gas-electric coupling: portfolios spanning multiple balancing authorities require normalizing temporal offsets and interval granularities before variance analysis. Gas nomination cycles (gas day 09:00–09:00 Central) rarely align with electric intervals, so an explicit bridge table translates gas-day boundaries into the corresponding electric DA/RT intervals — critical for fuel-cost recovery and cross-commodity P&L. The PJM-specific hour-24 conventions and LMP adjustments are worked end-to-end in How to map PJM settlement cycles to internal ledgers.

Threshold & Alerting Configuration #

Not every variance between a preliminary accrual and a final run warrants a human. The engine classifies each mapped delta against configurable tiers and routes accordingly, feeding the same alerting fabric described in Threshold Tuning & Alerts.

def classify_delta(delta: Decimal, run_total: Decimal) -> str:
    pct = abs(delta) / run_total if run_total else Decimal("1")
    if abs(delta) <= Decimal("50") or pct <= Decimal("0.005"):
        return "info"
    if abs(delta) <= Decimal("5000") or pct <= Decimal("0.02"):
        return "warn"
    return "break"

Thresholds are parameters, not constants — they are effective-dated per ISO and per charge type, because a $5,000 break on an ancillary-services run and on a day-ahead energy run carry very different materiality.

Testing & Reconciliation Verification #

Cycle mapping is verified by shadow calculation: an independent implementation recomputes each mapped interval from the raw run and the delta stream must net to the ISO’s published statement total to the cent. The DST boundary days and the negative-price case are the non-negotiable unit tests — they are where naive engines silently fail.

from decimal import Decimal
from zoneinfo import ZoneInfo
from datetime import datetime

def test_fall_back_day_has_25_hours():
    ct = ZoneInfo("America/Chicago")
    day = datetime(2026, 11, 1, tzinfo=ct)  # US fall-back
    intervals = operating_day_intervals(day, granularity_min=60)
    assert len(intervals) == 25            # the 01:00 hour repeats

def test_spring_forward_day_has_23_hours():
    ct = ZoneInfo("America/Chicago")
    day = datetime(2026, 3, 8, tzinfo=ct)  # US spring-forward
    intervals = operating_day_intervals(day, granularity_min=60)
    assert len(intervals) == 23            # the 02:00 hour is skipped

def test_negative_lmp_is_accepted():
    rec = SettlementRecord(
        node_id="AECO", interval_start=datetime(2026, 6, 1, 3, tzinfo=ZoneInfo("UTC")),
        settled_mwh=Decimal("40"), lmp_usd_mwh=Decimal("-12.50"), run_type="RT",
    )
    out = map_to_position(rec, "America/New_York")
    assert out["amount_usd"] == Decimal("-500.00")

def test_redelivery_is_idempotent(store):
    key, seq = "PJM_WEST|2026-06-01T00:00:00-04:00|FINAL", 3
    first = post_delta(store, key, seq, Decimal("1200.00"))
    second = post_delta(store, key, seq, Decimal("1200.00"))  # same run re-sent
    assert second == Decimal("0.00")       # exactly-once guarantee holds

Access to the mapped ledger — and to the raw runs feeding it — is governed by the RBAC and audit controls of the Security & Access Boundaries: only settlement analysts, risk managers, and compliance officers may view or approve mapped cycle data, every access event is written to an immutable audit store, and those controls satisfy the SOX, FERC, and NERC CIP traceability requirements that make the shadow reconciliation defensible under audit.

Frequently Asked Questions #

Why can’t the mapping engine assume 24 hours per operating day? #

Because two days a year do not have 24 hours. Under daylight-saving transitions the spring-forward operating day has 23 clock hours (the 02:00 hour never occurs) and the fall-back day has 25 (the 01:00 hour repeats). Interval indexing must be generated from real local time via zoneinfo, so the index is 23, 24, or 25 hours long as appropriate. A hard-coded 24-hour loop truncates or double-counts the boundary hour, and that error flows straight into imbalance and settlement totals.

How does cycle mapping keep a preliminary and a final run reconciled? #

Both runs collapse onto the same (node_id, operating_day) key, so the final run posts as a signed delta against the preliminary accrual rather than as a gross re-post. Posting is idempotent on a run-specific key, meaning re-delivery of the same run is a no-op, and each delta carries its inputs and a UTC timestamp to an append-only store — so a true-up months later is a further delta against a known, auditable accrual.

Should the mapping layer ever reject a negative settlement price? #

No. Congestion and oversupply routinely push nodal prices below zero, so a negative LMP is legitimate market data. Only volumes are non-negative. Validation constrains the settled MWh and never the price field; enforcing a non-negative price silently discards real intervals and understates congestion cost.

How are gas nomination cycles mapped to electric settlement intervals? #

Through an explicit bridge table. The NAESB gas day runs 09:00–09:00 Central and its nomination cycles do not align with electric DA/RT interval boundaries, so gas-day boundaries are translated into the corresponding electric intervals before any cross-commodity variance analysis. This alignment is what makes fuel-cost recovery, heat-rate optimization, and gas-electric P&L attribution reconcilable on a single timeline.

Related #

• ETRM System Architecture
• ISO/RTO Data Format Standards
• Security & Access Boundaries
• How to map PJM settlement cycles to internal ledgers
• Threshold Tuning & Alerts