Why does my LMP decomposition not sum to nodal price

Your three-part components reconcile perfectly on paper — energy, congestion, and loss — yet when you add them and compare against the ISO’s published nodal LMP the residual sits at a stubborn few cents, and every downstream charge inherits the break. This page walks the four faults that produce that residual and gives you a Decimal reconciliation diff that pins the cause to a specific component rather than leaving you to eyeball a spreadsheet. It is a debugging step within Pricing Logic Implementation, the layer that must prove every priced obligation traces back to a published market price before settlement acts on it.

The additive identity you are trying to satisfy is the standard node-level decomposition:

$$LMP_n = \lambda + \mu_n + \nu_n$$

where \(\lambda\) is the system energy price common to every node, \(\mu_n\) is the congestion component at node \(n\) (the line-limit duals mapped back through the shift factors), and \(\nu_n\) is the marginal-loss component. If your reconstructed \(LMP_n\) does not equal the published price, exactly one of those three terms — or the arithmetic that combines them — has drifted. The diagram below shows where each of the four common faults injects its error into that identity.

Prerequisites #

• Python packages: the standard-library decimal module (no third-party dependency for the arithmetic itself), pandas for joining computed components against the published feed on a shared key, and whatever client you already use to pull the operator’s nodal LMP export (PJM Data Miner 2, MISO Market Reports, CAISO OASIS, or ERCOT MIS).
• Data dependencies: your four computed vectors keyed by (node_id, settlement_interval) — energy_component, congestion_component, loss_component, and the total you persisted — plus the operator’s published nodal_lmp for the identical interval and market run. You also need the rounding rule the tariff fixes (increment and mode) and the effective timestamp of the PTDF and marginal-loss-factor snapshots your solver consumed.
• Permissions / access: read-only credentials for the market portal to pull the published prices. Reconciliation never writes to the market; it only reads, compares, and routes breaks to an exception queue.

Implementation #

The reconciliation harness below isolates which of the four faults is live. It re-quantizes every component with the tariff’s exact rounding rule, recomputes the sum in Decimal space, and reports a per-component residual against the published price so you can read the failing term directly instead of guessing. The four faults it disambiguates:

import logging
from decimal import Decimal, ROUND_HALF_EVEN, localcontext
from typing import Iterable

import pandas as pd

logger = logging.getLogger("lmp_reconcile")
logger.setLevel(logging.INFO)

# The tariff fixes both the increment and the rounding mode. PJM and MISO
# settle to the cent with banker's rounding; do NOT assume ROUND_HALF_UP.
SETTLEMENT_INCREMENT = Decimal("0.01")
SETTLEMENT_MODE = ROUND_HALF_EVEN


def _q(value: str | float | Decimal) -> Decimal:
    """Quantize a single value to the tariff increment. Cast via str to avoid
    inheriting binary-float noise (Decimal(0.1) != Decimal('0.1'))."""
    with localcontext() as ctx:
        ctx.rounding = SETTLEMENT_MODE
        return Decimal(str(value)).quantize(SETTLEMENT_INCREMENT)


def reconcile_decomposition(
    components: pd.DataFrame,   # cols: node_id, settlement_interval, market_run,
                               #       energy, congestion, loss, snapshot_ts
    published: pd.DataFrame,    # cols: node_id, settlement_interval, market_run,
                               #       nodal_lmp, se_run_ts
    tol: Decimal = Decimal("0.01"),
) -> pd.DataFrame:
    """Return per-node residuals that attribute a break to a specific fault.
    Every arithmetic step stays in Decimal so the diff is exact, never float."""

    key = ["node_id", "settlement_interval", "market_run"]

    # Fault 4 guard: a silent DA/RT mismatch is the most expensive miss because
    # it looks like a pricing bug. Refuse to reconcile across market runs.
    run_mix = set(components["market_run"]) ^ set(published["market_run"])
    if run_mix:
        raise ValueError(f"market-run mismatch (DA vs RT?) on: {sorted(run_mix)}")

    merged = components.merge(published, on=key, how="inner", validate="one_to_one")

    rows = []
    for rec in merged.itertuples(index=False):
        energy = _q(rec.energy)          # Fault 1: re-round with tariff mode
        congestion = _q(rec.congestion)  # Fault 2: sign preserved as computed
        loss = _q(rec.loss)
        reconstructed = _q(energy + congestion + loss)
        published_lmp = _q(rec.nodal_lmp)
        residual = reconstructed - published_lmp

        # Fault 3: flag when the solver's shift-factor/MLF snapshot predates the
        # state-estimator run behind the published price.
        stale_snapshot = rec.snapshot_ts < rec.se_run_ts

        rows.append({
            "node_id": rec.node_id,
            "settlement_interval": rec.settlement_interval,
            "reconstructed": reconstructed,
            "published_lmp": published_lmp,
            "residual": residual,
            "energy": energy,
            "congestion": congestion,
            "loss": loss,
            "stale_snapshot": stale_snapshot,
            "breach": abs(residual) > tol,
        })

    result = pd.DataFrame(rows)
    breaches = int(result["breach"].sum())
    if breaches:
        logger.warning("decomposition breaks on %d of %d nodes", breaches, len(result))
    else:
        logger.info("decomposition ties to published LMP across %d nodes", len(result))
    return result

The two design choices that matter: casting through str() before Decimal() so no component carries binary-float residue into the sum, and quantizing with the tariff’s declared mode inside a localcontext block so the reconciliation cannot accidentally use the interpreter default. If the operator settles with banker’s rounding and your solver used ROUND_HALF_UP, roughly half your nodes drift by a cent in a consistent direction — Fault 1’s fingerprint, and the one people misdiagnose as a solver bug most often.

Verification #

Confirm the diff is trustworthy before you act on it:

1. Row count survives the join. The validate="one_to_one" merge raises if a (node_id, settlement_interval, market_run) key is duplicated or unmatched — a length drop between components and result means a key collision, not a pricing break. Fix the join before reading any residual.
2. Clean case is exactly zero. On an interval you know reconciles, every residual must be Decimal("0.00"), not 1e-9. A non-zero float-shaped residue means a component slipped past the str() cast and is still carrying binary noise.
3. Fault attribution is consistent. Read the breach pattern against the table above: one sign across many nodes points at rounding mode; sign flips at constrained nodes point at the congestion dual; localized breaks near a topology change point at a stale snapshot; a whole-interval offset with stale_snapshot=False points at a DA/RT mix.
4. Hash the reconciled batch. Concatenate the sorted (node_id, residual) pairs and store a SHA-256 digest with the market run identifier so a later replay proves the same interval reconciled the same way.

Compliance Note #

Reconciliation must be validated against the operator’s tariff-defined pricing methodology — PJM Manual 11 and Manual 28, MISO BPM-002, CAISO’s Business Practice Manual for Market Operations, or ERCOT Nodal Protocols Sections 4 and 6 — each of which fixes the additive decomposition, the loss-factor reference bus, and the rounding increment and mode the diff above must mirror exactly. Under FERC’s Open Access rules and ISO/RTO market-design standards, every priced charge must trace to a metered interval, a market run identifier, and a published price version, so persist the residual, the snapshot timestamps, and the rounding mode for every node. A break that clears only because you loosened tolerance is a finding, not a pass: route it to the exception workflow used across the Settlement Calculation & Validation Engines framework and keep the loss inputs versioned through the Loss Factor Mapping Strategies engine rather than interpolating across a topology change.

Frequently Asked Questions #

My components sum correctly but still miss the published price — what now? #

If \(\lambda + \mu_n + \nu_n\) is internally consistent yet off the published LMP, the break is not in your arithmetic — it is in an input. Check the snapshot timestamps first: a PTDF or marginal-loss-factor file that predates the state-estimator run behind the published interval mis-prices exactly the nodes near the changed topology while leaving the rest clean. If the offset is uniform across the whole interval instead, you are almost certainly reconciling a day-ahead computation against a real-time published price, or vice versa.

How do I tell a rounding bug from a real pricing error? #

A rounding-mode mismatch has a distinctive signature: a small, stable residual — a fraction of a cent up to one cent — with the same sign across a large share of nodes, independent of whether they are congested. A genuine pricing error moves with the physics: it concentrates on constrained nodes, flips sign between import- and export-constrained buses, or scales with the loss factor. Re-quantize every component with the tariff’s exact increment and mode inside a Decimal context; if the residual collapses to zero, it was rounding.

Should I clamp the residual to zero when it is under a cent? #

No. Silently absorbing a sub-cent residual hides the very drift you need to catch — a systematic half-cent bias compounds across thousands of nodes and every settlement interval into a real dollar exposure. Record the exact Decimal residual, alert when it exceeds the tariff tolerance, and fix the responsible input. Clamping turns an auditable reconciliation into an unfalsifiable one.

Related #

• Calculating locational marginal pricing in Python
• Handling negative congestion in settlement
• Mapping transmission loss factors to settlement nodes
• Pricing Logic Implementation