Handling negative congestion in settlement

A validation rule fires because a node cleared at negative $18.40/MWh, someone flags it as bad data, and a well-meaning max(component, 0) gets added to the pricing path — now every counter-flow interval over-charges load and under-pays the generators that were relieving the constraint. Negative congestion is not a defect to be scrubbed; it is the market pricing the value of injecting against a binding limit, and settling it correctly means preserving its sign end to end. This page shows where negative congestion comes from and how to carry it through a Decimal settlement without clamping, as a component of Pricing Logic Implementation, the layer that converts signed market prices into signed financial obligations.

Recall the additive nodal price:

$$LMP_n = \lambda + \mu_n + \nu_n$$

The energy term \(\lambda\) is positive in normal conditions, but the congestion term \(\mu_n\) is signed: at a node whose injection would relieve a binding constraint (counter-flow), \(\mu_n\) is negative, and when it is large enough to overcome \(\lambda\) the whole \(LMP_n\) goes negative. The diagram traces how a binding line limit produces oppositely signed congestion at the two ends of the constraint and how that signed value flows into the charge.

Prerequisites #

• Python packages: the standard-library decimal module for all money arithmetic and pandas for joining priced components to metered volumes. No third-party numeric library is required for the settlement step — the signed prices already exist; the task is to carry them intact.
• Data dependencies: a per-node congestion component that retains its sign as computed from the netted flow-limit duals, the corresponding energy and loss components, the node’s metered imbalance_mwh for the interval (itself signed — net injection versus withdrawal), and the tariff’s rounding rule. If your upstream store has already applied a non-negative constraint to the congestion column, you cannot recover the sign here; fix it at the source.
• Permissions / access: read access to the priced-component store and the metered-volume feed for the interval. Settlement writes to the ledger, so this step needs append scope on the charge table plus the audit log, but never edits a published price.

Implementation #

The settlement function below multiplies each signed price component by the signed metered volume and sums to a net charge, quantizing only at the boundary. The critical rule is negative: there is no max(..., 0) anywhere in the path. A negative congestion component is a legitimate credit to the counter-flow party, and a negative total LMP paired with a negative (withdrawal) volume yields a positive charge — the double sign is arithmetic, not an anomaly to be corrected.

import logging
from decimal import Decimal, ROUND_HALF_EVEN, localcontext

import pandas as pd

logger = logging.getLogger("negative_congestion")
logger.setLevel(logging.INFO)

MONEY = Decimal("0.01")          # settlement rounds to the cent
PRICE_MODE = ROUND_HALF_EVEN     # match the tariff's declared rounding mode


def _money(value: str | float | Decimal) -> Decimal:
    """Quantize to the settlement increment, casting via str so no binary-float
    residue survives. Sign is always preserved — negatives quantize as negatives."""
    with localcontext() as ctx:
        ctx.rounding = PRICE_MODE
        return Decimal(str(value)).quantize(MONEY)


def settle_signed_congestion(priced: pd.DataFrame) -> pd.DataFrame:
    """Settle each node's charge from signed LMP components and signed volume.

    priced columns: node_id, settlement_interval, energy, congestion, loss,
                    imbalance_mwh  (negative = net withdrawal, positive = injection)
    Returns per-node signed charges. No component is ever clamped to zero.
    """
    rows = []
    for rec in priced.itertuples(index=False):
        energy = _money(rec.energy)
        congestion = _money(rec.congestion)   # may be negative — keep it
        loss = _money(rec.loss)
        nodal_lmp = _money(energy + congestion + loss)

        # Volume is signed; a Decimal-typed MWh keeps the product exact.
        volume_mwh = Decimal(str(rec.imbalance_mwh))
        charge = _money(nodal_lmp * volume_mwh)

        # A negative congestion component is a market signal, not an error.
        # Record it for audit rather than suppressing it.
        if congestion < 0:
            logger.info(
                "counter-flow credit at %s interval %s: congestion=%s",
                rec.node_id, rec.settlement_interval, congestion,
            )

        rows.append({
            "node_id": rec.node_id,
            "settlement_interval": rec.settlement_interval,
            "nodal_lmp": nodal_lmp,
            "congestion": congestion,
            "volume_mwh": volume_mwh,
            "charge": charge,
            "counter_flow": congestion < 0,
            "negative_lmp": nodal_lmp < 0,
        })

    ledger = pd.DataFrame(rows)
    logger.info(
        "settled %d nodes | %d counter-flow | %d negative-LMP",
        len(ledger), int(ledger["counter_flow"].sum()),
        int(ledger["negative_lmp"].sum()),
    )
    return ledger

Two invariants make this correct. First, imbalance_mwh is cast to Decimal before the multiply so a signed price times a signed volume never leaks into binary float — a negative-times-negative that should be an exact positive charge stays exact. Second, the only branch on sign is a logging branch: the code observes counter-flow and negative-LMP conditions for the audit trail but never alters the number. The moment you replace that log line with a clamp, you break the zero-sum property of congestion — the positive charges at constrained nodes no longer fund the negative credits at counter-flow nodes, and the market’s congestion rent stops balancing.

Verification #

Confirm signed settlement is intact before the charges post:

1. Congestion sign survives the round trip. Pick an interval with a known binding constraint and assert that the congestion column contains both positive and negative values. An all-non-negative column is proof that a clamp is still hiding upstream.
2. Counter-flow credits net against charges. Within a single binding constraint, the signed congestion rent should approximately balance — sum the congestion-times-volume products across the affected nodes and confirm the total is near the operator’s published congestion rent for that constraint, not inflated by suppressed negatives.
3. Negative-LMP charges have the right sign. For a node with a negative total LMP and a withdrawal (imbalance_mwh < 0), assert charge > 0 — the party is paid to consume, which lands as a credit or a negative charge depending on your sign convention; confirm it matches the tariff’s, and that it did not silently become a debit.
4. Hash the signed batch. Store a SHA-256 over the sorted (node_id, charge) pairs with the market run identifier so a replay reproduces the signed charges exactly and any later clamp is detectable as a digest change.

Compliance Note #

Signed congestion settlement must be validated against the operator’s tariff — PJM Manual 28, MISO BPM-002, CAISO’s Business Practice Manual for Market Operations, or ERCOT Nodal Protocols Section 6 — each of which explicitly permits negative LMPs and defines congestion as a signed rent that balances across a constraint. Suppressing negatives is a tariff violation, not a data-quality safeguard: under FERC’s Open Access rules and ISO/RTO market-design standards, the congestion component must settle at the published signed value so that congestion rent is neither created nor destroyed at the settlement boundary. Persist the signed component, the signed volume, and the resulting charge for every node, route genuinely anomalous prices (outside the tariff’s price floor, typically −$150/MWh) to the exception workflow used across the Settlement Calculation & Validation Engines framework rather than clamping them, and keep the loss inputs that ride alongside congestion versioned through the Loss Factor Mapping Strategies engine.

Frequently Asked Questions #

Is a negative LMP a data error I should filter out? #

Only if it falls below the market’s tariff-defined price floor — otherwise it is a real, settleable price. Negative LMPs occur when must-run or inflexible generation, transmission constraints, and low load coincide so that the system is willing to pay to have energy consumed. The floor (often around −$150/MWh) is the boundary between a valid signal and a suspected bad tick. Settle prices inside the floor exactly and route only floor-breaching values to an exception queue; blanket-filtering every negative price destroys correct settlements.

Why can’t I just clamp the congestion component to zero? #

Because congestion is a zero-sum rent: the positive charges collected at constrained nodes fund the negative credits paid at counter-flow nodes. Clamp the negatives to zero and the constrained-node charges no longer have offsetting credits — you have quietly created congestion rent that the market never collected, over-charging load and under-paying the generators that relieved the constraint. The sign is load-bearing; carry it through the multiply and the sum untouched.

How does a negative price interact with a negative metered volume? #

It multiplies exactly as written: a negative total LMP times a negative (withdrawal) volume is a positive product, which under a standard convention means the withdrawing party is compensated. Keep both operands as Decimal so the double-negative resolves without float rounding, and let the tariff’s sign convention decide whether the result posts as a credit or a negative charge. The arithmetic never needs a special case — special-casing the sign is exactly how clamp bugs get introduced.

Related #

• Calculating locational marginal pricing in Python
• Why does my LMP decomposition not sum to nodal price
• Mapping transmission loss factors to settlement nodes
• Pricing Logic Implementation