PolicyForge v0.7.0 tutorial
Tutorial · about 20 minutes

Your first stress test

You will write a sourcing policy as a typed rule with a mandate. Then you will run it against the current policy on the same 300 futures, and read what the numbers say about it. Everything here runs on your own machine.

Where things stand. PolicyForge's first screen, the Network Studio, builds and checks networks; the screens for writing and testing policies come in P7. In this tutorial you write the rule yourself, in Python, exactly as the compiler produces it from plain language. To see the compiler at work, run python -m examples.compile_intent: it compiles this tutorial's intent offline from a recorded answer, or any intent live with --live and an Anthropic API key.

1 · Set up

You need Python 3.12 and the code from the repository. uv installs Python 3.12 for you if you don't have it.

git clone https://github.com/kathuman/claude-projects.git
cd claude-projects/policyforge/backend
uv venv --python 3.12 .venv
uv pip install --python .venv -e ".[dev]"
.venv/bin/python -m pytest -q          # Windows: .venv\Scripts\python

You should see 179 tests pass in about five minutes. They include the checks that the simulator conserves stock and matches theory, so a green run means the engine you are about to use is sound.

2 · Meet FoodNet

FoodNet is a fictional food manufacturer. Material flows from suppliers to plants, then to distribution centres, then to markets. One period is a week.

SupplierCharacterCostLead timeFill rateCapacity
Acheap, slow, reliable0.90×21–24 days99 %6,000 /wk
Bmid-priced, variable1.00×12–15 days, CV 0.3593 % ± 66,000 /wk
Cexpensive, fast, small1.30×3–4 days98 %2,000 /wk

Cost is a multiplier on each SKU's unit cost. Fill rate is the share of each order actually shipped; the rest is never delivered.

Two plants make 10 SKUs. Two of them are chilled, with a shelf life of 21 and 28 days. Total demand is about 8,350 units a week. Normally the plants buy from A and B. C has no regular volume: it is the backup. The FoodNet README explains every field.

To see it, open the Network Studio, which starts on FoodNet. The diagram shows how material flows, the map shows where everything is, and the tables hold every number. Click any node or lane to see its fields with their units and bounds.

# in a Python shell, from backend/
from core.domain.network import load_foodnet
net = load_foodnet()              # reads and validates data/foodnet/network.yaml
net.demand_mean("MKT_N1", "S04")  # 440.0 units/week of dry pasta in North 1

3 · Write the policy

This is the intent, as a planner would say it:

"If Supplier B misses more than 10 % of orders for three weeks running, shift ambient volume to Supplier C, don't switch back for a month, and the sourcing agent can do this alone up to 30 % of volume."

As a rule in the DSL, from examples/first_stress_test.py:

fill_b = VarRef(var=Variable.supplier_fill_rate, supplier="SUP_B")

Policy(
    id="b-to-c-switch",
    scope=Scope(categories=["ambient"]),               # ambient SKUs only
    rule=SourcingSwitch(
        primary="SUP_B", secondary="SUP_C",
        trigger=Condition(var=fill_b, op="lt", threshold=0.90, for_periods=3),
        switch_back=Condition(var=fill_b, op="ge", threshold=0.90, for_periods=2),
        shift_fraction=0.30,                           # move 30 % of B's volume
        min_dwell_periods=4,                           # "don't switch back for a month"
    ),
    mandate=Mandate(
        owner="Planning manager", approver="Head of supply chain",
        agent_id="sourcing-agent-01",
        autonomy=[AuthorityLimit(kind=LimitKind.volume_shift_pct, value=30)],
        ...                                            # expires_at: 90 days by default
    ),
    ...
)

Four things to notice:

The incumbent, the rule running today, is plain base stock: order up to 10 days of cover at each DC (12 for chilled SKUs) and 35 days at each plant. It meets the constitution in normal times. The candidate is the incumbent plus the switch. Comparing the two isolates exactly what the switch adds.

4 · Pick the futures

A policy reacts to whatever happens, so you test it against futures that differ. This tutorial uses four of the eight scenarios in the default suite:

ScenarioWhat happens
baselineSeasonal, random demand; normal lead times and fill rates.
supplier-b-outageSupplier B loses 80 % of its capacity from week 20 for 8 weeks.
correlated-shockIn half the futures, A and B both lose 60 % of capacity, their lead times rise 50 %, and North and Central demand jumps 20 %, all at once, for 8 weeks.
lead-time-inflationSupplier B's lead times double, creeping up over 10 weeks from week 20.

Each scenario is a stack of components, so you can build your own: an outage and a promotion, for example.

5 · Run it

python -m examples.first_stress_test            # 300 futures × 104 weeks per scenario, about 10 s

The heart of it is three lines per scenario:

draws = generate(lay, scenario, R=300, T=104, seed=42)      # every random draw, made up front
incumbent = simulate(lay, draws, compile_program(lay, incumbent_policies))
candidate = simulate(lay, draws, compile_program(lay, candidate_policies))

Both policies run on the same draws: the same demand, the same lead times, the same supplier failures, week by week. This is called common random numbers. Any difference in the results comes from the policy, not from luck. Run it twice with the same seed and you get identical numbers.

6 · Read the result

The output, with seed 42:

ScenarioPolicyFillFill P5Chilled fillCost €/weekSwitches
baselineincumbent98.2 %97.8 %98.5 %30,6720.0
candidate98.2 %97.8 %98.5 %30,7634.1
supplier-b-outageincumbent98.0 %97.6 %98.4 %30,5760.0
candidate98.2 %97.8 %98.5 %30,7645.4
correlated-shockincumbent94.1 %88.1 %96.9 %31,0200.0
candidate96.6 %92.8 %97.4 %30,9774.6
lead-time-inflationincumbent96.7 %95.2 %98.4 %30,1790.0
candidate96.9 %95.6 %98.4 %30,2614.1

How to read it: Fill is the share of demand met from stock in the week it arose, averaged over the 300 futures; weeks 0–7 are left out while the system settles. Fill P5 is the 5th percentile: the fill rate of the worst 5 % of futures. Cost covers purchase, production, transport, holding, ordering, expediting, penalties and spoilage. Switches is the number of sourcing switches per future over two years.

What the numbers say

This is the point of evaluating a policy rather than a plan: the same rule looks almost free in one future, essential in another, and blind in a third. You see all three before it goes live.

7 · Get the verdict

The quick table above is your own reading. PolicyForge also evaluates a candidate formally, against the constitution and three baselines, on all eight scenarios:

python -m examples.evaluate_report              # 500 futures per scenario, about a minute

It writes docs/example-report.md (for people) and docs/example-report.json (every distribution, sample and fan chart series, for the interface to come). The report holds:

For this switch, the verdict is approve.

Rolling optimisation is the instructive comparator. It runs leaner than the incumbent, and the worst futures expose it: it misses the chilled target in 94 % of futures even in normal times. A plan optimised for the expected future has no answer to the unexpected one.

8 · Try it yourself

  1. Exceed the mandate. Set shift_fraction=0.6 but leave the mandate at 30 %. The simulator blocks the extra volume and counts an escalation each time the agent wants more than it may do alone. Pass SimConfig(on_limit="flag") to simulate and the extra volume moves anyway, flagged for review.
  2. Catch the slow supplier. Wrap the switch in a Composite with a second SourcingSwitch triggered by observed_lead_time above 21 days. Does lead-time creep improve?
  3. Make the incumbent leaner. Replace it with background_policies(lay, dc_days=7, plant_days=21). Fill falls to about 69 % even in normal times. 21 days at a plant does not even cover Supplier A's 21–24-day lead time plus a week of review, so the plant keeps running out and a backlog builds. Backlog is served first, so new demand waits. The order-up-to level has to exceed demand over the lead time, with room to spare.
  4. Add an expedite trigger on the DCs: ExpediteTrigger(condition=CoverageBelow(days=3), mode="express_road", cost_cap_per_period=4000). How much of the shock's damage does expediting buy back, and at what cost? Run the report and check whether the €20,000-a-month expedite budget holds.
  5. Break FoodNet in the Studio. Start the API (.venv/bin/python -m uvicorn api.app:app in backend/) and open http://127.0.0.1:8000/app/. Run the quick test, then set Supplier B's capacity to 2,000 and run it again, with the diagram coloured by fill rate. Which markets suffer first, and why those? The Health tab shows the supplier now overloaded. Undo (Ctrl+Z) restores FoodNet exactly.
  6. Watch it in production. Run .venv/bin/python -m examples.monitor_demo. It deploys the §10 demo policy, starts a live stream, and quietly takes 60 % of Supplier C's capacity from week 20. The assumption "C has spare capacity of at least 1,100 units a week" fires that same week and tops the inbox; the re-test runs on C as observed. Now set SEVERITY = 0.4 at the top of the file and run it again: C keeps 1,200 units, the assumption still holds, and nothing fires. The monitor watches the assumption, not the capacity.
  7. Delay the information. The bullwhip scenario delays what the agents see by two weeks and adds noise. Then try SimConfig(own_orders_known=False), where an agent forgets even its own recent orders. That is the classic beer-game instability, with order swings many times those of demand.