Propose rule changes from outcomes

When experts keep overriding a rule for the same reason, the rule is missing an exception. You declare which exceptions a rule may grow. The platform watches the overrides, proposes the exception clause that explains them, checks it against your golden cases and waits for a reviewer to admit it.

What proposes a rule change today#

The loop never invents a rule from nothing. It searches the changes you have allowed: threshold moves on the rules you wrote, and the exception clauses you declared. That keeps every proposal explainable as "this override, this clause, these golden cases still pass".

Steps#

1. Declare the exception the rule may grow

   Start from renewal_pack.json in Close the learning loop. Change pack_id and goal_id to renewals.term_exceptions.v1 and add an exception template:

   "exception_templates": [
     {
       "template_id": "multi-year-term",
       "description": "Multi-year renewals may go up to 25% without the VP",
       "applies_to_rule_ids": [
         "RULE-MAX-DISCOUNT-20"
       ],
       "note_pattern": "multi-year",
       "exempt_condition": "state.term_years >= 3 and state.discount_pct <= 25.0"
     }
   ]
   

   The template says: the 20% ceiling rule may gain an exemption for renewals of three years or more, up to 25%, and only an override whose note mentions "multi-year" is evidence for it. An override can match on note_pattern (a case-insensitive regular expression over the expert's note), on override_action (the action the expert chose), or both. exempt_condition is a condition over state, in the same language as the rule.

   Point the client at the new pack:

   import os
   
   import httpx
   
   API = httpx.Client(
       base_url=os.environ["HELIXOR_API_URL"],          # your reasoning API endpoint
       headers={"X-API-Key": os.environ["HELIXOR_API_KEY"]},
       timeout=10.0,
   )
   PACK_ID = "renewals.term_exceptions.v1"
   
   
   def call(method, path, **kwargs):
       resp = API.request(method, path, **kwargs)
       if resp.status_code >= 400:
           raise RuntimeError(f"{method} {path} -> {resp.status_code}: {resp.text}")
       return resp.json()
   
   
   def decide(state):
       return call("POST", "/api/v1/reasoning/playbooks/turn",
                   json={"pack_id": PACK_ID, "message": "Renewal offer", "state": state})
   

2. Admit the baseline, decide, and record the override

   import json
   
   from client import PACK_ID, call, decide
   
   pack = json.load(open("renewal_pack.json"))
   call("POST", "/api/v1/reasoning/playbooks/admit", json={
       "pack": pack,
       "reviewed_by": "pricing-lead@example.com",
       "review_note": "Baseline renewal policy with a declared multi-year exception template",
   })
   
   state = {"discount_pct": 23.0, "user_role": "account_manager",
            "annual_revenue": 90000, "term_years": 3}
   d = decide(state)
   print(f"23%, 3 years: verdict={d['verdict']} move={d['chosen_move']} rules={d['rule_ids']}")
   
   # The VP approved it anyway; record that as an override with a note.
   o = call("POST", f"/api/v1/reasoning/decisions/{d['workflow_run']['run_id']}/outcomes", json={
       "pack_id": PACK_ID,
       "outcome_label": "override",
       "decision_move": d["chosen_move"],
       "expert_override_action": "apply_discount_tier",
       "observed_value": state,
       "human_notes": "Approved: multi-year renewal, 23% is within the multi-year allowance",
   })
   print(f"override recorded: attribution={o['attribution']}")
   

   23%, 3 years: verdict=refuse move=escalate_to_vp rules=['RULE-MAX-DISCOUNT-20']
   override recorded: attribution=policy_defect
   

   A 23% discount on a three-year renewal breaches the 20% ceiling and escalates. The VP approves it anyway, and the note says why. The service classifies the override as a policy defect, so it becomes evidence for a change.

3. Ask for a proposal

   The golden cases pin down what must not change: a 23% single-year renewal still escalates, and a 30% multi-year renewal still escalates.

   import json
   
   from client import PACK_ID, call
   
   golden = [
       {"name": "single_year_23", "expected_verdict": "refuse",
        "state": {"discount_pct": 23.0, "user_role": "account_manager",
                  "annual_revenue": 90000, "term_years": 1}},
       {"name": "multi_year_30", "expected_verdict": "refuse",
        "state": {"discount_pct": 30.0, "user_role": "account_manager",
                  "annual_revenue": 90000, "term_years": 3}},
       {"name": "small_discount", "expected_verdict": "answer",
        "state": {"discount_pct": 10.0, "user_role": "account_manager",
                  "annual_revenue": 60000, "term_years": 1}},
   ]
   report = call("POST", "/api/v1/reasoning/playbooks/evolve",
                 json={"pack_id": PACK_ID, "golden_vectors": golden})
   
   for m in report["pareto_candidates"]:
       print(f"- {m['description']}")
       print(f"    resolved={m['gap_cases_resolved']} regressions={m['golden_regressions']}"
             f" admissible={m['is_admissible']}")
   champion = report["admitted_champion"]
   print("\nchampion     :", champion and champion["mutation_id"])
   print("new condition:", champion and champion["new_condition"])
   for line in report["diagnostics"]:
       print("diagnostic   :", line)
   if champion:
       json.dump(champion["candidate_pack"], open("proposal.json", "w"), indent=2)
   

   - Add exception 'multi-year-term' to RULE-MAX-DISCOUNT-20: Multi-year renewals may go up to 25% without the VP
       resolved=1 regressions=0 admissible=True
   - Adjust threshold discount_pct from 20.0 to 19.0
       resolved=0 regressions=0 admissible=False
   - Adjust threshold discount_pct from 20.0 to 19.6
       resolved=0 regressions=0 admissible=False
   - Adjust threshold discount_pct from 20.0 to 20.4
       resolved=0 regressions=0 admissible=False
   - Adjust threshold discount_pct from 20.0 to 21.0
       resolved=0 regressions=0 admissible=False
   
   champion     : mut-renewals.term_exceptions.v1-5
   new condition: (state.discount_pct > 20.0 and state.user_role != 'vp_sales') and not (state.term_years >= 3 and state.discount_pct <= 25.0)
   diagnostic   : EVOLUTION_SUCCESS: Champion 'mut-renewals.term_exceptions.v1-5' resolved 1/1 gap cases with ZERO golden suite regressions. Proposal only: admit it through the reviewed admission gate to make it live.
   

   Only the exception resolves the override without breaking a golden case. Raising the ceiling to 21% would not cover 23%, and any ceiling high enough to cover it would let the single-year 23% case through. The proposed rule is the original condition with the exemption carved out, so a reviewer can read exactly what changes.

4. Admit it as version 2

   import json
   
   from client import call, decide
   
   proposal = json.load(open("proposal.json"))
   r = call("POST", "/api/v1/reasoning/playbooks/admit", json={
       "pack": proposal,
       "reviewed_by": "pricing-lead@example.com",
       "review_note": "Adds the multi-year exception to the 20% ceiling",
   })
   print(f"admitted version {r['version']}")
   for years in (3, 1):
       d = decide({"discount_pct": 23.0, "user_role": "account_manager",
                   "annual_revenue": 90000, "term_years": years})
       print(f"23%, {years} year(s): verdict={d['verdict']} move={d['chosen_move']} "
             f"rules={d['rule_ids']} pack v{d['pack_version']}")
   

   admitted version 2
   23%, 3 year(s): verdict=answer move=offer_standard_renewal rules=['DISP-RENEWAL'] pack v2
   23%, 1 year(s): verdict=refuse move=escalate_to_vp rules=['RULE-MAX-DISCOUNT-20'] pack v2
   

   The three-year renewal now passes; the single-year one still escalates. Roll back exactly as in the learning-loop tutorial if the reviewer changes their mind.

Limits#

• The loop proposes only threshold moves and the exception templates you declare. It does not add rules, remove rules or change which action a rule selects.
• An admitted exception removes the escalation; the decision then falls through to the pack's disposition bands. Here the 23% renewal returns offer_standard_renewal, not the VP's apply_discount_tier. To select a different action, add a rule for it in a new version.
• The proposal is only as good as your golden cases. Add one for every boundary you care about before you ask.
• Proposals do not change anything until a reviewer admits them, and admission refuses a reviewer who is the same principal as the caller.

Rules drafted from a written policy#

Source integration preview Studio policy authoring takes a written policy and declared decision boundaries, then uses a configured model to propose typed rules with source citations. You review those rules and unresolved questions, save explicit test cases, and obtain separate approval of an immutable revision before installation. Imported drafts can follow the same review path without a model call. The approved policy evaluates structured facts without an authoring-model call. This workflow is a source integration, not a new public SDK API. See the purchasing example for a concrete threshold change.

Next steps#