Skip to content

F7 Prompt fragility

Small rewordings of the same prompt give different answers.

How counterfix finds it

Not live (it needs re-asks): FragilityCheck asks the same thing in 3 other wordings (yours, from metadata["variants"], or written by the model) and flags the call when a short answer changes meaning. Pass it to diagnose(detectors=[...]); profiles and fix() use it for F7. The linter's L2 flags conflicting constraints, a common cause.

Fixes it tries

In this order (a model profile can reorder them). Each one is kept only if it passes 2 of 3 re-runs.

Intervention Change
fixed_answer_format Pin the answer format so rewordings get the same answer

When no fix verifies

  • Add one or two worked examples (few-shot)
  • Use a larger model