F12 Code fails its tests

Code: the code in the answer fails its tests. counterfix names what went wrong: no code, a syntax error, the wrong function name or signature, a disallowed import, an error when run, wrong results, or a timeout. Code cut off by the token limit is F1.

How counterfix finds it

With the tests metric (opt-in, it runs the code in a sandbox): cheapest check first, ast parse, the name and signature the tests call, the import allow-list, then each test on its own in a subprocess or a Docker container with no network. See Code evaluation.

Fixes it tries

In this order (a model profile can reorder them). Each one is kept only if it passes 2 of 3 re-runs.

Intervention	Change
error_feedback	Send the error or failed test back once and ask for corrected code
check_examples	Ask the model to work the task's examples by hand before writing the code
restate_signature	Restate the exact function name and signature the tests call
one_code_block	Ask for the whole program in one code block

When no fix verifies

• Let the model run the tests itself (a code tool) and iterate
• Use a stronger coding model