Type-hint, mypy-verified.

What this recipe does

Reads an untyped function (parameters, return, and any internal locals where useful) and returns the same function with type hints added. The hard part is making the output actually type-check; the easy part is making it look like it does.

The verifier here is more interesting than usual: at compile time, every candidate output is run through mypy --strict. Outputs that mypy rejects are dropped from the training set entirely. This is the trick - the model learns to emit annotations that survive a real type checker, not just plausible-looking ones.

The spec

{
  "output_kind": "text",
  "verifier": {
    "mypy": { "strict": true, "python_version": "3.11" },
    "must_preserve_function_body": true,
    "must_add_return_annotation": true,
    "max_lines_added": 3
  }
}

Gold pairs (3 of 90 shown)

def average(xs):
    return sum(xs) / len(xs)

def average(xs: list[float]) -> float:
    return sum(xs) / len(xs)

def fetch(url, timeout=5):
    r = requests.get(url, timeout=timeout)
    r.raise_for_status()
    return r.json()

from typing import Any

def fetch(url: str, timeout: float = 5) -> dict[str, Any]:
    r = requests.get(url, timeout=timeout)
    r.raise_for_status()
    return r.json()

def first_match(xs, pred):
    for x in xs:
        if pred(x):
            return x
    return None

from typing import Callable, Iterable, TypeVar

T = TypeVar("T")

def first_match(xs: Iterable[T], pred: Callable[[T], bool]) -> T | None:
    for x in xs:
        if pred(x):
            return x
    return None

Compile

kolm compile "add python type hints, mypy --strict" \
  --base qwen2.5-coder-3b \
  --pairs pairs.jsonl \
  --verifier mypy:strict \
  --k-floor 0.85 \
  --output type-hint.kolm

ok wrote type-hint.kolm
   k_score=0.88  signature=hmac-sha256

K-score gate

3 of 45 outputs failed mypy --strict on first try (typically a missing import for TypeVar or Callable). Recompile with the offending pairs surfaced as hard negatives took the pass-rate to 94%.

Run-time profile

Deploy

# bulk-annotate a directory:
find src -name "*.py" -exec kolm run type-hint.kolm \
  --input-file {} --output-file {} \;

# run mypy after to verify the artifact's claim:
mypy --strict src/

Related recipes