kolm tune + kolm rag · v7.6

From skeleton LoRA,
to a living local model.

A compiled .kolm answers today's task. kolm tune and kolm rag are what make it answer tomorrow's: every production run becomes a training signal, every promotion goes through the same K-score gate that gated the compile, and every lookup hits a local BM25 index that knows your policies, your runbooks, your wiki. The base model never moves. Nothing phones home.

~/work - kolm tune cycle
$ kolm tune init --artifact redactor.kolm --base ~/models/qwen-coder-7b ok wrote skeleton v0 (zero-init, identity to base) $ kolm tune capture-on --artifact redactor.kolm ok captures on ~/.kolm/tune/redactor/captures.jsonl $ # ...time passes, agent calls the artifact 247 times... $ kolm tune step --artifact redactor.kolm --airgap --epochs 1 ok revision v1 elapsed 4m 31s · trainable 4.2M / 7B $ kolm tune eval --artifact redactor.kolm --rev v1 ok K-score 0.93 (gate 0.85 · pass 14/15) $ kolm tune promote --artifact redactor.kolm --rev v1 ok HEAD = v1 prev v0 snapshotted to head.prev
01The arc

Skeleton, capture, step, gate.
One closed loop.

Six verbs are the whole pipeline. init writes a zero-init LoRA so the cold start is identity to the base. capture-on tells the runner to record every successful call. step trains a candidate. eval recomputes K-score. promote flips HEAD only if the candidate clears the gate. rollback swaps HEAD with the snapshot. Each step is local, deterministic, logged.

v0

Initialize a zero-init adapter.

Skeleton LoRA at rank 8, alpha 16. Behaves exactly like the base on day one. Cold start is identity, not surprise.

$ kolm tune init --artifact redactor.kolm \ --base ~/models/qwen-coder-7b \ --rank 8 --alpha 16
on

Turn capture on.

Every successful kolm run writes (input, output, recipe, latency_us) to captures.jsonl under ~/.kolm/tune/<slug>/. No upload. No network code touches the file. Failed runs are skipped, bench runs are skipped.

$ kolm tune capture-on --artifact redactor.kolm
v(n+1)

Train one supervised pass, airgap-strict.

--airgap sets TRANSFORMERS_OFFLINE=1, HF_HUB_OFFLINE=1, HF_DATASETS_OFFLINE=1 and refuses any base_model that is not a local existing path. Output: a new revision directory revisions/v(n+1)/.

$ kolm tune step --artifact redactor.kolm \ --airgap --epochs 1 --lr 2e-4
eval

Recompute K-score on the artifact's own evals.

Same formula that gated the compile: K = 0.40·A + 0.15·(S+L+C+V). The eval set is the artifact's own held-out tests. One ruler.

$ kolm tune eval --artifact redactor.kolm --rev v1
gate

Promote, only if the gate agrees.

HEAD flips to the candidate only if K-score(v1) ≥ 0.85 and (with require_improvement) K-score(v1) ≥ K-score(current HEAD). Refusal logs gate_blocked and exits 2 with code K_GATE.

$ kolm tune promote --artifact redactor.kolm --rev v1
back

Rollback is a file rename.

HEAD swaps with head.prev. No retraining, no data loss. If a promotion turns out to underperform in production runs, the prior revision is one command away.

$ kolm tune rollback --artifact redactor.kolm
02Airgap, verified by source

Four mechanisms.
All readable. All cheap to audit.

"Airgapped" is not a marketing word here. Every claim below is a line of code your security team can grep for. The orchestrator imports no fetch and no http. The trainer refuses non-local base paths. The bench harness patches network primitives.

01 · env propagation

Offline flags into the trainer subprocess.

The orchestrator sets KOLM_AIRGAP=1, TRANSFORMERS_OFFLINE=1, HF_HUB_OFFLINE=1, HF_DATASETS_OFFLINE=1 in the Python env dict. See src/tune.js:runTuneStepWith.

02 · path validation

Base model must be a local existing path.

Before any model load, the trainer checks base_model. If KOLM_AIRGAP=1 and the path contains :// or does not exist, tune-step.py exits with a clean message. No partial load. No DNS lookup.

03 · no JS egress

Zero fetch / http imports in the loop.

The JS orchestrator only spawns Python and reads stdout. There is no fetch, no http import in src/tune.js. The only IO is local file IO under ~/.kolm/tune/.

04 · bench egress trap

Network calls during eval fail the benchmark.

The bench harness patches fetch, http, https, net, tls, dns. Any recipe that tries to call out during eval fails the benchmark, which fails the K-score, which fails promotion.

05 · captures stay local

No upload path exists.

The capture appender in src/tune.js:appendCapture is a pure file-append. No analytics. No telemetry. The training data physically cannot leave the box.

06 · rollback by rename

HEAD is a one-line file.

Promotion writes v1 into HEAD; the prior content moves to head.prev. Rollback swaps them. No retraining, no irrecoverable state.

03Lookup, where weights would be wrong

kolm rag: airgapped BM25 inside the sandbox.

Some facts change too often to bake into weights. Policies. Prices. Schemas. Runbooks. kolm rag ships the simplest thing that actually works: a deterministic, single-file BM25 index built in pure JavaScript. No embedder. No network at index time, query time, or attach time. The whole subsystem fits in ~250 lines.

build the index

One JSON file. Pure JS.

Walk a directory deterministically, tokenize (lowercase, strip non-alphanumeric, drop stopwords), build an inverted index, write index.json + manifest.json under ~/.kolm/rag/<name>/.

$ kolm rag index ./docs --name internal-docs
ok indexed 247 docs
    name:   internal-docs
    avgdl:  1195 tokens/doc
    size:   115.8 KB
attach + use inside a recipe

lib.rag, injected.

Attaching writes a sidecar (<art>.kolm.rag.json) next to the artifact. The artifact itself is not re-signed: the receipt stays byte-stable. The runner reads the sidecar at execution time and injects lib.rag.query() into the recipe sandbox.

$ kolm rag attach redactor.kolm --index internal-docs
ok wrote redactor.kolm.rag.json

// inside the recipe:
var hits = lib.rag.query(q, 5).matches;
04Optional autonomy

kolm tune watch.
The whole loop, while you sleep.

A daemon polls captures.jsonl. When the row count crosses threshold_rows (default 200), it auto-runs step, then eval, then gated promote. Every state change is one JSON line in ~/.kolm/logs/tune.jsonl. The gate refuses bad candidates without your help. The audit trail writes itself.

~/.kolm/logs/tune.jsonl
{"event":"watch_started","captures":198,"threshold":200} {"event":"step_triggered","captures":204} {"event":"step_complete","revision":"v3","elapsed_sec":271} {"event":"eval_complete","revision":"v3","k":0.9128,"accuracy":0.933} {"event":"promoted","revision":"v3","previous":"v2","k":0.9128} {"event":"step_triggered","captures":408} {"event":"step_complete","revision":"v4","elapsed_sec":258} {"event":"eval_complete","revision":"v4","k":0.81,"accuracy":0.86} {"event":"gate_blocked","revision":"v4","k":0.81,"gate":0.85}
05The file table

Where the loop touches disk.
The artifact itself never moves.

The .kolm file is signed at compile-time and never modified by the evolve loop. Every piece of evolving state lives in sidecar paths the runner reads at execution time. That is how the model evolves while the artifact's cryptographic receipt stays stable.

path owner when written
~/.kolm/tune/<slug>/tune-config.json kolm tune init Once at init. Updated by capture-on/off.
~/.kolm/tune/<slug>/captures.jsonl runner Append on every successful kolm run when captures are on.
~/.kolm/tune/<slug>/revisions/vN/ trainer One dir per kolm tune step.
~/.kolm/tune/<slug>/HEAD tune promote Single line, current revision id.
~/.kolm/tune/<slug>/head.prev tune promote The promotion before the current one.
~/.kolm/logs/tune.jsonl tune watch One JSON line per state change.
~/.kolm/rag/<name>/index.json kolm rag index Once per index. Clobbered on re-index.
<artifact>.kolm.rag.json kolm rag attach Sidecar pointing at a rag index. Detach by deleting it.