kolm  /  learn  /  from predibase

From Predibase to .kolm.

Predibase served your LoRAs through Lorax on managed GPUs. .kolm replaces both halves: the dataset becomes a signed compile artifact, and the serving stack becomes kolm serve or a single zipped file you ship to the device.

migration guide . v1.0 . Predibase Lorax + Ludwig configs supported

Mental model

Predibase lets you download both the adapter weights and the dataset. Pull them both; the dataset is what kolm needs, the adapter is reference for the base model and config.

pip install predibase
# Dataset
python -c "from predibase import Predibase; \
  pb = Predibase(api_token='$PREDIBASE_TOKEN'); \
  ds = pb.datasets.get('your-dataset'); \
  print(ds.download(local_path='./predibase-export'))"

# Adapter weights + config
python -c "from predibase import Predibase; \
  pb = Predibase(api_token='$PREDIBASE_TOKEN'); \
  adapter = pb.adapters.get('your-adapter'); \
  adapter.download(local_path='./predibase-adapter')"

ls predibase-export/ predibase-adapter/

predibase-export/
  train.csv
  validation.csv
  config.yaml
predibase-adapter/
  adapter_model.safetensors
  adapter_config.json
  README.md

Predibase exports as CSV or Parquet; kolm wants JSONL. The Ludwig config.yaml tells you which columns are input_features and which are output_features. Map them to {input, output}:

cat config.yaml

input_features:
  - name: prompt
    type: text
output_features:
  - name: response
    type: text

cat > convert.mjs <<'EOF'
import fs from 'node:fs';
import { parse } from 'csv-parse/sync';

const csv = fs.readFileSync('predibase-export/train.csv','utf8');
const rows = parse(csv, { columns: true, skip_empty_lines: true });
const inputCol = 'prompt';   // from config.yaml input_features
const outputCol = 'response'; // from config.yaml output_features

const out = rows.map(r => JSON.stringify({
  input: r[inputCol],
  output: r[outputCol]
})).join('\n');

fs.writeFileSync('train.jsonl', out + '\n');
console.log('wrote', rows.length, 'rows');
EOF
npm i -y csv-parse
node convert.mjs

wrote 12450 rows

kolm seeds validate train.jsonl

ok. 12450 rows, shape {input, output}

Ludwig YAML maps cleanly to the kolm spec. The translation:

Minimal translated spec:

{
  "task": "respond to user prompt in the established voice",
  "shape": { "input": "string", "output": "string" },
  "base": "meta-llama/Llama-3.1-8B-Instruct",
  "train": { "adapter": "lora", "rank": 16, "epochs": 3 },
  "gates": { "K_min": 0.85, "p50_ms_max": 200 }
}

If your spec includes a base model (LoRA-style), kolm compile rents a GPU for the fine-tune step. If your task can be expressed as a recipe (regex, lookup table, classifier head), it compiles in seconds with no GPU.

kolm compile \
  --spec my-migration.spec.json \
  --examples train.jsonl \
  --eval validation-converted.jsonl \
  --runtime cloud \
  --out my-migration.kolm

renting H100 . . . ready in 14s
training LoRA (rank=16, epochs=3) . . . . . . . . . done in 18m22s
quantizing to int8 . done
K-score for my-migration.kolm: 0.892  (gate >= 0.85 - pass)
size:        91.4 MB
p50 latency: 142 ms
verifying signature . ok
artifact written: my-migration.kolm

Same held-out cases, both models, side-by-side numbers:

# kolm
kolm eval my-migration.kolm --examples validation-converted.jsonl --json > kolm-eval.json

# Predibase (hit your existing endpoint)
python compare_predibase.py validation-converted.jsonl > predibase-eval.json

# Diff
node compare.mjs kolm-eval.json predibase-eval.json

cases:        2491
kolm pass:    2287 (91.8%)
predibase:    2271 (91.1%)
delta:        +0.7%
p50 latency:  142ms (kolm local) vs 380ms (predibase cloud)

Pick a serving mode that matches your old Lorax usage pattern:

# Drop-in HTTP server (Lorax-compatible /generate endpoint)
kolm serve my-migration.kolm --port 8080 --lorax-compat

# Or batch CLI
cat batch.jsonl | xargs -I {} kolm run my-migration.kolm '{}' > results.jsonl

# Or publish for the team
kolm publish my-migration.kolm --private

Things to know

• Base model availability: kolm pins the exact HF model. If Predibase used a custom or proprietary base, you need to either re-host it or pick a comparable open base in your spec.
• Lorax merge logic: Predibase Lorax serves multiple adapters against one base. kolm packages base+adapter per artifact. If you ran ten Lorax adapters, you ship ten .kolm files; each is independent.
• Quantization: kolm quantizes to int8 by default. Set "quantization": "int4" in the spec for smaller artifacts at slight K-cost.
• Ludwig features kolm does not have: nested feature configs, encoder ensembles, audio/image features. If you used these, the migration is a re-architecting, not a port.