Spectrum targeted training (v0.71.23)

Spectrum picks the layers worth training. Instead of a full fine-tune (every weight) or a blanket LoRA (every layer, low rank), soup spectrum scan ranks each weight matrix by its signal-to-noise ratio and lets you full-fine-tune only the high-signal layers. You get most of the quality of a full fine-tune at a fraction of the memory and compute.

The reference Spectrum implementation needs the model on a GPU to scan its layers. Soup's scanner streams the .safetensors shards one tensor at a time, so it never loads the model. Peak RAM is the largest single weight matrix, which means you can scan a 70B on a laptop CPU.

Scan a model

bash
soup spectrum scan --model meta-llama/Llama-3.1-8B --top-percent 50
bash
# write the patch straight to a file, restrict to specific module types
soup spectrum scan --model ./my-merged-model \
  --top-percent 25 \
  --modules mlp,attn \
  --output patch.yaml
FlagDefaultDescription
--model(required)HF Hub id or local path. Streamed shard-by-shard, never loaded.
--top-percent50Keep the top N% of layers by SNR, per module-type group.
--modulesallRestrict the scan to mlp, attn, or a comma list.
--output(stdout)Write the training.unfrozen_parameters block to a file (kept under cwd).
--no-cacheoffSkip the scan cache at ~/.soup/spectrum/<slug>.json.

The command prints a per-group SNR table and a ready-to-paste config block.

Train on the result

The scan emits a training.unfrozen_parameters list of regex patterns. Drop it into your soup.yaml:

yaml
base: meta-llama/Llama-3.1-8B
task: sft
data:
  train: ./data/train.jsonl
training:
  quantization: none
  unfrozen_parameters:
    - "model.layers.(2|5|8|11).mlp.down_proj"
    - "model.layers.(2|5|8).self_attn.v_proj"
bash
soup train --config soup.yaml

The SFT trainer freezes every parameter, then unfreezes only the matched set. This is a full fine-tune of a subset of layers (the matched weights train at full precision), not LoRA.

How the SNR ranking works

For each weight matrix, Spectrum takes the singular values and applies a Marchenko-Pastur noise threshold (arXiv:2406.06623): singular values above the random-matrix noise floor carry signal, the rest are noise. The ratio of signal energy to total energy is the layer's SNR. Layers are ranked within each module-type group, so attention and MLP layers compete with their own kind rather than against each other.

The kernel is pure-numpy and transpose-invariant (the singular values of W and W.T are identical), so GPT-2 Conv1D weights (c_attn / c_fc / c_proj) are recognised alongside Llama-style nn.Linear names.

Constraints

training.unfrozen_parameters requires a full-precision SFT run and is mutually exclusive with the LoRA stack and the other layer-selection knobs:

  • requires task: sft, backend: transformers, modality: text, quantization: none
  • mutually exclusive with LoRA features, freeze_layers, freeze_ratio, train_router_only, and expand_layers

Patterns are validated at parse time: nested-unbounded-quantifier regexes (ReDoS), null bytes, and empties are rejected, with caps on count (50k) and length (512). Hub downloads route through the SSRF-hardened, namespace-pinned loader; symlinked shards and matrices above a 2^31-element SVD cap are skipped, and --output stays under the working directory.

See also