Skip to content
Read the paper
Search

Poros

Poros documentation

Copy pageMarkdown

The base model streams through the GPU four blocks at a time. Only the LoRA adapters stay resident.

Fine-tune large models on small GPUs: 32B in ~11 GB of training peak, and 72B end-to-end on a single 32 GB card.

Poros streams the frozen quantized base through the GPU one block at a time, keeping only the LoRA adapters resident in VRAM. The streaming changes nothing — loss, adapter weights, and optimizer state all come out bit-for-bit identical to the same configuration trained fully resident.

Quickstart

from poros import PorosTrainer

trainer = PorosTrainer.from_pretrained(
    "Qwen/Qwen2.5-32B",
    quantization="nf4",
    lora_rank=16, lora_alpha=32,
)trainer.train("OpenAssistant/oasst1", steps=200, seq_len=512)  # any HF dataset, or "synthetic"trainer.save("./poros_output/qwen32b-lora")   # adapter + tokenizer + config

Five lines to a trained adapter. Full quickstart →

Getting started

02Installationpip install poros-train, poros doctor, and the pinned torch 2.10+ / CUDA 12.8 stack.03QuickstartZero to a trained adapter in under 5 minutes — PorosTrainer in 5 lines, the one-line poros.prepare HF drop-in, or the poros init / poros train CLI.04Train on your own dataPoint at a .jsonl/.json/.csv file and run poros train. Scaling and troubleshooting.

Usage

05poros.prepare Auto APIThe one-line drop-in — what prepare detects, the guarantee labels, and the gates that raise early.06Adapter supportPoros trains frozen-base adapters only. Status matrix for PEFT LoRA, DoRA, and experimental types.07API ReferencePorosTrainer, PorosModel, patch_hf_model, PorosConfig fields, the poros CLI, determinism, architecture registry, errors.08BenchmarkingThe bench harness: shared configuration per arm, what is and is not held identical, and how the campaign numbers were measured.

Reference

09Consumer GPUsMeasured peaks by model size and card.10Architecture supportWhich model families are validated.11Parity modesWhat each guarantee label means.12LimitationsWhat Poros does not do.13Official A100 resultsThe A100 parity + throughput matrix.14Official B300 resultsThe B300 parity + throughput matrix.

Key properties

  • Bitwise parity: 0.00e+00 max diff on loss and adapter weights (NF4 QLoRA, gated 0.00e+00 across Qwen2.5, Qwen3, Gemma3, and Gemma4 on the platforms in the paper's parity matrix, Gemma on RTX 5090; plus Qwen3.5/Qwen3.6 validated on RTX 4090).
  • 68–77% VRAM reduction vs fully-resident QLoRA; 45–60% less than Unsloth (NF4, 32B–72B scale).
  • Throughput tradeoff: hardware-dependent — about 1.6× slower per step on an RTX PRO 6000, up to ~2.7× on an A100, and ~3.5–4.3× on a B300 (where the resident baseline is so fast that PCIe streaming dominates the ratio). Memory for time.
  • Pure-python install: pip install poros-train works without torch. ML dependencies are optional extras.