Skip to content

TRLoom

TRLoom

TRLoom turns a single YAML file into an end-to-end Hugging Face TRL fine-tuning job.

Configure the model, dataset, trainer, Weights & Biases, and optional Modal GPU execution, then run one command.

Why TRLoom?

TRL is powerful, but wiring model loading, datasets, trainer configs, logging, and remote compute usually means a custom script every time. TRLoom keeps that in one config:

method: sft
model:
  model_name_or_path: Qwen/Qwen2.5-0.5B-Instruct
  use_peft: true
dataset:
  path: trl-lib/Capybara
  train_split: train
training:
  output_dir: ./outputs/sft
  learning_rate: 2.0e-4
  num_train_epochs: 1

trloom run sft.yaml

Install

pip install trloom
# or
pip install git+https://github.com/saqlain2204/trloom.git

See Getting started for extras and editable installs.

Features

  • TRL-native — discovers trainers from your installed TRL version (SFT, DPO, GRPO, KTO, Reward, RLOO, and experimental methods)
  • YAML-first — model, dataset, training args, W&B, and Modal in one file
  • Flexible datasets — Hub repos, local files, saved datasets dirs, mixtures
  • W&B — enable and configure logging from YAML
  • Modal — run the same YAML on remote GPUs with volume-backed outputs
  • CLI + Python API — trloom run or FineTuneJob.from_yaml(...)
Page What you get
Getting started Install, first job, validate + run
Configuration Full YAML reference
Datasets Hub, local, and mixture loading
Weights & Biases Experiment tracking
Modal Remote GPU execution
CLI Command reference
API reference Python API

License

Apache-2.0