trl-setup

Setup and installation of TRL (Transformer Reinforcement Learning) library with compatible torch/transformers versions. Use when setting up preference optimization training environments.

Files1

View on GitHub

SKILL.md

GitHub

TRL Setup

Installation

pip install torch transformers trl accelerate peft datasets --break-system-packages

Compatibility Notes

  • TRL imports from trl.trainer.utils: DPODataCollatorWithPadding, pad_to_length, etc.
  • Some imports may change across TRL versions (e.g., trl_sanitze_kwargs_for_tagging vs trl_sanitize_kwargs_for_tagging)
  • If running in externally-managed Python (Debian), use --break-system-packages flag

Common Import Errors

  • trl_sanitze_kwargs_for_tagging not found → check trl version, may be renamed or removed
  • Missing CPOTrainer → upgrade trl: pip install --upgrade trl

Running Tests from Project Root

cd /root/SimPO && python -m pytest unit_test/unit_test_1.py -v
# or
cd /root/SimPO && python unit_test/unit_test_1.py

Key TRL Trainer Utilities

  • DPODataCollatorWithPadding: handles padding for preference pairs
  • pad_to_length: pads tensor to specified length
  • disable_dropout_in_model: disables dropout during eval

Related in other

limits-colimits

Context management for Claude Code. Hooks maintain state via ledgers and handoffs. MCP execution without context polluti...

by parcadei
3,392

ddttom-allaboutv2-claude-skills-block-collection-and-party

The Block Collection and Block Party are repositories for existing AEM blocks, build tools, code snippets, integration p...

by mattnigh
20

nodnarbnitram-claude-code-extensions-claude-skills-esphome-confi

This skill should be used when the user asks to "create an esphome config", "set up an esp32 device", "configure esphome...

by mattnigh
20