LoRA

Use this repo skill when a task involves the loralib PyTorch package, low-rank adaptation, LoRA-only checkpoints, trainable-parameter selection, merged attention projections, RoBERTa/DeBERTa GLUE fine-tuning, or the repository's GPT-2 E2E/WebNLG/DART examples.

First checks

• Read repository provenance before deciding whether this skill still matches a checkout or should be refreshed.
• Read cross-cutting troubleshooting for missing PyTorch, state-dict mismatches, optional dependencies, CUDA/data assumptions, and stale-skill concerns.
• Run the safe core smoke helper before changing a model or debugging an import. It uses a tiny CPU fixture and does not download models, data, or checkpoints.
• Install the distribution with python -m pip install loralib; for a local source checkout, python -m pip install -e . is the editable equivalent. The public import is import loralib as lora.

Route by task

• Direct PyTorch integration: read core-lora-api for Linear, Embedding, MergedLinear, convolution wrappers, trainable-parameter marking, LoRA-only checkpoint state, bias policies, fan-in/fan-out weights, and eval-time merging.
• RoBERTa/DeBERTa GLUE: read nlu-glue-adaptation for the LoRA-specific run_glue.py flags, query/value insertion points, checkpoint transfer, launcher construction, and CUDA/multi-GPU caveats.
• GPT-2 data-to-text: read nlg-gpt2-adaptation for MergedLinear QKV adaptation, E2E/WebNLG/DART JSONL formats, training and beam-search command construction, decoding, and evaluation-file validation.

Shared operating rules

1. Decide whether the task is package use or repository maintenance. The generated skill is self-contained; do not require the original checkout for ordinary package use.
2. Keep the base model weights separate from the LoRA state. Load the base checkpoint first, then load LoRA parameters with strict=False; save only the adapter state when the goal is a small task-specific artifact.
3. Treat r=0 as the no-adapter case. A positive rank creates lora_A and lora_B; lora_alpha / r is the scaling factor.
4. Treat the large NLU/NLG recipes as optional, resource-heavy workflows. They assume model/data downloads and, for the documented benchmark settings, CUDA and distributed launchers. Do not claim benchmark reproduction from a CPU import or a command-only check.
5. Prefer the bundled helpers and the references in this skill over opening source-repository scripts. The original shell scripts are evidence, not runtime dependencies.

Quick smoke check

python scripts/check_lora_core.py --json

The helper checks the public import, tiny forwards for the supported layer families, eval/train merge transitions, trainable-parameter filtering, and the keys returned by lora_state_dict. Read its sub-skill references before changing defaults or using a nonstandard bias policy.

Common request patterns

Use this skill for requests such as:

• "add LoRA to this PyTorch model"
• "save only the adapter weights"
• "why do my LoRA keys not load"
• "build the GLUE LoRA command"
• "prepare the GPT-2 LoRA data-to-text flow"

If the user asks for a specific route, jump directly to the owning sub-skill instead of reading all three routes. If the user asks for a general question about rank, scaling, or merged weights, start in core-lora-api.

Out of scope

This skill does not replace modern PEFT/Transformers documentation, does not ship model weights or benchmark datasets, and does not make external download, Java/perl metric, credential, or multi-GPU actions safe by default. Those limits are intentional so future agents can reuse the guidance without mistaking historical benchmark settings for always-safe defaults.