Skip to main content
Two parameter-efficient fine-tuning methods surfaced from arXiv, drafted by Outrider, and taken to huggingface/peft, whose tuner surface is used directly by five of six major fine-tuning frameworks (per Scaling DoRA’s Appendix G). Both merged and are credited in the v0.21.0 release notes; the second has the paper’s first author co-credited.
Status as of November 2026. The linked PRs are the live record.

​
1. Riemannian Preconditioned LoRA: merged

Replaces the Euclidean gradient with the natural gradient under the Riemannian metric induced by the low-rank matrix manifold. Aligns optimizer steps with the intrinsic tangent space and quotients the (A, B) ↦ (A·C, C⁻¹·B) reparameterization gauge that Euclidean SGD and Adam depend on. Nine targeted tests cover the factory path, optimizer compatibility, DoRA compatibility, error handling, the paper formula, and bf16 finiteness. The disclosed benchmark did not beat LoRA in this setting: matching the reference damping default moved GSM8K accuracy from 0.450 to 0.475 against 0.490 for default LoRA in the same rank-32 MetaMathQA configuration, and the result stayed visible when the contribution merged. Review exposed ambiguity around optimizer closures, so the final implementation rejects them.
Riemannian Preconditioned LoRA geometric intuition

Riemannian Preconditioned LoRA: geometric intuition

​
2. Super-Tuning & Supra: merged

Freezes the base weight and trains only a sparse support of scalar entries selected by weight magnitude, a distinct point on the trainable-parameter Pareto front versus LoRA. Setting r > 0 additionally allocates a LoRA-style low-rank adapter composed additively (the paper’s Supra hybrid). Runs against method_comparison/MetaMathQA/ produce numbers directly comparable to the harness’s existing LoRA and SHiRA baselines. Five Super and Supra configurations ran against the existing LoRA and SHiRA baselines in PEFT’s MetaMathQA harness. Results were mixed and are disclosed on the PR: the best Supra row reached 0.4632 test accuracy against 0.4784 for LoRA, one run per configuration. A post-review custom-model run passed 262 tests with 10 skipped. Review focused on integration and testing; the API surface was reduced and checkpoint handling aligned with PEFT conventions over ten commits, and the maintainer ran the example before approval. The evidence does not establish a general performance gain for Super-Tuning or Supra.
Super-Tuning and Supra sparse plus low-rank subspaces

Super-Tuning & Supra: sparse ⊕ low-rank subspaces

​
Composite analytics for PR #3382

Composite analytics for the merged Riemannian LoRA PR
Six panels of the merged Riemannian LoRA PR against the year’s other merged tuner PRs: size (401 LOC, p82); composition (41% test / 48% src / 11% benchmark, above the cohort median’s 13% test / 1% benchmark); merge latency (32.7 days, p89; landed despite a first-time contributor); scope shape (focused and additive, no unrelated refactors); author recurrence (first merged PR to a repo where the top maintainer has 174); feature-utility signals (registry attachment, backwards-compat commitment, benchmark-harness registration under method_comparison/MetaMathQA/).

​
PR sizing against the year’s merged tuner PRs

Compared to huggingface/peft’s other tuner-adding PRs merged since 2025 (via gh pr view <n>): #3382 landed at the compact end (401 LOC against a cohort median around 1150): small, focused, all-additive. #3518 sits at the cohort median (1365 LOC, 28 files), the same shape as PSoFT, PEANuT and GraLoRA.

​
Outrider’s role

Outrider surfaced both candidates and drafted the first implementations and tests. Human review shepherded each through coordination-issue-first upstreaming:
  1. Method discovery. The arXiv-to-repo mapping identifies candidates whose mechanisms haven’t been ported to the host repo.
  2. Draft. Outrider produces an initial branch with tests, a docs page, and integration hooks.
  3. Human refinement and coordination. A coordination Issue upstream (per PEFT’s CLAUDE.md), with maintainer feedback incorporated before the PR is filed.
  4. Upstream PR. Filed against huggingface/peft:main referencing the coordination Issue and, where applicable, paper-author sign-off.
The composite above is the point of this rhythm: a first-time contributor’s PR landed through PEFT’s review process because it arrived shaped the way the maintainer expects. Focused, additive, test-heavy, with a durable feature-utility signal.

​
References