> ## Documentation Index > Fetch the complete documentation index at: https://docs.remyx.ai/llms.txt > Use this file to discover all available pages before exploring further. # huggingface/peft > Two parameter-efficient fine-tuning methods surfaced from arXiv, drafted by Outrider, and merged into huggingface/peft, both featured in the v0.21.0 release notes. Two parameter-efficient fine-tuning methods surfaced from arXiv, drafted by Outrider, and taken to `huggingface/peft`, whose tuner surface is used directly by five of six major fine-tuning frameworks (per Scaling DoRA's Appendix G). Both merged and are credited in the [v0.21.0 release notes](https://github.com/huggingface/peft/releases/tag/v0.21.0); the second has the paper's first author co-credited. > [!NOTE] > > Status as of November 2026. The linked PRs are the live record. ## 1. Riemannian Preconditioned LoRA: merged | | | | - | - | | **Paper** | Zhang & Pilanci, *Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models*, [arXiv:2402.02347](https://arxiv.org/abs/2402.02347) | | **Upstream PR** | [`huggingface/peft#3382`](https://github.com/huggingface/peft/pull/3382), merged 2026-08-03 (+401 / −3, 6 files, 33 days from open to merge) | | **Original attempt** | [`huggingface/peft#1807`](https://github.com/huggingface/peft/pull/1807), the authors' own PR, open May to September 2024 and closed unmerged with [maintainer follow-up on the unfinished work](https://github.com/huggingface/peft/pull/1807#issuecomment-2283410068) | | **Coordination** | [`huggingface/peft#3380`](https://github.com/huggingface/peft/issues/3380); the [original author supported a fresh PR](https://github.com/huggingface/peft/issues/3380#issuecomment-4858962847) and was co-credited | | **Review** | [First review](https://github.com/huggingface/peft/pull/3382#pullrequestreview-4625325791): "this PR is already quite mature"; [final implementation commit](https://github.com/huggingface/peft/pull/3382/commits/2c436f3fa122c562a7908d4357640c3158a8076c); five commits, two approvals | | **Release** | [PEFT v0.21.0](https://github.com/huggingface/peft/releases/tag/v0.21.0) | Replaces the Euclidean gradient with the natural gradient under the Riemannian metric induced by the low-rank matrix manifold. Aligns optimizer steps with the intrinsic tangent space and quotients the (A, B) ↦ (A·C, C⁻¹·B) reparameterization gauge that Euclidean SGD and Adam depend on. Nine targeted tests cover the factory path, optimizer compatibility, DoRA compatibility, error handling, the paper formula, and bf16 finiteness. The disclosed benchmark did not beat LoRA in this setting: matching the reference damping default moved GSM8K accuracy from 0.450 to 0.475 against 0.490 for default LoRA in the same rank-32 MetaMathQA configuration, and the result stayed visible when the contribution merged. Review exposed ambiguity around optimizer closures, so the final implementation rejects them. ![Riemannian Preconditioned LoRA geometric intuition](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/riemannian_lora.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=32f9bae15eedbbd8340c289fd7d67981) Riemannian Preconditioned LoRA: geometric intuition ## 2. Super-Tuning & Supra: merged | | | | - | - | | **Paper** | Ilin, Zmushko & Richtárik, *Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning*, [arXiv:2607.09287](https://arxiv.org/abs/2607.09287) | | **Upstream PR** | [`huggingface/peft#3518`](https://github.com/huggingface/peft/pull/3518), merged 2026-08-21 (+1365 / −6, 28 files, 16 days from open to merge) | | **Coordination** | [`huggingface/peft#3450`](https://github.com/huggingface/peft/issues/3450), direction confirmed by the maintainer | | **Author coordination** | [`vectozavr/SuperTuning#3`](https://github.com/vectozavr/SuperTuning/issues/3); the paper's first author is co-credited | | **Release** | [PEFT v0.21.0](https://github.com/huggingface/peft/releases/tag/v0.21.0) | Freezes the base weight and trains only a sparse support of scalar entries selected by weight magnitude, a distinct point on the trainable-parameter Pareto front versus LoRA. Setting `r > 0` additionally allocates a LoRA-style low-rank adapter composed additively (the paper's Supra hybrid). Runs against `method_comparison/MetaMathQA/` produce numbers directly comparable to the harness's existing LoRA and SHiRA baselines. Five Super and Supra configurations ran against the existing LoRA and SHiRA baselines in PEFT's MetaMathQA harness. Results were mixed and are disclosed on the PR: the best Supra row reached 0.4632 test accuracy against 0.4784 for LoRA, one run per configuration. A post-review custom-model run passed 262 tests with 10 skipped. Review focused on integration and testing; the API surface was reduced and checkpoint handling aligned with PEFT conventions over ten commits, and the maintainer ran the example before approval. The evidence does not establish a general performance gain for Super-Tuning or Supra. ![Super-Tuning and Supra sparse plus low-rank subspaces](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/supertuning.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=260a906c7b8e083f4b5306751f76d96b) Super-Tuning & Supra: sparse ⊕ low-rank subspaces ## Composite analytics for PR #3382 ![Composite analytics for the merged Riemannian LoRA PR](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/composite_pr3382_analytics.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=a3f1517d5a28c6c1bbdb5a384b90f67e) Six panels of the merged Riemannian LoRA PR against the year's other merged tuner PRs: **size** (401 LOC, p82); **composition** (41% test / 48% src / 11% benchmark, above the cohort median's 13% test / 1% benchmark); **merge latency** (32.7 days, p89; landed despite a first-time contributor); **scope shape** (focused and additive, no unrelated refactors); **author recurrence** (first merged PR to a repo where the top maintainer has 174); **feature-utility signals** (registry attachment, backwards-compat commitment, benchmark-harness registration under `method_comparison/MetaMathQA/`). ## PR sizing against the year's merged tuner PRs Compared to `huggingface/peft`'s other tuner-adding PRs merged since 2025 (via `gh pr view `): | PR | Method | Merged | +Additions | −Deletions | Files | | -: | - | :-: | -: | -: | -: | | [#2584](https://github.com/huggingface/peft/pull/2584) | SHiRA | 2025-07-14 | +1623 | −9 | 27 | | [#2851](https://github.com/huggingface/peft/pull/2851) | GraLoRA | 2025-11-18 | +1238 | 0 | 20 | | [#3037](https://github.com/huggingface/peft/pull/3037) | PSoFT | 2026-02-27 | +1556 | −1 | 23 | | [#3084](https://github.com/huggingface/peft/pull/3084) | PEANuT | 2026-03-16 | +1096 | 0 | 23 | | [#3195](https://github.com/huggingface/peft/pull/3195) | BEFT | 2026-04-30 | +773 | −5 | 22 | | [**#3382**](https://github.com/huggingface/peft/pull/3382) | **Riemannian LoRA** | **2026-08-03** | **+401** | **−3** | **6** | | [**#3518**](https://github.com/huggingface/peft/pull/3518) | **Super-Tuning & Supra** | **2026-08-21** | **+1365** | **−6** | **28** | **#3382** landed at the compact end (401 LOC against a cohort median around 1150): small, focused, all-additive. **#3518** sits at the cohort median (1365 LOC, 28 files), the same shape as PSoFT, PEANuT and GraLoRA. ## Outrider's role Outrider surfaced both candidates and drafted the first implementations and tests. Human review shepherded each through coordination-issue-first upstreaming: 1. **Method discovery.** The arXiv-to-repo mapping identifies candidates whose mechanisms haven't been ported to the host repo. 2. **Draft.** Outrider produces an initial branch with tests, a docs page, and integration hooks. 3. **Human refinement and coordination.** A coordination Issue upstream (per PEFT's `CLAUDE.md`), with maintainer feedback incorporated before the PR is filed. 4. **Upstream PR.** Filed against `huggingface/peft:main` referencing the coordination Issue and, where applicable, paper-author sign-off. The composite above is the point of this rhythm: a first-time contributor's PR landed through PEFT's review process because it arrived shaped the way the maintainer expects. Focused, additive, test-heavy, with a durable feature-utility signal. ## References * The full record, with evidence links: [remyx.ai/case-study](https://remyx.ai/case-study) * PEFT contribution guide: [https://huggingface.co/docs/peft/main/en/developer\_guides/contributing](https://huggingface.co/docs/peft/main/en/developer_guides/contributing) * PEFT `CLAUDE.md` (AI-assistance policy): [https://github.com/huggingface/peft/blob/main/CLAUDE.md](https://github.com/huggingface/peft/blob/main/CLAUDE.md) * The paper infographics were generated with [`llmsresearch/paperbanana`](https://github.com/llmsresearch/paperbanana). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.