# Agents and runs Source: https://docs.remyx.ai/api-reference/agents Outrider agents, implementation runs, workflows, and eval environments. ## Outrider agents The [agent](/platform/agents/overview) plane: run telemetry, the fleet report, the inbox, and dispatch. Base path `/api/v1.0/outrider`. These endpoints back the studio's Agents, Reports, and Inbox views and the MCP fleet tools, so what you read here matches the app exactly. Visibility is scoped to your own agents. Read routes need `recommendations:read`; install and dispatch routes need `provisioning:*`. ### Runs & telemetry ``` POST /api/v1.0/outrider/runs GET /api/v1.0/outrider/runs ``` `POST /runs` is called by the [Outrider action](/platform/agents/outrider-action) itself (authenticated with the repo's provisioned agent key) to report per-run telemetry: cost identity (agent, backend, model, tokens, cost basis), selection audit (pool sizes, per-candidate rejections, reasoning excerpt), routing and gate outcomes, chain phases, and file-touch lists. Idempotent on `run_id`. `GET /runs` lists your runs; params `repo`, `limit` (default 20, max 100), and `include=selection` to add the selection audit trail. ``` POST /api/v1.0/outrider/agents/runs/analyze ``` Root-cause analysis of one run from its job log: `{repo, run_id}` → cause, evidence bullets, suggested next step. Cached per run. ### Fleet report, activity, and inbox ``` GET /api/v1.0/outrider/agents/report GET /api/v1.0/outrider/agents/activity?repo=owner/name GET /api/v1.0/outrider/agents/inbox POST /api/v1.0/outrider/agents/inbox/dismiss POST /api/v1.0/outrider/agents/inbox/digest POST /api/v1.0/outrider/agents/artifact/diff POST /api/v1.0/outrider/agents/artifact/evidence POST /api/v1.0/outrider/agents/refresh POST /api/v1.0/outrider/agents/chat GET /api/v1.0/outrider/agents/chat/tasks/ POST /api/v1.0/outrider/agents/validations ``` * `/agents/report`: the [fleet report](/platform/review/reports): headline, KPIs, per-agent timelines and terminal states, findings, idea insights, deliverable pipeline, chronology. `since`/`until` (YYYY-MM-DD) and `repos` (intersected with your own footprint) scope it. * `/agents/activity`: one agent's runs, events, directions, and summary; backs [Agent detail](/platform/agents/overview#the-agent-report). * `/agents/inbox`: the derived [Inbox](/platform/review/inbox); `inbox/dismiss` archives `{id}` or `{ids: […]}` (or `undo: true`), `inbox/digest` generates once and persists the TLDR digest for a PR or Issue `{href}`, `artifact/diff` returns the per-file diff of an agent-authored PR `{href}`, and `artifact/evidence` returns the deterministic gate and fidelity facts for one artifact. * `/agents/refresh`: force a re-fetch of agent state from GitHub (`{full: false}` for a delta). * `/agents/chat`: one turn of the Ask chat: `{repo, messages[], focus_run_id?}`; long turns return a task to poll at `chat/tasks/`. Context is assembled server-side, and the transcript carries no privileged access. * `/agents/validations`: turn `@remyx-ai validate` on or off for one agent's repo: `{repo, enabled}`. The per-installation switch lives under [GitHub App](/api-reference/projects#github-app) and covers every repo the installation does. ### Installations, capabilities, and dispatch ``` GET /api/v1.0/outrider/installations POST /api/v1.0/outrider/installations//revoke GET /api/v1.0/outrider/agent-matrix POST /api/v1.0/outrider/trigger GET /api/v1.0/outrider/run-status?repo=owner/name ``` * `/installations`: one row per provisioned agent, including paused (revoked) ones. * `/installations//revoke`: **pause an agent**: marks the installation revoked and revokes its agent key, so the next run fails auth immediately. * `/agent-matrix`: the agent axis as capability data: every coding agent, the providers it can run against (already filtered to what this engine can provision), and the valid pairs. The same answer for every caller; what *you* have connected is `/integrations/providers`. * `/trigger`: the API twin of `remyxai outrider trigger`: `{repo, pin_arxiv?, interest_id?, ref?, agent?, provider?, model?, start_from_ref?, lead_content?, test_integration_policy?, fidelity_policy?}`. `agent` is validated against `provider` before dispatch; an impossible pair is a `400`, and the credential the run is armed with follows the pair. Dispatches via the remyx-ai\[bot] App token; returns `409 {pool_building: true}` while a fresh interest's candidate pool is still building. * `/run-status`: the latest run for a repo, live from GitHub (no webhook lag), enriched with its outcome kind and artifact URL. > [!NOTE] > > Outcome tracking (merge or close of the artifacts a run opened) is **webhook-driven**: it flows back through the GitHub webhooks rather than being polled here. *** ## Implementation Manage the experiment implementation pipeline: context assembly, implementation runs, and status tracking. Scope: `experiments:*`. ### Get Experiment Context ``` GET /api/v1.0/implement/experiments//context ``` Returns the full experiment context formatted for a coding agent: resource metadata, Docker environment, task description, and implementation plan. ### Get Reference Implementation ``` GET /api/v1.0/implement/experiments//reference ``` Returns reference implementation sources: GitHub repo, Docker image, key methods, and implementation plan extracted from the launch context. ### List Actionable Experiments ``` GET /api/v1.0/implement/experiments/actionable ``` | Parameter | Type | Description | | - | - | - | | `project_id` | string | Filter by project UUID | | `limit` | integer | Max results | Returns experiments that have a `launch_context` but no PR yet. ### Run Implementation ``` POST /api/v1.0/implement/experiments//run ``` Triggers the server-side implementation pipeline. Returns `202 Accepted` with a task reference. ### Get Implementation Status ``` GET /api/v1.0/implement/experiments//status ``` Returns the current pipeline status: `pipeline_status`, `pr_url`, `pr_status`, and timing information. ### Report Implementation Result ``` POST /api/v1.0/implement/experiments//report ``` | Field | Type | Description | | - | - | - | | `pr_url` | string | URL of the created PR | | `branch_name` | string | Git branch name | | `summary` | string | Implementation summary | | `status` | string | Result status | | `artifacts` | object | Additional artifact references | The sandboxed coding-agent pipeline has its own endpoints under `/api/v1.0/pipeline/experiments//{run, context, progress, report}`; they are called by the pipeline container, not by clients. *** ## Workflows Scheduled deliveries such as a daily digest to Slack. Scope: `workflows:read` / `workflows:write`. ``` GET /api/v1.0/workflows POST /api/v1.0/workflows GET /api/v1.0/workflows/ PATCH /api/v1.0/workflows/ # only the fields sent change; accepts `active` DELETE /api/v1.0/workflows/ POST /api/v1.0/workflows//test # deliver once, now ``` | Field | Type | Description | | - | - | - | | `workflow_type` | string | `digest` | | `schedule_hour`, `schedule_minute` | integer | Local time of delivery | | `schedule_days` | string | Comma-separated ISO weekdays, e.g. `1,2,3,4,5` | | `schedule_timezone` | string | IANA zone, e.g. `America/Los_Angeles` | | `interest_ids` | array | Interests to include | | `limit_per_interest` | integer | Items per interest | | `include_github` | boolean | Include GitHub repos alongside papers | | `delivery_channel` | string | `slack` | | `delivery_target` | string | Channel id | *** ## Eval Environments Build and run containerized evaluation environments. Base path `/api/v1.0/eval-env`. Scope: `evals:read` / `evals:write`. This is the experiment-level evaluation path; PR validation has its own [Validate](/platform/configure/account#validate) surface. ### Builds ``` POST /api/v1.0/eval-env/builds GET /api/v1.0/eval-env/builds/ GET /api/v1.0/eval-env/builds//context POST /api/v1.0/eval-env/builds//progress POST /api/v1.0/eval-env/builds//report GET /api/v1.0/eval-env/lookup ``` `POST /builds` starts a new environment build. `GET /builds/` returns build status; `/context` returns the assembled build context; `/progress` and `/report` are called by the builder. `/lookup` finds an existing build for a repo and ref. ### Runs ``` GET /api/v1.0/eval-env/runs POST /api/v1.0/eval-env/runs GET /api/v1.0/eval-env/runs/ POST /api/v1.0/eval-env/runs//cancel ``` List or create eval runs, fetch a single run, or cancel an in-flight run. ### Templates ``` POST /api/v1.0/eval-env/templates GET /api/v1.0/eval-env/templates GET /api/v1.0/eval-env/templates/ PUT /api/v1.0/eval-env/templates/ POST /api/v1.0/eval-env/templates//lock ``` CRUD for reusable eval-environment templates. `POST /templates//lock` freezes a template so it can't be edited once it's in use. ### Runner webhooks ``` POST /api/v1.0/eval-env/webhooks/results POST /api/v1.0/eval-env/webhooks/progress POST /api/v1.0/eval-env/webhooks/artifact-url ``` Inbound calls from the runner, authenticated with the run's own HMAC secret rather than a key: `results` delivers scored results (idempotent per run, variant, seed, and nonce), `progress` is a heartbeat the engine relays to the PR comment and Check, and `artifact-url` hands the job a short-lived upload URL for one artifact. *** ## Other endpoints Routes that exist but are either internal or rarely called directly: | Route | Status | | - | - | | `GET /api/v1.0/user`, `GET /api/v1.0/user/credits` | Profile and balance summary (`account:read`); `GET /billing` is the fuller view | | `GET /api/v1.0/evaluation/*` | Legacy evaluation downloads, `evals:*` | | `/api/v1.0/webhooks/github/repo-events` | GitHub webhook receiver; internal | | `/api/v1.0/pipeline/experiments//*` | Called by the implementation container; internal | | `/api/v1.0/model/*`, `/datasets/*`, `/myxboard/*`, `/task/*`, `/deployment/*` | Deprecated model-hosting surface, reachable only with a full-access key | # API reference Source: https://docs.remyx.ai/api-reference/api-reference Authenticate and call current Remyx endpoints with exact request and response contracts. Use the Remyx API to access supported workflows from code. ## Authenticate Use the documented API key and the scopes required by each endpoint. [Authentication](/api-reference/authentication) · [Scopes](/api-reference/scopes) ## Choose an endpoint Browse the endpoints by task. Each page specifies the HTTP method and path, parameters, request and response bodies, errors and side effects. [Validate API](/api-reference/validate) · [Projects and repositories](/api-reference/projects) · [Research and recommendations](/api-reference/research) · [Agents and runs](/api-reference/agents) · [Billing](/api-reference/billing) ## Examples Copy a tested request and compare its response with the documented output shape. [Guides](/introduction) · [CLI & tools](/cli) # Authentication Source: https://docs.remyx.ai/api-reference/authentication API keys, scopes, key management, and response codes for the Remyx API. The Remyx REST API is available at `https://engine.remyx.ai/api/v1.0`. Pass your scoped API key in the `Authorization` header as a Bearer token. Set `REMYXAI_API_KEY` in your shell before running the example. ```bash theme={null} curl -H "Authorization: Bearer $REMYXAI_API_KEY" \ https://engine.remyx.ai/api/v1.0/interests ``` > [!NOTE] > > Create a named API key under **Account > API keys** at [studio.remyx.ai/account?tab=keys](https://studio.remyx.ai/account?tab=keys). Each route is gated by [scopes](/api-reference/scopes). Keys come in two flavors, distinguished by prefix: | Prefix | Type | Typical use | Expiry | | - | - | - | - | | `rmxu_…` | Personal | CLI, MCP, ad-hoc REST calls | None by default; optional 30, 90, or 365 days | | `rmxa_…` | Automation | Minted by Outrider provisioning into repo secrets | 365 days | Each key carries a set of **scopes** that gate which routes it can call (a route returning `403` means the key lacks the required scope). See [API Scopes](/api-reference/scopes) for the full catalog, presets, and which surface needs what. ### Managing keys Keys are created and managed under **Account > API keys** in the studio. The routes behind that page are **session-only**: they live under `/account/api-keys` (not `/api/v1.0`), require a browser login, and cannot be called with an API key. ``` GET /account/api-keys # list keys, plus the scope catalog and presets POST /account/api-keys # {name, scopes?, expires_in_days?}; plaintext returned once PATCH /account/api-keys/ # {name?, scopes?}; automation keys: rename only POST /account/api-keys//revoke # stops working now; stays listed for audit DELETE /account/api-keys/ POST /account/delete # delete the account: revokes every key, disconnects integrations ``` You can only create personal (`rmxu_`) keys; automation keys are minted by provisioning. Omitting `scopes` creates a full-access key; an empty list is an error. Revoking an agent (`rmxa_`) key **pauses the agent that uses it**. See [Account](/platform/configure/account). > [!WARNING] > > The legacy single-token endpoints under `/account/tokens/*` are removed and now return `410 Gone`. Migrate to named keys via `/account/api-keys`. ### Response Codes | Code | Meaning | | - | - | | `200` | Success | | `201` | Created | | `202` | Accepted (async task started) | | `400` | Bad request (invalid parameters) | | `401` | Unauthorized (missing or invalid token) | | `403` | Forbidden (key lacks the required scope, or `validate_gated`) | | `404` | Not found | | `409` | Conflict (duplicate resource, stale `key_id`, pool still building, Checkout not configured) | | `410` | Gone (legacy `/account/tokens/*` endpoints; use `/account/api-keys`) | | `503` | A dependency is not configured on this server (for example the GitHub App) | | `500` | Server error | # Billing Source: https://docs.remyx.ai/api-reference/billing Balance, top-ups, and the price list. One prepaid Remyx balance, spent pay as you go. Scope: `billing:read` / `billing:write`. See [Billing](/platform/configure/billing). ``` GET /api/v1.0/billing ``` Your balance, recent transactions, whether a grant covers you, Checkout availability, and the top-up presets and limits. A service token gets the price list instead. ``` POST /api/v1.0/billing/checkout ``` | Field | Type | Description | | - | - | - | | `amount_usd` | number | Between the minimum and maximum top-up returned by `GET /billing` | | `success_url`, `cancel_url` | string | Optional return URLs; default to the Studio Account page | Returns `{url, session_id, amount_usd}`; send the browser to `url`. `409` when Checkout is not configured on the server, `400` for an amount out of range, `403` for anything but a user-scoped token. The Stripe webhook at `/billing/stripe/webhook` credits the balance once per event. # CLI Reference Source: https://docs.remyx.ai/api-reference/cli-reference Complete command reference for the Remyx CLI # CLI Reference Quick reference for all `remyxai` commands. For usage examples and guides, see [Using the CLI](/cli). *** ## `remyxai interests` Manage Research Interests — the context that drives personalized recommendations. | Command | Description | | - | - | | `interests list` | List all Research Interests | | `interests get -i ` | Get details for one interest | | `interests create` | Create a new interest (interactive or with flags) | | `interests from-repo ` | Create an interest from a GitHub repo profile | | `interests from-project ` | Create an interest from a project's experiments | | `interests update -i ` | Update an interest's fields | | `interests toggle -i ` | Toggle active/inactive status | | `interests delete -i ` | Delete an interest | ### `interests create` ``` remyxai interests create [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--name, -n` | string | *(prompted)* | Short label | | `--context, -c` | string | *(prompted)* | Description or URLs (HuggingFace, GitHub, arXiv, free text) | | `--daily-count, -d` | int | 2 | Recommendations per day (1-10) | | `--inactive` | flag | — | Create as inactive | | `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating | | `--wait, -w` | flag | — | Block until the refresh completes | | `--format, -f` | choice | `text` | `text` or `json` | ### `interests from-repo` ``` remyxai interests from-repo [OPTIONS] ``` | Argument | Type | Description | | - | - | - | | `repo_url` | string | **Required.** GitHub repo URL | | Flag | Type | Default | Description | | - | - | - | - | | `--name, -n` | string | *(from repo)* | Short label | | `--daily-count, -d` | int | 2 | Recommendations per day (1-10) | | `--inactive` | flag | — | Create as inactive | | `--automate` | choice | `none` | `none`, `review`, or `auto`. `review`/`auto` provision Outrider on the repo like `outrider init` | | `--timeout` | int | 300 | Seconds to wait on Outrider provisioning when automating | | `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating | | `--wait, -w` | flag | — | Block until the refresh completes | | `--format, -f` | choice | `text` | `text` or `json` | ### `interests from-project` ``` remyxai interests from-project [OPTIONS] ``` | Argument | Type | Description | | - | - | - | | `project` | string | **Required.** Project name or UUID | | Flag | Type | Default | Description | | - | - | - | - | | `--name, -n` | string | *(from project)* | Short label | | `--daily-count, -d` | int | 2 | Recommendations per day (1-10) | | `--inactive` | flag | — | Create as inactive | | `--include-experiment, -e` | string | *(all)* | Limit context to a specific experiment (repeatable) | | `--no-auto-update` | flag | — | Snapshot the project once instead of tracking it | | `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating | | `--wait, -w` | flag | — | Block until the refresh completes | | `--format, -f` | choice | `text` | `text` or `json` | ### `interests update` ``` remyxai interests update [OPTIONS] ``` | Flag | Type | Description | | - | - | - | | `--interest, -i` | string | **Required.** Name or UUID | | `--name, -n` | string | New name | | `--context, -c` | string | New context (invalidates recommendation pool) | | `--daily-count, -d` | int | New daily count | | `--activate` | flag | Set active | | `--deactivate` | flag | Set inactive | | `--format, -f` | choice | `text` or `json` | ### `interests delete` ``` remyxai interests delete [OPTIONS] ``` | Flag | Type | Description | | - | - | - | | `--interest, -i` | string | **Required.** Name or UUID | | `--yes, -y` | flag | Skip confirmation prompt | | `--format, -f` | choice | `text` or `json` | *** ## `remyxai papers` Get personalized recommendations from the Remyx resource index. | Command | Description | | - | - | | `papers digest` | Today's top recommendations grouped by interest | | `papers list` | Flat list of recommendations with filtering | | `papers refresh` | Trigger async recommendation refresh | | `papers refresh-status ` | Poll a refresh task's status | ### `papers digest` ``` remyxai papers digest [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--limit, -n` | int | 5 | Max items per interest | | `--period, -p` | choice | `today` | `today`, `week`, or `all` | | `--format, -f` | choice | `text` | `text` or `json` | | `--full` | flag | — | Show full reasoning without truncation | ### `papers list` ``` remyxai papers list [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--interest, -i` | string | *(all)* | Filter by interest name or UUID | | `--limit, -n` | int | 20 | Max results (1-50) | | `--period, -p` | choice | `all` | `today`, `week`, or `all` | | `--source-type, -s` | string | *(all)* | Filter by source: `arxiv_paper`, `github_repo`, etc. | | `--format, -f` | choice | `text` | `text` or `json` | | `--full` | flag | — | Show full reasoning without truncation | ### `papers refresh` ``` remyxai papers refresh [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--interest, -i` | string | *(all active)* | Refresh one interest by name or UUID | | `--num-results, -n` | int | *(daily\_count)* | Items per interest | | `--wait, -w` | flag | — | Block until all tasks complete | | `--format, -f` | choice | `text` | `text` or `json` | ### `papers refresh-status` ``` remyxai papers refresh-status [OPTIONS] ``` | Argument | Type | Description | | - | - | - | | `task_id` | string | **Required.** UUID from `papers refresh` output | | Flag | Type | Default | Description | | - | - | - | - | | `--format, -f` | choice | `text` | `text` or `json` | *** ## `remyxai search` Search across the Remyx resource index. | Command | Description | | - | - | | `search query ` | Semantic search across papers, repos, models | | `search info ` | Detailed info for a specific resource | | `search list` | Browse recently added resources | | `search stats` | Catalog statistics | ### `search query` ``` remyxai search query [OPTIONS] ``` | Argument | Type | Description | | - | - | - | | `query_text` | string | **Required.** Natural language search query | | Flag | Type | Default | Description | | - | - | - | - | | `--max-results, -n` | int | 10 | Maximum results | | `--category, -c` | string | *(all)* | arXiv category filter (repeatable) | | `--docker` | flag | — | Only resources with Docker environments | | `--no-docker` | flag | — | Only resources without Docker | ### `search info` ``` remyxai search info [OPTIONS] ``` | Argument | Type | Description | | - | - | - | | `arxiv_id` | string | **Required.** arXiv ID (e.g., `2308.12345`) | | Flag | Type | Default | Description | | - | - | - | - | | `--format, -f` | choice | `text` | `text` or `json` | ### `search list` ``` remyxai search list [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--limit, -n` | int | 20 | Number of results | | `--offset, -o` | int | 0 | Pagination offset | | `--category, -c` | string | *(all)* | Category filter (repeatable) | | `--docker` | flag | — | Only Docker-ready resources | | `--no-docker` | flag | — | Only non-Docker resources | ### `search stats` ``` remyxai search stats ``` No options. Prints total assets, Docker availability counts, top categories, and citation coverage. *** ## `remyxai outrider` Install [automated discovery PRs](/platform/agents/outrider-action) (the Outrider GitHub Action) on a repo. | Command | Description | | - | - | | `outrider init` | Provision via the Remyx GitHub App (`remyx-ai[bot]`) | | `outrider setup-local` | Provision via your own `gh` CLI, without the GitHub App | | `outrider trigger` | Fire a one-shot `workflow_dispatch` run of an installed workflow | | `outrider set-provider-secret` | Set a provider's API-key secret on a repo, safely | ### `outrider init` ``` remyxai outrider init [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL | | `--interest, -i` | string | — | Research Interest UUID to wire in | | `--auto-interest` | flag | — | Auto-create the interest from the repo (exclusive with `--interest`) | | `--bulk-repos` | string | — | Path to a TSV of `owner/nameinterest_uuid` rows; loops with per-row error capture. Exclusive with `--repo` / `--interest` / `--auto-interest` | | `--pace` | int | 3 | Seconds between repos in a `--bulk-repos` run | | `--mode` | choice | `auto` | `auto`, `review`, or `off` | | `--agent` | choice | `claude` | Coding-agent CLI to provision: `claude`, `codex`, `backboard`. Confirmed against the provisioned workflow | | `--anthropic-key` | string | `$ANTHROPIC_API_KEY` | Model-provider key (used only if none is connected) | | `--no-wait` | flag | — | Don't block on App install / provisioning | | `--dry-run` | flag | — | Print the plan and exit | | `--yes, -y` | flag | — | Skip the confirmation prompt | ### `outrider setup-local` ``` remyxai outrider setup-local [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL | | `--interest, -i` | string | — | Research Interest UUID to wire in | | `--auto-interest` | flag | — | Auto-create the interest from the repo | | `--bulk-repos` | string | — | Path to a TSV of `owner/nameinterest_uuid` rows; loops with per-row error capture. Exclusive with `--repo` / `--interest` / `--auto-interest` | | `--pace` | int | 3 | Seconds between repos in a `--bulk-repos` run | | `--mode` | choice | `auto` | `auto` or `review` | | `--no-cron` | flag | — | Render the workflow with the cron schedule commented out, keeping only `workflow_dispatch` | | `--agent` | choice | `claude` | Coding-agent CLI the generated workflow runs by default: `claude`, `codex`, `backboard` | | `--backend` | choice | *(the agent's own vendor)* | Default model backend: `anthropic`, `openai`, `zai`, `moonshot`, `openrouter`. Scoped to the single-file setup | | `--model` | string | *(the agent's own default)* | Default model id for the install. Required with `--agent backboard` | | `--anthropic-key` | string | `$ANTHROPIC_API_KEY` | Key set as the `ANTHROPIC_API_KEY` repo secret | | `--dry-run` | flag | — | Print the plan + rendered workflow and exit | | `--yes, -y` | flag | — | Skip the confirmation prompt | Requires an authenticated `gh` CLI and admin on the target repo. ### `outrider trigger` ``` remyxai outrider trigger [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL | | `--interest, -i` | string | *(configured interest)* | Research Interest to run against | | `--pin-method` | string | — | A method query, or a literal arXiv ID, to pin for this run; bypasses the selection pass. Exclusive with `--pin-arxiv` | | `--pin-arxiv` | string | — | An arXiv ID already in the candidate pool to pin. Exclusive with `--pin-method` | | `--ref` | string | *(repo default branch)* | Git ref to dispatch the workflow on | | `--agent` | string | *(workflow default)* | Coding-agent CLI for this dispatch (`claude`, `codex`, `backboard`); an invalid agent/provider pair is rejected locally. The workflow must declare an `agent` dispatch input | | `--agent-timeout` | int | *(workflow default)* | Wall-clock seconds for each agent phase on this dispatch. `--claude-timeout` is the old name and still works | | `--provider` | string | *(workflow default)* | Model backend for this dispatch (`anthropic`, `openai`, `zai`, `moonshot`, `openrouter`); the workflow must declare a `provider` dispatch input | | `--model` | string | *(provider default)* | Model to request from the provider (e.g. `glm-5.3`, `gpt-5.4-mini`), forwarded as the selected agent's model variable | Refuses if no Outrider workflow is registered on the repo. Requires an authenticated `gh` CLI. ### `outrider set-provider-secret` ``` remyxai outrider set-provider-secret [OPTIONS] ``` | Flag | Type | Default | Description | | - | - | - | - | | `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL | | `--provider` | string | **required** | Provider the key is for; selects the secret name (`anthropic` → `ANTHROPIC_API_KEY`, `zai` → `ZAI_API_KEY`, and the same convention for `openai`, `moonshot`, `openrouter`) | | `--key-from` | path | **required** | File containing only the API key. Read from file to avoid the `gh secret set --body -` truncation trap | *** ## Global Options All commands support `--help` for detailed usage: ```bash theme={null} remyxai --help remyxai interests --help remyxai papers digest --help ``` # Projects and repositories Source: https://docs.remyx.ai/api-reference/projects Projects, experiments, GitHub App installations, and connected integrations. ## Projects Projects scope experiments, agents, and the quality gate. Scope: `projects:read` / `projects:write`. ``` GET /api/v1.0/projects # non-archived projects for your team POST /api/v1.0/projects # create GET /api/v1.0/projects/ # full config PUT /api/v1.0/projects/ # name, description, or config (merged) DELETE /api/v1.0/projects/ # hard delete; experiments are unlinked, not deleted ``` ### Create from repos ``` POST /api/v1.0/projects/from-repos ``` | Field | Type | Required | Description | | - | - | - | - | | `name` | string | Yes | Project name | | `repos` | array | No | `[{url, name?}]`; each linked repo gets an experiment history | | `description` | string | No | | | `team_id` | string | No for users, yes for service tokens | | | `scope_context` | string | No | Free-text scope that steers ranking and gates | | `skip_extraction` | boolean | No | Create the history rows without dispatching extraction | ### Quality gate ``` POST /api/v1.0/projects//eval-templates/ # body = the template definition GET /api/v1.0/projects//decision-policy # null when unset POST /api/v1.0/projects//decision-policy # body = full DecisionPolicy; 400 with `detail` on a bad payload DELETE /api/v1.0/projects//decision-policy GET /api/v1.0/projects//metrics # per-metric trend: current, baseline, direction ``` ### Experiment history ``` GET /api/v1.0/projects//experiment-history # merged rows across linked histories, with per-history status POST /api/v1.0/projects//experiment-history/refresh # re-run extraction on every linked repo ``` The underlying histories are also addressable directly: `POST /experiment-histories`, `GET|DELETE /experiment-histories/`, `GET .../status`, `POST .../extract`, `POST .../report`, `POST .../score-candidates`, `POST .../refit-preference-model`. Scope: `experiments:*`. *** ## Experiments Manage experiments, track outcomes, and log decisions. Scope: `experiments:read` / `experiments:write`. ### List Experiments ``` GET /api/v1.0/experiments ``` | Parameter | Type | Description | | - | - | - | | `project_id` | string | Filter by project UUID | | `status` | string | Filter by status | | `ticket_source` | string | Filter by ticket source (e.g., `linear`, `jira`) | | `external_ticket_id` | string | Filter by external ticket ID | | `created_via` | string | Filter by creation method (e.g., `mcp`, `web`, `seed`) | | `limit` | integer | Max results to return | **Response:** Array of experiment objects with all ExperimentOps fields. ### Get Experiment ``` GET /api/v1.0/experiments/ ``` **Response:** Single experiment object with full detail including `source_type`, `source_ref`, `hypothesis`, `target_metric`, `observed_delta`, `delta_confidence`, `decision`, `tags`, `project_id`, `project_name`, `launch_context`, `ab_test_config`, `ab_test_results`, and linked resources. ### Update Experiment ``` PUT /api/v1.0/experiments/ ``` | Field | Type | Description | | - | - | - | | `name` | string | Experiment name | | `status` | string | Lifecycle state: `draft`, `running`, `complete`, `analyzing`, `shipped`, `iterating`, `rejected`, `abandoned` | | `hypothesis` | string | Hypothesis text | | `outcome` | string | `ship`, `iterate`, `abandon` | | `decision` | string | Decision text with reasoning | | `observed_delta` | float | Measured metric change | | `delta_confidence` | string | `significant`, `inconclusive`, `not_tested` | | `target_metric` | string | Target business metric | | `tags` | array | Free-form tags for grouping | | `pr_url` | string | Link to implementation PR | | `pr_status` | string | PR status | | `external_ticket_id` | string | Linked external ticket ID | | `ticket_source` | string | Ticket provider | | `ticket_url` | string | Ticket URL | ### External data ``` GET /api/v1.0/experiments//external-data ``` The A/B or production data attached to an experiment from external platforms. ### Relationships ``` GET /api/v1.0/experiments//relationships POST /api/v1.0/experiments//relationships PUT /api/v1.0/experiments//relationships/ DELETE /api/v1.0/experiments//relationships/ POST /api/v1.0/experiments//attach-to-history ``` Typed links between experiments (`borrows_from`, `variant_of`, `replicates`, `references`) with optional notes. `attach-to-history` files an experiment under an extracted experiment history. ### Insights ``` GET /api/v1.0/experiments/insights GET /api/v1.0/experiments/insights//recommendations ``` Pattern clusters across a project's experiments, and the recommended resources for one cluster. Scope: `papers:read`. *** ## GitHub App Endpoints for the Remyx GitHub App: install status, the installations your account owns, and the identity behind agents and Validate. Base path `/api/v1.0/github`. Scope: `github:read` / `github:write`. Provisioning a repo is under [Research Interests](/api-reference/research#provision-an-agent-outrider). ### Get Install URL ``` GET /api/v1.0/github/app/install-url ``` Returns the URL to install the Remyx GitHub App on a user's account or org, with the CSRF `state` for the caller to echo. ### Get Installation ``` GET /api/v1.0/github/app/installation?repo=owner/name ``` Reports whether the App covers a repo. Always asks GitHub, so it can be polled while a person installs the App in a browser. | Field | Description | | - | - | | `installed` | `true` when the App covers this repo | | `installation_id`, `account`, `account_type`, `repository_selection` | The covering installation (`all` or `selected` repos) | | `record` | The durable installation row: owner, Validate switch, `mine`, `claimable`, `manage_url` | | `account_installed` | `true` when the owner account has an installation that does not cover this repo | | `reason` | Why not, when `account_installed` is true (below) | | `manage_url` | The App's repository-access settings page, the place to fix `repo_not_selected` | | `reason` | Meaning | Fix | | - | - | - | | `repo_not_selected` | A selected-repos install that this repo is not in | Open `manage_url` and add the repo | | `repo_unavailable` | An all-repos install that still does not cover it: GitHub has not propagated a just-created repo (a fork followed by an immediate `init` hits this), or the repo is misspelled, deleted, or invisible | Wait and retry, or check the name | | `suspended` | Installed but suspended | Unsuspend it in GitHub settings | Returns `503 {configured: false}` when the server has no App configured. ### Installations ``` GET /api/v1.0/github/app/installations PATCH /api/v1.0/github/app/installations/ # {validations_enabled: bool}; owner only POST /api/v1.0/github/app/installations//claim ``` `GET` lists the installations your account owns, plus the ones your GitHub authorization can reach: unclaimed ones you may claim, and ones another Remyx account owns (read-only). An install made on github.com shows up here to be claimed. `PATCH` flips the per-installation Validate switch for every repo the installation covers; the model Validate uses is a [Validate setting](/api-reference/validate#model), not a field here. `claim` makes you the owner of an unclaimed installation you can reach through your own GitHub authorization; it never transfers one another account owns. ### Get Installation Token ``` POST /api/v1.0/github/installation-token ``` Mints a short-lived remyx-ai\[bot] token scoped to one repo (`{repo}`), for the provisioned action key. The caller must own a merged, non-revoked provisioned action for that repo. Requires `github:write`. ### Fork a Repo ``` POST /api/v1.0/github/fork ``` Forks a repo into **your own account** (your identity, never the bot's), optionally seeding `branches` from the source network. This is what the Quickstart's set-example track uses to warm-start curated drafts. Idempotent. Requires `provisioning:write`. | Field | Type | Description | | - | - | - | | `repo` | string | Repository in `owner/name` form | | `branches` | array | Branch names to seed into the fork (optional) | *** ## Integrations Manage connections to external tools (GitHub, Linear, Jira, Slack, Hugging Face, model providers, and more). Scope: `integrations:read` / `integrations:write`. ### List Providers ``` GET /api/v1.0/integrations/providers ``` Returns available integration providers with configuration status. ### Connect ``` GET /api/v1.0/integrations//auth-url # OAuth providers: the authorization URL POST /api/v1.0/integrations//callback # {code, redirect_uri} POST /api/v1.0/integrations//connect # API-key providers: {api_key, ...} validated against the provider before it is stored ``` OpenRouter connects through an OAuth PKCE flow rather than a pasted key. Slack has its own routes under `/api/v1.0/slack/{install-url, callback, configure, installations}`. ### Connections and status ``` GET /api/v1.0/integrations/connections GET /api/v1.0/integrations//status DELETE /api/v1.0/integrations/ ``` ### Get stored credentials ``` GET /api/v1.0/integrations//token ``` Returns a valid token for the provider: a decrypted, auto-refreshed access token for OAuth providers, or the decrypted secret plus provider data for API-key providers. A service token may pass `user_id` to read another user's credentials; a user key reads only its own. > [!WARNING] > > This route is gated by `integrations:read`, so a key with that scope can read your stored third-party credentials, not only connection status. Grant `integrations:read` only to keys you would trust with those credentials. ### Identity Resolution ``` GET /api/v1.0/integrations/identity/resolve ``` | Parameter | Type | Description | | - | - | - | | `source` | string | External platform (e.g., `github`, `linear`) | | `external_id` | string | External user ID | | `external_email` | string | External user email (alternative to external\_id) | ``` POST /api/v1.0/integrations/identity/link ``` Creates an identity link between a Remyx user and an external platform identity. ``` GET /api/v1.0/integrations/identity/links ``` Lists all identity links for the current user. # Research and recommendations Source: https://docs.remyx.ai/api-reference/research Research Interests, recommendations, and search. ## Research Interests Manage your Research Interests, the context that drives personalized resource recommendations. Scope: `interests:read` / `interests:write`. > [!NOTE] > > Two interests surfaces exist. The programmatic API uses **`/interests`** (documented here). `/research-interests` is the slimmer v1 surface the Outrider action reads (`GET /research-interests`, `GET /research-interests/`, `POST /research-interests//refresh`, `DELETE /research-interests/`); an automation key's resource-qualified `interests:read:` grant is what it checks. When calling with a personal key, use `/interests`. ### List Interests ``` GET /api/v1.0/interests ``` **Response:** Array of Research Interest objects with `name`, `context`, `daily_count`, `is_active`. ### Create Interest ``` POST /api/v1.0/interests ``` | Field | Type | Required | Description | | - | - | - | - | | `name` | string | Yes | Interest label (e.g., "Retrieval-Augmented Generation") | | `context` | string | Yes | URLs and descriptions defining the interest | | `daily_count` | integer | No | Recommendations per day (default: 3) | | `is_active` | boolean | No | Whether the interest is active (default: true) | ### Get Interest ``` GET /api/v1.0/interests/ ``` **Response:** The full interest object. Includes an additive `experiment_history` field, a bullet list summarizing the experiments linked to this interest. ### Update Interest ``` PUT /api/v1.0/interests/ ``` Accepts same fields as Create. Updating `context` invalidates the recommendation pool and triggers re-ranking. ### Delete Interest ``` DELETE /api/v1.0/interests/ ``` Deletes the interest and cascading recommendations. ### Toggle Interest ``` POST /api/v1.0/interests//toggle ``` Toggles the `is_active` status without deleting the interest or its history. ### Interests from a repo ``` POST /api/v1.0/interests/analyze-repo # {repo_url} → 202 {task_id, status_url} GET /api/v1.0/interests/analyze-repo/ # poll; returns the report + structured analysis when complete POST /api/v1.0/interests//regenerate # re-run analysis on the interest's source repo (optional {repo_url}) GET /api/v1.0/interests//project-experiments # a project-mode interest's experiments, with included/excluded flags GET /api/v1.0/interests/github/repos # repos reachable through your GitHub connection ({connected: false, repos: []} when none) ``` ### Provision an agent (Outrider) ``` POST /api/v1.0/interests//provision-action ``` Provisions the [Outrider action](/platform/agents/outrider-action) on a repo: issues an automation (`rmxa_`) key, pushes it to the repo's Actions secrets, enables Actions PR permissions, writes the workflow file, and opens a setup PR. Requires `provisioning:write`. Returns `202 { task_id, status_url }`. | Field | Type | Description | | - | - | - | | `repo_url` | string | GitHub URL. Defaults to the interest's source repo | | `agent` | string | Coding agent CLI: `claude` (default), `codex`, or `backboard`. Validated against the model provider up front; an impossible pair is a `400` | | `model_provider` | string | Provider id (`claude_code`, `zai`, `moonshot`, `openai`, `openrouter`). Defaults to the first connected provider the agent can use | | `phases` | object | Two-tier drafter/refiner configuration (see [Model backends](/platform/agents/backends#the-two-tier-drafter-refiner-pattern)) | | `branch` | string | Base branch for the setup PR | | `workflow_filename` | string | Override the workflow file name | | `auto_merge` | boolean | Merge the setup PR automatically (default `true`) | | `dispatch` | boolean | Fire the first run once installed (defaults to `auto_merge`) | | `first_run` | string | `"user"` to own the first run yourself; otherwise the engine fires a cold run when recommendations land | | `force` | boolean | Re-drive an already-enabled repo so new `phases` rewrite the YAML, without revoking its key | | `sealed_provider_secrets` | array | BYOK relay: `[{secret_name, key_id, encrypted_value}]` sealed against the repo's Actions public key (below). Remyx forwards the ciphertext and stores no credential | | `reclaim_key_custody` | boolean | Hand a BYOK repo back to Remyx-held keys on re-provision | The task result carries `pr_url`, `secret_set`, `key_label`, and, for a two-tier install with a tier whose provider has no connected key, `providers_missing_keys`. ``` GET /api/v1.0/interests//provision-action/ # poll one task GET /api/v1.0/interests//provision-action # latest state; {provisioned: false} if none GET /api/v1.0/interests//provision-action/actions-public-key?repo_url= ``` `actions-public-key` returns `{repo, key_id, key, sealable_secret_names}`: the repo's GitHub Actions public key, so a client can seal a provider key locally before sending it as `sealed_provider_secrets`. A stale `key_id` on the POST returns `409`. Requires `provisioning:read`. ``` POST /api/v1.0/interests//provision-cli-token ``` Mints the scoped automation key for a CLI-installed Outrider (`remyxai outrider setup-local`). Body `{repo_full_name}`; the plaintext key is returned once. Requires `provisioning:write`. > [!NOTE] > > The provisioned workflow keys its GitHub Actions concurrency group per target (pinned paper, start-from ref, run id), so dispatches at different targets run in parallel instead of cancelling each other. Installs provisioned before this change should be re-provisioned with `force: true`. *** ## Recommendations Get personalized resource recommendations based on your Research Interests. Scope: `recommendations:read` / `recommendations:write`. ### List Recommendations ``` GET /api/v1.0/papers/recommended ``` | Parameter | Type | Description | | - | - | - | | `interest_id` | string | Filter by interest UUID | | `limit` | integer | Max results | | `period` | string | `today`, `week` | **Response:** Array of recommended resources with titles, abstracts, relevance scores, and links. ### Get Digest ``` GET /api/v1.0/papers/recommended/digest ``` | Parameter | Type | Description | | - | - | - | | `limit` | integer | Max results per interest; `0` returns every alerted recommendation with a `truncated` flag | | `period` | string | `today`, `week` | **Response:** Structured digest grouped by Research Interest. ### Refresh Recommendations ``` POST /api/v1.0/papers/recommended/refresh ``` | Field | Type | Description | | - | - | - | | `interest_id` | string | Refresh a specific interest (optional; refreshes all if omitted) | | `num_results` | integer | Number of results to generate | Returns `202 Accepted` with a task reference for polling. ### Poll Refresh Status ``` GET /api/v1.0/papers/recommended/refresh/ ``` Returns the status of an async recommendation refresh task. *** ## Search Search across research assets with hybrid retrieval. Scope: `papers:read`. ### Search Assets ``` POST /api/v1.0/search/assets ``` | Field | Type | Description | | - | - | - | | `query` | string | Natural language search query | | `max_results` | integer | Max results to return | | `has_docker` | boolean | Filter for resources with runnable Docker environments | | `categories` | array | Filter by categories | | `use_llm` | boolean | Enable LLM-enhanced search | **Response:** Array of matching assets with titles, abstracts, categories, Docker availability, and relevance scores. ### Get Asset Details ``` GET /api/v1.0/search/assets/ ``` Returns detailed information for a specific asset: full abstract, Docker status, Dockerfile, reference summary. ### List Recent Assets ``` GET /api/v1.0/search/assets/list?limit=20&offset=0&category=cs.LG&category=cs.CV&has_docker=true ``` | Parameter | Type | Description | | - | - | - | | `limit` | integer | Page size (default 20, max 100) | | `offset` | integer | Pagination offset | | `category` | string | Repeatable; a paper matches if any requested category hits | | `has_docker` | boolean | `true` or `false`; applied in the database query, so paging and `total` stay correct | **Response:** `{ assets, total, limit, offset }`, newest first. ### Get Asset Statistics ``` GET /api/v1.0/search/stats ``` Returns aggregate statistics: total assets, counts by Docker availability, by category, and citation classification coverage. # API Scopes Source: https://docs.remyx.ai/api-reference/scopes Key prefixes, the scope catalog, presets, and which surface needs what Remyx API keys are opaque, named tokens that carry an explicit set of **scopes**. Each scope gates which routes a key can call: a request to a route the key isn't scoped for returns `403 Forbidden`. Create and manage keys under **Account > API keys**. See [Account](/platform/configure/account) for that, and the [API authentication](/api-reference/authentication) for how scopes appear in REST calls. *** ## Key prefixes Every key's prefix tells you what it's for and how long it lives: | Prefix | Type | Use it for | Expiry | | - | - | - | - | | `rmxu_…` | **Personal** | Your CLI, MCP server, ad-hoc REST calls, CI scripts | None by default; optional 30, 90, or 365 days at creation | | `rmxa_…` | **Automation** | Minted by Outrider provisioning into the repo's Actions secrets | 365 days | Both are opaque (prefix + base62 entropy + CRC32 checksum). Remyx stores only `sha256(key)`. The plaintext is shown **once** at creation, then rendered as `rmxu_…last4` for identification. You create personal keys only. Automation keys are minted for you when you provision an agent (`remyxai outrider init`, `setup-local`, or the studio), with scopes fixed to what that repo's workflow calls (see [Automation key scopes](#automation-key-scopes)). You can rename or revoke an automation key; revoking it pauses the agent. A key's name and scopes can be edited after creation (`PATCH /account/api-keys/`); automation keys accept a rename only. Revoking keeps the key listed for audit; deleting removes it. *** ## Scope catalog Scopes are `category:access`. Access is either `read` or `write`, and **`write` implies `read`** (and `refresh`, where a route declares it) for the same category. This table is generated from the engine's scope registry and is what the key form shows. | Category | `read` grants | `write` grants (includes read) | | - | - | - | | `account` | Your profile and credit balance (`/user`, `/user/credits`) | *(no write scope)* | | `projects` | Projects, decision policies, eval templates, and project metrics | Create and modify projects, policies, and templates | | `interests` | Research interests and their experiment-history context; `/research-interests//refresh` needs `write` | Create, edit, refresh, toggle, and delete interests | | `recommendations` | Recommendations, digests, agent runs, the fleet report, the Inbox, and the Ask chat | Refresh recommendation pools; report run telemetry | | `papers` | Paper catalog, semantic search, and `/experiments/insights` | *(no write scope)* | | `experiments` | Experiments, experiment histories and relationships, and implementation runs | Create, update, decide, and run implementations | | `evals` | Evaluation environments, runs, builds, templates, and [Validate](/platform/agents/validate) runs, specs, and settings | Start validations, push specs, change Validate settings; create, run, and cancel evals | | `billing` | Prepaid balance, rates, transactions, and grant status (`GET /billing`) | Open a Stripe Checkout session to add credits | | `workflows` | Scheduled agent workflows (e.g. "weekdays at 9am, send the digest to #research") | Create, edit, test, and delete workflows | | `integrations` | Connected services, and **retrieval of stored third-party credentials** | Connect and disconnect services; link identities | | `github` | Remyx GitHub App install status and your installations | Mint repo-scoped bot tokens; flip the per-installation Validate switch; claim an installation | | `provisioning` | Provisioning status, your GitHub repos, the agent matrix | Install Outrider on repos, mint automation keys, dispatch runs, fork repos | > [!WARNING] > > `integrations:read` is broader than its name suggests. Besides connection status, it gates `GET /integrations//token`, which returns the decrypted credential for a connected service (an OAuth access token, or an API key you pasted). Grant it only to keys you would trust with those credentials. The **Read-only** preset deliberately leaves it out. > [!NOTE] > > `account` and `papers` are read-only; there is no `account:write` or `papers:write`. Profile and team settings change in the studio, not through a key. The deprecated `models`/`datasets` surface is **not** in the catalog; those routes are reachable only with a full-access key. ### Full access A key created without a scope list is **full access** (`*`): every route, including the key-management and provisioning routes that take no scoped alternative. The studio's key form asks you to pick features or choose full access explicitly; an empty scope list is rejected rather than silently becoming full access. ### Resource-qualified grants A scope can be narrowed to a specific resource by appending its UUID: ``` interests:read:3f2a9c1e-… # read only this one interest ``` Use resource-qualified grants when a key should touch a single resource rather than the whole category. Routes that resolve a resource (for example `GET /research-interests/`) honour them; the provisioning flows mint them for automation keys. *** ## Presets When creating a key you can apply a one-click preset instead of picking scopes by hand. Presets are starting points; the stored scopes are ordinary catalog entries you can adjust before creating the key. | Preset | Scopes granted | What it is for | | - | - | - | | **Outrider setup (CLI)** | `provisioning:write`, `interests:write`, `recommendations:write`, `github:read`, `integrations:write` | Everything `remyxai outrider init` does: create or pick an interest, check and install the App, connect the model provider, provision the repo, read and refresh recommendations. The repo secret gets its own narrower automation key | | **Experiment CI** | `experiments:write`, `evals:write`, `projects:read` | Run implementations and evaluations from CI against your projects | | **Digest bot** | `recommendations:read`, `papers:read`, `interests:read` | Read recommendations, papers, and interests for a Slack digest or dashboard | | **Read-only** | `account:read`, `projects:read`, `interests:read`, `recommendations:read`, `papers:read`, `experiments:read`, `evals:read`, `workflows:read`, `github:read`, `provisioning:read` | Read everything except integration credentials | All presets are personal (`rmxu_`) keys, since that is the only kind you can create. The Outrider setup preset needs only `github:read`: the `github:write` bot-token mint belongs to the automation key the provisioning flow creates. *** ## Automation key scopes The `rmxa_` key Outrider provisioning writes into a repo's secrets carries exactly the surface the workflow exercises: | Scope | Used for | | - | - | | `recommendations:write` | Read and refresh the interest's recommendations (`/papers/recommended`, `/refresh`), and report run telemetry | | `papers:read` | The coding agent's `remyxai search query` | | `github:write` | `POST /github/installation-token`, the remyx-ai\[bot] token the workflow commits and opens PRs with | | `interests:read:` | Only the interest the repo was provisioned for | The same set is minted by the App-install path and `setup-local`, so the two installs stay interchangeable. *** ## Which surface needs what | Surface | Scopes | | - | - | | **CLI / MCP, personal use** | A full-access key, or **Read-only** if you only consume data | | **`remyxai outrider init` / `setup-local`** | **Outrider setup (CLI)** preset | | **Validate from the API or CLI** | `evals:write`, plus `billing:read` to check the balance | | **CI pipelines** | **Experiment CI** preset | | **Digest / notification bot** | **Digest bot** preset, optionally resource-qualified to one interest | | **Billing top-ups** | `billing:write` on a user key (`rmxu_`); an automation key gets `403`. Studio uses the same route with its session | > [!TIP] > > Prefer a narrowly scoped key per use over one broad key everywhere. To rotate, create a new key and revoke the old one. *** ## Related > [card] [Account](/platform/configure/account) > > Create, scope, expire, and revoke named keys > [card] [API Reference](/api-reference/api-reference) > > REST endpoints and authentication > [card] [Connectors](/platform/configure/connectors) > > OAuth and API-key connections > [card] [CLI](/cli) > > Authenticate the CLI with a personal key # Validate API Source: https://docs.remyx.ai/api-reference/validate Request a validation, inspect a run, and read results through the Remyx API. Use the API to request and inspect validations for connected repositories. The API requires the documented authentication scheme and the appropriate `evals:read` or `evals:write` scope. ## Endpoints The endpoint definitions below describe accepted requests, status responses, run fields, artifacts, and errors. For a first run in GitHub, see [Validate](/platform/agents/validate). For manual configuration, see [Validation spec](/platform/agents/validate-spec). ### Runs [Request a validation](/api-reference/validate-request) · [Get a run](/api-reference/validate-get-run) · [List results and artifacts](/api-reference/validate-results) ### Model ``` GET /api/v1.0/validate/model?repo=owner/name # which provider would design a validation for this repo, and where that choice came from GET /api/v1.0/validate/settings?repo= # the account's Validate settings; with repo, that repo's row and the effective source PUT /api/v1.0/validate/settings # {repo?, installation_id?, model_provider?, model?, settings?}; omit both scopes for the account default; null clears a field DELETE /api/v1.0/validate/settings?repo= # remove a repo override, or the account default without repo ``` `PUT` returns `400 {code: "provider_not_connected", available_providers}` when the provider is not connected on the account, and `403 {code: "not_owner"}` for an installation setting from someone other than the installation's owner. ### Specs ``` GET /api/v1.0/validate/specs # runs grouped by spec_hash, each with the reconstructed .remyx/validation.yaml POST /api/v1.0/validate/specs/push # commit an edited spec back as remyx-ai[bot], always on a new branch with a PR POST /api/v1.0/validate/specs/lint # check a draft spec with the same parser a run uses; returns what the engine understood ``` ### Summary, signals, entitlement ``` GET /api/v1.0/validate/summary?repo= # verdict, calibration, and spend rollup behind the Studio Validations view POST /api/v1.0/validate/signals # production evidence for a validated change, scored against its offline prediction GET /api/v1.0/validate/entitlement?repo= # would a validation be admitted now, on what basis, how many are left; the refusal line and checkout link when not ``` `/validate/grants` and `/validate/proposals//{context, report}` and `/validate/runs//{fix-context, fix-report}` are service-only: they serve the proposer and fix jobs and the Remyx team. # Get a run Source: https://docs.remyx.ai/api-reference/validate-get-run One run's state, record, and artifact download links. Related guide: [Evaluate a pull request](/platform/agents/validate) · Example: [Run your first validation](/tutorials/validate-first-run) ``` GET /api/v1.0/validate/runs/ # one run's state and artifacts, with download URLs minted per request ``` ```bash Read one run theme={null} RUN_ID="YOUR_RUN_ID" curl -H "Authorization: Bearer $REMYXAI_API_KEY" \ "https://engine.remyx.ai/api/v1.0/validate/runs/$RUN_ID" ``` The response contains a `run` object, a `record` object (or `null` while in flight), and an `artifacts` array with time-limited download links. `record` is null while the run is in flight. Artifact download URLs are minted per request and expire. The full route list, including proposals, settings, specs and the entitlement check, is in the [Validate API reference](/api-reference/validate). # Request a validation Source: https://docs.remyx.ai/api-reference/validate-request Start or confirm a validation of a pull request. Related guide: [Evaluate a pull request](/platform/agents/validate) · Example: [Run your first validation](/tutorials/validate-first-run) ``` POST /api/v1.0/validate/runs ``` Start, or confirm, a validation of a PR. This is the same state machine the `@remyx-ai validate` comment drives, so confirming a proposal is invoking this again once the spec is committed. Returns `202`; progress and the verdict are posted on the PR. | Field | Type | Required | Description | | - | - | - | - | | `repo` | string | Yes | `owner/name` | | `pr_number` | integer | Yes | | | `guidance` | string | No | What to measure, in plain language; ignored with a warning when the PR carries a committed spec | | `model_provider` | string | No | One-off override of the provider that designs the test | | `model` | string | No | One-off model override | The caller must own the repo's App installation or be able to reach it. ```bash theme={null} curl -X POST https://engine.remyx.ai/api/v1.0/validate/runs \ -H "Authorization: Bearer $REMYXAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"repo": "owner/name", "pr_number": 42, "guidance": "does the new scheduler keep eval loss within 1% at half the steps"}' ``` The response carries an `outcome` and the `message` that was posted on the PR. `started` and `proposed` return `202`. Every other outcome maps to the status in the [Troubleshooting reference](/platform/agents/validate-troubleshooting#error-reference). Fields shown are a subset. ```json 202, a proposal was launched theme={null} {"outcome": "proposed", "message": "…", "repo": "owner/name", "pr_number": 42, "head_sha": "…", "proposal_id": "…", "proposal_status": "proposing"} ``` ```json 202, the run started theme={null} {"outcome": "started", "message": "…", "repo": "owner/name", "pr_number": 42, "head_sha": "…", "run_id": "…", "status": "…", "compute_tier": "cpu", "estimate": "…"} ``` ```json 402, refused theme={null} {"outcome": "gated", "message": "⛔ **Not started — …** … Nothing was used."} ``` # List results and artifacts Source: https://docs.remyx.ai/api-reference/validate-results Decision records, proposals, and the artifacts a run produced. Related guide: [Evaluate a pull request](/platform/agents/validate) ``` GET /api/v1.0/validate/runs?project_id=&repo= # decision records, newest first, each with the spec_hash it ran against GET /api/v1.0/validate/runs/ # one run's state and artifacts, with download URLs minted per request GET /api/v1.0/validate/proposals/ # a proposal as structured fields plus a tldr ``` # huggingface/diffusers Source: https://docs.remyx.ai/case-studies/diffusers Two training-free high-resolution methods for FLUX, one shipped as a community pipeline and one proposed as a hook, with the placement decision raised with maintainers first. Two complementary routes to 4K generation from off-the-shelf **FLUX** checkpoints, with no fine-tuning and no new weights, surfaced from arXiv, prototyped by Outrider, and taken toward `huggingface/diffusers`. The pair puts the same problem, training-free high resolution on a RoPE DiT, into two different library shapes. The choice of shape is itself the contribution. > [!NOTE] > > Status as of November 2026. The linked PRs and Issues are the live record. ## 1. HRDiT: community pipeline, closed unmerged | | | | - | - | | **Paper** | HRDiT, *Training-free high-resolution generation*, [arXiv:2608.07003](https://arxiv.org/abs/2608.07003); reference repo [zylwithxy/HRDiT](https://github.com/zylwithxy/HRDiT) (MIT) | | **Upstream PR** | [`huggingface/diffusers#14480`](https://github.com/huggingface/diffusers/pull/14480), closed 2026-09-02 by the repo's automation because no issue was linked; it can be reopened once the placement question in [#14520](https://github.com/huggingface/diffusers/issues/14520) settles | 4K (up to 4096²) from off-the-shelf FLUX.1-dev, with no fine-tuning, via NTK-aware RoPE scaling, monotonic bundle coarsening (SPA), and a structure-guided progressive ladder. Users get 4K straight from the base checkpoint instead of the generate-then-upscale or train-a-high-res-model workarounds. Submitted as a **self-contained community pipeline** (`examples/community/pipeline_flux_hrdit.py`) with no core-library changes. Built minimal-first: a position-alignment port established parity, then NTK RoPE scaling and structure guidance were layered onto only the stages that needed them. The speed-oriented HAP and DWT stages were documented and deferred, not shipped. ## 2. DyPE (plus optional spectral attention): hook, proposed | | | | - | - | | **Papers** | DyPE, *Dynamic Position Extrapolation*, [arXiv:2510.20766](https://arxiv.org/abs/2510.20766); SEGA, *Spectral-Energy Guided Attention*, [arXiv:2605.22668](https://arxiv.org/abs/2605.22668) | | **Upstream Issue** | [`huggingface/diffusers#14520`](https://github.com/huggingface/diffusers/issues/14520), placement proposal | `apply_dype(pipe.transformer)` swaps the transformer's rotary embedding for a timestep-aware YaRN / NTK-by-parts schedule (`method="yarn"`), a no-op at or below the trained resolution. `method="spectral"` adds a per-frequency, content-aware attention temperature computed from the latent's spectral energy at each denoising step. Validated on FLUX.1-Krea-dev at 4096²: * `method="yarn"` reproduces the reference implementation ([guyyariv/DyPE](https://github.com/guyyariv/DyPE), MIT) bit for bit (Δ=0 on the positional-embedding output). * `method="spectral"` cuts flat-sky high-frequency speckle about 6× (Laplacian variance 76.5 → 12.7) while keeping fine detail, a separation a single scalar attention temperature can't achieve. ![Sky speckle comparison between the yarn and spectral methods](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/diffusers/spectral_speckle.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=6d13e30bdfeb0f145d4dc6b929410ce9) DyPE spectral attention: sky speckle, yarn vs spectral State (the timestep, and for `spectral` the latent's spectral profiles) is fed through a native `register_forward_pre_hook`, so it survives `enable_model_cpu_offload`. The spectral mode implements SEGA but is exposed as `method="spectral"` to avoid colliding with the existing Semantic-Guidance "SEGA" in diffusers. ## Calibrated shape: the real contribution HRDiT self-contains cleanly as a community pipeline. DyPE, by contrast, modifies a model internal (the positional embedding) at inference, the same surface as diffusers' core hooks (`apply_faster_cache`, `apply_pyramid_attention_broadcast`). But training-free high-resolution methods have also historically landed as community pipelines (DemoFusion). Precedent points both ways, so the placement is raised with the maintainers in [#14520](https://github.com/huggingface/diffusers/issues/14520) **before** opening the PR rather than assumed. A pipeline-level caveat surfaced during validation: above about 2K, FLUX's default flow-matching shift (`mu`) grows with sequence length and collapses the sigma schedule, so both methods pin `base_shift == max_shift`. It's documented as a usage step, since a positional-embedding hook can't reach the pipeline's scheduler. ## Outrider's role Both methods were surfaced from arXiv and drafted by Outrider; DyPE through a [brief-anchored run](/platform/agents/outrider-run-modes#run-from-a-design-brief). The draft is the start. Parity checking, the spectral port, the shift and offload fixes, the rebase onto current `main`, and the maintainer-facing framing were the substantial follow-on. What Outrider removes is the cold start: turning "this paper looks relevant" into a wired, testable branch to harden. ## References * Listed under other public work on [remyx.ai/case-study](https://remyx.ai/case-study) * HRDiT: [arXiv:2608.07003](https://arxiv.org/abs/2608.07003) · [huggingface/diffusers#14480](https://github.com/huggingface/diffusers/pull/14480) * DyPE: [arXiv:2510.20766](https://arxiv.org/abs/2510.20766) · [guyyariv/DyPE](https://github.com/guyyariv/DyPE) (MIT) * SEGA: [arXiv:2605.22668](https://arxiv.org/abs/2605.22668) · [wildminder/ComfyUI-DyPE](https://github.com/wildminder/ComfyUI-DyPE) (Apache-2.0) * Placement proposal: [huggingface/diffusers#14520](https://github.com/huggingface/diffusers/issues/14520) # Case studies Source: https://docs.remyx.ai/case-studies/overview Real code changes, evaluations, and review outcomes from Remyx workflows. Real changes and evaluations, with links to the underlying code and evidence. ## Merged contributions Review examples of Outrider-assisted changes that passed upstream maintainers' standards. [Read the upstream case study](https://remyx.ai/case-study) ## Validation evidence SAELens compared TopK and AbsTopK against two offline criteria. Both passed in a single-seed CPU run. The result does not establish production impact. [Read the SAELens validation](https://github.com/salma-remyx/SAELens/pull/5) # huggingface/peft Source: https://docs.remyx.ai/case-studies/peft Two parameter-efficient fine-tuning methods surfaced from arXiv, drafted by Outrider, and merged into huggingface/peft, both featured in the v0.21.0 release notes. Two parameter-efficient fine-tuning methods surfaced from arXiv, drafted by Outrider, and taken to `huggingface/peft`, whose tuner surface is used directly by five of six major fine-tuning frameworks (per Scaling DoRA's Appendix G). Both merged and are credited in the [v0.21.0 release notes](https://github.com/huggingface/peft/releases/tag/v0.21.0); the second has the paper's first author co-credited. > [!NOTE] > > Status as of November 2026. The linked PRs are the live record. ## 1. Riemannian Preconditioned LoRA: merged | | | | - | - | | **Paper** | Zhang & Pilanci, *Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models*, [arXiv:2402.02347](https://arxiv.org/abs/2402.02347) | | **Upstream PR** | [`huggingface/peft#3382`](https://github.com/huggingface/peft/pull/3382), merged 2026-08-03 (+401 / −3, 6 files, 33 days from open to merge) | | **Original attempt** | [`huggingface/peft#1807`](https://github.com/huggingface/peft/pull/1807), the authors' own PR, open May to September 2024 and closed unmerged with [maintainer follow-up on the unfinished work](https://github.com/huggingface/peft/pull/1807#issuecomment-2283410068) | | **Coordination** | [`huggingface/peft#3380`](https://github.com/huggingface/peft/issues/3380); the [original author supported a fresh PR](https://github.com/huggingface/peft/issues/3380#issuecomment-4858962847) and was co-credited | | **Review** | [First review](https://github.com/huggingface/peft/pull/3382#pullrequestreview-4625325791): "this PR is already quite mature"; [final implementation commit](https://github.com/huggingface/peft/pull/3382/commits/2c436f3fa122c562a7908d4357640c3158a8076c); five commits, two approvals | | **Release** | [PEFT v0.21.0](https://github.com/huggingface/peft/releases/tag/v0.21.0) | Replaces the Euclidean gradient with the natural gradient under the Riemannian metric induced by the low-rank matrix manifold. Aligns optimizer steps with the intrinsic tangent space and quotients the (A, B) ↦ (A·C, C⁻¹·B) reparameterization gauge that Euclidean SGD and Adam depend on. Nine targeted tests cover the factory path, optimizer compatibility, DoRA compatibility, error handling, the paper formula, and bf16 finiteness. The disclosed benchmark did not beat LoRA in this setting: matching the reference damping default moved GSM8K accuracy from 0.450 to 0.475 against 0.490 for default LoRA in the same rank-32 MetaMathQA configuration, and the result stayed visible when the contribution merged. Review exposed ambiguity around optimizer closures, so the final implementation rejects them. ![Riemannian Preconditioned LoRA geometric intuition](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/riemannian_lora.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=32f9bae15eedbbd8340c289fd7d67981) Riemannian Preconditioned LoRA: geometric intuition ## 2. Super-Tuning & Supra: merged | | | | - | - | | **Paper** | Ilin, Zmushko & Richtárik, *Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning*, [arXiv:2607.09287](https://arxiv.org/abs/2607.09287) | | **Upstream PR** | [`huggingface/peft#3518`](https://github.com/huggingface/peft/pull/3518), merged 2026-08-21 (+1365 / −6, 28 files, 16 days from open to merge) | | **Coordination** | [`huggingface/peft#3450`](https://github.com/huggingface/peft/issues/3450), direction confirmed by the maintainer | | **Author coordination** | [`vectozavr/SuperTuning#3`](https://github.com/vectozavr/SuperTuning/issues/3); the paper's first author is co-credited | | **Release** | [PEFT v0.21.0](https://github.com/huggingface/peft/releases/tag/v0.21.0) | Freezes the base weight and trains only a sparse support of scalar entries selected by weight magnitude, a distinct point on the trainable-parameter Pareto front versus LoRA. Setting `r > 0` additionally allocates a LoRA-style low-rank adapter composed additively (the paper's Supra hybrid). Runs against `method_comparison/MetaMathQA/` produce numbers directly comparable to the harness's existing LoRA and SHiRA baselines. Five Super and Supra configurations ran against the existing LoRA and SHiRA baselines in PEFT's MetaMathQA harness. Results were mixed and are disclosed on the PR: the best Supra row reached 0.4632 test accuracy against 0.4784 for LoRA, one run per configuration. A post-review custom-model run passed 262 tests with 10 skipped. Review focused on integration and testing; the API surface was reduced and checkpoint handling aligned with PEFT conventions over ten commits, and the maintainer ran the example before approval. The evidence does not establish a general performance gain for Super-Tuning or Supra. ![Super-Tuning and Supra sparse plus low-rank subspaces](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/supertuning.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=260a906c7b8e083f4b5306751f76d96b) Super-Tuning & Supra: sparse ⊕ low-rank subspaces ## Composite analytics for PR #3382 ![Composite analytics for the merged Riemannian LoRA PR](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/case-studies/peft/composite_pr3382_analytics.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=a3f1517d5a28c6c1bbdb5a384b90f67e) Six panels of the merged Riemannian LoRA PR against the year's other merged tuner PRs: **size** (401 LOC, p82); **composition** (41% test / 48% src / 11% benchmark, above the cohort median's 13% test / 1% benchmark); **merge latency** (32.7 days, p89; landed despite a first-time contributor); **scope shape** (focused and additive, no unrelated refactors); **author recurrence** (first merged PR to a repo where the top maintainer has 174); **feature-utility signals** (registry attachment, backwards-compat commitment, benchmark-harness registration under `method_comparison/MetaMathQA/`). ## PR sizing against the year's merged tuner PRs Compared to `huggingface/peft`'s other tuner-adding PRs merged since 2025 (via `gh pr view `): | PR | Method | Merged | +Additions | −Deletions | Files | | -: | - | :-: | -: | -: | -: | | [#2584](https://github.com/huggingface/peft/pull/2584) | SHiRA | 2025-07-14 | +1623 | −9 | 27 | | [#2851](https://github.com/huggingface/peft/pull/2851) | GraLoRA | 2025-11-18 | +1238 | 0 | 20 | | [#3037](https://github.com/huggingface/peft/pull/3037) | PSoFT | 2026-02-27 | +1556 | −1 | 23 | | [#3084](https://github.com/huggingface/peft/pull/3084) | PEANuT | 2026-03-16 | +1096 | 0 | 23 | | [#3195](https://github.com/huggingface/peft/pull/3195) | BEFT | 2026-04-30 | +773 | −5 | 22 | | [**#3382**](https://github.com/huggingface/peft/pull/3382) | **Riemannian LoRA** | **2026-08-03** | **+401** | **−3** | **6** | | [**#3518**](https://github.com/huggingface/peft/pull/3518) | **Super-Tuning & Supra** | **2026-08-21** | **+1365** | **−6** | **28** | **#3382** landed at the compact end (401 LOC against a cohort median around 1150): small, focused, all-additive. **#3518** sits at the cohort median (1365 LOC, 28 files), the same shape as PSoFT, PEANuT and GraLoRA. ## Outrider's role Outrider surfaced both candidates and drafted the first implementations and tests. Human review shepherded each through coordination-issue-first upstreaming: 1. **Method discovery.** The arXiv-to-repo mapping identifies candidates whose mechanisms haven't been ported to the host repo. 2. **Draft.** Outrider produces an initial branch with tests, a docs page, and integration hooks. 3. **Human refinement and coordination.** A coordination Issue upstream (per PEFT's `CLAUDE.md`), with maintainer feedback incorporated before the PR is filed. 4. **Upstream PR.** Filed against `huggingface/peft:main` referencing the coordination Issue and, where applicable, paper-author sign-off. The composite above is the point of this rhythm: a first-time contributor's PR landed through PEFT's review process because it arrived shaped the way the maintainer expects. Focused, additive, test-heavy, with a durable feature-utility signal. ## References * The full record, with evidence links: [remyx.ai/case-study](https://remyx.ai/case-study) * PEFT contribution guide: [https://huggingface.co/docs/peft/main/en/developer\_guides/contributing](https://huggingface.co/docs/peft/main/en/developer_guides/contributing) * PEFT `CLAUDE.md` (AI-assistance policy): [https://github.com/huggingface/peft/blob/main/CLAUDE.md](https://github.com/huggingface/peft/blob/main/CLAUDE.md) * The paper infographics were generated with [`llmsresearch/paperbanana`](https://github.com/llmsresearch/paperbanana). # CLI & tools Source: https://docs.remyx.ai/cli Install and authenticate the CLI, configure Outrider, and use supported MCP tooling. Use Remyx from the terminal, GitHub Actions, or a supported agent client. ## Start with the CLI Install and authenticate the CLI, then select a supported command. Use the reference to check required flags, permissions and expected output. [Install the CLI](/cli/install) · [Authenticate](/cli/authenticate) ## Outrider commands Set up a repository, trigger Outrider and inspect the resulting GitHub Actions run. [Outrider CLI commands](/cli/outrider) · [Set up a repository](/cli/outrider-setup) · [Trigger a run](/cli/outrider-trigger) ## Other commands The reference below includes the supported discovery, recommendation, account and workflow commands. [Interests and discovery](/cli/interests) · [Recommendations and search](/cli/recommendations) · [CLI reference](/api-reference/cli-reference) ## Supported command groups | Group | What it does | | - | - | | `remyxai interests` | Manage Research Interests — the context that drives personalized recommendations | | `remyxai papers` | Get daily recommendations, view digests, and trigger refreshes | | `remyxai search` | Search across papers, repos, models, and datasets | | `remyxai outrider` | Install and drive [Outrider agents](/platform/agents/overview) — the [Outrider action](/platform/agents/outrider-action) — on your repos | Run `remyxai GROUP --help` for the options in a group. Current definitions are in the [CLI source](https://github.com/remyxai/remyxai-cli). ## Related interfaces [Quickstart](/quickstart) · [API reference](/api-reference/api-reference) · [MCP server](/resources/mcp-server) # Authenticate Source: https://docs.remyx.ai/cli/authenticate Export your Remyx API key for the CLI. ## Authentication Remyx authenticates API calls with a bearer token. Export it before running commands: ```bash macOS / Linux theme={null} export REMYXAI_API_KEY="YOUR_REMYX_API_KEY" ``` ```powershell Windows (PowerShell) theme={null} $env:REMYXAI_API_KEY = "YOUR_REMYX_API_KEY" ``` The client uses this variable when calling `https://engine.remyx.ai/api/v1.0`. Do not commit keys or print them in terminal output. Use an appropriate environment or secret store, and keep any local `.env` file out of version control. > [!NOTE] > > Your CLI auth is always a single personal key (`rmxu_`) in `REMYXAI_API_KEY`. When Outrider provisions a repo (`outrider init` / `setup-local`, or `interests` with `--automate review`/`auto`), it writes a *scoped* automation key (`rmxa_`) into the repo's secrets as `REMYX_API_KEY`. Your personal key stays on your machine. See [API key scopes](/api-reference/scopes) for the difference. # Install the CLI Source: https://docs.remyx.ai/cli/install Install the Remyx CLI, configure an API key, and verify the commands. Use the Remyx CLI to set up Outrider and access supported research and repository commands from your terminal. ## Requirements * Python 3 and `pip` * A Remyx API key from [Studio → Account → API keys](https://studio.remyx.ai/account?tab=keys) ## Install ```bash theme={null} python -m pip install remyxai remyxai --help ``` ## Authenticate Set your key in the same shell session where you run the CLI. See [Authenticate](/cli/authenticate) for macOS, Linux, and PowerShell examples. ```bash theme={null} export REMYXAI_API_KEY="YOUR_REMYX_API_KEY" ``` Keep your key out of committed files and terminal logs. Use the secret store appropriate for your environment. ## Troubleshooting > [accordion] REMYXAI\_API\_KEY is missing or invalid > > Check whether the variable is set without printing its value: > > ```bash theme={null} > if [ -n "${REMYXAI_API_KEY:-}" ]; then > echo "API key is set" > else > echo "API key is missing" > fi > ``` > > If it is missing, export the key in the shell running `remyxai`. If authentication fails, check the key and its [scopes](/api-reference/scopes) in Studio. > [accordion] Command not found: remyxai > > Verify the package is installed in your active Python environment: > > ```bash theme={null} > python -m pip show remyxai > ``` > > Use the same Python environment for installation and execution. Ensure its scripts directory is on your `PATH`. Avoid using `sudo pip install` to solve a user environment issue. ## Next steps [Set up a repository](/cli/outrider-setup) · [CLI command reference](/api-reference/cli-reference) · [Quickstart](/quickstart) # Interests and discovery Source: https://docs.remyx.ai/cli/interests Manage Research Interests from the terminal. ## Research Interests Research Interests define what you care about. They drive the recommendation engine — each interest has its own context and daily recommendation cadence. ### List Interests ```bash theme={null} remyxai interests list ``` Shows all your Research Interests with name, context preview, daily count, and active status. ```bash Text output (default) theme={null} remyxai interests list ``` ```bash JSON output theme={null} remyxai interests list --format json ``` ### Create an Interest ```bash theme={null} remyxai interests create \ --name "Retrieval-Augmented Generation" \ --context "https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B" \ --daily-count 3 ``` | Flag | Description | Default | | - | - | - | | `--name, -n` | Short label for the interest | *(prompted if omitted)* | | `--context, -c` | Natural language description, or URLs to HuggingFace models, GitHub repos, arXiv papers | *(prompted if omitted)* | | `--daily-count, -d` | Recommendations per day (1-10) | 2 | | `--inactive` | Create as inactive (excluded from daily digest until toggled) | — | | `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` | | `--wait, -w` | Block until the refresh completes | — | | `--format, -f` | Output format: `text` or `json` | `text` | > [!TIP] > > Run `remyxai interests create` with no flags for an interactive prompt that walks you through each field. ### Create from a GitHub repo ```bash theme={null} remyxai interests from-repo https://github.com/your-org/your-project ``` Builds an interest from a repo's profile (languages, topics, README, dependencies) instead of free text. Pass `--automate review` or `--automate auto` to also provision [Outrider](/platform/agents/outrider-action) on that repo in the same step, the same flow as `outrider init`. | Flag | Description | Default | | - | - | - | | `--name, -n` | Short label for the interest | *(derived from the repo)* | | `--daily-count, -d` | Recommendations per day (1-10) | 2 | | `--inactive` | Create as inactive | — | | `--automate` | `none`, `review`, or `auto`. `review`/`auto` provision Outrider on the repo like `outrider init` | `none` | | `--timeout` | Seconds to wait on Outrider provisioning when automating | 300 | | `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` | | `--wait, -w` | Block until the refresh completes | — | | `--format, -f` | Output format: `text` or `json` | `text` | ### Create from a project's experiments ```bash theme={null} remyxai interests from-project "My RAG Project" ``` Builds an interest from a project's experiments. The `` argument accepts a project name or UUID. The interest tracks the project by default, so new experiments keep it current; pass `--no-auto-update` to take a one-time snapshot instead. | Flag | Description | Default | | - | - | - | | `--name, -n` | Short label for the interest | *(derived from the project)* | | `--daily-count, -d` | Recommendations per day (1-10) | 2 | | `--inactive` | Create as inactive | — | | `--include-experiment, -e` | Limit context to a specific experiment (repeatable) | *(all experiments)* | | `--no-auto-update` | Snapshot the project once instead of tracking it | — | | `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` | | `--wait, -w` | Block until the refresh completes | — | | `--format, -f` | Output format: `text` or `json` | `text` | ### Get Interest Details ```bash theme={null} remyxai interests get --interest "Retrieval-Augmented Generation" ``` The `--interest` flag accepts either a **name** (case-insensitive) or a **UUID**. ### Update an Interest ```bash theme={null} remyxai interests update \ --interest "Retrieval-Augmented Generation" \ --context "https://github.com/your-org/your-rag-system" \ --daily-count 5 ``` | Flag | Description | | - | - | | `--interest, -i` | Interest name or UUID (required) | | `--name, -n` | New name | | `--context, -c` | New context (invalidates recommendation pool and triggers re-ranking) | | `--daily-count, -d` | New daily count | | `--activate` | Set active | | `--deactivate` | Set inactive | > [!NOTE] > > Updating the `context` invalidates the pre-ranked recommendation pool. Run `remyxai papers refresh` afterward to get fresh results immediately. ### Toggle Active Status ```bash theme={null} remyxai interests toggle --interest "Retrieval-Augmented Generation" ``` Toggles between active and inactive without deleting the interest or its recommendation history. ### Delete an Interest ```bash theme={null} remyxai interests delete --interest "Retrieval-Augmented Generation" ``` Prompts for confirmation. Pass `--yes` to skip: ```bash theme={null} remyxai interests delete --interest "Retrieval-Augmented Generation" --yes ``` # Outrider CLI commands Source: https://docs.remyx.ai/cli/outrider Initialize a repository, trigger Outrider, and inspect the supported run options. Use the CLI to configure a repository and trigger Outrider runs. ## Initialize Set up the workflow and repository configuration using the supported init command. ## Trigger Start a run against the configured repository. You can let Outrider select a change or pin a supported input. ## Options See the full reference below for required flags, backend selection, run status, and output behavior. ## Review Use the GitHub Actions run and any generated PR, Issue, or branch to review the work. *** Install [automated discovery PRs](/platform/agents/outrider-action) on a GitHub repo without leaving the terminal. The CLI drives the same "set it up for me" flow as the web app — your local git is never touched. ## Quick Reference | Intent | Command | | - | - | | Get today's digest | `remyxai papers digest` | | Get this week's digest | `remyxai papers digest --period week` | | Recommendations for one interest | `remyxai papers list --interest "RAG" --period today` | | Refresh all interests | `remyxai papers refresh --wait` | | Refresh one interest | `remyxai papers refresh --interest "RAG" --wait` | | List interests | `remyxai interests list` | | Create an interest | `remyxai interests create --name "RAG" --context "..." --daily-count 3` | | Create an interest from a repo | `remyxai interests from-repo https://github.com/your-org/your-project` | | Create an interest from a project | `remyxai interests from-project "My RAG Project"` | | Toggle an interest | `remyxai interests toggle --interest "RAG"` | | Delete an interest | `remyxai interests delete --interest "RAG" --yes` | | Search for resources | `remyxai search query "your query" --max-results 10` | | Search Docker-ready only | `remyxai search query "your query" --docker` | | Get resource details | `remyxai search info 2308.12345` | | Browse recent resources | `remyxai search list --limit 20` | | Catalog stats | `remyxai search stats` | | Install an agent | `remyxai outrider init --repo owner/name --auto-interest` | | Install without the GitHub App | `remyxai outrider setup-local --repo owner/name --auto-interest` | | Fire a one-shot Outrider run | `remyxai outrider trigger --repo owner/name` | | Implement an exact paper | `remyxai outrider trigger --repo owner/name --pin-arxiv 2402.02347v3` | | Implement a brief you wrote | `remyxai outrider trigger --repo owner/name --brief @design.md` | | Run a method via z.ai GLM | `remyxai outrider trigger --repo owner/name --search-method "..." --provider zai --model glm-5.3` | | Run a method via Moonshot Kimi | `remyxai outrider trigger --repo owner/name --search-method "..." --provider moonshot --model kimi-k3 --agent-timeout 3600` | | Run a method on Codex instead of Claude Code | `remyxai outrider trigger --repo owner/name --search-method "..." --agent codex --provider openai` | | Keep the provider key in GitHub secrets only | `remyxai outrider init --repo owner/name --auto-interest --github-secrets-only` | | Two-tier drafter/refiner install | `remyxai outrider setup-local --repo owner/name --auto-interest --two-tier` | | Set a provider's API key on a repo | `remyxai outrider set-provider-secret --repo owner/name --provider zai --key-from ~/zai-key` | # Set up a repository Source: https://docs.remyx.ai/cli/outrider-setup Install Outrider on a repository with or without the Remyx GitHub App, and set provider secrets. Task guide: [Find and draft changes](/platform/agents/outrider-action) ```bash theme={null} remyxai outrider init --repo OWNER/REPO --auto-interest ``` ## Install on a repo ```bash theme={null} remyxai outrider init --repo owner/name --auto-interest ``` This drives the Remyx engine so the **Remyx GitHub App** (`remyx-ai[bot]`) sets the repo secrets, writes the workflow, opens a bot-authored setup PR, and — in `auto` mode — merges it and fires the first run. No personal `gh` token is needed; only your `REMYXAI_API_KEY`. | Flag | Description | Default | | - | - | - | | `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from the current directory's git remote)* | | `--interest, -i` | Research Interest UUID to wire in | *(prompted / see `--auto-interest`)* | | `--auto-interest` | Auto-create a Research Interest from the repo (mutually exclusive with `--interest`) | — | | `--bulk-repos` | Path to a TSV of `owner/nameinterest_uuid` rows; provisions each in turn with per-row error capture. Mutually exclusive with `--repo` / `--interest` / `--auto-interest` | — | | `--pace` | Seconds to wait between repos in a `--bulk-repos` run | 3 | | `--mode` | `auto` (provision + merge setup PR + first run), `review` (provision + open setup PR to review), `off` (create the interest only) | `auto` | | `--agent` | Which coding-agent CLI runs the implementation: `claude`, `codex`, or `backboard`. A separate axis from the model provider. The CLI reads the provisioned workflow back and confirms the agent it was written with | `claude` | | `--anthropic-key` | Anthropic API key to connect as the model provider when none is connected (use [Connectors](/platform/configure/connectors#model-providers-and-coding-agents) for any other provider). Falls back to `$ANTHROPIC_API_KEY` | *(only used if no provider is connected)* | | `--github-secrets-only` | Keep the provider key in the repo's GitHub Actions secrets only: the CLI seals it against the repo's Actions public key on your machine and the engine relays the ciphertext, so no copy is stored on Remyx. See [Where the model key lives](/platform/agents/security#where-the-model-key-lives) | — | | `--force` | Re-drive provisioning on a repo that is already installed (rewrite the workflow, pick up new dispatch inputs) without revoking its automation key | — | | `--no-wait` | Don't block polling for App install / provisioning; print next steps and return | — | | `--dry-run` | Print the plan and exit without making changes | — | | `--yes, -y` | Skip the confirmation prompt | — | > [!NOTE] > > Connect the [GitHub](/platform/configure/github) and a [model provider](/platform/configure/connectors#model-providers-and-coding-agents) (Anthropic, Z.ai, Moonshot, OpenAI, or OpenRouter) integration in [Connectors](/platform/configure/connectors) first; `outrider init` uses those connections. If the GitHub App isn't installed on the repo, the command surfaces the install link, and you can pass `--anthropic-key` to set the Anthropic key the first time. ```bash Auto-create the interest and set it all up theme={null} remyxai outrider init --repo remyxai/RepoRanger --auto-interest ``` ```bash Wire in an existing interest, review the setup PR theme={null} remyxai outrider init --repo owner/name --interest --mode review ``` ```bash Non-interactive (CI) theme={null} REMYXAI_API_KEY=... ANTHROPIC_API_KEY=... \ remyxai outrider init --repo owner/name --interest --yes ``` ## Install without the GitHub App For organizations that can't grant a third-party GitHub App yet (e.g. a pending security review), `setup-local` does the same provisioning using **your own authenticated `gh` CLI** instead of the Remyx App — nothing new to security-review. The running Action opens PRs with the repo's built-in `GITHUB_TOKEN`. ```bash theme={null} remyxai outrider setup-local --repo owner/name --auto-interest ``` | Flag | Description | Default | | - | - | - | | `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* | | `--interest, -i` | Research Interest UUID to wire in | — | | `--auto-interest` | Auto-create a Research Interest from the repo | — | | `--bulk-repos` | Path to a TSV of `owner/nameinterest_uuid` rows; provisions each in turn with per-row error capture. Mutually exclusive with `--repo` / `--interest` / `--auto-interest` | — | | `--pace` | Seconds to wait between repos in a `--bulk-repos` run | 3 | | `--mode` | `auto` (open + merge setup PR + dispatch first run) or `review` (open the setup PR for you to merge) | `auto` | | `--no-cron` | Render the workflow with the cron schedule commented out, keeping only `workflow_dispatch` (manual / `outrider trigger` runs) | — | | `--anthropic-key` | Anthropic key to set as the `ANTHROPIC_API_KEY` repo secret. Falls back to `$ANTHROPIC_API_KEY`, then prompts | — | | `--agent` | Which coding-agent CLI the generated workflow runs by default: `claude`, `codex`, or `backboard`. Sets the `agent` dispatch input's default, so one install can still switch agents per run — provided that pair's secret is on the repo | `claude` | | `--backend` | Default model backend for the single-file setup: `anthropic`, `openai`, `zai`, `moonshot`, or `openrouter` — on either API family, since the agent axis decides which one is spoken. Unset, it follows the agent to its own vendor (`codex` → `openai`). The rendered workflow declares `agent`/`provider`/`model` dispatch inputs either way, so any install can be re-routed per run | *(the agent's own vendor)* | | `--model` | Model id the install runs by default, from the backend you picked (e.g. `gpt-5.4-mini` on `openai`, `glm-5.3` on `zai`). Becomes the `model` dispatch input's default. Required with `--agent backboard`, which addresses models as `/` and has no default of its own | *(the agent's own default)* | | `--zai-key` | Z.ai GLM Coding Plan key, set as the `ZAI_API_KEY` repo secret when a GLM model is in play. Falls back to `$ZAI_API_KEY` | — | | `--two-tier` | Write the [two-tier drafter/refiner](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) layout: a dispatch-capable main workflow, a daily branch-mode drafter, and a weekly refiner | — | | `--drafter-model` / `--refiner-model` | Models for the two tiers — any tier from any supported provider, mixed freely (e.g. a `glm-5.2` drafter with a `claude-opus-4-8` refiner, or a `kimi-k3` drafter) | — | | `--dry-run` | Print the plan + rendered workflow and exit | — | | `--yes, -y` | Skip the confirmation prompt | — | > [!NOTE] > > `setup-local` needs an authenticated `gh` (run `gh auth login`, or set `$GITHUB_TOKEN` with `repo` + `workflow` scopes) and admin on the target repo. The only runtime Remyx dependency is the `REMYX_API_KEY` the workflow uses. > > It writes only the selected backend's secret; add another with [`set-provider-secret`](#set-a-providers-api-key-on-a-repo) to switch per dispatch. Claude Code and Codex read no credential of their own — the action derives theirs from the provider's key — so don't set an agent-named secret for them. Backboard is the exception: its own key *is* the credential (`gh secret set BACKBOARD_API_KEY`). ## Set a provider's API key on a repo `set-provider-secret` writes a provider's API key into the repo's secrets safely. It reads the key from a file, strips the trailing newline, and sidesteps the `gh secret set --body -` truncation trap that otherwise stores a literal `-` and breaks every run with a `401`. ```bash theme={null} remyxai outrider set-provider-secret \ --repo owner/name --provider zai --key-from ~/zai-key ``` | Flag | Description | Default | | - | - | - | | `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* | | `--provider` | Which provider the key is for; selects the secret name by the `_API_KEY` convention (`anthropic` → `ANTHROPIC_API_KEY`, `openai` → `OPENAI_API_KEY`, `zai`, `moonshot`, `openrouter`). Covers every provider the action understands, which is wider than the set `init` can provision through the engine. **Required** | — | | `--key-from` | Path to a file containing only the API key. **Required** | — | See [Automated discovery PRs](/platform/agents/outrider-action) for what the action does once installed, its configuration inputs, and how to review the PRs it opens. # Trigger a run Source: https://docs.remyx.ai/cli/outrider-trigger Dispatch a one-shot Outrider run, route it at a provider, or switch the coding agent. Task guide: [Find and draft changes](/platform/agents/outrider-action) ```bash theme={null} remyxai outrider trigger --repo OWNER/REPO ``` ## Trigger a one-shot run Once Outrider is installed on a repo, `trigger` fires a single `workflow_dispatch` run of the workflow through your local `gh`. Useful for testing or kicking a run between cron cycles without waiting for the schedule. Four ways to say what to implement, in ascending order of override: nothing (the ranked pool), `--search-method`, `--pin-arxiv`, or `--brief`. ```bash theme={null} # the normal selection pass over the interest's ranked pool remyxai outrider trigger --repo owner/name # search for a method family and implement the top hit remyxai outrider trigger --repo owner/name --search-method "riemannian preconditioning LoRA optimizer" # implement an exact paper (any published arXiv id, not only pool members) remyxai outrider trigger --repo owner/name --pin-arxiv 2402.02347v3 # implement a brief you wrote: inline, or @path to a file remyxai outrider trigger --repo owner/name --brief "add exponential backoff to the HTTP client" remyxai outrider trigger --repo owner/name --brief @design.md ``` | Flag | Description | Default | | - | - | - | | `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* | | `--interest, -i` | Research Interest to run against | *(the repo's configured interest)* | | `--search-method` | A free-text method query. Runs an engine search and implements the top hit, bypassing the ranked pool and the selection pass. Mutually exclusive with `--pin-arxiv` and `--brief` | — | | `--pin-arxiv` | An exact arXiv id (e.g. `2402.02347v3`). Bypasses selection and implements that paper. It need not be in the candidate pool: the action looks it up in the Remyx catalog, then on arxiv.org. Mutually exclusive with `--search-method` and `--brief` | — | | `--brief` | A design brief the agent implements directly: inline text, or `@path` to read a file. No arXiv anchor; the brief is the spec. Shorthand for `--mode brief` with the brief as the lead content. Mutually exclusive with `--pin-arxiv`, `--search-method`, `--lead-content` and `--lead-content-file`; combines with `--start-from-ref` and `--publish branch`. Refused on an install whose workflow lacks the `mode` and `lead-content` inputs (re-run `init --force`), rather than silently dispatched as a paper run | — | | `--mode` | Run mode: `recommend` (the workflow default), `brief` (set for you by `--brief`), `smoke`, or any mode the installed action supports | *(workflow default)* | | `--ref` | Git ref to dispatch the workflow on; this picks which branch the workflow *file* comes from, not what the agent builds on | *(repo default branch)* | | `--start-from-ref` | Branch the agent **builds on** instead of implementing from scratch: a drafter branch, or a third party's stalled PR branch | — | | `--lead-content` / `--lead-content-file` | Markdown fed to the agent as leading context (a gap analysis) in recommend mode; `--lead-content-file` reads it from disk. Mutually exclusive with each other and with `--brief`. Capped at about 60k characters by GitHub's dispatch payload limit | — | | `--staged-synthesis` | Turn on the multi-pass staged-synthesis flow (what the two-tier refiner uses for its promotion pass) | off | | `--publish` | `pr` opens a draft PR or Issue; `branch` pushes the drafter branch without opening anything, so you can review before publishing | *(workflow default, `pr`)* | | `--fidelity-policy` | Pre-PR fidelity gate for this run: `block`, `advisory`, or `off`. `advisory` suits a human-vetted warm-start branch | *(workflow default)* | | `--test-integration-policy` | Override the run's test-integration policy (`strict` / `soft` / `off`) | *(workflow default)* | | `--wait-for-slot` | If a run is already pending on the repo, wait for it to start instead of dispatching (which would cancel it on an install with a flat concurrency group). Use when firing several dispatches at one repo | off | | `--base-url` | Route the coding agent at an Anthropic-compatible endpoint of your own (a proxy, an on-prem gateway). Requires the workflow to declare a `base-url` input | — | | `--agent` | Which coding-agent CLI runs the implementation on this dispatch (`claude`, `codex`, `backboard`) — a separate axis from `--provider`. An impossible agent/provider pair is rejected locally, before the dispatch is spent. The target workflow must declare an `agent` input; re-run `init --force` or `setup-local` on installs predating it | *(workflow default)* | | `--agent-timeout` | Wall-clock seconds for each agent phase on this dispatch (preflight and implementation share the budget). Raise for very large monorepos or thinking-mode backends. Only Claude Code has a round cap, so on `codex` and `backboard` this is the only bound on spend. `--claude-timeout` is the old name and still works | *(workflow default)* | | `--provider` | Route the coding agent at a specific model backend for this dispatch (`anthropic`, `openai`, `zai`, `moonshot`, `openrouter`). The target workflow must declare a `provider` dispatch input, which `setup-local` workflows do | *(workflow default)* | | `--model` | Specific model to request from the provider (e.g. `claude-opus-4-8`, `glm-5.3`, `kimi-k3`, `gpt-5.4-mini`), forwarded as the selected agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, …) | *(provider's default)* | > [!NOTE] > > `trigger` needs an authenticated `gh` (run `gh auth login`). It refuses up front if no Outrider workflow is registered on the repo; install one first with `outrider init` or `setup-local`. > > Inputs the installed workflow doesn't declare are dropped with a warning and the dispatch retried, so older installs still run on their baked defaults. The exceptions are the inputs a dispatch cannot do without: `--brief` refuses rather than running a paper pass, and `--provider` without `--agent` refuses when the install's own agent cannot speak to that provider. `remyxai outrider init --repo owner/name --force` rewrites the workflow with the current inputs. ## Route a run at a different model provider Outrider's coding runtime speaks whichever model-API family its coding agent speaks, so runs can route at any compatible provider. A `setup-local` workflow declares `agent`, `provider`, and `model` as dispatch inputs — the action's [`provider` input](/platform/agents/backends#the-provider-input) resolves the credential, base URL, and auth style per run — so one workflow can A/B Anthropic against Z.ai's GLM or Moonshot's Kimi: ```bash theme={null} # one run on z.ai's GLM; the workflow's own provider stays the default remyxai outrider trigger --repo owner/name \ --search-method "speculative decoding" --provider zai --model glm-5.3 # the same run on Moonshot's Kimi (raise the timeout for thinking-mode latency) remyxai outrider trigger --repo owner/name \ --search-method "speculative decoding" --provider moonshot --model kimi-k3 \ --agent-timeout 3600 ``` `--provider` names the API endpoint; `--model` picks the model from it. Omit either to fall back to the workflow's (or the provider's) default. See [Model backends](/platform/agents/backends) for the full matrix — secrets, auth styles, recommended timeouts, and cost telemetry. ## Switch the coding agent for a run `--agent` is the second axis: which coding-agent CLI does the implementing, independent of which model serves it. ```bash theme={null} # Codex on OpenAI remyxai outrider trigger --repo owner/name \ --search-method "speculative decoding" --agent codex --provider openai --model gpt-5.4-mini # Backboard's router — it addresses models as /, so name one remyxai outrider trigger --repo owner/name \ --agent backboard --provider openrouter --model z-ai/glm-5.3 ``` An agent speaks one API family and a provider serves one or more, so the CLI checks the pair locally before spending a dispatch — and names the agent that does serve your provider: ``` $ remyxai outrider trigger --agent codex --provider anthropic Error: agent=codex speaks openai-responses, which Anthropic does not serve. Use --agent claude for this provider. ``` Omit `--agent` and the check still runs, against the agent the installed workflow bakes rather than the CLI's default. The pair table is vendored from the action's [published matrix](https://github.com/remyxai/outrider/blob/main/docs/agent-matrix.json), so the CLI and the action can't disagree about what's supported. To confirm a pairing actually reaches the vendor before spending a real run, dispatch the action's wiring check — it resolves the routing and makes one trivial call, no clone and no artifact, in about 30 seconds: ```bash theme={null} remyxai outrider trigger --repo owner/name \ --mode smoke --agent codex --provider openai --model gpt-5.4-mini ``` # Recommendations and search Source: https://docs.remyx.ai/cli/recommendations Daily recommendations, refreshes, and search across papers, repos, models, and datasets. ## Recommendations Get personalized resource recommendations based on your Research Interests. ### Daily Digest ```bash theme={null} remyxai papers digest ``` Shows today's top recommendations grouped by Research Interest. Each entry includes the resource title, source type, relevance score, and why it was recommended. | Flag | Description | Default | | - | - | - | | `--limit, -n` | Max items per interest | 5 | | `--period, -p` | `today`, `week`, or `all` | `today` | | `--format, -f` | `text` or `json` | `text` | | `--full` | Show full reasoning text without truncation | — | ```bash Today's digest theme={null} remyxai papers digest ``` ```bash This week, JSON output theme={null} remyxai papers digest --period week --format json ``` ```bash Full reasoning, 3 per interest theme={null} remyxai papers digest --full --limit 3 ``` ### List Recommendations ```bash theme={null} remyxai papers list ``` Flat list of all recommendations (not grouped by interest). Supports more granular filtering. | Flag | Description | Default | | - | - | - | | `--interest, -i` | Filter by interest name or UUID | *(all interests)* | | `--limit, -n` | Max results (1-50) | 20 | | `--period, -p` | `today`, `week`, or `all` | `all` | | `--source-type, -s` | Filter by source type (`arxiv_paper`, `github_repo`, etc.) | *(all types)* | | `--format, -f` | `text` or `json` | `text` | | `--full` | Show full reasoning text | — | ```bash theme={null} # Papers for a specific interest from this week remyxai papers list --interest "Retrieval-Augmented Generation" --period week # Only GitHub repos remyxai papers list --source-type github_repo ``` ### Refresh Recommendations ```bash theme={null} remyxai papers refresh ``` Triggers an async recommendation refresh. By default, refreshes all active interests. Use `--wait` to block until complete. | Flag | Description | Default | | - | - | - | | `--interest, -i` | Refresh a specific interest (name or UUID) | *(all active)* | | `--num-results, -n` | Items per interest | *(uses interest's daily\_count)* | | `--wait, -w` | Block until all tasks complete | — | | `--format, -f` | `text` or `json` | `text` | ```bash Refresh all, wait for results theme={null} remyxai papers refresh --wait ``` ```bash Refresh one interest theme={null} remyxai papers refresh --interest "Reinforcement Learning" --wait ``` ### Check Refresh Status ```bash theme={null} remyxai papers refresh-status ``` Poll the status of an async refresh task (from `papers refresh` without `--wait`). ## Search Search across the Remyx resource index — papers, repos, models, and datasets with Docker availability filtering. ### Query ```bash theme={null} remyxai search query "retrieval augmented generation for code" ``` | Flag | Description | Default | | - | - | - | | `--max-results, -n` | Maximum results | 10 | | `--category, -c` | Filter by category (can be specified multiple times) | *(all)* | | `--docker` | Only show resources with runnable Docker environments | — | | `--no-docker` | Only show resources without Docker | — | ```bash theme={null} # Search for Docker-ready resources only remyxai search query "vision language models" --docker --max-results 5 # Filter by category remyxai search query "reinforcement learning" --category cs.LG --category cs.AI ``` ### Get Resource Details ```bash theme={null} remyxai search info 2308.12345 ``` Returns detailed information for a specific resource by arXiv ID, including full abstract, Docker status, Dockerfile contents, and reference summary. | Flag | Description | Default | | - | - | - | | `--format, -f` | `text` or `json` | `text` | ### List Recent Resources ```bash theme={null} remyxai search list ``` Browse recently added resources with pagination. | Flag | Description | Default | | - | - | - | | `--limit, -n` | Number of results | 20 | | `--offset, -o` | Pagination offset | 0 | | `--category, -c` | Filter by category (multiple allowed) | *(all)* | | `--docker` / `--no-docker` | Filter by Docker availability | *(all)* | ### Catalog Statistics ```bash theme={null} remyxai search stats ``` Shows aggregate stats: total assets, Docker availability counts, category breakdown, and citation classification coverage. # How Remyx works Source: https://docs.remyx.ai/concepts/experimentops How Remyx prepares changes, measures outcomes, and retains evidence for the next decision. # How Remyx works Remyx helps AI teams find promising changes and turn them into review-ready code. It helps teams measure and capture the outcomes, using those results to improve the next recommendation and guide where to spend engineering time and compute. Outrider searches for candidate changes, checks their fit against the repository, and prepares code for review. Validate runs defined evaluations on pull requests and records the measured criteria, artifacts, and cost. Each can be used independently. The longer-term approach, sometimes called *ExperimentOps*, connects those decisions and outcomes across projects. The sections below describe this direction, including capabilities still on the [roadmap](/roadmap). Cross-project recommendations and causal analysis should not be treated as available for every account. > [!NOTE] > > **Available today:** Outrider, PR-based Validate, and the corresponding run and result records. Production comparisons require connected signals. See [Outrider](/platform/agents/outrider-action), [Validate](/platform/agents/validate), and [Reports](/platform/review/reports). *** ## Why This Matters Now Writing code is only one part of improving an AI system. Teams also need a way to decide which changes matter, test them against defined criteria, and retain the evidence behind each decision. The sections below describe recurring problems this approach addresses: > [accordion] Context stays trapped in people's heads > > A data scientist spends two months testing retrieval strategies. The reasoning behind the final choice (why hybrid search won, what alternatives were tested, what tradeoffs were considered) lives in their memory and a few Slack messages. MLflow logged the runs but not the interpretation. When that person leaves, the next engineer starts from scratch. > > Every experiment without captured context is knowledge that walks out the door when someone changes teams or leaves the company. > [accordion] Experiments are isolated, patterns are invisible > > A team runs 14 experiments in a quarter. Five explored retrieval improvements and all produced positive results. Three explored routing and none worked. But nobody sees this pattern because each experiment is tracked in a different tool: a Jira ticket, an MLflow run, a Notion page. > > The strategic signal ("retrieval is our strongest direction, routing is not working") is invisible unless someone manually reviews all experiments and connects the dots. Nobody has time for that meta-analysis. > [accordion] Leadership can't see what's working > > A CTO managing three AI projects needs to know: which are producing results? Which are stalled? Where should the team invest more? Today, getting that answer requires being in every room or scheduling status meetings with each team lead. > > Without a portfolio view across AI projects, resource allocation is based on status reports instead of measured outcomes. > [accordion] The pace of change is outrunning teams > > What was best practice weeks ago may already be suboptimal. New techniques, architectures, and tooling ship faster than any team can evaluate. The result: a persistent backlog of improvements that never gets touched because maintenance, firefighting, and stakeholder management consume all the bandwidth. *** ## How ExperimentOps Works ExperimentOps adds three layers that traditional experiment tracking doesn't capture: ### 1. The Decision Layer Every experiment in Remyx records *what happened* (metrics, artifacts), *why it happened*, and *what the team decided to do about it*. | Field | What it captures | Example | | - | - | - | | **Source** | Where the idea came from | Paper, GitHub repo, HuggingFace model, hypothesis, incident, Remyx recommendation | | **Hypothesis** | What the team expects | "Adding temporal decay to embeddings will increase 7-day retention by 1-2%" | | **Target metric** | The business outcome that matters | Resolution rate, conversion rate, CSAT | | **Decision** | What the team decided after seeing results, and why | "Ship to 100%. Explore temporal features in search ranking next." | The decision field captures the reasoning that artifact-tracking tools miss. When a team logs "Ship to 100%. The re-ranker specifically helps with multi-topic tickets where the old retriever returned tangentially related articles," that reasoning persists through team changes, onboarding, and leadership transitions. MLflow tracks what you did. Remyx tracks what it meant and what to do next. ### 2. The Pattern Layer After enough experiments, Remyx groups them by direction and surfaces which directions are producing results: ``` Retrieval quality (5 experiments): 5 of 5 positive, avg +3.2% → HIGH SIGNAL Prompt engineering (2 experiments): 2 of 2 positive, avg +1.2% → MODERATE Routing / model selection (2 experiments): 0 of 2 significant → LOW SIGNAL ``` This meta-analysis is what no individual on the team can do on their own. It requires looking across all experiments, grouping them by theme, and computing which themes consistently produce results. For a team running 10+ experiments per quarter, this is the difference between iterating randomly and iterating strategically. > [card] [Patterns and Insights](/platform/review/reports) > > See how pattern detection works in the Insights view ### 3. The Recommendation Layer For each high-signal direction, Remyx searches its resource index for techniques, models, and methods that align with the cluster's theme. Your team's own experiment history feeds the ranker directly: candidates that align with the direction you've actually shipped rank above ones that are merely topically related, and a learned **preference model** fit over your past experiments breaks ties. Past experiments inform future discovery, so recommendations improve as the team's data grows. That same loop can run on a schedule — surfacing recommendations daily and opening reviewable draft PRs into your repo, with a person still approving every merge. > [card] [Automated discovery PRs](/platform/agents/outrider-action) > > How Outrider runs the discovery-to-draft loop on a schedule, with the human at the merge *** ## The Experiment Lifecycle **1. Origin** An experiment starts from one of several sources: * **Research**: A relevant paper, repo, or model surfaced by Remyx or found by the team * **Hypothesis**: An engineer has an idea based on domain knowledge * **Incident**: A production issue revealed a gap worth investigating * **Recommendation**: Remyx detected a pattern and suggested a next step For research-sourced experiments, Remyx assembles a **launch context** with resource metadata, Docker environment, target repo structure, and an AI-generated implementation plan grounded in actual file paths. **2. Implementation** The team implements the experiment using the tools they already have: * **GitHub**: PRs linked to experiments, status synced via webhooks * **Linear / Jira**: Tickets created automatically, bidirectional status sync * **Claude Code**: AI agent generates a draft PR via MCP, implementing the technique in the team's codebase **3. Validation** The experiment runs against the target metric. Remyx tracks A/B test configuration, traffic splits, duration, and results. The observed delta and statistical confidence are recorded and tied to the business outcome. **4. Decision** The team logs their interpretation: ship, iterate, or abandon, and why. A good decision captures reasoning, context, and next steps that help the next person (or the same person six months later) understand what happened and what it means. **5. Patterns** As experiments accumulate, Remyx surfaces cross-experiment patterns. Tag-based clustering identifies which directions produce consistent results. Recommendations suggest what to try next, grounded in the team's own data. *** ## ExperimentOps vs. Traditional Experiment Tracking | | MLflow / W\&B | Remyx ExperimentOps | | - | - | - | | **Tracks** | Model runs, hyperparameters, artifacts | Full experiment lifecycle including decisions and business outcomes | | **Entry point** | `mlflow.log_param()` in training code | Hypothesis, research resource, or production incident | | **Decision capture** | Not supported | First-class `log_decision` with reasoning and next steps | | **Cross-experiment patterns** | Manual analysis | Automatic tag-based clustering with hit rates | | **Resource discovery** | None | Papers, repos, and models matched to experiment history | | **Leadership view** | Dashboard of model metrics | Portfolio of projects with health indicators | | **Integrations** | Model registry, artifact store | GitHub, Linear, Jira, Slack, Claude Code MCP | | **Question answered** | "What hyperparameters produced the best loss?" | "Which directions are working, and what should we try next?" | > [!NOTE] > > Remyx operates at a higher level than MLflow or W\&B. Link your MLflow runs as artifacts on a Remyx experiment to connect the training details with the business decision. *** ## Who Uses ExperimentOps > [tab] ML Engineers & Data Scientists > > Track every experiment from hypothesis through results. Link PRs, tickets, and datasets. Log decisions so the next person understands why things were done this way. See which directions are producing results across the team's work. > > Frees up time spent on maintenance triage and context reconstruction so engineers can focus on the experiments that actually improve the product. > [tab] Team Leads > > See which projects are on track and which need attention. Understand hit rates across experiment directions. Know when a direction has been sufficiently explored or when it's worth doubling down. > > Turns "which direction should we invest in?" from a debate into a data-informed decision. > [tab] CTOs / VPs > > **Get visibility without being the bottleneck.** The Portfolio view shows every project's experiment velocity, positive hit rate, and metric trends. Know which teams are producing results without reviewing every experiment or scheduling syncs. Let the system be the structured experimentation process so you don't have to be in every room. > > Replaces status meetings with a live portfolio view that shows which projects are producing results and which need attention. *** ## Core Principles **Decisions over metrics.** The most valuable artifact of an experiment is the team's interpretation: "This worked because X, and next we should try Y." **Patterns over individual results.** A single experiment result is a data point. Ten experiments grouped by direction reveal a strategy. **Compounding over starting fresh.** Every experiment builds institutional knowledge. That knowledge persists through team changes and informs future recommendations. **Visibility over status meetings.** Leadership sees which projects are working on one screen, without booking a round of syncs. *** ## Get Started > [card] [Quick Start](/quickstart) > > Your first Outrider run — a draft PR in minutes > [card] [Outcomes](/platform/configure/projects) > > Explore the experiment timeline and detail views > [card] [Connectors](/platform/configure/connectors) > > Connect GitHub, Linear, Jira, Slack, and Claude Code > [card] [MCP Server](/resources/mcp-server) > > Use Remyx tools from Claude Code or Slack # Remyx documentation Source: https://docs.remyx.ai/introduction Evaluate a pull request, find promising changes, and prepare review-ready code with Remyx. Find changes worth testing, prepare code for review, and measure the results. Use Remyx from GitHub, Studio, the CLI, or supported APIs. The guides below start with the task you want to complete. ## Choose a workflow > [card] [Evaluate a pull request](/platform/agents/validate) > > Run a test against a code change and review the evidence before merging. > [card] [Find and draft changes](/platform/agents/outrider-action) > > Use Outrider to find relevant changes for a repository and prepare an implementation for review. > [card] [Browse tutorials](/tutorials/overview) > > Work through setup, commands, and a result you can inspect. ## Start here [Quickstart](/quickstart) · [API reference](/api-reference/api-reference) · [CLI & tools](/cli) ## Ways to use Remyx | Interface | Start with | | - | - | | GitHub PR comments | [Evaluate a pull request](/platform/agents/validate) · [Commands](/platform/agents/validate-commands) | | Studio | [Quickstart](/quickstart) · [Review results](/platform/review/reports) | | CLI | [CLI & tools](/cli) · [Find and draft changes](/platform/agents/outrider-action) | | API | [API reference](/api-reference/api-reference) · [Validate API](/api-reference/validate) | ## Where the work runs Outrider runs through GitHub Actions in your repository. Validate runs an approved PR evaluation on Remyx compute. Both return reviewable results, and your team controls merges. ![Studio Reports showing the SAELens PR validation, result status, linked spec, and recorded compute spend.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/studio-validations-report.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=e3a7f439606cf5bbffc3f0f6f23c0038) A PR validation appears in Studio's results history. # Model backends Source: https://docs.remyx.ai/platform/agents/backends Configure the coding agent and model provider used by Outrider. Outrider uses a configured coding agent and model provider to prepare changes. Studio shows which provider is connected and which backend the repository's agent uses. ## Choose a backend Connect a supported provider in Studio, then choose the backend when setting up an agent. Changing the backend may update the repository's workflow or credentials. ## Supported combinations The compatibility matrix below lists supported coding agents, providers, and setup requirements. *** Outrider's coding runtime speaks whichever model-API family its [coding agent](#the-agent-input) speaks, so it can be pointed at any compatible endpoint. That makes the model backend a first-class choice per agent. The loop, guardrails, and refinement chain are identical on every backend; what changes is cost, latency, and the models on offer: | Backend | `provider` value | Secret | Base URL | Default model | Recommended `agent-timeout` | | - | - | - | - | - | - | | **Anthropic (Claude)** | `anthropic` | `ANTHROPIC_API_KEY` | api.anthropic.com | `claude-opus-4-8` | action default (`1500`; see note) | | **Moonshot (Kimi)** | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/anthropic` (`/v1` for Codex) | `kimi-k3` | `3600` | | **OpenAI** | `openai` | `OPENAI_API_KEY` | api.openai.com | *(the agent's own)* | action default (`1500`) | | **OpenRouter** | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api` (`/v1` for Codex) | *(the agent's own)* | `3600` | | **Z.ai (GLM)** | `zai` | `ZAI_API_KEY` | `https://api.z.ai/api/anthropic` | `glm-5.3` | `3600` | | **Custom / on-prem** | `custom` | your choice | set `model-base-url` | varies | depends on backend | An unset `provider` leaves the caller's own auth untouched, which is what installs written before the input existed rely on. Which providers a run can reach depends on the [coding agent](#the-agent-input) it runs: an agent speaks one model-API family and a provider serves one or more. OpenRouter serves every agent. The higher timeout for Z.ai and Moonshot exists because their flagship models' thinking modes add per-turn latency; a GLM session routinely takes 15–20 minutes. The `1500` action default applies to hand-written and `setup-local` workflows. Installs provisioned by the app or `remyxai outrider init` bake `agent-timeout: 3600` for every provider, with a 90-minute job wall derived from it, so you only need to set a timeout on those when you want a lower cap. The `provider` and `model` inputs are independent: `provider` names the endpoint, `model` picks the model from it (e.g. `glm-5.3` / `glm-5.2` / `glm-4.6` on Z.ai; `kimi-k3` / `kimi-k2.7-code` on Moonshot; `claude-opus-4-8` / `claude-sonnet-4-6` / `claude-haiku-4-5` on Anthropic). In the app, the agent and its backend are set in the [agent setup wizard](/platform/agents/overview#set-up-an-agent) and editable later under the agent's [Configuration](/platform/agents/overview#configuration). Connect each key once under [Connectors](/platform/configure/connectors); Remyx pushes it to the repo as a secret at provisioning time. *** ## The `provider` input In workflow YAML, one input wires the whole backend — auth env var, base URL, and mutual exclusion between the two auth styles: ```yaml theme={null} - uses: remyxai/outrider@v1 with: interest-id: ${{ vars.REMYX_INTEREST_ID }} provider: ${{ inputs.provider }} # anthropic | openai | zai | moonshot | openrouter | custom model: ${{ inputs.model }} agent-timeout: 3600 env: REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} ZAI_API_KEY: ${{ secrets.ZAI_API_KEY }} MOONSHOT_API_KEY: ${{ secrets.MOONSHOT_API_KEY }} OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} ``` Pass every backend's secret in the step's `env:` block — the action reads only the one matching `provider`, so a single workflow can switch backends per dispatch. This input replaced the fork-side "Configure provider auth" case-switch step older workflows carried; an empty `provider` preserves the old behavior exactly, so existing installs need no migration. The action fails clean with a specific error if the selected provider's secret is missing (e.g. `provider=zai requires ZAI_API_KEY in the caller's env block`). > [!WARNING] > > Don't set both `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` yourself: with both set the runtime sends *both* headers, so an unrelated Anthropic key travels to whichever gateway the run is pointed at. Naming a `provider` is what prevents it, and is most of why the input exists. The action builds the agent's launch environment explicitly and puts exactly one credential in it, the one that provider needs; nothing in a caller's workflow can add the other back. Use `model-base-url` directly (instead of `provider`) only for custom or on-prem endpoints: Bedrock, Vertex, or a proxy of your own. Set alongside a named provider it still wins, since a gateway in front of that vendor is the usual reason to pass both. The run warns that it has overridden the registry endpoint and that the provider's credential is going there. *** ## The `agent` input `provider` picks the model; `agent` picks the **coding-agent CLI** that drives the implementation. The two are independent axes. Each agent speaks one model-API family, so which providers it can reach follows from that: | Agent | `agent` value | Speaks | Credential | Installed by the action | | - | - | - | - | - | | **Backboard R-CLI** | `backboard` | its own router | `BACKBOARD_API_KEY`, which is also the model-routing credential | the vendor installer (`https://app.backboard.io/api/cli`) | | **Claude Code** | `claude` | Anthropic Messages | derived from the provider's key | `@anthropic-ai/claude-code` at the exact version pinned in the action's `package.json` | | **OpenAI Codex** | `codex` | OpenAI Responses | derived from the provider's key | `@openai/codex` at the exact version pinned in the action's `package.json` | Set the agent in the [setup wizard](/platform/agents/overview#set-up-an-agent), in the workflow file, from the [CLI](/cli/outrider-trigger#switch-the-coding-agent-for-a-run) (`--agent`), or per dispatch. An unset `agent` resolves to `claude`, so workflows written before the input existed keep running unchanged. ```yaml theme={null} - uses: remyxai/outrider@v1 with: interest-id: ${{ vars.REMYX_INTEREST_ID }} agent: codex # claude | codex | backboard provider: openai model: gpt-5.4-mini agent-timeout: 3600 env: REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} ``` ### Which pairs work | `agent` | `provider` | Secret you set | Endpoint | Default model | | - | - | - | - | - | | `backboard` | *(any, the router resolves it)* | `BACKBOARD_API_KEY` | | none; name a `model` | | `claude` | `anthropic` | `ANTHROPIC_API_KEY` | vendor default | *(the agent's own)* | | `claude` | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/anthropic` | `kimi-k3` | | `claude` | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api` | *(the agent's own)* | | `claude` | `zai` | `ZAI_API_KEY` | `https://api.z.ai/api/anthropic` | `glm-5.3` | | `codex` | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/v1` | `kimi-k3` | | `codex` | `openai` | `OPENAI_API_KEY` | vendor default | *(the agent's own)* | | `codex` | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/v1` | *(the agent's own)* | | `claude` / `codex` | `custom` | your own | you set `model-base-url` | *(the agent's own)* | Every pair above except `custom` has been verified end-to-end against the vendor. A pair whose two halves don't share an API family is refused before any work starts, naming the agent that does serve that provider: ``` agent=codex speaks openai-responses, which Anthropic does not serve. Use agent=claude for this provider. ``` The pairs come from the action's own registry, published as [`agent-matrix.json`](https://github.com/remyxai/outrider/blob/main/docs/agent-matrix.json). The CLI and the engine read that contract instead of keeping their own copies, so the three stay in step. > [!WARNING] > > OpenRouter reserves the requested `max_tokens` against your balance *before* calling the model, and both CLIs request a lot by default (Codex's is 131,072), so a thin balance answers HTTP 402 before the model is reached. ### What each agent can report An agent can be partially capable and still usable: the affected telemetry degrades, the run doesn't fail. | Capability | `backboard` | `claude` | `codex` | | - | - | - | - | | `cost_usd`, the CLI's own dollar figure | ✓ | ✓ | | | `turn_cap`, a round limit besides the timeout | | ✓ | | | `guardrail_policy`, the injection-hardening tool gate | | ✓ | | | `web_research`, the staged research pass | ✓ | ✓ | vendor endpoint only | | `output_schema`, schema-constrained verdict passes | | | ✓ | | `stream_transcript`, `token_usage`, `oneshot_json` | ✓ | ✓ | ✓ | What a missing capability costs: | Missing | Effect on the run | | - | - | | `cost_usd` | Cost resolves from the per-model rate table instead; with no rate row the run reports `unavailable` instead of \$0.00. Token counts stay exact either way | | `turn_cap` | `agent-timeout` becomes the only bound on spend, so keep it tight on cron-driven installs | | `guardrail_policy` | The tool gate is not in effect and the run logs a warning; the post-hoc diff validators still apply | | `web_research` | The staged research phase is skipped; the coding session runs without web context | | `output_schema` | Verdict passes fall back to extracting JSON from the model's prose | ### Models, per agent All three agents take the same `model` input; the action sets that agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, `BACKBOARD_MODEL`). Backboard addresses models as `/` and has no default of its own, so name a `model` with it. The action composes `provider: openai` plus `model: gpt-5.4-mini` into `openai/gpt-5.4-mini`, and leaves an already-qualified id alone. Routing Codex off OpenAI's own endpoint pins two request fields for you, because Codex fills them from its own model catalog and an unrecognized model leaves them in a shape strict gateways reject: `web_search="disabled"` (the server-side tool is OpenAI's, not part of the protocol) and `model_reasoning_effort="medium"` (OpenRouter answers `Reasoning is mandatory for this endpoint` without it). Neither is applied on OpenAI, where Codex's per-model defaults win. A Chat-Completions-only provider needs a translating gateway in front of it, since `codex exec` requires a **Responses** endpoint. Z.ai is the case people hit first: it serves Anthropic Messages, and its OpenAI-compatible surface is Chat Completions (`/v4/responses` answers 404), so `agent: codex` with `provider: zai` is refused rather than failing mid-run. GLM under Codex goes through a router that speaks Responses, `provider: openrouter` with `model: z-ai/glm-5.3`, or through your own gateway on `provider: custom`. ### Verify a pairing before you spend a run `mode: smoke` resolves the routing, runs the agent's preflight, and makes one trivial call in about 30 seconds, without cloning the repo or opening anything. It reaches the vendor for real, so it catches a wrong secret, a bad endpoint, a model id from another vendor's vocabulary, or an exhausted quota. The run reports `smoke_ok` or `smoke_failed`: ```bash theme={null} remyxai outrider trigger --repo owner/name \ --mode smoke --agent codex --provider openrouter --model openai/gpt-5.4-mini ``` Run it after changing `agent`, `provider`, `model`, or a secret, instead of waiting on a full dispatch to find out. A `setup-local` workflow declares `mode`, `agent`, `provider`, and `model` as dispatch inputs, so the same check runs from the Actions tab or `gh workflow run`. Every key above can be connected once under [Connectors](/platform/configure/connectors), and Remyx pushes it to the repo at provisioning time. A [`setup-local`](/cli/outrider-setup#install-without-the-github-app) install skips that path and sets the secret itself: ```bash theme={null} remyxai outrider set-provider-secret --repo owner/name --provider openrouter --key-from ~/key gh secret set BACKBOARD_API_KEY --repo owner/name < ~/backboard-key ``` *** ## The two-tier drafter-refiner pattern The recommended default for a repo you want worked continuously. It splits exploration and commitment into two workflows: **Workflow 1: Drafter** — daily, budget tier, `publish: branch` * Pushes draft branches only: no PRs, zero maintainer-facing noise. \~\$0.20–0.40/run. * Each run accumulates repo intel — landing zones that worked, mechanism shapes that got rejected. **Workflow 2: Refiner** — weekly, strong tier, `publish: pr` * Picks the best drafter branch and generates a mechanism-level **gap analysis** against its source paper. * Re-runs from that branch (`start-from-ref`) with the gap analysis as its brief (`lead-content`), through the full refinement chain, to a draft PR. \~\$5–10/week including the gap generation. Why the split works: the drafter's branch anchors an extension point and test scaffolding, so the refiner commits to PR-shape on borderline cases; a budget tier makes daily exploration of the arXiv frontier (\~200 ML papers/day) rational; even failed branches contribute negative-space signal to the next pick. Set it up from the wizard's **Advanced — models per phase → Two-tier** option (writes three workflows: the dispatch-capable main one, the drafter, the refiner), or from the CLI. Tier pairings are cross-provider and switchable — the app defaults to a GLM drafter with an Opus refiner: ```bash theme={null} remyxai outrider setup-local --repo owner/name --auto-interest \ --two-tier --drafter-model glm-5.3 --refiner-model claude-opus-4-8 ``` Runs report their `tier` (main / drafter / refiner) in fleet telemetry, so [Reports → Costs](/platform/review/reports#costs) prices each tier separately. *** ## Cost telemetry Every run reports `cost_usd`, `input_tokens`, and `output_tokens` as action outputs and into fleet telemetry. How the dollar figure is computed depends on the backend: * **Anthropic**: the runtime's own `total_cost_usd` figure is authoritative. * **Z.ai / Moonshot**: cost is computed from a per-model rate table keyed by the model actually used (`glm-5.2`, `glm-4.6`, `kimi-k3`, `kimi-k2.7-code`, `kimi-k2.7-code-highspeed`, …), overriding the CLI's Anthropic-rate estimate. An unknown model on a known host is priced at that host's flagship rate and reported as approximated, which can be a 3–4× error across tier pairs, so set `model` explicitly. `glm-5.3` sits there today: it's the `zai` default and has no rate row yet. * **Custom endpoints**: the reported figure is an estimate. * **Agents differ in what they report.** One that reports its own dollar figure is used directly; one that reports none resolves through the rate table, and reports `unavailable` when the host has no rate row. Token counts stay exact in every case. The [capability table](#what-each-agent-can-report) says which is which. Each run's step summary names the pair it ran, such as `Agent: Codex → OpenRouter`, alongside the cost basis it used. The [Costs report](/platform/review/reports#costs) turns this into unit economics: runs per day by backend against **\$ per code artifact**. *** ## Choosing a backend There's no house pick. Every backend runs the same loop behind the same guardrails; the practical variables are per-run cost, session latency, and the draft quality you observe on *your* repo: * **Anthropic (Claude)** — Claude tiers over `x-api-key` auth; a full PR run with the refinement chain is roughly \$5–6. * **Z.ai (GLM)** — GLM tiers at roughly 20× lower per-run cost; budget the longer timeout for thinking-mode latency. * **Moonshot (Kimi)** — Kimi tiers, same Bearer-auth pattern and timeout guidance as Z.ai. * **Custom / on-prem** — anything that speaks the API family your agent speaks, via `model-base-url`. The cheap way to decide is evidence, not defaults: per-dispatch routing (`remyxai outrider trigger --provider … --model …`) makes A/B-ing providers on the same paper nearly free, and [Reports → Costs](/platform/review/reports#costs) shows each backend's cost per code artifact against what its runs actually produced. The [two-tier pattern](#the-two-tier-drafter-refiner-pattern) sidesteps the choice entirely: explore on a budget tier daily, promote on the tier your merges say you trust. *** ## Next steps [Run Outrider on a repository](/tutorials/outrider-repository) · [Use the CLI](/cli/outrider) · [Read the GitHub Action configuration](/platform/agents/outrider-configuration) ## Related > [card] [Actions](/platform/agents/outrider-action) > > All configuration inputs, guardrails, and run outcomes > [card] [Connectors](/platform/configure/connectors) > > Connect Anthropic, Z.ai, and Moonshot keys > [card] [CLI: provider routing](/cli#route-a-run-at-a-different-model-provider) > > Per-dispatch provider/model overrides > [card] [Backends guide (repo)](https://github.com/remyxai/outrider/blob/main/docs/backends.md) > > The action's own backends reference # Find and draft changes Source: https://docs.remyx.ai/platform/agents/outrider-action Use the Outrider GitHub Action to find relevant changes and prepare code for review. Outrider finds candidate changes for your repository and prepares code for maintainers to review. Outrider is an open-source GitHub Action. It checks a candidate against repository context, implements the change, runs available checks, and reviews the work before publication. ## How it works 1. Choose a repository and configure a supported model provider. 2. Let Outrider select a candidate or specify a method to implement. 3. Inspect the resulting code, tests, review notes, and publication decision. [Start in Studio](https://studio.remyx.ai/start) · [Use the CLI](/cli/outrider) · [View the source](https://github.com/remyxai/outrider) ## What a run can return A run may open a draft PR, create a discussion Issue, leave a branch for further work, or stop before publication when its checks fail. A completed workflow does not guarantee a merged contribution. ![GitHub draft PR for the AbsTopK SAE architecture showing a bot-authored implementation description and links to commits, checks, and changed files.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/outrider-draft-pr.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=65f776b1f3c9bc89949601c8ed7d7f67) An Outrider-assisted draft PR includes implementation context and linked checks. [View the pictured draft PR](https://github.com/salma-remyx/SAELens/pull/5) [Run Outrider on a repository](/tutorials/outrider-repository) · [Run modes](/platform/agents/outrider-run-modes) ## Execution and cost Outrider runs in the repository's GitHub Actions environment using the configured provider. Outrider is free and open source. The account running GitHub Actions and the selected model provider determine any associated usage costs. ## Reference [Configuration](/platform/agents/outrider-configuration) · [Run modes](/platform/agents/outrider-run-modes) · [Model backends](/platform/agents/backends) · [Security](/platform/agents/security) Want to measure an existing PR? [Use Validate](/platform/agents/validate). # Outrider configuration Source: https://docs.remyx.ai/platform/agents/outrider-configuration Install and configure Outrider as a GitHub Action for a repository. Outrider runs as a GitHub Action in the repository you configure. For the shortest setup path, use [Studio Quickstart](https://studio.remyx.ai/start). It connects your provider, configures the repository, and starts the first run. ## Manual setup The reference below shows how to install the workflow, provide credentials, and configure its inputs. ## Review the installation Check the workflow file and permissions before enabling scheduled or manual runs. The GitHub Actions log shows each run's execution and outcome. ## Manual install The CLI and the UI do this for you. Install by hand when neither is an option, or to see exactly what they do. **1. Create a Research Interest and an API key** Connect the repo in [studio.remyx.ai](https://studio.remyx.ai) so Remyx ingests its history and creates a Research Interest; note its UUID. Create a Remyx API key under [Account → API keys](/platform/configure/account#api-keys). **2. Add two repo secrets** Under *Settings → Secrets and variables → Actions*: `REMYX_API_KEY`, and the secret for the provider you route at (`ANTHROPIC_API_KEY`, `ZAI_API_KEY`, `MOONSHOT_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, or `BACKBOARD_API_KEY` for the Backboard agent). Set by hand, the keys only ever exist in your repo; `remyxai outrider init --github-secrets-only` gets the same property without the manual steps. **3. Allow Actions to open PRs** *Settings → Actions → General → Workflow permissions* → tick **Allow GitHub Actions to create and approve pull requests**. Without it, PR creation fails with `HTTP 403`. **4. Add the workflow** Save this as `.github/workflows/outrider.yml`. The `workflow_dispatch` inputs are what `remyxai outrider trigger` sets per run. ```yaml theme={null} name: Outrider on: schedule: - cron: '0 14 * * 1' # Mondays 14:00 UTC; pick any cadence workflow_dispatch: inputs: mode: description: 'recommend (default), brief, smoke, weekly-summary' required: false default: '' pin-arxiv: required: false default: '' search-method: required: false default: '' lead-content: description: 'A gap analysis in recommend mode; the whole spec in brief mode' required: false default: '' publish: description: 'pr (default) or branch' required: false default: 'pr' claude-timeout: description: 'Seconds per agent phase (the name remyxai outrider trigger sends; agent-timeout is the same ceiling)' required: false default: '1500' concurrency: group: outrider-${{ inputs['pin-arxiv'] || inputs['start-from-ref'] || github.run_id }} cancel-in-progress: false jobs: recommend: runs-on: ubuntu-latest timeout-minutes: 55 permissions: contents: write pull-requests: write issues: write # discussions: write # only for mode: weekly-summary env: REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # or ZAI_API_KEY, MOONSHOT_API_KEY, ... steps: - uses: remyxai/outrider@v1 with: interest-id: 'YOUR-INTEREST-UUID-HERE' mode: ${{ inputs.mode }} pin-arxiv: ${{ inputs.pin-arxiv }} search-method: ${{ inputs.search-method }} lead-content: ${{ inputs.lead-content }} publish: ${{ inputs.publish }} claude-timeout: ${{ inputs.claude-timeout }} ``` To route per run at a different provider, add `agent`, `provider` and `model` dispatch inputs and pass each backend's secret in `env:`; see [Model backends](/platform/agents/backends). **5. First run** *Actions → Outrider → Run workflow*. A draft PR, an Issue, or a branch appears when it completes. The permissions above are the minimum: `contents: write` to push the branch, `pull-requests: write` and `issues: write` to open artifacts, plus `discussions: write` for the weekly digest. With a same-repo install the action mints a `remyx-ai[bot]` token for the artifacts and uses the workflow token only as a fallback. *** > [!WARNING] > > Connect Remyx to your GitHub account in [Connectors](/platform/configure/connectors). The [**GitHub**](/platform/configure/github) connector installs the Remyx GitHub App in your account or organization; include access to the repos you want. > > A connected [**model provider**](/platform/configure/connectors#model-providers-and-coding-agents) is required to draft PRs. Model usage is billed to your provider account. > > For the CLI, also create a [Remyx API key](/platform/configure/account#api-keys). Either path provisions the workflow and fires the first run. A draft PR or Issue lands in minutes. ## From the CLI ```bash theme={null} pip install remyxai # authenticate the CLI with your Remyx API key export REMYXAI_API_KEY="YOUR_REMYX_API_KEY" # install Outrider on the repo and auto-create a Research Interest from it remyxai outrider init --repo owner/name --auto-interest ``` `--auto-interest` creates a Research Interest from the repo; drop it and pass `--interest ` to use an existing one. If the GitHub App isn't installed yet, the command prints the install link. Add `--github-secrets-only` to seal your provider key against the repo's own Actions public key on your machine, so it reaches GitHub as ciphertext and no copy is stored anywhere else (see [Security](/platform/agents/security#where-the-model-key-lives)). **No GitHub App?** `setup-local` provisions with your own authenticated `gh` CLI instead; PRs then come from your account, not **remyx-ai\[bot]**. ([details](/cli/outrider-setup#install-without-the-github-app)) ```bash theme={null} remyxai outrider setup-local --repo owner/name --auto-interest ``` ## From the UI Use [**+ Set up an agent**](/platform/agents/overview#set-up-an-agent) on the Agents page, or create a [Research Interest from your repo](/platform/discover/feed#research-interests-from-a-repo) and choose **Set it up for me**. No terminal required. ## By hand When the CLI isn't an option (CI-restricted environments, org policy against helper tooling), install the action yourself. See [Manual install](#manual-install) below. ## Identity and install Either path pins the marketplace action at `remyxai/outrider@v1`. A same-repo install needs no `github-token`: the action self-mints a **remyx-ai\[bot]** App token at runtime. Commits and PRs are authored by the bot (GitHub identity `remyx-ai[bot]`), and the setup PR carries a `Co-Authored-By: remyx-ai[bot]` trailer so credit survives a squash-merge. `@v1` floats: a published Outrider release moves the `v1` tag to the released commit, so a merged fix reaches every install at the next release. Change-controlled orgs can pin an exact release instead (`remyxai/outrider@v1.9.0`) and bump deliberately. Inside the runner, the coding-agent CLIs are installed at exact versions pinned in the action's own `package.json`, Python dependencies come from a pinned requirements file, and no system packages are installed. See [Supply chain](/platform/agents/security#supply-chain). You can also talk to a run by mentioning `@remyx-ai` on the PR or Issue. Research questions get read-only answers (see [Deep Research](/platform/discover/briefs)); `@remyx-ai validate` starts a [Validate](/platform/agents/validate) run on the PR. > [!TIP] > > Outrider's artifacts carry [Open Knowledge Format](https://github.com/remyxai/outrider) frontmatter. A repo-root `CONTEXT.md`, a `REVIEW.md`, and agent-instruction files (`CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`) sharpen its picks. See [Steer Outrider from your repo](#steer-outrider-from-your-repo). *** ## Steer Outrider from your repo The selection and drafting passes read your repo before the model picks anything. The more signal the repo emits, the more precise the pick. * **`REVIEW.md`** (or `.github/REVIEW.md`) is the highest-precedence signal: your review conventions, test bar, scope stance, benchmark expectations and anti-patterns, threaded into the drafter and the downstream fidelity and convention passes. A template is in the action repo at [`docs/REVIEW-template.md`](https://github.com/remyxai/outrider/blob/main/docs/REVIEW-template.md). * **Agent instruction files**, in this order: `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`, `CONTEXT.md`. Lint and house-style rules are read from the repo's own config (ruff, pre-commit, black, flake8) rather than hardcoded. * **`ENVIRONMENTS.md`** (or `ENVIRONMENT.md`), written by the workflow, tells the agent how to set up and run the repo's environment. * **Discussions**, ranked by overlap with the candidate paper. A fork without Discussions falls back to its parent repo's. * **Recent merged PRs** supply the PR-body template and contributor conventions the convention pass applies; `.github/ISSUE_TEMPLATE/` shapes Issues. * **Direction signals** gate the strictest pick shape, a new capability your repo lacks: a README roadmap heading, a recurring theme in `CONTEXT.md`, or an open maintainer-authored `[RFC]` Issue. Without at least one, extension-shaped candidates are rejected. * **Not re-recommending a paper.** A paper is treated as discharged when an Outrider-opened Issue exists for it (open or closed), or a maintainer-opened Issue links its arXiv id. Reopen the Issue to put it back in play. *** ## Configuration The defaults are conservative; most repos only touch `interest-id` and the schedule. | Input | Default | What it does | | - | - | - | | `interest-id` | `''` | The Research Interest UUID the run draws candidates from. Required in every mode except `brief`, which uses it (when set) only for repo context | | `mode` | `recommend` | What the run produces: `recommend` (rank, validate, PR or Issue), `brief` (draft a PR from `lead-content`, no arXiv anchor; see [Run from a design brief](/platform/agents/outrider-run-modes#run-from-a-design-brief)), `smoke` (verify the run's `agent` / `provider` / `model` wiring against the vendor in about 30s and exit, without cloning the repo or opening anything; see [Verify a pairing](/platform/agents/backends#verify-a-pairing-before-you-spend-a-run)), `weekly-summary` (post a digest, see [Weekly digest](#weekly-digest)), `fidelity` / `convention` / `test` (run a single [refinement phase](/platform/agents/outrider-run-modes#refinement-pass-chain) standalone), or `issue-convention` (fold an Issue body to the repo's `ISSUE_TEMPLATE`) | | `chain` | `true` | After opening a draft PR, run the [refinement chain](/platform/agents/outrider-run-modes#refinement-pass-chain) inline in the same run (fidelity, convention, test). Set `false` for recommend-only: cheaper, leaves the draft as Outrider first wrote it | | `min-confidence` | `moderate` | Skip recommendations below `high` / `moderate` / `low` | | `draft-mode` | `always` | `always` (draft), `on_test_failure` (ready if tests pass), or `never` (skip on test failure). A diff that touches a risky surface opens as a draft regardless | | `rate-limit-days` | `7` | Cadence guard. Any value above 0 skips the run while an **open** Remyx PR or Issue exists on the target; merging or closing it releases the gate. The numeric value is otherwise ignored, kept as an on/off bit for workflow files written for the earlier sliding-window semantics. `0` disables | | `lookback` | `week` | Candidate pool window: `today` / `week` / `month` | | `candidate-pool` | `25` | How many candidates the selection pass chooses from | | `test-integration-policy` | `strict` | `strict` demotes to an Issue if new tests don't import an existing module; `soft` opens a draft with a warning; `off` skips the gate. Use `soft` for component or library repos where a standalone module *is* the contribution | | `guardrails-blocklist` | `''` | Path globs the agent must **not** touch (e.g. `secrets/**`, `*.lock`), added on top of the fixed default block (`.github/workflows/**`). Blocklist matches always win | | `guardrails-allowlist` | `''` | Deprecated since v1.7.24 and a no-op: path access is permissive by default. Setting it logs a warning. Removal planned in v2; use `guardrails-blocklist` | | `publish` | `pr` | `pr` opens a draft PR and runs the inline chain; `branch` pushes the branch only: no PR, no chain, no maintainer-facing artifact until the team opens the PR themselves. Branch mode is the drafter tier of the [two-tier pattern](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) | | `agent` | `''` | Which coding-agent CLI drives the implementation: `backboard` (Backboard R-CLI), `claude` (Claude Code), or `codex` (OpenAI Codex); unset resolves to `claude`. A separate axis from `provider`, which picks the model. An agent speaks one API family, so an impossible pair is refused up front, naming the agent that does serve your provider. See [the `agent` input](/platform/agents/backends#the-agent-input) | | `provider` | `''` | Route the coding agent at a named backend: `anthropic`, `openai`, `zai`, `moonshot`, `openrouter`, or `custom`. Resolves that vendor's credential, endpoint, and auth style, and hands the agent exactly one credential; pass each backend's secret in the step's `env:`. Empty preserves legacy behavior. See [Model backends](/platform/agents/backends#the-provider-input) | | `model` | `''` | Model to request from the provider (e.g. `claude-opus-4-8`, `glm-5.3`, `kimi-k3`, `gpt-5.4-mini`), forwarded as the selected agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, `BACKBOARD_MODEL`); also keys the per-model cost rate table. Required for `agent: backboard`, which has no default of its own | | `model-base-url` | `''` | Override the selected agent's base-URL variable (`ANTHROPIC_BASE_URL` or `CODEX_BASE_URL`) to reach a **custom** endpoint that speaks the agent's API family (AWS Bedrock, GCP Vertex, an on-prem proxy). For the known backends prefer the `provider` input; set alongside a named provider it still wins, with a warning that it has overridden the registry endpoint and that the provider's credential is going there. Ignored for a native router, whose base-URL variable addresses its own control plane. Cost telemetry becomes an estimate | | `agent-timeout` | *(falls back to `claude-timeout`)* | Wall-clock seconds for each coding-agent phase; every phase shares the ceiling. The generalized name for `claude-timeout`: pass either, and this one wins if both are set. For an agent with no `turn_cap` ([capability table](/platform/agents/backends#what-each-agent-can-report)) it is the **only** bound on spend, so keep it tight on cron-driven installs. Remyx-provisioned installs bake `3600` with a 90-minute job wall | | `claude-timeout` | `1500` | The original name for the same ceiling, supported indefinitely. The `1500` default applies to hand-written and `setup-local` workflows; raise to `3600` for backends whose thinking modes add per-turn latency, lower to cap cost | | `pin-arxiv` | `''` | Implement one specific `arxiv_id` and skip the selection pass, for reproducible re-runs and warm starts. The paper need not be in the candidate pool: it is looked up in the Remyx catalog, then fetched from arxiv.org (`skipped_pin_arxiv_unresolvable` if neither resolves). Mutually exclusive with `search-method` | | `search-method` | `''` | A free-text method query. Resolves the top match via engine search and implements it, bypassing the candidate pool and the selection pass. Mutually exclusive with `pin-arxiv`. (The CLI flag is `--search-method`) | | `start-from-ref` | `''` | Branch, tag or SHA on the target to start the coding session from. The agent sees that ref's diff already applied, which is how a refinement run builds on a prior draft. The result pushes to a `-refined` branch | | `lead-content` | `''` | Markdown the agent reads as leading context. In `recommend` mode it replaces the paper's suggested experiment in the run's SPEC.md (a gap analysis, or a LEAD Issue's content); in `brief` mode it is the whole spec. A `linear.app//issue/` URL is resolved to the issue body when `linear-api-key` is set | | `linear-api-key` | `''` | Linear API key used to resolve `lead-content` URLs of the form `linear.app//issue/` into the issue's body and metadata. Empty: the URL falls through to a plain web fetch | | `fidelity-policy` | `block` | Pre-PR fidelity gate: `block` (one scoped patch attempt on a needs-judgment verdict, then skip; keeps hallucinated implementations out of public PRs), `advisory` (the audit attaches its coverage matrix but never blocks; best for curated warm-start runs), or `off` | | `staged-synthesis` | `false` | Opt-in two-stage session: a bounded research pass first (target-repo state, prior attempts, conventions), then coding reads its findings. Adds about \$0.15–0.50 and 1–3 minutes | | `maintain-state` | `false` | Opt-in per-fork cross-run learning via `.remyx/repo_intel.yaml`: confirmed landing zones, rejected mechanism shapes, exploration budget | | `enable-cocoindex` | `true` | Installs the `ccc` AST-based semantic code search binary on the runner's PATH for every agent (the matching skill is installed for Claude Code only), grounding call-site claims on real paths. Adds about 60–90s and about 1 GB of dependencies to the run, and also powers the "Convention precedents" enrichment on Issue bodies. Disable to slim the runner | | `pr-number` | `''` | PR number for the standalone chain phases (`fidelity` / `convention` / `test`), from the event payload or a dispatch input. Empty in `recommend` mode; the inline chain threads the just-opened PR through automatically | | `issue-number` | `''` | Issue number for `mode: issue-convention`. Empty in `recommend` mode; the inline chain threads the just-opened Issue through automatically | | `weekly-discussion-id` | `''` | Discussion number (from its URL) or GraphQL node ID for `mode: weekly-summary`. Read only in that mode, which skips cleanly when it's empty | | `github-token` | `''` | Token for git push, PR and Issue creation, and the dedup check. Leave empty for a same-repo install: the action self-mints a `remyx-ai[bot]` token through the Remyx engine and falls back to the workflow's own `github.token` when minting isn't available. Set a PAT with `repo` scope only for a cross-repo controller (the action runs in repo A and writes to repo B) | ### Outputs Every output below is declared on the action, so a downstream step can read it as `steps..outputs.`. | Output | When | Description | | - | - | - | | `status` | always | The run outcome; see [Run outcomes](/platform/agents/outrider-run-modes#run-outcomes) | | `pr_url` / `pr_number` | `pr_opened*` | The opened PR | | `issue_url` / `issue_number` | `issue_opened*` | The opened Issue | | `arxiv` | a paper was picked | The arXiv id (empty in brief mode) | | `tier` | a paper was picked | `high` / `moderate` / `low` / `noise` | | `candidates_considered` / `selection_rejected` | recommend mode | How many candidates the selection pass weighed, and how many it rejected | | `chain_fidelity_status` / `chain_convention_status` / `chain_test_status` | the chain ran on a PR | Per-phase outcome of the inline [refinement chain](/platform/agents/outrider-run-modes#refinement-pass-chain) | | `chain_draft_dropped` | the chain ran | `true` when the test gate passed and moved the draft PR to ready for review; `false` when it stayed a draft | | `cost_usd` / `input_tokens` / `output_tokens` | always | Model spend and token usage for the run, on whichever backend ran it; see [Costs](#costs) | | `discussion_comment_url` | `weekly_summary_posted` | The posted weekly digest comment | ```yaml theme={null} steps: - uses: remyxai/outrider@v1 id: outrider with: interest-id: ${{ vars.REMYX_INTEREST_ID }} - name: Ping the channel when a draft is ready for review if: steps.outrider.outputs.chain_draft_dropped == 'true' run: echo "ready: ${{ steps.outrider.outputs.pr_url }}" ``` ### Concurrency Workflows written by `remyxai outrider init` and `setup-local` key their concurrency group by what the dispatch works on (`pin-arxiv`, then `start-from-ref`, then the run id), so distinct papers or branches run in parallel while a true duplicate of the same target queues behind its predecessor. A flat `group: outrider` permits exactly one pending run and silently cancels the rest, which reads as a flake; installs written before this change should re-provision (`init --force`), and the CLI's `trigger --wait-for-slot` waits for a pending run instead of cancelling it. ### Guardrails Outrider constrains what the agent may change so a draft PR is reviewable: * **Fixed block:** `.github/workflows/**` is always off-limits. An agent editing its own workflow file is the one change that compounds silently across future runs. `commit_and_push` strips any workflow edit before the push and lists the dropped paths in the step summary. `guardrails-blocklist` adds your own globs on top, and blocklist matches always win. * **Risky surface (flag, not block):** a diff that touches dependency manifests or lockfiles, CI pipelines, git hooks, container or shell scripts is forced to draft even under `draft-mode: never`, labelled `risky-surface`, and gets a "review before landing" section in the PR body listing the files. A new method that genuinely needs a dependency still opens, just flagged. * **Integration checks** (post-session, pre-push): at most 3 new `.py` files per run, and at least one newly added function or class must be invoked from another changed file; an import alone doesn't count. Violations terminate the run before any push (`rejected_path_violations`); orphan code downgrades the run to an Issue. * **Canary:** the agent must echo a per-run canary token in its final commit. A missing or wrong canary is the fingerprint of an injection that hijacked the agent's output, and downgrades routing from PR to Issue (`issue_opened_canary_missing`). * **Validators:** stub density, the test-integration gate, a self-review orphan check, a diff-risk score gate, and an outbound secret scan that aborts the run rather than redacting (`aborted_secret_in_payload`). Each has its own Issue-route or abort status. These gates are why a run picks an Issue over a PR when an honest implementation would be larger or riskier than a reviewable draft. The full model, including what the agent process can and cannot reach, is on the [Security](/platform/agents/security) page. *** ## Costs You bring your own model-provider key ([any backend](/platform/agents/backends)), so model usage is billed to you by that provider. Outrider runs are not charged to your Remyx balance. * **Model usage:** on Anthropic, a full PR-route run **with the inline [chain](/platform/agents/outrider-run-modes#refinement-pass-chain)** is about **\$5–6**; recommend-only (`chain: false`) is **\$1–3**. Other backends scale with their per-token rates, which differ by an order of magnitude between a budget model and a flagship one, so compare backends on your own repo rather than on a rule of thumb: every run reports its own `cost_usd`, and [Reports → Costs](/platform/review/reports#costs) breaks spend down by agent, provider and model. A run that skips after the selection pass costs less still: it reads the repo to verify fit, then exits. * **GitHub Actions:** minutes on `ubuntu-latest`. A run takes a few minutes to around twenty depending on the backend, the repo, and whether the chain runs; provisioned installs cap each agent phase at 60 minutes and the job at 90. * **Remyx API:** included. Each run reports `cost_usd`, `input_tokens`, and `output_tokens` as outputs and in the run summary, with a `cost_basis` that says where the dollars came from: `backend_rate_table` (computed from the vendor's published per-token rates for the named model), `backend_rate_table_approx` (the model has no rate row, so the host's default model's rates were used), `claude_code_envelope` or `agent_envelope` (the agent CLI's own figure), or `unavailable` (the backend reported no dollars; tokens are still exact). The [two-tier drafter/refiner pattern](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) is the structured way to spend less per artifact. ### Weekly digest Run a second weekly cron with `mode: weekly-summary` and a `weekly-discussion-id` input to post a 7-day digest to a GitHub Discussion: outcomes, verified costs, license distribution, and open Issues with their next actions. It requires the `discussions: write` permission and makes one model call per week (about **\$0.10–0.20** on Anthropic); the rest is GitHub API reads. ## Next steps [Run Outrider on a repository](/tutorials/outrider-repository) · [Use the CLI](/cli/outrider) · [Read the GitHub Action configuration](/platform/agents/outrider-configuration) # Outrider run modes Source: https://docs.remyx.ai/platform/agents/outrider-run-modes Understand scheduled, manual, pinned, and prepared-example Outrider runs. Outrider can select a candidate for a repository or run against a change you specify. ## Automatic selection Outrider reviews available candidates and repository context, then chooses whether a change is ready to implement. ## Pinned change Use the documented input or command to request a specific method or research item. ## Prepared example Studio's Quickstart can start from a prepared implementation branch. This demonstrates the review-and-refine workflow and can finish faster than an implementation from scratch. ## Outcomes A run may produce a draft PR, Issue, branch, or a decision not to publish. Use the GitHub Actions log to inspect the outcome and checks. ## Exact inputs The tables below define each supported run mode and its configuration. *** ## Ways to trigger a run Four ways to say what Outrider should implement. The harness downstream (guardrails, tests, license scan, PR body) is the same for all of them; only where the "why" comes from differs. | Trigger | Where the intent comes from | CLI | Action input | | - | - | - | - | | **Alerts** | The ranker picks from arXiv against your Research Interest on the workflow's cron | none (scheduled) | none; `interest-id` scopes the pool | | **Search** | You supply a method-family query; the engine search returns the top arXiv hit | `--search-method "riemannian preconditioning LoRA"` | `search-method` | | **Pin** | You name an exact arXiv id | `--pin-arxiv 2410.20305v2` | `pin-arxiv` | | **Brief** | You write the design brief yourself: an issue body, a spec, a notebook. No arXiv anchor | `--brief "add exp backoff to the HTTP client"` or `--brief @design.md` | `mode: brief` + `lead-content` | ```bash theme={null} # paper-anchored: exact arXiv id, or a method-family search remyxai outrider trigger --repo owner/name --pin-arxiv 2410.20305v2 remyxai outrider trigger --repo owner/name --search-method "riemannian preconditioning LoRA optimizer" # brief-anchored: a design brief you supply, inline or from disk remyxai outrider trigger --repo owner/name --brief "add exponential backoff to the HTTP client" remyxai outrider trigger --repo owner/name --brief @design.md ``` A pin does not have to be in the candidate pool: the action looks the id up in the Remyx catalog, then on arxiv.org, so any published arXiv paper works (`skipped_pin_arxiv_unresolvable` if neither resolves). An explicit pin or search also bypasses the don't-re-recommend throttle, so you can re-run a paper that already has an open artifact. The full flag list is in the [CLI reference](/cli/outrider-trigger#trigger-a-one-shot-run). ### Run from a design brief `mode: brief` is the paper-less path. The brief you pass as `lead-content` **is** the spec: no ranking, no selection pass, no paper preflight, and no refinement chain. The run clones the repo, writes the brief into the agent's spec bundle, implements it, runs the same post-session gates as a paper run (path policy, integration check, stub density, self-review, canary, secret scan), and opens a draft PR whose body carries the brief under **Design brief** instead of a paper attribution. GitHub, Hugging Face, and dataset links in the brief are license-scanned (the first five) and the result lands in the PR's License section. `remyxai outrider trigger --brief` is shorthand for `--mode brief` with the brief as the lead content. `@path` reads the brief from a file. It can't be combined with `--pin-arxiv`, `--search-method`, `--lead-content` or `--lead-content-file`; it can be combined with `--start-from-ref` to build the brief on top of an existing branch, and with `--publish branch` to review the output before anything opens. If the installed workflow predates the `mode` and `lead-content` inputs, the CLI refuses rather than silently running a paper pass; re-provision with `remyxai outrider init --repo owner/name --force`. Brief mode is the one mode that runs without an `interest-id`; when one is configured it is read only for repo context. | Status | Meaning | | - | - | | `pr_opened_brief` | A draft PR was opened from the brief | | `branch_pushed_no_pr` | `publish: branch`: the branch was pushed, nothing opened | | `brief_declined_open_as_issue` | The agent judged the brief too broad for a PR and wrote an Issue-shaped write-up instead. No Issue is opened yet: the branch is left unpushed for you to review the write-up in the run log | | `issue_opened_canary_missing` / `branch_pushed_canary_missing` | The per-run canary was missing from the final commit; routed to an Issue (or kept as a branch) for a human to inspect | | `skipped_brief_missing_lead_content` | `mode: brief` ran with an empty `lead-content` | | `brief_failed` | The run threw (for example a rejected push). Exits red | In the action directly: ```yaml theme={null} - uses: remyxai/outrider@v1 with: mode: brief lead-content: ${{ inputs.brief }} # pass a Linear issue URL as the brief and set linear-api-key to have its body fetched ``` *** ## How it works A run has two stages. The Remyx engine ranks a candidate pool server-side, then Outrider checks candidates against a clone of your repo and decides what to open. **Ranking.** The pool is ranked from three inputs: your interest's context, your team's shipping history (extracted from the merge log), and a learned preference model. Candidates that match the direction your team has shipped rank above topically similar ones. **Validation and decision.** A high relevance score does not mean a candidate is implementable; the top-ranked paper often has no call site in your repo. A selection pass checks each candidate against the codebase (code search and file reads) and picks the most implementable; if none fit, the run skips. Filters that do not call the model run first, so a run that cannot produce a useful result exits before any LLM cost. Each run ends in one of three outcomes: a draft PR, an Issue, or a skip. ![The Outrider pipeline: external sources and your repo feed the Remyx engine's ranking; the action fetches the candidate pool, gates it, selects the most implementable candidate, implements within guardrails, validates, and routes to a draft PR, an Issue, or a skip, with the refinement chain promoting PRs and run telemetry feeding back into ranking](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/outrider/outrider-pipeline.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=489774aaa92c57bd77dc3ab43689e805) > [card] Draft PR > > The candidate integrates into an existing call site and passes the validators (tests, stub density, reachability). Outrider opens a draft PR with code and tests. > [card] Discussion Issue > > The candidate is relevant but not directly implementable (no call site, too many stubs, or unreachable code). Outrider opens an Issue with the attempted diff. > [card] Skip > > No candidate cleared the confidence, dedup, or fit gates. No artifact is created; the run records a [status](#run-outcomes). *** ## Refinement-pass chain When `chain` is `true` (the default), opening a draft PR isn't the end of the run. Outrider keeps going in the same run and walks the draft through three phases before it asks for review: **1. Fidelity** Diffs the draft against the paper's reference implementation and emits a **coverage matrix**: what the diff covers, what it defers, and where it deviates from the source. When no public reference exists, it anchors the audit against the arXiv abstract instead; that's lower precision and the PR body says so. **2. Convention** Rewrites the PR body to the repo's canonical PR template, folds Outrider's own scaffolding into a collapsible **Discovery context** block, and applies contributor conventions learned from recent merged PRs. Runs `ruff` auto-fix and force-pushes the result as the bot. **3. Test** Runs lint plus a focused `pytest` on the touched files. On a pass it flips the draft to **ready-for-review**; on failure the PR stays a draft, labelled `outrider:test-failed`, for you to take over. The Issue path gets a convention pass too: `mode: issue-convention` folds an Issue body to the repo's `ISSUE_TEMPLATE` shape. Set `chain: false` to stop after the first draft: Outrider opens the PR exactly as it first wrote it and runs none of the phases above. That's the cheaper recommend-only mode. ### Labels The chain records its progress as labels on the PR, which is also how a separate-workflow chain is triggered. | Label | Set when | | - | - | | `outrider:draft` | The draft PR opens (the fidelity phase's trigger) | | `outrider:fidelity-done` | The fidelity audit posted its coverage matrix | | `outrider:needs-judgment` | The audit flagged at least one item for human review | | `outrider:convention-done` | The convention pass rewrote the body | | `outrider:test-failed` | Lint or tests failed; the PR stays a draft | | `outrider:issue-convention-done` | An Issue's body was folded to the repo template | | `risky-surface` | The diff touched a dependency manifest, CI, hook, container or shell file (see [Guardrails](/platform/agents/outrider-configuration#guardrails)) | ### Run the chain from separate workflows Instead of the inline chain, you can drive each phase from a `pull_request: labeled` workflow with `pr-number` from the event payload. This is how a phase is re-run against an existing PR. ```yaml theme={null} on: pull_request: types: [labeled] jobs: fidelity: if: github.event.label.name == 'outrider:draft' runs-on: ubuntu-latest permissions: { contents: write, pull-requests: write } env: REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # or the secret of the provider you route at steps: - uses: remyxai/outrider@v1 with: interest-id: ${{ vars.REMYX_INTEREST_ID }} mode: fidelity # convention / test take the same shape pr-number: ${{ github.event.pull_request.number }} ``` *** ## Run outcomes Every run ends in a `status` in the Action log and the `status` output. Statuses marked **red** make the workflow step exit non-zero; everything else, including Issues and skips, is a legitimate green outcome. ### Recommend mode | Status | Meaning | | - | - | | `pr_opened` | PR opened ready for review (tests passed and `draft-mode` is not `always`) | | `pr_opened_draft` | PR opened as a draft | | `issue_opened` | The agent elected Issue mode (wrote a write-up instead of code) | | `issue_opened_preflight` | The pre-flight pass routed to an Issue before implementation | | `issue_opened_no_integration` | The diff adds code that nothing invokes | | `issue_opened_stub_density` | The new module is at least 50% stubs | | `issue_opened_no_test_integration` | New tests don't import from any pre-existing module | | `issue_opened_self_review` | The self-review judged the new code an orphan; the Issue preserves the diff | | `issue_opened_substitution` | A replacement or extension candidate that needs dependency changes the guardrails block, or has no call site to anchor on | | `issue_opened_high_risk` | The diff-risk gate routed to a human-review Issue; the diff is preserved in the body | | `issue_opened_canary_missing` | The canary trailer was missing from the final commit; the branch is kept for inspection | | `skipped_low_confidence` | Best candidate below `min-confidence` | | `skipped_open_artifact` | An open Remyx PR or Issue from a prior run still exists (`rate-limit-days`) | | `skipped_by_selection_verification` | The selection pass checked every candidate against the repo and rejected all; the reasoning renders in the step summary | | `skipped_pr_exists` / `skipped_issue_exists` | Every candidate already has an open PR, or a prior Issue (open or closed) referencing its arXiv id | | `skipped_external_issue_exists` | The selection pass surfaced an out-of-pool candidate the team already has in an Issue | | `skipped_arxiv_already_landed` | The pinned paper has already been merged on the target | | `skipped_pin_arxiv_unresolvable` | `pin-arxiv` resolved neither in the Remyx catalog nor on arxiv.org | | `skipped_no_method_match` | `search-method` returned no usable hit | | `skipped_issues_disabled` | The Issue route was needed but Issues are disabled (the default on forks); enable them with `gh repo edit --enable-issues` | | `skipped_fidelity_fabrication` / `skipped_fidelity_fabrication_after_patch` | The pre-PR fidelity gate (`fidelity-policy: block`) found the implementation did not match the reference, before or after its one patch attempt | | `skipped_test_failure` | Tests failed and `draft-mode` is `never` | | `rejected_path_violations` | The agent touched a blocked path; nothing was pushed | | `aborted_secret_in_payload` | The outbound secret scanner matched a credential in the draft payload; nothing was opened. **Red** | | `claude_failed` | The coding-agent CLI exited non-zero. **Red** | | `error` | Unhandled exception. **Red** | ### Brief mode, branch mode, smoke, weekly summary | Status | Meaning | | - | - | | `pr_opened_brief` / `brief_declined_open_as_issue` / `skipped_brief_missing_lead_content` | See [Run from a design brief](#run-from-a-design-brief) | | `brief_failed` | A brief run threw. **Red** | | `branch_pushed_no_pr` | `publish: branch`: the branch was pushed and nothing was opened | | `branch_pushed_canary_missing` | Branch mode, with the canary missing; review before promoting | | `lead_captured_no_issue` | Branch mode would have filed an Issue; the content is in the step summary instead | | `smoke_ok` / `smoke_failed` | Whether the configured agent, provider and model reached the vendor (`mode: smoke`) | | `weekly_summary_posted` / `weekly_summary_skipped_no_discussion_id` | The digest was posted, or ran with no `weekly-discussion-id` | | `weekly_summary_failed` | Nothing was posted. **Red** | ### Chain phases These appear in the `chain_*_status` outputs of an inline run, or as the run `status` when a phase runs standalone. The pre-PR gate's verdict (`pre_pr_fidelity_clean` / `pre_pr_fidelity_advisory` / `pre_pr_fidelity_needs_judgment` / `pre_pr_fidelity_skipped_*` / `pre_pr_fidelity_failed_*`) is recorded in the run summary and telemetry rather than as an output; a blocking verdict surfaces as `skipped_fidelity_fabrication*` above. | Phase | Status | Meaning | | - | - | - | | Fidelity | `fidelity_audited`, `fidelity_audited_needs_judgment`, `fidelity_audited_paper_anchored`, `fidelity_audited_advisory` | Coverage matrix posted (reference-anchored, or abstract-anchored when no reference exists) | | | `fidelity_skipped_no_reference` / `fidelity_skipped_not_bot` / `fidelity_skipped_no_pr` / `fidelity_skipped_reference_mismatch` | Nothing to audit against, or the PR isn't a `remyx-ai[bot]` draft | | | `fidelity_failed_clone` / `fidelity_failed_claude` | **Red** | | Convention | `convention_aligned` | Body rewritten and non-algorithmic alignment patches applied | | | `convention_skipped_no_corpus` / `convention_skipped_no_pr` / `convention_skipped_not_bot` | No merged-PR corpus to learn from, or nothing to act on | | | `convention_failed_extraction` / `convention_failed_misalignment` / `convention_failed_patch` / `convention_failed_push` | **Red** | | Test | `test_passed` | Lint and tests passed; the draft was moved to ready for review | | | `test_failed` | Lint or tests failed; `outrider:test-failed` applied, the PR stays a draft | | | `test_skipped_no_pr` / `test_skipped_not_bot` | Nothing to test | | | `test_failed_setup` | The test environment could not be set up. **Red** | | Issue convention | `issue_convention_aligned` / `issue_convention_aligned_no_fitting_template` / `issue_convention_skipped_no_templates` | The Issue body was folded to the best template, or folded with scaffolding collapsed when none fit | | | `issue_convention_skipped_no_issue` / `issue_convention_skipped_not_bot` | Nothing to act on | | | `issue_convention_failed_claude` / `issue_convention_failed_patch` | The model call or the body PATCH failed (`_claude` is **red**) | The same tables, kept in step with the action, are in the action repo's [`docs/configuration.md`](https://github.com/remyxai/outrider/blob/main/docs/configuration.md). *** ## Anatomy of an Outrider PR As first opened, a PR body is built in a fixed order so you know where to look: 1. **Tests and self-review**: what the integration covers, what it left out, and the self-review's verdict on reachability. 2. **License & code availability**: the paper's reference code and any repos, models or datasets the implementation drew on, with their SPDX ids and a flag when one is copyleft, non-commercial, or missing. 3. An italic *Implements \[paper]* line linking the arXiv id. A brief-mode PR says it was drafted from a design brief instead, lists the Issues the brief referenced, and carries the brief itself in a collapsed **Design brief** block. 4. **Discovery context**, a collapsed `
` block with Outrider's scaffolding: the research interest, which agent implemented it, the selection reasoning, and the suggested experiment. Absent on a brief-mode PR, since there was no discovery. 5. A footer naming the Remyx orchestrator that opened it. 6. **Review before landing**, appended only when the diff touched a risky surface; lists the files. The refinement chain then edits the body in place: the fidelity phase appends a **Coverage** matrix (covers / defers / deviates against the reference, with a note when the audit was anchored on the abstract only), and the convention phase rewrites the body to your repo's PR template and folds the scaffolding into the Discovery context block. The body's last line is a `Co-Authored-By: remyx-ai[bot]` trailer, kept last through every rewrite, so attribution survives a squash-merge. Relevance scores and confidence lines are not in the body; they live in the run telemetry and the [agent report](/platform/agents/overview#the-agent-report). ### Reviewing it Treat it like a first-pass proposal from a teammate: 1. **Read the self-review and the coverage matrix** in the PR body: what the integration covers, what it deferred, and where it deviates from the source. 2. **Run the diff and tests** locally; the action runs `pytest` plus an integration check, but your full suite is the real bar. 3. **Decide**: merge, push follow-ups, or close. Your call feeds the agent's [Direction](/platform/agents/overview#direction) either way, and the [Inbox](/platform/review/inbox) card clears itself when GitHub records the merge or close. Merged and closed PRs feed back into the agent's [Direction](/platform/agents/overview#direction); every decision re-weights what it proposes next. To measure a PR's effect before deciding, mention `@remyx-ai validate` on it ([Validate](/platform/agents/validate)). ## Next steps [Run Outrider on a repository](/tutorials/outrider-repository) · [Use the CLI](/cli/outrider) · [Read the GitHub Action configuration](/platform/agents/outrider-configuration) # Manage agents Source: https://docs.remyx.ai/platform/agents/overview Review Outrider installations, run status, schedules, and repository configuration. In Studio, an agent is an Outrider installation connected to a repository. Use the Agents view to see installed repositories, inspect runs, and manage the configured model provider and schedule. ## Open an agent 1. Go to [Agents in Studio](https://studio.remyx.ai/agents). 2. Select the repository. 3. Review its current status, recent runs, and configuration. The linked GitHub Actions run shows what executed. A completed run can open a draft PR or Issue, leave a branch, or skip publication. ## Change configuration Use the agent settings in Studio or the documented repository configuration. Changes to credentials, workflow installation, or schedules may require an updated setup step. [Outrider configuration](/platform/agents/outrider-configuration) *** **Nav:** Agents | **URL:** [`/agents`](https://studio.remyx.ai/agents) An **agent** is an autonomous worker on your repos. It does a job, reports evidence for everything it does, and holds the calls a human should make for your [Inbox](/platform/review/inbox). Two ship today: * **Outrider**, a GitHub Action installed on one repo, turns recent research into draft PRs on a schedule. This page is its fleet view. * **[Validate](/platform/agents/validate)** measures a pull request when you mention `@remyx-ai validate` on it: it designs a test, runs it on Remyx compute, and posts the numbers against your bar. It needs only the Remyx GitHub App, not an Outrider install, and the first validation on each installation is free. The Agents view is the fleet index: every repo you've installed on, its state, and its last outcome, with **+ Set up an agent** to add another. Click any agent to open its report. *** ## Set up an agent Click **+ Set up an agent**. The setup wizard is the in-app twin of `remyxai outrider init`: ![The Set up an agent wizard: repository picker with the Remyx GitHub App confirmed installed, the research-interest dropdown on 'Create one from this repo automatically', Review vs Auto mode radios, the coding-agent and model-backend selectors with their connected providers marked, and the Advanced models-per-phase fold with Two-tier selected](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/agent-setup-wizard.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=64b4f21327736839824ee347cc865a3b) **1. Repository** Pick from your GitHub repos or paste a URL. The wizard preflights the **Remyx GitHub App** on that repo. The App is how agents author PRs and Issues as **remyx-ai\[bot]** (without it they'd come from `github-actions[bot]`). If it's missing, the wizard shows the one-click install link and unlocks automatically once GitHub records it. **2. Research interest driving the agent** **Create one from this repo automatically** (the default — Remyx derives the interest from the repo's code and history) or pick an existing active interest. **3. Mode** * **Review** — the setup PR and every draft wait for you. * **Auto** — the setup PR merges itself; drafts still wait for you. **4. Coding agent and model backend** Two independent choices. The **coding agent** is the CLI that writes the implementation: **Backboard R-CLI**, **Claude Code**, or **OpenAI Codex**. The **model backend** is the vendor serving it: **Anthropic (Claude)**, **Moonshot AI (Kimi)**, **OpenAI**, **OpenRouter**, or **Z.ai (GLM)**. The wizard preselects a connected provider and flags one that isn't connected yet. Each agent speaks one model-API family, so the backend list narrows to the providers that agent can reach; the [pair table](/platform/agents/backends#which-pairs-work) has the full set. Picking Backboard resolves the backend inside your Backboard account instead, and needs an explicit model. Under **Advanced — models per phase** you can choose the [two-tier drafter/refiner layout](/platform/agents/backends#the-two-tier-drafter-refiner-pattern), whose tiers can run different agents as well as different models. Provisioning issues the repo a scoped **agent key** (`rmxa_`), writes the workflow file, and opens a bot-authored **setup PR** — merge it (or let Auto mode merge it) to activate the agent. ```bash From the terminal theme={null} remyxai outrider init --repo owner/name --auto-interest --mode review ``` *** ## Provisioning states Each agent row shows its status chip, inline **Run now** / **Pause** actions, and its last outcome: ![Agent rows in the fleet index: each repo with its status chip (active, or setup PR open linking to the PR), Run now and Pause actions, and the last outcome — a discussion, a skip with its reason, or 'Provisioned · awaiting first run'](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/agents-fleet-index.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=e00a34296feb7e44cbe685eafdea4e6c) The chip is derived the same way the CLI's preflights reason about it: | State | Meaning | What to do | | - | - | - | | **setup PR open** | The bot-authored setup PR hasn't been merged | Merge the PR (the chip links to it) | | **needs model key** | No key is connected for its agent/backend pair | Connect the one that pair needs under [Connectors](/platform/configure/connectors) | | **awaiting first run** | Provisioned, no run dispatched yet | **Run now**, or wait for the schedule | | **active** | Running on its schedule | Nothing — review its drafts in the [Inbox](/platform/review/inbox) | | **paused** | Its key is revoked; the next run stops immediately | **Resume** re-provisions with a fresh key | ## Run now / Pause * **Run now** dispatches a one-shot run outside the schedule. Available once the setup PR is merged and the agent isn't paused. * **Pause** revokes the agent's key: the next run fails auth immediately. The workflow file stays in the repo; nothing else is touched. Resuming is a fresh re-provisioning (new key, new setup PR). Pausing here, revoking the agent key under [Account](/platform/configure/account#agent-keys), and the MCP `pause_agent` tool all do the same thing. ```bash From the terminal theme={null} remyxai outrider trigger --repo owner/name ``` *** ## The agent report Click an agent to open its report at `/agents/`. It leads with a KPI strip and **Findings** (each finding deep-links into the evidence it came from) over a shared **date window** (default last 7 days, presets up to All time) that rides the URL, so any view is shareable. Two header buttons: **Ask about this agent** and **Configure**. Anything blocking the agent is a banner up front; healthy agents get **Run now**. ![One agent's report: KPI strip (27 dispatches, 1 draft opened, 2 leads awaiting a call, \$17.04 spend at \$5.68 per artifact), findings such as '7 runs failed before reporting' and ''already tried / duplicate' dominates demotions' — each with an evidence link — and the Evidence list of Runs, Direction, and Activity trace sections](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/agent-report.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=60a5eef01717687e4c1cba1059734d8b) ### Runs Every run the agent has reported, in weekly rollups, filterable by outcome and searchable by title, arXiv id, or status: | Outcome | Meaning | | - | - | | **Draft PR** | The run opened a draft pull request | | **Discussion** | The run opened an Issue — relevant, but safer to discuss first | | **Branch** | The run pushed a draft branch; refinement promotes it to a PR | | **Failed** | The run errored — see **Analyze** | | **Skipped** | The run held publication (low confidence, cadence guard, no structural fit, …) | Each row shows the artifacts, the run's **cost**, and an **Analyze ›** action that pulls the run's job log and returns a root-cause read with a suggested next step. ### Direction What the agent has learned from your decisions. Every merge, skip, and discussion re-weights what it proposes next: | Marker | Meaning | | - | - | | **↑ Promoted** | A preference your merges have strengthened | | **↓ Demoted** | A direction your skips and closures have weakened | | **→ Held** | Signal exists but isn't decisive yet | Each direction is evidence-backed: expand it to see the specific PRs, Issues, and runs it was learned from. ### Activity trace The agent's event stream, grouped by week: **Merged**, **Closed**, **Drafts**, **Discussed**, **Failed runs**, **Skipped** — each event linked to its artifact, with skip reasons attached. ### Configuration The agent's operating setup, editable in place: repository, interest, workflow file, setup PR, **coding agent** and **model backend** (changing either re-provisions with a fresh setup PR), and status with **Pause** / **Resume**. A single run can override both without re-provisioning, via `remyxai outrider trigger --agent --provider`. The **Signal** panel lists what feeds this agent's decisions: GitHub merge history (the ranking baseline), the repo's [research brief](/platform/discover/briefs), validation runs, Linear tickets (interfaces under active work route risky candidates to discussion), and, when connected, A/B ship-rate feedback. ### Ask about this agent A chat drawer grounded in the agent's real data (runs, telemetry, failing-run log tails, open artifacts, direction) via read-only tools. It explains; actions like triggering or pausing stay behind the app's confirm-gated buttons. [Inbox](/platform/review/inbox) **Ask** links land in this same drawer, focused on the relevant run. *** ## Where agents surface elsewhere * The fleet rolls up in [Reports](/platform/review/reports); items needing you land in the [Inbox](/platform/review/inbox). * An agent can belong to several [Projects](/platform/configure/projects) — link it under the project's **Agents** section. * The full fleet is scriptable via the [MCP tool plane](/resources/mcp-server#outrider-fleet-tools) and the [REST API](/api-reference/agents#outrider-agents). *** ## Related > [card] [Actions](/platform/agents/outrider-action) > > What actually runs in the repo's GitHub Actions > [card] [Model backends](/platform/agents/backends) > > The coding agent and model backend axes, and the two-tier pattern > [card] [Inbox](/platform/review/inbox) > > Where an agent's drafts and held calls wait for you > [card] [Reports](/platform/review/reports) > > The fleet in one view # Outrider security Source: https://docs.remyx.ai/platform/agents/security Review Outrider repository permissions, execution boundaries, and credentials. Outrider runs through GitHub Actions in a repository you configure. Review the workflow permissions, model-provider credentials, and code changes before allowing automated runs. ## Repository access The workflow uses the repository permissions and secrets documented below. Its access depends on the installation method and repository settings. ## Review before merge Draft PRs and other run artifacts are inputs to maintainer review. Your team controls merges and repository protection rules. ## Exact permissions and controls The reference below lists the supported security controls and their defaults. *** Outrider drives an autonomous coding agent from **untrusted input**. The candidate brief is assembled from arbitrary GitHub Issue and PR text, which is the classic vector for indirect prompt injection. No single control is trusted to stop a determined injection. Six independent layers each fail safe, so an attack has to defeat all of them to cause harm. The durable controls sit outside the model's influence (deterministic code, a stripped environment, path enforcement); the model-facing layers (input wrapping, the canary) are treated as signals, never guarantees. This page covers the agent runtime. For what the agent is allowed to change in your repo, see [Guardrails](/platform/agents/outrider-configuration#guardrails). ## The layers **1. L1: Trust boundary at ingestion (signal)** Every piece of untrusted text (Issue bodies, PR text, Discussions, fetched lead content) is wrapped in `` tags with a preamble instructing the agent to treat it as data, not instructions. This reduces naive injections; it is not relied on alone. **2. L2: Capability reduction (durable)** The spawned agent runs under a **PreToolUse Bash deny gate** delivered through `claude --settings`, fail-closed, that blocks package installs and any outbound network command. Its subprocess environment is allow-listed at launch: `GITHUB_TOKEN`, `REMYX_API_KEY` and every `INPUT_*` variable are stripped, so `printenv` or `cat` cannot surface them. The tool gate is a Claude Code feature; Codex and Backboard run with the stripped environment but without a per-tool deny gate, so on those agents the timeout and the post-session gates are the controls. **3. L3: Credential-leak prevention (durable)** The clone's origin is rewritten **token-less** so no credential persists in `.git/config`; the push re-authenticates with a one-shot URL. Every outbound payload (PR and Issue bodies, API calls) passes a fail-closed scrubber: a credential match **aborts** the run with `aborted_secret_in_payload`, which exits red, rather than being silently redacted. **4. L4: Per-run canary (signal)** The invocation template requires the agent to acknowledge the spec with a per-run canary token derived from the run id and the target repo. A missing or wrong canary is the fingerprint of an injection that hijacked the agent's output, and downgrades routing from PR to Issue (`issue_opened_canary_missing`) for a human to inspect. **5. L5: Deterministic post-session verification (durable)** After the session and before any push: the path block (`.github/workflows/**` is the one hard block; workflow edits are stripped from the commit and listed), the **risky-surface** flag that forces draft and labels diffs touching dependency manifests, CI, hooks, containers or shell scripts, the integration gate (new code must be invoked by a changed file), and the stub-density gate. **6. L6: Human-in-the-loop (external)** Nothing auto-merges. Output is a **draft PR** or a **human-review Issue**. A guard trip aborts rather than filing, and a maintainer can disable the workflow outright. Before the agent starts, the action pre-installs the target repo's dependencies itself, so the agent never needs install permissions during the session. ### What we deliberately do not rely on The two model-facing layers (L1 wrapping, L4 canary) are signals, not guarantees; a sufficiently clever injection can evade either. The security argument rests on the durable layers (L2, L3, L5, L6), which are deterministic code the model cannot reach. ## Where the model key lives Outrider runs on your own provider key. Where that key sits depends on how you installed: | Install path | Where the key goes | Who holds a copy | | - | - | - | | `remyxai outrider init` (default) | The provider key you connected in [Connectors](/platform/configure/connectors#model-providers-and-coding-agents) is pushed to the repo as an Actions secret by the Remyx engine | Remyx (encrypted at rest) and the repo's secret store | | `remyxai outrider init --github-secrets-only` | Sealed against the repo's own Actions public key **on your machine**, so it reaches GitHub as ciphertext | The repo's secret store only | | BYOK from the app (client-sealed relay) | Studio fetches the repo's Actions public key, seals the key in your browser, and sends only the ciphertext to the engine, which forwards it to GitHub | The repo's secret store only | | Manual install | You set the secret yourself | The repo's secret store only | In every case the key is decrypted only inside the ephemeral runner, and the agent subprocess never sees it as an environment variable the model can print. ### BYOK: the client-sealed relay For the app path, the engine exposes the repo's Actions public key so the browser can seal the secret before it leaves your machine: * `GET /api/v1.0/interests//provision-action/actions-public-key` (scope `provisioning:read`) returns `repo`, `key_id`, `key`, and `sealable_secret_names`, the provider secrets the registry allows (`ANTHROPIC_API_KEY`, `ZAI_API_KEY`, `MOONSHOT_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`). * `POST /api/v1.0/interests//provision-action` accepts `sealed_provider_secrets`, a map of allowed secret name to sealed ciphertext plus the `key_id` it was sealed against. A stale `key_id` (GitHub rotated the repo key) returns `409`; fetch the key again and re-seal. * The agent's `model_key_source` records custody: `remyx` when the engine pushed a key it holds, `byok` when you sealed it. Custody is sticky across re-provisions; pass `reclaim_key_custody: true` to switch a BYOK repo back to the engine-pushed key. BYOK serves Outrider only. [Validate](/platform/agents/validate) runs the model server-side on the key you connected in Connectors and never pushes a secret to the repo, so a sealed key cannot serve it. ## Supply chain * Consumers pin `remyxai/outrider@v1`. That tag moves automatically when a release is published, so a fix reaches every install at the next release. Change-controlled orgs can pin an exact release (`remyxai/outrider@v1.9.0`) and bump deliberately. * The coding-agent CLIs (`@anthropic-ai/claude-code`, `@openai/codex`) are installed at the exact versions pinned in the action's `package.json`, with a check that the runner's Node meets the CLI's declared floor; a floating install would pull whatever shipped overnight into every scheduled run at once. Backboard R-CLI is installed from its vendor installer. * Python dependencies come from a pinned requirements file. No system packages are installed (`apt` is never called); PDF handling uses a pure-Python library. * Steps that only improve the agent's inputs (optional PDF support, code-search indexing) are best-effort and never fail the run. ## What the Remyx engine holds For an `init` install the engine holds a hash of your Remyx API key, the repo's scoped automation key (which can read one interest, refresh recommendations, and mint `remyx-ai[bot]` tokens for that repo), and, unless you used `--github-secrets-only` or BYOK, the provider key you connected, encrypted at rest. Each run reports its telemetry back (status, cost and tokens, the selection reasoning, the artifact URLs) for the [agent report](/platform/agents/overview#the-agent-report) and [Reports](/platform/review/reports). The repo clone stays on the ephemeral runner. ## Next steps [Run Outrider on a repository](/tutorials/outrider-repository) · [Use the CLI](/cli/outrider) · [Read the GitHub Action configuration](/platform/agents/outrider-configuration) ## Related > [card] [Guardrails](/platform/agents/outrider-configuration#guardrails) > > What the agent may change in the repo > [card] [Model backends](/platform/agents/backends) > > Which key each agent and provider pair needs > [card] [Connectors](/platform/configure/connectors#choose-a-model-provider) > > Connect a provider once; the engine pushes it per repo > [card] [Defense-in-depth in the action repo](https://github.com/remyxai/outrider/blob/main/docs/security.md) > > The action repo's security page, with code references # Validate a pull request Source: https://docs.remyx.ai/platform/agents/validate Request a PR validation, review the test and inspect metrics, artifacts and cost. Measure a change against defined criteria before merging it. Validate proposes and runs an evaluation on a pull request in a connected GitHub repository. It works on PRs written by your team, another coding agent, or Outrider. ## Run a validation 1. Connect the repository through the Remyx GitHub App and enable Validations. 2. Comment `@remyx-ai validate` on an open pull request to request a test. 3. Review the proposed test and criteria. Confirm the run when the request requires a separate approval. 4. Inspect the measured results, executed notebook, and cost on the pull request. A committed validation spec can start a run directly. An explicit request to revise and run may also skip separate confirmation. [Run your first validation](/tutorials/validate-first-run) · [Command reference](/platform/agents/validate-commands) ![GitHub PR conversation showing an @remyx-ai validate request and the bot acknowledging that it is designing a proposed test.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/validate-pr-request.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=6143a1ef7ae742276e8143a3dd8ed3d2) Request an evaluation directly from a GitHub pull request. ## What gets measured Some validations compare the base branch with the PR. Others test a new capability against an absolute threshold or reference. The result identifies the comparison used and which criteria were met. ## See a completed result The SAELens validation compared TopK and AbsTopK sparse autoencoders under two offline criteria. In the recorded single-seed CPU run, both criteria were met. | Measure | TopK | AbsTopK | | - | -: | -: | | Explained variance | 0.9666 | 0.9677 | | Negative selected latents | 0% | 49.2% | ![Validate's GitHub result for SAELens, with a passing verdict, parity-gap metric, negative-latent metric, and evidence table.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/validate-sae-result.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=4d1db7eb14a421f1a38a4f786f47a203) The SAELens validation met both specified offline criteria in one CPU run. This result supports the specified test criteria. It does not establish production improvement or downstream steering and monitoring performance. [Read the GitHub result](https://github.com/salma-remyx/SAELens/pull/5) · [Open the notebook](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/.remyx/validations/abstopk-vs-topk-tinystories.ipynb) ## What changes on the PR Validate commits proposed test files and configuration to the PR branch. Executed notebooks are committed by default. Automatic repair may also edit repository code by default, but cannot change the confirmed test or criteria. [Execution and safeguards](/platform/agents/validate-execution) ## Cost The first completed eligible validation is free per GitHub App installation. Later completed runs use prepaid credits or an applicable grant. A valid inconclusive result may still count as a completed, chargeable run. [Billing](/platform/configure/billing) · [Results and evidence](/platform/agents/validate-results) · [Compute and limits](/platform/agents/validate-limits) # Validate commands Source: https://docs.remyx.ai/platform/agents/validate-commands Request, revise, run, check, and cancel a validation from a GitHub pull request. ## Request or run a validation Comment `@remyx-ai validate` on a connected pull request. If no spec is committed, Validate proposes a test for review. If a valid spec is already committed, the command can start execution. A request that explicitly asks to revise and run can also skip a separate confirmation. Review the spec and any instructions that will trigger execution. The command reference below lists supported actions, aliases, authorization rules, response states and cancellation behavior. ## Request a test Comment on an open pull request: ```text theme={null} @remyx-ai validate ``` If the PR has no validation spec, Validate proposes a test and commits the proposal to the PR branch for review. Add a description of what to measure when the default question is too broad. ## Run an approved test After reviewing the proposal and `.remyx/validation.yaml`, comment again: ```text theme={null} @remyx-ai validate ``` When a committed validation spec already exists, the command can begin the run without generating a proposal first. ## Revise a proposal Use the documented `@remyx-ai revise` command to request a change to the proposed test or criteria. Review the updated proposal before running it. ## Check progress ```text theme={null} @remyx-ai status ``` Read the PR comment and the Remyx Validation GitHub Check for the latest state. ## Cancel a run ```text theme={null} @remyx-ai cancel ``` Cancellation requests stop an in-flight validation and its repair loop. Review the resulting status before starting another run. ## Full command reference The tables below list the supported aliases, comment-routing rules, and run-state behavior. The handle is the App's slug, `@remyx-ai`. `@remyxai` and `@remyx` also work, in any case. | You want to | Comment | What happens | | - | - | - | | **Start or confirm** | `@remyx-ai validate` | With a proposal waiting: confirms it and runs it, smoke test first. With a committed spec and no proposal: runs the spec. With neither: proposes a test from the PR itself. | | **Start with your own ask** | `@remyx-ai validate ` | Designs a test for that claim, commits it to the PR branch, and posts it for review. Nothing runs yet. | | **Revise** | `@remyx-ai validate ` or `@remyx-ai revise ` on a proposed test | Revises the proposal from the previous attempt plus your feedback. Add "and run it" to run as soon as it is revised. `redesign` and `refine` also work. | | **Check status** | `@remyx-ai status` | Where the run is and how long it has been going. | | **Cancel** | `@remyx-ai cancel` | Stops the in-flight run and the fix loop behind it. `stop` and `abort` also work. | | **Ask about a run** | `@remyx-ai ` | Answered from the run's own telemetry, with no action on the validation. | > [!NOTE] > > A committed `.remyx/validation.yaml` takes precedence over guidance in a new PR comment. To change its test or criteria, edit the file on the branch before you run. Comment feedback alone does not revise a hand-written spec. > [accordion] Advanced routing behavior > > * **The bare commands never go to a model.** `validate`, `cancel`, `revise` and `status` on their own are matched literally and act immediately. > * **Everything else is routed by a model** that reads the comment, the thread, and the PR's state (spec committed, proposal pending, run in flight) and picks one intent: propose, revise, run, cancel, status, question, or unclear. Below 0.5 confidence it asks you instead of guessing. If the model is unreachable, the literal commands are the fallback. > * **Informal asks with an action verb** ("please run a validation vs the upstream benchmark") route to the same pipeline as `validate`. Questions about a validation go to Ask. > * **Feedback on a proposal is routed as a revision.** If the request also explicitly asks to "and run it", the revised test can start immediately when the revision lands, without another approval comment. A `validate` comment that reports a change you already made (for example, "fixed the version pin") requests a run. > * **Every reply ends with the next step** for the state the PR is in. > [accordion] Who may trigger, caps, and acknowledgement > > | Rule | Value | > | - | - | > | Who may trigger | Comment authors whose GitHub association is `OWNER`, `MEMBER` or `COLLABORATOR`. Others are ignored and logged. | > | Daily cap | 20 answered mentions per repo per day. | > | Acknowledgement | A 👀 reaction on your comment. No reaction means the mention was ignored; the reason is in the engine log. | > | One run per commit | A second `validate` on a commit already being measured points at that run. Push a new commit to measure that instead. | > | One run per repo | A run in flight on another PR of the same repo refuses this one at start. Nothing is used. | ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Execution and safeguards Source: https://docs.remyx.ai/platform/agents/validate-execution Review pull request branch writes, confirmation rules, code repair controls and cancellation. ## What Validate can change Validate writes proposed test files and configuration to the pull request branch. Executed notebooks are committed by default. Automatic repair may edit repository code unless you disable that behavior in the validation spec. A committed spec or an explicit request to revise and run may execute without an additional confirmation. Automatic repair cannot change the confirmed test or acceptance criteria. Review the exact permissions, repair settings, write boundaries, run states and cancellation behavior below. ## Before you run Validate uses the repository's GitHub App installation to write tests, configuration, repairs, and results to the PR branch. It never merges the PR. An owner, member, or collaborator must request execution, but a committed spec or an explicit request to revise and run may start without an additional confirmation. To prevent automatic code edits, set `loop.fix_code: false` in `.remyx/validation.yaml`. To stop an active run and its repair loop, comment `@remyx-ai cancel`. [Command reference](/platform/agents/validate-commands) · [GitHub App permissions](/platform/configure/github) ## Detailed execution lifecycle The reference below covers proposals, comparison selection, execution, repair bounds, artifacts, cancellation, and status transitions. ### From proposal to result **1. Resolve the spec** In order: a committed `.remyx/validation.yaml` at the PR head, then a locked eval template on the project, otherwise Validate proposes one and waits. A yaml that only carries `model:` is configuration, not a spec. **2. Propose** The reply is **Designing the test now, nothing for you to do yet**. Validate then commits a proposed test to the PR branch and posts it: what it measures, the metric and bar, its rationale, and what the design could not cover. Nothing runs until you say so. With "and run it" in your ask it runs as soon as the proposal lands. **3. Confirm** `@remyx-ai validate`. The reply is **Validation started** with a cost estimate on the first line, and the **Remyx Validation** check opens on the PR. Your instructions are ignored at this point if the PR already has committed criteria. The started comment says so. **4. Smoke (conditional, never charged)** For a harness with a parameter file, the same test runs first on a few steps. For a notebook that reads `REMYX_SMOKE=1`, the notebook runs its smallest end-to-end version of itself. The smoke run proves the environment, the imports and the results file before the budget is spent. It posts no result, and it is never charged. A notebook with no `REMYX_SMOKE` path goes straight to the full run. **5. Measure** Validate chooses a comparison based on the question and the available reference. Some tests compare the base branch with the PR branch. Others test a new capability against an absolute threshold or a published baseline. The run records which comparison it used. The full measurement may use one or more arms, times the declared seeds. Progress is edited into the started comment and streamed to the check. **6. Result** The result posts on the PR, the check completes (success, failure, or neutral for an abstention), the executed notebook is committed, and the run is charged. An errored run posts one comment and completes the check as neutral. A cancelled run marks it cancelled. ### What gets committed to your PR | File | When | Notes | | - | - | - | | `.remyx/validation.yaml` | On proposal | The spec. Edit it; a hand-edited spec is what runs next. | | `.remyx/validations/.ipynb` or the test file | On proposal | The test itself. The fix loop cannot edit the confirmed test. | | `.remyx/validations/.Dockerfile` | On proposal | The environment. Edit it and the next run builds from your version. | | `.remyx/validations/.ipynb` (executed) | On result | The executed notebook with outputs, overwritten each run. Trimmed to a 2 MB budget, plots dropped first, text streams capped. | With `commit: false` on the notebook suite, the executed notebook is served from the artifact store instead (presigned links, 7 days) and linked from the result. Each run may keep up to 12 artifacts of 64 MB each, readable from `GET /validate/runs/`. Every commit is authored by **remyx-ai\[bot]** on the PR branch, never on the default branch. Revert any you do not want. ### Fix loop and trust guards | Bound | Default | Where to change | | - | - | - | | Iterations per stage (smoke, then measurement) | 8, hard ceiling 10 | `loop.max_iterations` | | Same failure after a fix | One repeat is fed back to the fixer; a second identical return stops the loop | not configurable | | Runs on one PR in a window | 12 in 30 minutes stops the chain as a safeguard | not configurable | | Spend | `loop.budget_cap_usd` | the spec | | Scope | `fix_env` installs packages, `fix_code` edits the repo, `fix_spec` adjusts the spec | `loop.fix_env`, `loop.fix_code`, `loop.fix_spec` (all `true`) | | Cancel | `@remyx-ai cancel` ends the whole chain, not one job | the PR | What the loop can never do: * **Edit the confirmed test or the criteria.** A loop allowed to move the thresholds it is judged against would optimise the judge. * **Write the metrics.** A harness or notebook that assigns the result literals is blocked as forged before it runs, and the rejection goes back to the fixer. * **Install a look-alike.** `from utils import x` that resolved to a PyPI package instead of the repo's own file is triaged as a missing file, not a missing dependency, and never pip-installed. Hard stops the loop does not retry: out of memory, out of time, a device the pool cannot provide, and a run whose arms cannot be told apart. Set `loop: {fix_code: false}` to go back to reporting only. ### Logging to W\&B Logging is triggered by `logging.wandb: {entity, project, base_url}` or `requires: [wandb]` in the spec, and detected from the PR template, the harness's own config, and the repo's contribution docs. Runs are tagged `remyx-validate`. Your own key is injected into the measurement pods, never a Remyx key. Before the run, the started comment shows **Where results will land** with the provenance of each value. A spec that logs to W\&B on an account with no W\&B connection, a bad entity or an invalid project is refused at start. ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Compute and limits Source: https://docs.remyx.ai/platform/agents/validate-limits Runtime hardware, concurrency, timeouts, seed settings, and validation constraints. ## Plan the run Validate uses Remyx-hosted compute. CPU is the default. Available GPU tiers, concurrency, timeouts, seeds, size limits, and budget caps are defined below. A timeout means a run did not finish. It does not prove that the change failed its criteria. Validate runs on Remyx-hosted compute. CPU is the default. Tests that need accelerators can use an available GPU tier when supported by the current configuration. Only one validation can run at a time per repository. A second request for the same commit may link to the existing run rather than launch a duplicate. A timeout means the requested evaluation did not finish within its configured limit. It does not establish that the change failed its criteria. ## Current limits The tables below list the supported hardware, resource quotas, concurrency, timeouts, seed defaults, size limits, and command restrictions. Check the active configuration before planning a costly run. | | Default | Notes | | - | - | - | | Compute tier | CPU | `compute.tier: gpu` or a `lighteval:` suite selects GPU | | GPU available | One L40S (48 GB) per run | One GPU eval runs at a time across the platform; the next one queues | | Measurement timeout | 1800 s | `compute.timeout_s`; `auto` is the estimate times `timeout_padding` (1.5), clamped to 60 s and `timeout_max_s` (4 h) | | Pod deadline | measurement + 900 s CPU, + 3600 s GPU | Absorbs image pull, GPU scale-up and upload | | Environment build timeout | 1200 s per cold arm, on CPU | Warm re-runs of the same commit skip it | | Default seeds | 1 | `seeds: N` multiplies the arms | | Concurrency | One run per commit, one run per repo | A second mention points at the running one or refuses | A run that hits its deadline reports **Ran out of time**. That says nothing about your code. Raise `compute.timeout_s` or narrow the corpus. ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Model selection Source: https://docs.remyx.ai/platform/agents/validate-models Choose a model for proposal and validation tasks, with applicable billing rules. ## Choose a model Connect a supported provider in Studio or use an eligible Remyx-funded allowance. The model selected for a task depends on the documented precedence rules. Your own provider bills calls using your key. Hosted validation compute and applicable Remyx-funded model calls use Validate credits on paid runs. Connect a provider under Studio → Connectors → LLM providers. Select a default or repository-specific model under Studio → Account → Validate. ## How a model is chosen The configured precedence determines which model a task uses. Review the ordered rules below when an explicit preference does not take effect. ## Allowances The first eligible validation may use a Remyx-funded model without connecting your own provider. Availability and caps apply. This is not an unlimited, permanent model allowance. ## Precedence and allowance reference The first source that names a provider wins. | Order | Source | Where to set it | | - | - | - | | 1 | Per-call override | `model_provider` / `model` in `POST /validate/runs` | | 2 | `model:` in the committed `.remyx/validation.yaml` | the spec | | 3 | Validate settings: repository override | Account → Validate → Repository overrides | | 4 | Validate settings: the App installation (every repo it covers) | Account → Validate, or `PUT /validate/settings` with `installation_id` | | 5 | Validate settings: account default | Account → Validate → Default model | | 6 | The backend Outrider was provisioned with for this repo | fallback only | | 7 | The account's connected providers | Connectors | Two rules at the end of the chain: * **Several providers connected and no default: Validate will not pick one.** It asks you to choose under Account → Validate. One connected provider is a choice; two is a question. * **Nothing connected: the first run designs on a Remyx-funded model.** Connecting your own key takes over from the next mention. The key is probed with one call before the run starts. A key with no credit or a rejected key is refused at start and nothing is used; the reply names the fix. The same applies to a rate limit and to the funded cap. **The Remyx-funded allowance** applies only when no provider is connected and only on the free validation. Code defaults, which production may override: | Cap | Default | Env var | | - | - | - | | Per job | \$2 | `REMYX_FUNDED_CAP_USD` | | Per account | \$5 | `REMYX_FUNDED_ALLOWANCE_TRIAL_USD` | | Per day, all accounts | \$50 | `REMYX_FUNDED_DAILY_CAP_USD` | On a paid run, funded model calls are priced into the debit at list price times the platform multiplier; calls on your own key are never charged. ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Results and evidence Source: https://docs.remyx.ai/platform/agents/validate-results Interpret validation criteria, metrics, artifacts, passing and inconclusive outcomes. A validation result connects the question, executed test, observed metrics, criteria, artifacts, and cost. Read the PR comment for the summary. Use the GitHub Check for execution status and the notebook and run artifacts for the underlying work. Results also appear in Studio under Reports → Validations. ## How to read a result Read the question, comparison, measured metrics, acceptance criteria and supporting artifacts together. A passing result means the required criteria were met in that run. It does not prove production improvement. An inconclusive result, a no-signal result, an execution error and a timeout have different meanings. Use the detailed status definitions below before deciding what to change or rerun. [Interpret evaluation results](/tutorials/interpret-results) | Measure | Measured value | Required criterion | Scope of evidence | | - | - | - | - | | Explained variance | TopK 0.9666, AbsTopK 0.9677 | AbsTopK no more than 0.02 worse | Single-seed CPU run | | Negative selected latents | TopK 0%, AbsTopK 49.2% | AbsTopK at least 1% | Single-seed CPU run | A missed criterion means the test did not meet its specified target. A diagnostic metric supplies context but does not determine the outcome unless the spec makes it required. An inconclusive or no-signal result means the run did not establish a reliable answer under its chosen test and data. Review missing metrics, comparability, uncertainty, or execution limitations before treating the result as a decision input. ## Comparisons Some validations compare base and PR branches. Others test a capability against an absolute threshold or reference value. The result identifies which comparison ran. ## Artifacts Executed notebooks are committed to the PR branch by default. If the notebook suite sets `commit: false`, Validate provides time-limited artifacts instead. Review artifacts before relying on the summary. ## GitHub checks and merging A passing or failing result is not an automatic merge decision. The Remyx Validation check affects merge protection only when your repository requires it. ## Result fields and status reference The tables below define exact result fields, outcome labels, and status semantics. | Status | Meaning | | - | - | | ✅ **Met your criteria** | The target passed and no guardrail crossed. | | ⚠️ **Met your criteria, with a warning** | The target passed; something else needs a look. | | ❌ **Missed your criteria** | The target fell short of your bar. | | ❌ **Missed your criteria, guardrail crossed** | A guardrail moved past its line. Flagged for review. | | ⚠️ **Inconclusive, `` was not measured** | The test did not produce a metric the result needs. | | 📊 **Measured** | The target's bar is a floor or a sanity check, not a goal. Read the numbers. | | 🔍 **No signal** | The run cannot answer the question. See [No signal](#no-signal) below. | | ⏱️ **Ran out of time** | The budget ended before a result. Says nothing about your code. | | ✅ / ❌ **Your tests pass / do not pass on this PR** | A `checks:` only spec. | Every result comment follows one layout: the status line, your ask, a one-sentence answer, what was found, the metric table, then the details. | Part | What it tells you | | - | - | | **Status** | The outcome in one line, from the table above. | | **You asked** | Your own words, when this thread is where the ask came from. An inherited spec reports what it ran instead. | | **Answer** | The target metric, the comparison, and whether your bar was met. A crossed guardrail or a metric the test never produced is named here. | | **What we found** | Findings from the run. A finding whose metric was not emitted is dropped and listed under **Not measured**. | | **Table** | One row per metric: direction, baseline, this PR, delta, your bar, pass mark. Guardrail, cost and diagnostic rows carry a legend. | | **Links** | The executed notebook on GitHub, Colab and nbviewer; W\&B dashboards when the run logged there. | | **Footer** | What the run cost and what it was charged to. See [the cost footer](#the-cost-footer). | When a fix loop ran, a **What I tried** table lists each attempt, what failed, and the commit that tried to fix it. When the loop stopped, the first line says why: the test itself failed, a file is missing from the branch, or the circuit breaker tripped. > [accordion] Illustrative example of a result comment > > Illustrative example. The numbers are invented to show the layout. For a real one, open the [completed validation](/platform/agents/validate#see-a-completed-result) above. > > ```markdown theme={null} > ## ✅ Met your criteria > > **You asked:** does spectral targeting halve trainable params without losing fit vs standard LoRA? > > **Answer:** `gsm8k_accuracy` came in at **0.412**, vs **0.407** on `main` (+0.005). Your bar was ≥ 0.40 — met. > > **What we found** > - Trainable parameters fell from 6.8M to 3.4M with the same rank. > - Eval loss moved inside seed noise across 3 seeds. > > | Metric | Goal | main | this PR | Δ | Your bar | | > |---|---|---|---|---|---|---| > | `gsm8k_accuracy` | ↑ | 0.407 | 0.412 | +0.005 | ≥ 0.40 | ✅ | > | `trainable_params` | ↓ | 6.8M | 3.4M | -3.4M | ≤ 3.5M | ✅ | > | `eval_loss` 🛡️ | ↓ | 1.21 | 1.22 | +0.01 | ≤ 1.30 | ✅ | > > 🛡️ guardrail — a metric that must not regress, even if the target improves. > > Notebook: [GitHub](…) · [Colab](…) · [nbviewer](…) > > --- > 💳 **$1.84 debited from your Remyx balance — $18.16 left.** > ``` ### Evidence strength Tier A is the default answer: a scaled workload at a matched budget, labelled directional. Tier B is the repo's own harness at its own regime, what a maintainer asks for. Declare `compute.rigor: A|B`; otherwise a reused harness is Tier B and everything else is Tier A. A smoke run is neither. ### No signal Before any result, deterministic checks compare what each arm declared with what it reported. Two arms that declare different settings and report identical provenance, a factor that drifted, or a difference inside the noise produce **No signal, this run can't answer the question** instead of a confident number. ### Targets, guardrails, and diagnostics | Role | Gates the result | Shown as | | - | - | - | | `target` | Yes (warn) | The headline row | | `guardrail` | Yes (fail, with `policy.guardrail_veto`) | 🛡️ must not regress even if the target improves | | `cost`, `latency` | No | ℹ️ measured and reported | | `diagnostic` | No | 📊 measured to inform the reading | A metric's `bar` is a `goal` (the default), a `floor` (a bound the measurement cannot fall below; missing it points at the harness) or a `sanity` check on the harness. ### The cost footer | Basis | Footer | | - | - | | Free validation | 🎁 That was this installation's free validation. What it would have cost, and the two ways on: add credits, or a team pilot. | | Balance | 💳 What was debited and what is left. **Add credits** appears when the balance is below the larger of that debit and \$5. | | Counted grant | How many validations are left on the grant. | | Unlimited grant | No footer. | ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Validation spec Source: https://docs.remyx.ai/platform/agents/validate-spec Configure the question, test, metrics, criteria, compute and repair behavior for a PR validation. A validation spec records what to measure, how to run the test, and how the result will be judged. A PR can use a committed `.remyx/validation.yaml` file or ask Validate to propose one. Review the comparison, required metrics and criteria before execution. ## Start with a working example The minimal example below uses a test or command that exists in the linked repository. Copy its files together and check the expected metric output before adapting it. ```yaml .remyx/validation.yaml theme={null} question: kind: capability ask: "Does AbsTopK recover negative signals without hurting reconstruction? Train one TopKTrainingSAE and one AbsTopKTrainingSAE on tiny-stories-1M activations with the same k, width, seed and tokens under 30 minutes on CPU, and compare run_evals explained_variance and L0 side by side." loop: max_iterations: 8 fix_code: true benchmarks: - name: abstopk-vs-topk-tinystories suite: notebook: path: "eval/eval_abstopk_vs_topk.ipynb" packages: [wandb] baseline: none scorer: explained_variance_gap__topk_minus_abstopk policy: guardrail_veto: true metrics: - name: explained_variance_gap__topk_minus_abstopk direction: min threshold: 0.02 role: target bar: goal reads_as: "How much of TopK's held-out explained_variance AbsTopK fails to recover; 0 or negative means parity or better." - name: abstopk_negative_latent_share direction: max threshold: 0.01 role: guardrail bar: goal reads_as: "Share of AbsTopK's k selected latents that are negative on held-out tokens; TopK's is 0 by construction." compute: tier: cpu timeout_s: 2700 requires: [wandb] ``` This example includes the test files it runs. Copy the spec and its referenced files together, then adjust the repository paths, data and criteria for your change. [Full spec on the branch](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/.remyx/validation.yaml) · [The notebook it runs](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/eval/eval_abstopk_vs_topk.ipynb) · [Executed notebook](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/.remyx/validations/abstopk-vs-topk-tinystories.ipynb) · [Read the GitHub result](https://github.com/salma-remyx/SAELens/pull/5) ## Full schema The reference below defines supported questions, suites, metrics, policy, compute, repair, model selection, artifacts and defaults. ## Location and precedence | Where | Used when | | - | - | | `.remyx/validation.yaml` at the PR head | Always first. A spec on the branch is what runs. | | A locked eval template on the project | When the branch has no spec. | | A proposal | When neither exists: the bot designs one, commits it to the branch, and waits for `@remyx-ai validate`. | Rules the parser applies before anything runs: * A file that only carries `model:` is configuration, not a spec. Its `model:` still applies to the design. * A flat spec (benchmark keys at the top level) is lifted into a one-entry `benchmarks:` list. Both shapes are accepted. * Keys the proposer sometimes writes under the benchmark (`compute`, `metrics`, `policy`, `baseline`, `variants`) are hoisted; top level wins on collision. * A hand-written spec is never revised by feedback on the PR. Edit the file and mention the bot with `validate`. * With a committed spec, instructions in your mention are ignored and the started comment says so. Lint before you commit: `POST /api/v1.0/validate/specs/lint {yaml}` runs the same parser and returns problems; the Studio spec editor uses it on every keystroke. *** ## Key reference ### Top level | Key | Type | Default | Meaning | | - | - | - | - | | `name` | string | `validation` | Label for the run. | | `question` | map | derived | `kind` and `ask`. See below. | | `checks` | map or `true` | none | "Does it work?" `command` to run on the feature arm; `true` means `pytest -x -q`. | | `benchmarks` | list | none | "Does it do what it claims?" Only the first entry runs today. | | `variants` | map | none | Per-arm env overrides: `{feature: {env: {LR: "1e-4"}}}`. Reserved platform keys are dropped. | | `seeds` | int | `1` | How many seeds per arm. Multiplies the arms. | | `compute` | map | `{tier: cpu}` | Tier, budget and rigor. See below. | | `loop` | map | all on | The fix loop's bounds. See below. | | `model` | map or string | none | `provider` and `name` of the model that designs and repairs. `model: zai` is shorthand. | | `requires` | list or string | none | Connected accounts the run needs, e.g. `[wandb]`. A `logging.` block implies it. | | `logging` | map | none | `wandb: {entity, project, base_url}`. See [W\&B](/platform/agents/validate-execution#logging-to-wb). | | `held_constant` | list | none | Factors the design holds fixed. Checked against provenance by the no-signal check. | | `avoid` | list | none | What the design must not do. | | `provenance` | map | none | Free-form provenance the report echoes. | | `production` | map | none | Production signal wiring for calibration. | | `report` | map | none | `headline`, `caveat`, `next`, `findings[]`, `establishes[]`, `does_not_establish[]`, `not_measured[]`. Templates may reference metrics with `{metric}`; a finding whose metric was never emitted is dropped. | ### `question` | Key | Values | Meaning | | - | - | - | | `kind` | `compare`, `parity`, `capability`, `regression` | How many arms run and what the baseline is. Derived when absent: `baseline: none` → capability, a published baseline → parity, benchmarks → compare, checks only → works. | | `ask` | string | The request this test answers, in the requester's words. Reported only when this PR is where the ask came from. | | `question.kind` | Asks | Arms | Baseline | | - | - | - | - | | `compare` | Did the PR change a measured outcome against the base branch? | baseline + feature | the base branch | | `parity` | Does it match a published reference or known result? | feature only | published numbers | | `capability` | Does the new capability meet an absolute criterion? | feature only | none, absolute bars | | `regression` | Does a refactor stay within the allowed tolerance? | baseline + feature | the base branch | **Baselines.** The baseline arm is skipped when the repo publishes its own numbers (`baseline:` with a reference dict, or a results corpus the harness reads) or when a prior validation on the same base commit and the same spec wiring already measured it. Reuse is exact-match only; anything less re-runs the arm. `baseline: none` runs one arm with absolute bars. ### `benchmarks[0]` | Key | Type | Meaning | | | - | - | - | - | | `name` | string | Names the committed test, notebook and Dockerfile stems. | | | `suite` | string or map | What runs. Forms below. | | | `baseline` | string or map | A ref (`main`), `none` / `self` / `feature` / `single-arm` for one arm with absolute bars, or `{source, values: {metric: number}}` for published numbers that replace the baseline arm. | | | `metrics` | list | The criteria. See below. | | | `policy` | map | `thresholds: {metric: value}` (per-metric `threshold` wins), \`auto\_promote: off | on\_pass`, `guardrail\_veto: true\`. | | `dataset_ref` | string | Recorded on the run. | | | `compute` | map | Accepted here as well as at the top level. | | **`suite` forms** | Form | Runs | | - | - | | `lighteval:` | A lighteval task. Defaults the tier to GPU. | | `command:` or a bare command | A shell command on each arm. It must print JSON metrics. | | `path/to/script.py` | A script on each arm. | | `notebook: {path, parameters, packages, commit, commit_path, max_bytes}` or `notebook:path.ipynb` | A notebook under papermill. `commit` defaults to `true`; `max_bytes` to 2,000,000. | | `harness: {runner, notebook, experiments, workdir, results_glob, method, packages, base_image, preimport, smoke}` or `harness:path/run.py` | The repo's own eval apparatus. `results_glob` is read as a published baseline and as a duration source. `smoke: false` skips the smoke stage. | **`metrics[]`** | Key | Values | Default | Meaning | | - | - | - | - | | `key` or `name` | string | required | The metric the test emits. | | `direction` | `>=`, `<=`, or words like `increase` / `lower` | `>=` | Which way is better. | | `threshold` | number | `0`, or `policy.thresholds[key]` | Your bar. | | `role` | `target`, `guardrail`, `cost`, `latency`, `diagnostic` | `target` | `primary` / `main` read as target; `report` / `observe` as diagnostic. | | `bar` | `goal`, `floor`, `sanity` | `goal` | How the threshold reads in the verdict. | | `reads_as` | string | none | One line the table shows next to the metric. | Target rows gate with warn severity, guardrails with fail severity. Cost, latency and diagnostic rows are measured and shown but never gate; a threshold on them is a bound shown for context. ### `compute` | Key | Default | Meaning | | - | - | - | | `tier` | `cpu` | `cpu` or `gpu`. | | `gpu` | follows `tier` | Kept consistent with `tier`. | | `timeout_s` | `auto` | Measurement budget in seconds. `auto` is the estimate times `timeout_padding`, else 1800. Clamped to 60 s and `timeout_max_s`. | | `timeout_padding` | `1.5` | Multiplier applied to an estimate. Not applied to a pinned `timeout_s`. | | `timeout_max_s` | `14400` | The ceiling `auto` may never exceed. Raise it yourself to consent to more. | | `rigor` | derived | `A` (scaled, directional) or `B` (the repo's own harness at its own regime). | | `budget_cap_usd` | none | A spend ceiling the fix loop checks before each dispatch. | ### `loop` | Key | Default | Meaning | | - | - | - | | `max_iterations` | `8` | Per stage (smoke, then measurement). Hard ceiling 10. | | `fix_env` | `true` | Install missing packages and re-run. | | `fix_code` | `true` | Edit the repo and re-run. `false` returns Validate to reporting only. | | `fix_spec` | `true` | Adjust the spec (a wrong results path, for instance). | | `budget_cap_usd` | none | Spend ceiling for the loop. | The loop can never edit the confirmed test or the criteria, and a second identical failure after a fix stops it regardless of these settings. *** ## Examples ### 1. Does it work? Tests only. The verdict is **Your tests pass** or **do not pass**. ```yaml theme={null} checks: command: pytest tests/test_scheduler.py -x -q ``` ### 2. Compare with a guardrail Both arms run on the base branch and the PR; the target must improve and the guardrail must not regress. ```yaml theme={null} question: kind: compare ask: "does the fused attention kernel speed up decoding without changing outputs?" benchmarks: - name: decode-bench suite: command:python bench/decode.py --json baseline: main metrics: - key: tokens_per_s direction: ">=" threshold: 120 role: target - key: output_match_rate direction: ">=" threshold: 0.999 role: guardrail - key: peak_memory_gb direction: "<=" role: cost policy: guardrail_veto: true seeds: 2 compute: tier: gpu timeout_s: 1200 ``` ### 3. Parity with published numbers No baseline arm. The reference values come from the repo's own results corpus. ```yaml theme={null} question: kind: parity ask: "does the new config reach the MetaMathQA accuracy the maintainers publish?" benchmarks: - name: metamath-parity suite: harness: runner: method_comparison/MetaMathQA/run.py experiments: experiments/supertuning results_glob: results/*.json baseline: source: method_comparison/MetaMathQA/results/full-finetuning.json values: {metamath_accuracy: 0.62, peak_memory_gb: 31.2} metrics: - key: metamath_accuracy direction: ">=" threshold: 0.60 role: target - key: peak_memory_gb direction: "<=" threshold: 32 role: guardrail compute: tier: gpu rigor: B timeout_s: 7200 timeout_max_s: 10800 ``` ### 4. Capability with no baseline The base branch cannot run this code, so one arm runs against absolute bars. Illustrative configuration. ```yaml theme={null} question: kind: capability ask: "does the new forecaster train and beat the naive baseline on the sample set?" benchmarks: - name: forecaster-smoke suite: python-script baseline: none metrics: - key: mase direction: "<=" threshold: 1.0 role: target reads_as: "below 1.0 beats the seasonal naive forecast" - key: train_steps direction: ">=" threshold: 50 role: diagnostic bar: sanity ``` ### 5. GPU notebook with a smoke path and W\&B A notebook that reads `REMYX_SMOKE=1` gets a smoke stage before the full run. The W\&B block pulls your connected key into the pod. ```yaml theme={null} question: kind: compare ask: "does rank-16 LoRA match rank-32 on eval loss at half the trainable parameters?" benchmarks: - name: lora-rank16 suite: notebook: path: .remyx/validations/lora-rank16.ipynb parameters: {max_steps: 400, eval_every: 50} packages: [peft==0.14.0] commit: true max_bytes: 2000000 baseline: main metrics: - key: eval_loss direction: "<=" threshold: 1.25 role: target - key: trainable_params direction: "<=" threshold: 3500000 role: guardrail variants: feature: env: {LORA_RANK: "16"} baseline: env: {LORA_RANK: "32"} seeds: 3 compute: tier: gpu timeout_s: auto timeout_padding: 1.5 logging: wandb: entity: my-team project: lora-rank-study loop: max_iterations: 6 budget_cap_usd: 15 ``` The notebook reads `REMYX_SMOKE` and, when set, runs a handful of steps and still emits every declared metric. The smoke run posts no verdict and is never charged. *** ## Lint, push, and the PR Edit the spec from the repo's spec card in Studio. The editor lints against the engine's own parser as you type and **Open a PR** commits the yaml to a new branch and opens a pull request as remyx-ai\[bot]. It never writes to the default branch; nothing changes until that PR merges. The same path is `POST /api/v1.0/validate/specs/push {repo, yaml, project_id?}`. > [!NOTE] > > **`auto_promote` never changes PR state unless you opt in.** The default `off` reports only. `on_pass` marks the PR ready for review when the verdict is a pass. *** ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) ## Related > [card] [Validate](/platform/agents/validate) > > How a validation runs and how to read the verdict > [card] [Billing](/platform/configure/billing) > > What a run costs and when it is charged # Troubleshooting Source: https://docs.remyx.ai/platform/agents/validate-troubleshooting Diagnose missing installations, unavailable models, blocked runs, and errors. ## Find why a run stopped Check the installation, command permissions, selected model, available credits, active run status, and the specific error reported in GitHub. Use the exact error code and stage from the PR comment or GitHub Check to select a recovery step. If Validate does not start, check the repository connection, permissions, credits, model selection, and current run state. ## Check the repository Install the Remyx GitHub App on the repository and connect the installation in Studio. If the installation has not been claimed, complete that step with an authorized GitHub account. Confirm that Validations are enabled for the installation. ## Check the request The command must be on an open pull request and submitted by an authorized repository owner, member, or collaborator. If another validation is running on the repository, read its status before submitting another request. ## Check credits and models If the free completed run is used, add credits under Studio → Account → Billing. Check whether a supported model is available through your configured provider or an eligible Remyx-funded allowance. ## Read the error The PR comment and GitHub Check identify the failed stage when available. A timeout, missing metric, insufficient balance, and invalid spec require different fixes. ## Error reference Use the exact error codes and HTTP statuses below to identify the cause. | The reply said | Outcome | HTTP | What to do | | - | - | - | - | | This repo isn't connected to Remyx yet | `not_connected` | 404 | Install the App on the repo under Connectors → GitHub. | | The Remyx App is installed, but no account has claimed the installation | `not_connected` | 404 | Connect GitHub from Studio with an account that can reach that GitHub login. | | Validations are turned off for this repo | `disabled` | 409 | Flip the installation's Validations switch under Connectors → GitHub. | | Not started, this installation's free validation has been used, and the balance is empty | `gated` | 402 | Add credits under Account → Billing. Nothing was used. | | Not started, the free run's model allowance is used up | `allowance` | 402 | Add credits or connect a model provider. | | Not started, Remyx's shared allowance for free runs is used up for today | `allowance` | 402 | Mention again tomorrow, add credits, or connect a provider. | | This account has more than one model provider connected and no Validate default | `no_model` | 409 | Pick one under Account → Validate. | | I need a model to design a validation and this account has no provider connected | `no_model` | 409 | Connect one under Connectors → LLM providers. | | Your connected model key is out of credit / was rejected / is rate-limited | `model_key` | 409 | Top up or reconnect the key, or switch Validate to another provider. | | This commit is already being validated | `already_running` | 409 | Wait, or push a new commit. | | Not started, a validation is already running on this repo | `busy` | 409 | Mention again when it finishes. | | There is no proposal of mine awaiting confirmation to revise | `nothing_to_revise` | 409 | Edit the yaml and mention `validate`, or delete it and ask for a fresh design. | | Nothing is running | `nothing_running` | 409 | A `cancel` with nothing in flight. | | This spec logs to W\&B and the account has no W\&B connection | `no_wandb` | 409 | Connect W\&B, or drop the `logging.wandb` block. | | This doesn't look like a PR I can validate | `not_a_pr` | 400 | The PR has no head commit. | | I couldn't synthesize a validation for this change | `error` | 502 | Add a `.remyx/validation.yaml` yourself, or try again. | A refusal at start always says "Nothing was used". It is never charged. ## Next steps [Run a validation tutorial](/tutorials/validate-first-run) · [Read the validation spec](/platform/agents/validate-spec) · [Use the API](/api-reference/validate) # Account Source: https://docs.remyx.ai/platform/configure/account Profile, billing, API keys, Validate's model, workspaces, and notifications: one tab at a time **Nav:** Sidebar > Account (bottom left) | **URL:** [`/account`](https://studio.remyx.ai/account) The Account page is tabbed: **Profile**, **Billing**, **API keys**, **Validate**, **Workspaces**, **Notifications**. The selected tab rides the URL (`?tab=billing`), so links and browser history work. Every form has a visible label and an explicit **Save** that stays disabled until something changes. *** ## Profile `?tab=profile`. Your avatar, name and email come from your sign-in; your **title** is editable. **Delete account** lives here. It deletes everything, immediately: every API and agent key is revoked, so agents stop mid-loop; all connected integrations are disconnected; your interests, projects and history are removed. Type `DELETE` to confirm. It cannot be undone. *** ## Billing `?tab=billing`. One prepaid Remyx balance, spent pay as you go. Validate is the metered agent today; Outrider runs bill your own model provider. | Section | What it shows | | - | - | | **Credit balance** | The balance and **Add credits**: presets of $20, $50 and $100, or any amount from $10 to \$2,000, paid through Stripe Checkout with promotion codes accepted. "Top-ups aren't enabled on this deployment" appears when Stripe is not configured. | | **Usage rates** | The current per-hour compute rates by tier, the model-call multiplier, and the per-validation fee. Budget from this, not from the docs. | | **Recent transactions** | Every credit and debit with the repo and PR it paid for. | Arriving with `?credits=paid` shows that the balance updates once Stripe confirms, usually within a minute; `?buy=credits` (the link in a refusal on a PR) opens the top-up flow directly. An account on a grant reads "Covered by a Remyx grant. Tasks are recorded, not charged." Full detail, including what admits a validation and when it is charged: [Billing](/platform/configure/billing). *** ## API keys `?tab=keys`. Named keys authenticate the CLI, the MCP server, and GitHub Actions, passed as a Bearer token. Keys are shown **once** at creation (Remyx stores only a hash) and can be scoped, revoked, or deleted at any time. Create one key per machine or workload so you can revoke them independently. ### Key types | Prefix | Type | Use it for | | - | - | - | | `rmxu_…` | **Personal** | Your own CLI, MCP server, and ad-hoc REST calls. The only type you can create here. | | `rmxa_…` | **Agent (automation)** | Unattended automation; see [Agent keys](#agent-keys) below. | ### Creating a key 1. Click **+ Create API key** and name it (for example `laptop-cli`, `ci-experiments`). 2. Pick an expiry: **Never expires**, 30 days, 90 days, or 1 year. 3. Choose access: **Full access**, or **Limited** with per-category read/write grants ("write" includes the matching read). One-click presets fill the grid for common workloads. See [API Scopes](/api-reference/scopes) for the catalog, including the new `billing` category. 4. Copy the key from the show-once modal and use it as `Authorization: Bearer `. ### Managing keys Each row shows the key's access summary, created / last-used / expiry dates, and its display id (`rmxu_…last4`). **Edit** changes the name and scopes. **Revoke** disables a key immediately across the REST API, CLI, and MCP server; it stays listed for audit until you **Delete** it. ```bash theme={null} # CLI authentication (personal rmxu_ key) export REMYXAI_API_KEY=rmxu_xxxxxxxxxxxxxxxxxxxxxxxx # REST curl -H "Authorization: Bearer rmxu_xxxxxxxxxxxxxxxxxxxxxxxx" \ https://engine.remyx.ai/api/v1.0/interests ``` ### Agent keys Agent keys (`rmxa_`) are **created and rotated by the agent's Action for each repo** when you provision an agent; you don't mint them by hand. Their scopes are managed by provisioning and can't be edited (rename only). > [!WARNING] > > **Revoking an agent key pauses that agent immediately.** Its next run fails auth. This is the same operation as **Pause** on the [Agents](/platform/agents/overview#run-now--pause) page; resuming requires re-provisioning. *** ## Validate `?tab=validate`. Which model writes and repairs validations when you run `@remyx-ai validate`. | Section | What it does | | - | - | | **Default model** | The account-wide provider and model, chosen from your connected providers. Required before the bot will run on an account with more than one provider connected; it will not pick one for you. | | **Repository overrides** | A provider and model for one repository, or for one App installation (every repo it covers). Only the installation's owner can set an installation-level override. | The full resolution order, from a per-call override down to connected providers, is in [Validate → Choosing the model](/platform/agents/validate-models). *** ## Workspaces `?tab=workspaces`. Projects belong to a workspace; owners and admins manage members and invitations. Expand a workspace to manage member roles (**admin**, **member**, **read-only**), invite by email, and see pending invites. A workspace's only admin must promote someone else before stepping down, and its projects must be deleted or re-assigned before the workspace can be deleted. *** ## Notifications `?tab=notifications`. Your [Inbox](/platform/review/inbox) is the source of truth; email is a mirror of it. | Section | Setting | | - | - | | **Agent activity** | Draft PRs ready, new discussions, run failures. Email for these isn't configurable yet; they appear in Inbox. | | **Digests and updates** | Weekly project summary and the monthly newsletter, each with its own toggle. | *** ## Security best practices * **One narrowly-scoped key per use.** Grant only the scopes a tool needs; don't reuse one broad key everywhere. * **Rotate by replacement**: create a new key, revoke the old. * **Never commit tokens.** Use environment variables, `.env` files with a loader, or CI secret stores. * Keys authenticate as you: any action taken with a key is attributed to your account, gated by that key's scopes. *** ## Related > [card] [Billing](/platform/configure/billing) > > What admits a validation and when it is charged > [card] [API Scopes](/api-reference/scopes) > > Scope catalog, presets, and key prefixes > [card] [Validate](/platform/agents/validate) > > The agent the Validate tab configures > [card] [Connectors](/platform/configure/connectors) > > Model-provider keys live there, not here # Billing Source: https://docs.remyx.ai/platform/configure/billing Free validation eligibility, credits, compute rates, model charges and billing outcomes. Validate uses Remyx credits for hosted validation compute and applicable Remyx-funded model calls. ## What costs money The first completed eligible validation is free per GitHub App installation. Later completed runs use prepaid credits or an applicable grant. A valid inconclusive or no-signal result may still be chargeable. Calls made through your own model-provider key are billed by that provider. A run that fails before a valid measurement result follows the documented non-charge and error rules. ## See the rate and the charge Validate posts an estimate before measurement. After a completed run, the result records the actual cost. Open Studio → Account → Billing to see your current credit balance, usage rates and transactions. [Open Billing in Studio](https://studio.remyx.ai/account?tab=billing) ![Remyx Studio Billing with a grant-covered balance, CPU and GPU hourly rates, and a recent transactions section.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/billing-credits-and-rates.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=4bf69f6b07a8ccb366eafa5e6b1ec90f) Studio Billing shows the available credits, current compute rates, and transaction section. ## Details The tables below define eligibility, rates, usage accounting, grants, failed runs and other charge conditions. | Case | Billing result | | - | - | | Free validation (one completed validation per App installation) | The cost is recorded; nothing is debited; the installation's free run is marked used. | | Completed run on a balance | The actual cost is debited. | | Valid abstention (**No signal**, **Inconclusive**) | Charged once, when the measurement run finalizes, the same as a verdict. | | Smoke run that precedes the measurement | Not charged. | | Error or infrastructure failure | Not charged; does not consume the free validation. | | Cancelled before a result | Not charged. | | Refused at start | Nothing was used. | | Counted grant | One validation is decremented from the grant. | | Unlimited grant | The cost is recorded; nothing is debited. | **Nav:** Account → Billing | **URL:** [`/account?tab=billing`](https://studio.remyx.ai/account?tab=billing) ## What admits a validation When you mention the bot, the account that owns the repo's App installation is checked, in this order. The first that applies admits the run. | Order | Basis | What it is | | - | - | - | | 1 | **Free validation** | One completed validation per App installation. | | 2 | **Balance** | A positive Remyx balance. | | 3 | **Grant** | A grant on the account or on the installation's GitHub login: design partners, sponsored open source, pilots. Counted, unlimited, or expiring. | Nothing else admits. A refusal is one line on the PR that names what fixes it, and it always ends with "Nothing was used". The gate fails closed: a lookup error is a refusal, never a free run. ## When you are charged Once, when the measurement run finalizes with a verdict or a valid abstention (**No signal**, **Inconclusive**). Never: * on the smoke run that precedes the measurement; * on an error or an infrastructure failure, which also does not consume the free validation; * mid-run. Everything that could refuse a run is checked before it starts: entitlement, the funded allowance, a run already in flight on the repo, a dead model key. | Basis | At the verdict | | - | - | | Free validation | The cost is recorded; nothing is debited; the installation's free run is marked used. | | Balance | The actual cost is debited. The balance may go below zero by at most one validation; the next run is refused until you top up. | | Counted grant | One validation is decremented from the grant. | | Unlimited grant | The cost is recorded; nothing is debited. | ## What it costs Check the current rates in Studio before running. CPU, GPU, model calls funded by Remyx, and validation fees may contribute to a charge. Calls made with your own model key are billed by your provider. The example numbers in this guide are not a quote for your run. | Component | How it is priced | | - | - | | Compute | Per second, at the tier's hourly rate (CPU or GPU), with a per-job floor. A first-time environment build is priced on CPU. | | Model calls on Remyx's allowance | Provider list price times the platform multiplier. Calls on your own connected key are never charged. | | Flat fee | A per-validation fee per product. | The current rates are on **Account → Billing → Usage rates** and in the `prices` block of `GET /billing`, which any key with `billing:read` can call. Production sets its own rates (CPU $0.60/h and GPU $4.00/h as of November 3, 2026, multiplier 1.0, fee \$0). The code defaults are lower and are not what you are charged. Do not budget from the defaults or from this page. Read the rates page. The started comment on the PR quotes a ceiling ("up to \~\$0.90 on CPU, includes first-time environment build"): the per-job cost at the run's own deadline, times arms and seeds, plus a build allowance for each arm with no cached image. The verdict footer shows the actual. ## Free first run Each App installation gets one free validation. It is used when a validation completes with a verdict or a valid abstention (**No signal**, **Inconclusive**), so a run that dies on infrastructure leaves it intact. With no model provider connected, the free run's test is designed and repaired on a Remyx-funded model, within caps that are code defaults production may change: $2 per job, $5 per account, \$50 per day across all accounts. When a cap is reached the bot says so and names the two ways on: add credits, or connect your own provider. On a paid run, funded model calls are priced into the debit instead. ## Add credits **1. Open Account → Billing** The **Credit balance** section shows the balance and **Add credits**. On a deployment with no Stripe key it says "Top-ups aren't enabled on this deployment". **2. Pick an amount** Presets of $20, $50 and $100, or any amount from $10 to \$2,000. **3. Pay in Stripe Checkout** Promotion codes are accepted. You return to the account page with `?credits=paid` or `?credits=cancelled`. **4. The balance updates once Stripe confirms** Usually within a minute. The credit is keyed on the Stripe event id, so a redelivered event never credits twice. A refusal on a PR links to `studio.remyx.ai/account?buy=credits`, which opens the same flow signed in. The verdict footer shows **Add credits** when the balance is below the larger of the last debit and \$5. ## Statement **Account → Billing → Recent transactions** lists every credit and debit. The balance is derived from that ledger, not stored, so it cannot drift from the statement. Each debit names the repo and PR it paid for and the cost breakdown (compute, model calls, fee). An account covered by a grant sees "Covered by a Remyx grant. Tasks are recorded, not charged." ## What you see on the PR | Moment | Line | | - | - | | Started, free run | *This is this installation's free validation.* | | Started, balance | *Pay as you go: the actual cost is debited from your Remyx balance (\$18.16 now) when the verdict posts.* | | Started, counted grant | *This uses 1 of the N validations left on your Remyx grant.* | | Verdict, free run | 🎁 **That was this installation's free validation.** What it would have cost, then **Add credits** or a team pilot. | | Verdict, balance | 💳 **$1.84 debited from your Remyx balance, $18.16 left.** | | Verdict, counted grant | *N validations left on your Remyx grant.* | | Refused | ⛔ **Not started**, the reason, the fix, "Nothing was used". | ## Grants and teams Grants are written by the Remyx team, not self-served: design partners, sponsored open-source maintainers, pilots. [Contact us](mailto:hello@remyx.ai) with the repo or GitHub organization. A grant can be counted (N validations), unlimited, or expiring, and it can sit on the account or on the installation's GitHub login so every repo under that organization is covered. A **trusted** grant also unlocks the Docker-in-Docker sidecar for runs that need to build or run containers inside the measurement. Trust is a property of the account, so it applies to that account's free run too. The free validation and the balance are untrusted by default. ## API | Route | Scope | Returns | | - | - | - | | `GET /api/v1.0/billing` | `billing:read` | Whether top-ups are configured, the checkout URL, the presets and limits, and for a user token the balance summary and whether a grant covers the account. A service token gets the price list instead. | | `POST /api/v1.0/billing/checkout` | `billing:write` | `{amount_usd, success_url?, cancel_url?}` → `{url, session_id, amount_usd}`. Personal tokens only (`403` for a service token), `409` when Stripe is not configured, `400` when the amount is outside $10 to $2,000. | | `GET /api/v1.0/validate/entitlement?repo=` | `evals:read` | Whether a validation on that repo would be admitted now, on what basis, how many are left, and the refusal line when not. | The `billing` scope category (read and write) is new; keys created before it need the scope added under Account → API keys. The Stripe webhook and the grants routes are internal. ## Related > [card] [Validate](/platform/agents/validate) > > What a validation is and how to run one > [card] [Account](/platform/configure/account#billing) > > The Billing tab > [card] [API scopes](/api-reference/scopes) > > `billing:read` and `billing:write` > [card] [Model providers](/platform/configure/connectors#choose-a-model-provider) > > Outrider bills your provider, not Remyx # Connectors Source: https://docs.remyx.ai/platform/configure/connectors Connect GitHub and the model providers needed for your Remyx workflow. Connect GitHub to work with repository changes. Add a model provider when your workflow requires one. ## What to connect | Workflow | Start with | Add when needed | | - | - | - | | Evaluate a pull request | The Remyx GitHub App installed for the repository | A model provider, W\&B, or other evaluation integrations | | Find and draft changes | GitHub access and a supported model provider | Additional repository context or integrations | The first eligible Validate run can use a Remyx-funded model allowance when available. Outrider runs in GitHub Actions with its configured provider. ## Connect GitHub For Validate, install the Remyx GitHub App on the repositories you want to evaluate. In Studio, connect or claim the installation and confirm that Validations are enabled. For Outrider, use the Studio Quickstart to authorize GitHub, choose a repository, and configure the workflow. [Open Connectors in Studio](https://studio.remyx.ai/connectors) · [Quickstart](/quickstart) ![Remyx Studio GitHub settings showing a connected account, Change repo access button, and Validate on toggle.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/connect-github-app.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=6a5aed1260bf1ad155bed2021a6ad185) Manage GitHub repository access and turn on Validate for the connected installation. [Connect GitHub guide](/platform/configure/github) ## Choose a model provider A model provider supplies the model used for eligible tasks. The coding-agent backend determines how Outrider prepares code. The available combinations and credential requirements are documented separately. [Model providers](/platform/agents/backends) · [Validate model selection](/platform/agents/validate-models) ## Other integrations Use the catalog below to find available integrations, setup requirements, and any permissions or usage costs. An integration shown as planned or requiring team setup is not part of the standard self-service flow. **Nav:** Manage > Connectors | **URL:** [`/connectors`](https://studio.remyx.ai/connectors) Each connector card shows a status chip — **connected**, **not connected**, **enterprise**, or **coming soon** — and connected cards offer **Test connection**, **Replace credentials** (for key-based connectors), and **Disconnect**. ### Catalog | Category | Connectors | | - | - | | **Code** | GitHub | | **Model providers** | Anthropic (Claude), Z.ai (GLM), Moonshot AI (Kimi), OpenAI, OpenRouter | | **Coding agents** | Claude Code, Codex, Backboard R-CLI | | **Project management** | Linear, Jira | | **Communication** | Slack | | **ML lifecycle** | Weights & Biases | | **Data** | Hugging Face | | **Compute** | Modal | | **Coming soon** | Langfuse, Arize, LaunchDarkly, Statsig | Enterprise connectors (MLflow, Databricks, Snowflake, AWS, GCP, Azure) are private-network setups done with the Remyx team — use the **Contact us** card. ### Linear **What it feeds:** draft-vs-discussion routing — open tickets flag interfaces under active work, which routes risky candidates to discussion instead of drafts. Decisions link back to issues, and `@remyx` mentions in comments get agent replies. **Setup:** click **Connect**, authorize Remyx, then link Linear where you use it — as a project [source](/platform/configure/projects#sources), or as the durable store for two-tier gap analyses via the action's `linear-api-key` input. ### Jira Same routing signal as Linear, for Jira Cloud teams. Sync uses the Jira Cloud v3 webhook API (`offline_access` + `manage:jira-webhook` scopes); inbound deliveries are verified against the Atlassian webhook identifier header. If auth errors appear after an extended period, disconnect and reconnect to refresh. ### Slack Inbox items, daily digests, and weekly portfolio summaries in your channels, plus a managed agent: mention `@Remyx` to get your digest, list agents' output, or ask about a run. Install the app, pick a default channel, and `/invite @Remyx`. ### Weights & Biases **What it feeds:** two things. Gate evidence: eval runs and metrics feed the [decision policy](/platform/review/evaluation) at the gate, and W\&B Launch is a compute backend for experiment validation. And [Validate](/platform/agents/validate-execution#logging-to-wb) runs: when a spec declares `logging.wandb` or `requires: [wandb]`, or the PR template and harness call for W\&B, your key is injected into the measurement pods so the repo's own `wandb.init()` lands in your project, tagged `remyx-validate`. The started comment says where results will land; the verdict links the dashboards. Never a Remyx key. **Setup:** paste a W\&B API key from [wandb.ai/authorize](https://wandb.ai/authorize) (plus an optional entity and self-hosted URL). The key is validated against the W\&B GraphQL `viewer` query before it's saved. A Validate spec that logs to W\&B on an account with no connection, a wrong entity, or an invalid project is refused before anything runs. ### Hugging Face Connects via OAuth so Remyx can read model and dataset metadata. Your own Hugging Face token lives on this card; Validate injects it into measurement pods so a test can pull gated models and datasets on your access, never Remyx's. ### Modal **What it feeds:** experiment-validation compute for the eval-template path: Remyx dispatches scored jobs to Modal Sandboxes (GPU-capable) on your account. Enter your Modal token ID and secret. See [Evaluation](/platform/review/evaluation#experiment-validation-modal--wb-launch). PR [Validate](/platform/agents/validate) runs on Remyx compute and does not use this connector. ## Permissions and credential handling Review the detailed scopes, secret-handling rules, and installation controls before connecting a repository or provider. ### Security * Credentials and OAuth tokens are encrypted at rest; keys are validated against their provider before being stored. * Webhook payloads are verified for authenticity on every request. * Disconnecting is immediate: agents and gates that reference the connection stop working until it's reconnected. A **Fix** card in the [Inbox](/platform/review/inbox) (`model_key_push`) appears when an agent's provider key could not be pushed to its repo and needs a reconnect. ## Model providers and coding agents Outrider runs in your repository's GitHub Actions environment. Its coding agent uses a compatible model provider, with the selected key stored in the repository's Actions secrets. With `--github-secrets-only` or client-sealed credentials, Remyx does not retain a copy of that provider key. Validate uses a connected provider key on the server to design or repair tests. It does not push that key to your repository, and a client-sealed key cannot be used for Validate. Calls made with your own model-provider key are billed by that provider; hosted validation compute is charged under the [Validate billing rules](/platform/configure/billing). Backboard R-CLI is a coding-agent backend with its own routing credential, not a model provider. For supported agent/provider combinations, API families, secret names, default models, timeout settings, and routing behavior, use the [Model backends reference](/platform/agents/backends). The [Validate model-selection reference](/platform/agents/validate-models) explains how it chooses a provider. ### Connect a model provider Select the provider in [Studio → Connectors](https://studio.remyx.ai/connectors). The connection flow validates credentials and stores them encrypted at rest. OpenRouter uses an OAuth flow to mint a revocable provider key. For a local Outrider installation that does not use the Remyx GitHub App, put the key directly into the repository's GitHub Actions secrets. The following commands read a key from a local file; they do not print it to the terminal. ```bash theme={null} remyxai outrider set-provider-secret --repo OWNER/REPO --provider openrouter --key-from ~/openrouter-key gh secret set BACKBOARD_API_KEY --repo OWNER/REPO < ~/backboard-key ``` The action reads the credential required by the chosen backend. Ensure the provider account has sufficient API credit; a paid Claude web subscription, for example, does not include Anthropic API credit. [Set up Outrider from the terminal](/cli/outrider-setup) · [MCP server](/resources/mcp-server) ## Related > [card] [Model backends](/platform/agents/backends) > > How the LLM providers compare per run > [card] [Validate](/platform/agents/validate) > > Set up, the Validations switch, and the model it uses > [card] [Projects](/platform/configure/projects) > > Reference connected sources per project > [card] [Agents](/platform/agents/overview) > > What provisioning pushes to the repo > [card] [MCP Server](/resources/mcp-server) > > Use Remyx from your own MCP client # Connect GitHub Source: https://docs.remyx.ai/platform/configure/github Connect a repository to Remyx and manage its GitHub App installation. Connect the repositories where you want to review changes or run validations. ## For Validate 1. Open [Connectors in Studio](https://studio.remyx.ai/connectors). 2. Install the Remyx GitHub App on the repositories you select. 3. Return to Studio and connect or claim the installation if prompted. 4. Confirm that Validations are enabled for that installation. To start a validation, comment `@remyx-ai validate` on an open pull request in the connected repository. [Read the Validate guide](/platform/agents/validate) ## For Outrider Use [Studio Quickstart](https://studio.remyx.ai/start) to authorize GitHub, choose a repository, and install the Outrider workflow. The Quickstart's GitHub connection and repository setup have distinct permission steps. Follow the prompts in Studio rather than installing a workflow manually. ## Permissions The exact permissions, installation-claim rules, and repository controls are documented below. **What it feeds:** the ranking baseline (every merge and revert re-weights the candidate pool), the write path for everything agents author, and the mention surface for [Validate](/platform/agents/validate). **Setup:** 1. Click **Connect** on the GitHub card. 2. Install the **Remyx GitHub App** on your organization or account. 3. Select the repositories to grant access to. > [!NOTE] > > The connector is a **GitHub App**, not a plain OAuth grant. Remyx authenticates as the App and mints short-lived, repo-scoped tokens on demand; the App's private key never leaves the Remyx engine. The base token covers `contents`, `pull-requests`, `issues`, and `workflow` (needed to install the agent workflow), and permissions like `discussions:write` are layered on only when your installation actually granted them. ### The `remyx-ai[bot]` identity Because the connector is a GitHub App, everything Remyx does on your repos is authored by **remyx-ai\[bot]**: setup PRs, draft PRs, Issues, and Validate's commits to a PR branch. Never as you, so it's clear at a glance which artifacts an agent originated. Commits show as **Verified**. The bot answers `@remyx-ai` mentions in two ways, and they have different reach: * **Research questions** on a PR or Issue an agent opened get a grounded, read-only reply. See [Briefs](/platform/discover/briefs#the-remyx-bot). * **`@remyx-ai validate`** works on **any** PR in a repo the App covers, whoever opened it, and it does write: it commits a proposed test, its environment and the executed notebook to the PR branch. Nothing lands on the default branch. See [Validate](/platform/agents/validate). ### Installations Below the card, one row per App installation Remyx can see: | Column | Meaning | | - | - | | **Account** | The GitHub user or organization the App is installed on, and its repository selection. | | **Owner** | The Remyx account that claimed it. Validate gates, charges and attributes runs to this account. | | **Claim** | Shown when nobody has claimed the installation. The claimer must be able to reach it through their own GitHub authorization. An installation another account owns is not transferred. | | **Validations** | The per-installation switch, on by default. Off, the bot replies that validations are turned off for the repo and does nothing. A legacy Outrider agent's own toggle under Agents → repo → Validations is honoured alongside it. | Connecting GitHub from Studio claims the installations you can reach. The API is `GET /github/app/installations`, `PATCH /github/app/installations/ {validations_enabled}` and `POST /github/app/installations//claim` (`github:write`). > [!NOTE] > > The in-app [Quickstart](/quickstart) uses a repo-scoped OAuth connection (it needs fork rights for the demo repo); agent setup outside the quickstart runs on the App. If both exist, the App is preferred. # Projects Source: https://docs.remyx.ai/platform/configure/projects The namespace each initiative runs in: its team, the agents feeding it, and the sources it references — plus the Pro quality gate **Nav:** Manage > Projects | **URL:** [`/projects`](https://studio.remyx.ai/projects) A **project** is the namespace an initiative runs in: its **team**, the **agents** feeding it, and the **sources** it references. The project switcher at the top of the sidebar scopes the Inbox, feeds, Agents, and Reports to the active project's repos and experiments. The Projects index lists each project with its team, agent count, and source count. **+ New project** creates one inline; open a project to edit it. *** ## Scope Free text, edited any time — e.g. *"Focus on the retrieval and prompting pipeline; ignore CI/infra changes."* Recommendations, agents, and gates all read this: saving it refreshes the project's interests, so it's the cheapest way to steer what the whole loop pays attention to. ## Agents The [agents](/platform/agents/overview) linked to this project, as chips — add from your installed agents, remove with a click. An agent can belong to several projects. ## Sources The connected accounts this project references, grouped by role: | Group | Sources | | - | - | | **Code** | GitHub | | **Project management** | Linear, Jira | | **LLM providers** | Anthropic (Claude), Z.ai (GLM) | | **Signal & data** | Weights & Biases, Hugging Face, Slack | Linking references a connected account for this project; the account itself is connected once under [Connectors](/platform/configure/connectors). ## Experiments The project's experiment history, extracted from its repos' merge logs — each row a change with its status, decision, and date. **Refresh extraction** re-reads the merge log; individual rows can be excluded (and restored) to curate what feeds ranking. Agent drafts file here as proposed experiments; merged work and extracted history join them. This history is what agents rank candidates against — it's the "what your team has actually shipped" signal. *** ## The Pro quality gate Three sections put a quality gate between agent drafts and your main branch. They carry a **Pro** badge — Remyx sets them up with you (self-service is coming; [contact us](mailto:contact@remyx.ai)): **1. Target metrics** What "better" means for this project: each metric with a **baseline** and a **target**. Metrics frame what agents recommend and what validation gates against. **2. Evaluation** Locked [eval templates](/platform/review/evaluation): the scoring harness is committed **before results exist, so nobody moves the bar after seeing them**. Future experiments inherit the most recently locked template. **3. Decision policy** Three cards — **Ship**, **Reject**, **Iterate** — rendering the promote bar at the gate. Ship and Reject are rules over your metrics (≥, improves-by, no-regression, …); Iterate is the automatic fallback when neither matches. Evidence is posted right on the PR. See [Evaluation & decision policy](/platform/review/evaluation) for the full reference. *** ## Danger zone **Delete this project** removes its configuration and unlinks its experiments. It cannot be undone. *** ## Related > [card] [Evaluation & decision policy](/platform/review/evaluation) > > The Pro gate in detail > [card] [Agents](/platform/agents/overview) > > The agents a project links > [card] [Connectors](/platform/configure/connectors) > > Connect the accounts projects reference > [card] [Reports](/platform/review/reports) > > Fleet output, scoped by project # Briefs Source: https://docs.remyx.ai/platform/discover/briefs Typed, cited research briefs that ground recommendation ranking and your agents' drafting context, plus the Remyx bot **Nav:** Research > Briefs | **URL:** [`/research?mode=briefs`](https://studio.remyx.ai/research?mode=briefs) ## Read cited research summaries and candidate changes before deciding what to implement. Research Q\&A may be read-only; Validate commands on an authorized pull request can commit files and run tests. ## Research briefs A **research brief** is a typed, cited summary that Remyx produces and stores **section-grained** for a repo, so each section is addressable on its own. A brief contains: | Section | What it holds | | - | - | | **Domain framing** | What problem space the repo sits in and why it matters | | **Approaches survey** | The methods in use across the field, with tradeoffs | | **Future directions** | Where the work could plausibly go next | | **Candidate changes** | Concrete, implementable changes worth trying | | **Citations** | The sources behind every claim above | Because the brief is typed and cited, ranking and drafting can pull just the section they need, and you can trace any claim back to its source. *** ## How briefs are created and used A brief is created automatically when you create a [Research Interest from a GitHub repo](/platform/discover/feed#research-interests-from-a-repo). Remyx runs the deep-research pass in the background; you don't trigger it separately. Once stored, the brief is used in two places: > [card] [Recommendation ranking](/platform/discover/feed#how-recommendations-are-ranked) > > The brief's **future directions** become a forward-looking ranking axis alongside your shipping history, so the feed can weigh where the work is headed. The **candidate changes** seed recommendations directly. > [card] [Agent drafting](/platform/agents/outrider-action) > > When an [agent](/platform/agents/overview) drafts an implementation against your repo, the stored brief is part of the context it works from, alongside the recommendation and your repo's history. The agent's [Signal panel](/platform/agents/overview#configuration) shows the brief feeding it. *** ## The Remyx bot You can ask about a brief and a run from the pull request or issue itself: any PR or Issue one of your [agents](/platform/agents/overview) opened. This section is about those research questions. `@remyx-ai validate` is a different, write-capable path that works on any PR in a connected repo; see the callout below and [Validate](/platform/agents/validate). **1. Mention the bot** In a comment on such a pull request or issue, mention **`@remyx-ai`** (aliases `@remyxai` and `@remyx` also work) and ask your question. **2. It acknowledges** The bot posts a 👀 reaction so you know the mention was picked up. **3. It replies once** It posts a single grounded reply as `remyx-ai[bot]`, retrieved over the stored research brief plus that run's telemetry: the recommendation runs, their outcomes, and rejected candidates. Because it answers from the brief and the run's own telemetry, you can ask things like: * "Why was this recommendation surfaced over the others?" * "What did the brief say about alternatives to this approach?" * "Which candidate changes were considered and rejected for this run?" > [!WARNING] > > **Research replies are read-only.** On a question, the bot answers; it does not edit code, push commits, or change PR state. A human stays at every decision. > [!NOTE] > > **Validate does write.** `@remyx-ai validate ` on any PR in a repo the Remyx App covers, whoever opened it, commits a proposed test to the PR branch, runs it on confirmation, and commits the executed notebook with the verdict. Nothing lands on the default branch. On a PR that already has a proposal or a run, every mention is read as being about that validation: `status` reports where the run is, and a question is answered from the run's own results. See [Validate](/platform/agents/validate-commands). A research question gets a reply when **all** of these hold: * The comment **mentions the bot** (`@remyx-ai` / `@remyxai` / `@remyx`). * The comment is on a pull request or issue an [agent](/platform/agents/overview) opened. * The **permission check** passes: the author is an owner, member, or collaborator of the repo. If any condition fails, no reply is posted. *** ## Paper enrichment and PDF code-link recovery A recommendation is only actionable if you can find the code behind it. Two background steps make more papers implementable. Both need nothing from you and run at ingest. > [accordion] Paper enrichment > > For a paper with no code link, an enrichment agent traverses the arXiv abstract and HuggingFace papers pages through to project pages, classifies the URLs it finds, resolves licenses, and recovers GitHub, HuggingFace, and SPDX links back into the paper record. Existing fields are left alone; it only fills empties. > [accordion] PDF code-link extraction > > Remyx fetches the paper's PDF and promotes a `Code:`, `Project page:`, or `Repository:` URL directly into the paper record at ingest. Downstream, that named reference repo is lifted into the agent's implementation context as the **source of truth**, so the agent translates from the reference and diffs against it before opening a PR. More papers then arrive with a named reference repo already attached. That is the difference between a recommendation you can act on now and one you'd have to go find the code for first. *** ## Related > [card] [Feed](/platform/discover/feed) > > Repo-backed interests trigger a brief that feeds ranking > [card] [Actions](/platform/agents/outrider-action) > > The agent runtime that drafts PRs from the brief > [card] [Continuous Experimentation](/concepts/continuous-experimentation) > > How discovery, briefs, and implementation form a loop # Feed Source: https://docs.remyx.ai/platform/discover/feed Personalized daily recommendations grouped by your Research Interests **Nav:** Research > Feed | **URL:** [`/research?mode=feed`](https://studio.remyx.ai/research?mode=feed) ## Track research interests and review recommendations associated with your repositories. Each recommendation is a candidate to inspect, not a guarantee that a change should be implemented. ## Research Interests Research Interests are the context that drives your recommendations. Each interest has its own name, context description, and recommendation cadence. ### Creating an Interest Click **+ New Interest** at the top of the Feed view. | Field | Description | Example | | - | - | - | | **Name** | Short label (appears as a pill tab) | "RAG & Retrieval" | | **Context** | Natural language description, URLs to HuggingFace models / GitHub repos / arXiv papers / blog posts, a GitHub repo, or an existing [Project](/platform/configure/projects) | `https://huggingface.co/your-org/your-model` | | **Daily Recommendations** | Resources per day (1-10) | 3 | You can seed an interest from free text, from a GitHub repo (see [below](#research-interests-from-a-repo)), or **from an existing Project**. The Project source draws on that project's experiments, so the ranker scores candidates against the work you've actually run. > [!TIP] > > **Richer context = better recommendations.** Link GitHub repos with detailed READMEs, HuggingFace models with comprehensive cards, or specific arXiv papers that define your area. Resources that cite or benchmark against your work get boosted. ### Interest Pill Tabs Your interests appear as **pill tabs** at the top of the Feed: * **All** — Combined feed from all interests * **\[Interest Name]** — Filtered to that interest only, with badge count showing how many new recommendations are available ### Managing Interests Click the **gear icon** (⚙️) next to an interest name to: * Edit the name, context, or daily count * Toggle active/inactive (inactive interests stop generating recommendations but preserve history) * Delete the interest ### Research Interests from a repo Instead of a free-text context, you can create an interest **from a GitHub repo** (pick one from your connected GitHub, or paste a public URL). Remyx extracts the repo's merge history into a structured shipping record in the background, then ranks recommendations against the work your team has actually shipped. This is the same extracted history that powers a [Project](/platform/configure/projects); a repo-driven interest and a project that links the same repo converge on one history. Profiling reads the whole tree, not a truncated path list: a directory-level inventory of what the repo covers (largest groups first), the high-signal files, and a commit timeline sampled across the repo's history rather than only its head. The report it produces is editable and becomes the interest's context. A repo-backed interest also triggers a typed, cited **[deep-research brief](/platform/discover/briefs)** for the repo. Its "future directions" become a forward-looking ranking axis alongside your shipping history, and its "candidate changes" seed recommendations, so the feed can weigh where the work is headed. When you create an interest from a repo, the dialog also offers an **auto-PR** setting that can provision [automated discovery PRs](/platform/agents/outrider-action) for that repo: | Choice | What happens | | - | - | | **Set it up for me** | Remyx installs the recommendation GitHub Action, opens and merges the setup PR, and fires the first run automatically once recommendations exist. | | **Let me review the setup PR** | Same provisioning, but the setup PR is left open for you to review and merge first. | | **Not now** | No provisioning — turn it on later from the interest's settings. | > [!TIP] > > Editing a repo-sourced interest lets you **curate its extracted history** (include/exclude individual changes), the same way you curate a project's experiment list. Keeping direction-setting work and dropping pure refactors sharpens what gets recommended. *** ## Recommendation Cards Each recommendation shows: | Element | Description | | - | - | | **Title** | Resource title with link to detail viewer | | **Source type** | Badge indicating `arxiv_paper`, `github_repo`, etc. | | **Relevance score** | How well this matches your interest context | | **Reasoning** | One-line explanation of why this was recommended | | **Interest tag** | Which interest surfaced this resource | Cards are sorted by recency by default. Use the **Sort** control to change ordering. *** ## Refreshing Recommendations Each interest has a **Refresh** button in the toolbar. Click it to trigger an immediate re-ranking — don't wait until the next scheduled run. Refresh runs asynchronously. A progress indicator shows the status. Navigate away and come back; the job continues in the background. > [!NOTE] > > Scheduled recommendations refresh daily at **9:30 AM UTC** for all active interests. Use **Refresh** for instant updates, or update your context sources before the daily run. ### How recommendations are ranked For interests backed by a shipping history (any [repo-sourced interest](#research-interests-from-a-repo) or a project), ranking goes beyond matching your context text: * Your team's extracted shipping history is fed into the ranker, so candidates that align with the methodological direction you've actually shipped rank above ones that are merely topically related. * A learned **preference model**, fit over your past experiments, scores candidates and breaks ties behind relevance. * A **Semantic Scholar** candidate source: the Semantic Scholar Recommendations API is seeded from your merge history (shipped work as positive examples, rejected work as negative, plus the brief's candidate changes) and its results are merged with the nearest-neighbour and recency candidates before ranking. Items from this source carry a "Powered by Semantic Scholar" attribution. All three make the feed sharper as your history grows. *** ## Resource Detail & Chat Click **View** on any recommendation to open the detail page with: * **Details tab** — Full abstract, resource links, citations, Docker availability * **Chat tab** — Ask questions about the resource in natural language with streaming responses See [Search > Resource Detail Viewer](/platform/discover/search#resource-detail-viewer) for full details. *** ## From Feed to agent The feed is the same candidate stream your [agents](/platform/agents/overview) draw from: a repo-backed interest is the input to an agent, and the auto-PR setting above is how a feed becomes a running agent. Once installed, the agent ranks this stream against your shipping history, drafts the best fit, and your merges and skips sharpen what the feed surfaces next. *** ## CLI & API You can also access your feed programmatically. By default the digest honors each interest's per-day limit; the API's **unlimited mode** (`limit=0`) returns every alerted recommendation instead, along with a `truncated` flag indicating whether anything was held back. ```bash CLI theme={null} # Today's digest grouped by interest remyxai papers digest # Recommendations for one interest remyxai papers list --interest "RAG & Retrieval" --period today # Trigger a refresh remyxai papers refresh --interest "RAG & Retrieval" --wait ``` ```bash API theme={null} # Get digest curl -H "Authorization: Bearer $REMYXAI_API_KEY" \ "https://engine.remyx.ai/api/v1.0/papers/recommended/digest?period=today" # Unlimited: every alerted recommendation across active interests curl -H "Authorization: Bearer $REMYXAI_API_KEY" \ "https://engine.remyx.ai/api/v1.0/papers/recommended/digest?limit=0" # Refresh an interest curl -X POST -H "Authorization: Bearer $REMYXAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"interest_id": "uuid"}' \ "https://engine.remyx.ai/api/v1.0/papers/recommended/refresh" ``` *** ## Related > [card] [Agents](/platform/agents/overview) > > Turn a repo-backed interest into a running agent > [card] [Search](/platform/discover/search) > > On-demand semantic search across all resources > [card] [CLI: Interests & Papers](/cli#research-interests) > > Manage interests and recommendations from the terminal # Discover changes Source: https://docs.remyx.ai/platform/discover/overview Review candidate changes and research matched to your repository. Review research and proposed changes that may be relevant to your repositories. Use Studio to inspect candidate context before deciding whether a change is worth implementing. Outrider checks repository fit and prepares code when a candidate is suitable. A surfaced paper or method is a starting point, not evidence that the change will improve your system. [How Outrider works](/platform/agents/outrider-action) ## Reference [Search](/platform/discover/search) · [Feed](/platform/discover/feed) · [Briefs](/platform/discover/briefs) # Search Source: https://docs.remyx.ai/platform/discover/search Semantic search across papers, repos, models, and datasets with runnable Docker environments **Nav:** Research > Search | **URL:** [`/research?mode=search`](https://studio.remyx.ai/research?mode=search) > [!TIP] > > Search is also the app's **public surface**: signed out, [studio.remyx.ai](https://studio.remyx.ai) serves real search results (no fixtures) — the rest of the app asks you to sign in. Every paper has a shareable, anonymous-readable URL at `studio.remyx.ai/paper/`, and legacy engine paper links redirect there. Press `/` anywhere in the app to jump to search. ## Find research and methods relevant to your work. Open a result to inspect its source, context and available links before deciding whether it is useful for a repository. ## Search Interface Type a natural-language query into the search bar. Remyx runs hybrid retrieval (BM25 + dense vector + re-ranking) across its resource index and returns ranked results. ``` retrieval augmented generation for customer support ``` > [!NOTE] > > `/search/assets` enforces a server-side deadline. If a query runs long, it returns a structured timeout (with a request id and the elapsed time) instead of hanging. Retry the request, and include the request id if you report a slow query. Results appear in a card grid. Each card shows: | Element | Description | | - | - | | **Title** | Resource title with link to detail viewer | | **Authors** | Author list (for papers) | | **Abstract** | Truncated summary | | **Categories** | arXiv categories or source tags | | **Runnable badge** | Blue "Runnable environment" badge with Docker icon if a pre-built Docker image is available | | **Source indicators** | Icons for GitHub, HuggingFace, Docker availability | > [!TIP] > > Resources with the **Runnable environment** badge have pre-built Docker images with dependencies installed and example scripts ready to run. These are the fastest path from discovery to experiment. *** ## Filtering and Sorting A toolbar appears above results after you search: | Control | What it does | | - | - | | **Tabs** | **All** (default), **Runnable** (Docker-ready only) | | **Sort** | Relevance (default), Most Recent, Most Popular | The **Runnable** tab filters to resources with verified Docker environments — useful when you want to test a technique immediately without environment setup. *** ## Personalized Suggestions ### Authenticated Users When you're logged in and haven't searched yet, the page shows: * **Suggestion cards** built from your experiment tags — topics you've already explored, surfaced as quick-search shortcuts * **"Relevant to your work"** section with resources matched to your team's experiment history ### Anonymous Users When not logged in, the page shows: * **Trending suggestion cards** populated from live search queries * **Recent resources** section with highlights from the index *** ## Resource Detail Viewer Click **View** on any result to open the resource detail page: > [tab] Details > > * Full abstract / README > * Resource links (arXiv, GitHub, HuggingFace) > * Citation count and categories > * Docker image reference (if available) > * Dockerfile contents (if available) > [tab] Chat > > Ask questions about the resource in natural language. Responses stream in real-time and conversation history is preserved. > > Example questions: > > * "What's the main contribution compared to prior work?" > * "How would I adapt this for a 3B parameter model?" > * "What datasets did they use for evaluation?" > [tab] Annotations > > Highlight passages and add notes directly on the resource. Annotations help your team collaborate on ideas that emerge during review and carry context forward into experiments. > > * **Highlights**: Select text to highlight key methods, results, or claims > * **Notes**: Add written observations, questions, or hypotheses to any highlight > * **Personal / Team mode**: Annotations can be private or shared with your team. In team mode, you can filter between your own annotations and those from teammates. > * **Experiment context**: When you create an experiment from an annotated resource, your annotations are carried forward. The hypothesis field is pre-filled from your notes, so the reasoning captured during review becomes part of the experiment record. *** ## Briefs When you create a repo-backed [Research Interest](/platform/discover/feed#research-interests-from-a-repo), Remyx runs a typed, cited deep-research pass and stores a section-grained **research brief** for that repo: domain framing, a survey of approaches, future directions, candidate changes, and citations. The brief feeds recommendation ranking and grounds your [agents'](/platform/agents/overview) drafting context. You can also ask questions about a brief and its runs by mentioning `@remyx-ai` on a pull request or issue an agent opened. > [card] [Briefs](/platform/discover/briefs) > > How research briefs are built and stored, the Remyx bot, and paper enrichment *** ## Related > [card] [Feed](/platform/discover/feed) > > Personalized daily recommendations > [card] [Agents](/platform/agents/overview) > > The agents that draw on this same index # Inbox Source: https://docs.remyx.ai/platform/review/inbox Waiting on you: drafts to review, calls your agents held for you, and anything blocking the loop **Nav:** Inbox | **URL:** [`/inbox`](https://studio.remyx.ai/inbox) The Inbox is titled **"Waiting on you"**, and that's the whole contract: drafts to review, calls your agents held for you, and anything blocking the loop — in your own time. It's a **derived** queue, not a task tracker you maintain: items appear from live agent state, and anything resolved on GitHub clears itself. ![The Inbox: 'Waiting on you' with Review cards for draft PRs across repos, filter chips showing All 48 / Review 21 / Decide 27, a date window set to All time, and per-card Review the draft, Ask, Run log, Details, and Mark done actions](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/inbox-waiting-on-you.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=c4f12f10a65696e22d026ba3a4362a87) *** ## Card types Every card carries one of three tags, filterable with chips at the top: | Tag | What it asks of you | Typical items | | - | - | - | | **Review** | Read a diff and merge or close | A draft PR opened by an agent | | **Decide** | Make a judgment call | A discussion Issue the agent opened instead of a PR; a saturated candidate pool (widen the interest or retire the agent?) | | **Fix** | Unblock the loop | A blocked agent (failing runs), an unmerged setup PR, a missing model key | *** ## Auto-resolution Item state follows GitHub. When a PR or Issue is merged or closed, its card auto-completes into the **Done** pile, Linear-style, labeled **Merged** / **Closed**. You never mark those done — the system does. Cards you clear by hand get **Done by you** and can be restored to the inbox anytime; auto-resolved ones can't (there's nothing left to do). Three status views: **Todo** (open and waiting on you), **Done** (resolved on GitHub — merged or closed), **Archived** (marked done by you). A **Mark all as done** chip batch-archives. > [!NOTE] > > The Inbox's date window defaults to **All time** — old waiting items must not hide. Health cards (setup PR, missing key) are never hidden by the window. *** ## Working a card * **Details** expands a **TLDR digest** — a server-generated summary built from the run's telemetry on first expand — plus deterministic **run evidence**: gate check chips, a diff-risk band, flagged callouts, and scope, reported from the run's own passes rather than a recommendation. * **Ask** opens a run-grounded chat scoped to that agent, seeded with the right question for the card kind (e.g. for a blocked agent: *"Why is this repo blocked — what is failing in the logs, and what should I change?"*). It's the same drawer as [Ask about this agent](/platform/agents/overview#ask-about-this-agent). * **Analyze** (on Fix cards with a run) pulls the failing run's log and returns a root-cause read. * The primary button does the obvious thing — open the PR, open the Issue, open Connectors — and **Run log ↗** / **View in agent** take you to the evidence. ![An expanded Inbox card: the TLDR digest summarizing what the draft proposes and what to verify, then 'From the run' evidence — Tests passed and Diff risk low chips, and an All-run-detail fold with fidelity and self-review statuses, stub density, diff-risk band, checks, and the selection rationale — footed by 'Reported from the run's own passes, not a recommendation'](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/inbox-card-details.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=ff7ad3bc48873c32e32f28959f408e3c) The sidebar badge counts all open items, so it lights up when a run lands. The footer links out to the [fleet report](/platform/review/reports) and [Agents](/platform/agents/overview). When there's nothing left: *"You're all caught up."* *** ## Programmatic access The inbox is also exposed on the [MCP server](/resources/mcp-server#outrider-fleet-tools) (`get_inbox`, `dismiss_inbox_item`, `get_artifact_digest`) and the REST API (`GET /outrider/agents/inbox`, `POST /outrider/agents/inbox/dismiss`), so an agent or script can triage the same queue you see. *** ## Related > [card] [Reports](/platform/review/reports) > > The fleet view behind the queue > [card] [The agent report](/platform/agents/overview#the-agent-report) > > Where evidence links land > [card] [Agents](/platform/agents/overview) > > Fix provisioning states surfaced by Fix cards > [card] [Evaluation](/platform/review/evaluation) > > Pre-committed bars for the Decide calls (Pro) # Reports Source: https://docs.remyx.ai/platform/review/reports Find validation results, Outrider activity, run artifacts, history and cost in Studio. Review the work Remyx ran and the evidence it returned. ## Validation results Reports → Validations shows linked pull requests, evaluation questions, outcomes, artifacts and costs. Validate works independently of Outrider, so a PR validation does not require an Outrider installation. ![Studio Reports → Validations showing the SAELens PR with a pass result, linked spec, run history, and spend.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/studio-validations-report.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=e3a7f439606cf5bbffc3f0f6f23c0038) Studio Reports records PR validation outcomes and associated compute spend. A passing offline result means the required criteria were met in that evaluation. It does not establish production impact. [Results and evidence](/platform/agents/validate-results) ### Validations **URL:** [`/reports/validations`](https://studio.remyx.ai/reports/validations) The Validations view records the repository, pull request, question, result, artifacts, and cost of each run. Use the date window to narrow the results. Production comparisons appear only when the corresponding signals have been recorded. | Panel | What it shows | | - | - | | **KPIs** | Validations run, verdict mix, and spend in the window. | | **Calibration: predicted vs production** | For validated changes that merged, the offline verdict against what production reported: **confirmed**, **no effect**, **regressed**, **mixed**, or **window open** while the readout is pending. Production deltas arrive only through `POST /validate/signals` or a merge webhook; a validation with no linked signal stays out of this panel. | | **Records** | One row per validation: repo, PR, question, verdict, cost, and links to the PR, the executed notebook, and the run. | A repository's **spec card** shows its current `.remyx/validation.yaml`, associated runs, and a linting editor. **Open a PR** proposes spec edits on a new branch as remyx-ai\[bot]; the default branch does not change until that PR merges. Without an entitlement (no free validation left, no balance, no grant) the page shows **Validate is a paid feature** with a link to [Billing](/platform/configure/billing). ## Outrider activity Review selected candidates, published drafts, held changes, and execution outcomes across configured repositories. **Nav:** Reports | **URL:** [`/reports`](https://studio.remyx.ai/reports) Reports is the fleet dashboard: what your agents produced, where pools are depleting, and what's waiting on you. It opens with a written **headline** — a one-line thesis about the window — then a **KPI strip** and **findings**, each finding deep-linking into the agent evidence it came from. ![The Reports dashboard: headline '15 drafts opened and 6 merged and 6 leads routed to discussion', a KPI strip with dispatches, drafts opened, merged, leads, needs-attention, and spend with cost per artifact, findings that deep-link to agent evidence, and the fleet-activity chart stacked by outcome kind per day](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/reports-overview.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=d3b7d804742d918c3df97c4fde0be631) Two controls scope everything and ride the URL (so any view is shareable): * **Agent picker** — all agents or a subset; the scope carries into every chart and drill-down. * **Date window** — the same window control as agent views, from 7 days to All time. A **↻ Refresh** button pulls the latest agent state from GitHub and rebuilds the report; the "Updated N minutes ago" stamp tells you how fresh the data is. ### Fleet activity A daily stacked-bar chart of run volume by outcome kind — Draft PR / Discussion / Branch / Skipped / Failed — with spend overlaid. Legend chips toggle outcome kinds; zoom presets narrow to the last 90/30/14/7 days. ### Deliverable pipeline Everything currently in flight, as cards: drafts and discussions with title, repo, age, and a direct **Open ↗** link. This is the maintainer-facing work the fleet has put in front of you, in one place. ### Per-agent breakout Each agent classified into one of six states, filterable by chips (the summary counts how many **need attention** — blocked plus saturated): | State | Meaning | | - | - | | **blocked** | Runs are failing — something needs fixing | | **saturated** | The candidate pool is exhausted — widen the interest or retire the agent | | **shipped** | Work merged in the window | | **active** | Running normally | | **new** | Recently provisioned | | **retired** | Deliberately wound down | Two views of the same data: * **Outcomes** — proportional stacked bars per agent, scaled either absolute (bar length by busiest agent) or as share (100% rows). * **Timeline** — a **contribution grid**: one cell per day (per week on long windows), colored by the highest-signal outcome that day and shaded by run count. A season of agent work reads at a glance, like a GitHub contribution graph for your fleet. ![The per-agent breakout in Outcomes view, share scale: one 100% stacked bar per agent splitting its runs across draft PRs, discussions, and branches, with state filter chips (All 64 / shipped 12 / active 17 / new 35) and each agent's state labeled at the row's end](https://mintcdn.com/remyxai/2lznviOD5A07VqzC/images/studio/reports-per-agent-breakout.png?fit=max\&auto=format\&n=2lznviOD5A07VqzC\&q=85\&s=a0d2a39203bd31980da9522813933ee7) Sortable by status, name, run count, draft count, draft share, or failure share. A **Chronology** fold lists the discrete events — Enabled, Retired, Merged, Draft, Discussion — each linked to its artifact. ### Insights Below the findings, **Insights** groups the window's work into named idea clusters, each with a verdict, under four headers: | Header | Meaning | | - | - | | **Landing** | The direction is producing merged work | | **Held for a decision** | Drafts or discussions are waiting on a call from you | | **Mixed signal** | Some artifacts landed, others were closed or skipped | | **Not fitting** | The direction keeps getting routed to discussion or closed; consider steering the interest | A **standouts** list calls out the single findings worth reading first. Every cluster links to the artifacts behind it. ## Costs and history Use the available filters and run records to inspect activity and spend for the relevant period. ### Costs The **Spend** KPI reads *median $X per opened PR/issue (N) · $Y overhead*: the median cost of the runs that produced an artifact, and what the runs that produced none added up to. It drills into `/reports/costs` (carrying your window and agent scope): unit economics for the fleet. Runs per day stacked by series, with grain chips for **Agent** (the coding-agent CLI), **Provider** (the model backend), or **Model** (each provider split down to its models), against a cost-per-artifact line and per-series KPIs. Runs carry their coding agent, backend, model, and tier (main / drafter / refiner), so a [two-tier setup](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) shows exactly what exploration costs versus promotion. A run whose [agent reports no dollar figure](/platform/agents/backends#what-each-agent-can-report) is left out of spend rather than counted as \$0, and the number of those runs rides along — so a partial total says so instead of quietly understating. ### Programmatic access The same report is available from the [MCP server](/resources/mcp-server#outrider-fleet-tools) (`get_fleet_report`) and the REST API (`GET /outrider/agents/report`), scoped to your own agents. ## Production outcomes When production signals have been connected and recorded, Reports can compare later outcomes with offline validation results. The presence of an offline result alone does not supply production evidence. ## Related > [card] [Inbox](/platform/review/inbox) > > The action items behind "needs attention" > [card] [Agents](/platform/agents/overview#the-agent-report) > > Drill into one agent's runs, direction, and trace > [card] [Model backends](/platform/agents/backends) > > What drives the cost-per-artifact differences > [card] [Projects](/platform/configure/projects) > > Scope the fleet to an initiative # Quickstart Source: https://docs.remyx.ai/quickstart Connect GitHub, choose a provider and repository, and start an Outrider run from Studio or the terminal. Start an Outrider run from Studio or the terminal. This guide covers setup and dispatching the first run. The implementation may take longer. [Open Studio Quickstart](https://studio.remyx.ai/start) ## See it in Studio The walkthrough follows GitHub connection, provider setup, repository selection, and the first run. [Remyx Studio Quickstart walkthrough](https://www.youtube.com/embed/hPdtk47tUBg) [Watch the Quickstart walkthrough](https://www.youtube.com/watch?v=hPdtk47tUBg) ## Before you start You need a GitHub account with repository access and a supported model provider. For a prepared example, Studio can fork the demonstration repository. For your own code, select a repository you control. ## 1. Connect GitHub Studio: Open Quickstart and finish the GitHub authorization. Terminal: Install the Remyx CLI and authenticate with your Remyx API key. CLI authentication does not authorize GitHub; the repository's GitHub App access is checked during Outrider setup in Step 3. ```bash Install the CLI theme={null} pip install remyxai ``` ```bash Authenticate with your Remyx API key theme={null} export REMYXAI_API_KEY="YOUR_REMYX_API_KEY" ``` > [!WARNING] > > Keep API keys out of committed source, workflow logs, and documentation examples. Use the supported secret store for the chosen installation method. Expected result: GitHub is connected in Studio, or the CLI is authenticated to Remyx. Repository access is verified when you configure Outrider. [Install and authenticate the CLI](/cli/install) ## 2. Connect a model provider Studio: Choose a supported provider and connect it. Terminal: Configure a supported provider through the documented account or repository-owned credential path. Provider setup may still require Studio; do not use an unsupported CLI command. Expected result: The configured Outrider backend has access to a supported model provider. [Model backends](/platform/agents/backends) ## 3. Set up a repository Studio: Choose a prepared example or one of your repositories. Studio configures the Outrider workflow. Terminal: Initialize Outrider. The default `--mode auto` opens and merges the setup PR and dispatches the first run after you confirm the CLI prompt. Use `--mode review` if you want to inspect and merge the setup PR yourself. ```bash Configure Outrider for a repository theme={null} remyxai outrider init --repo OWNER/REPO --auto-interest ``` Expected result: In auto mode, Outrider is installed and the first run is dispatched. In review mode, the setup PR is ready for your review. [Outrider CLI reference](/cli/outrider) · [Configuration](/platform/agents/outrider-configuration) ## 4. Start or dispatch a run Studio: Start the prepared example or run Outrider on your repository. Terminal: If you used `init` with its default `--mode auto`, the first run has already been dispatched. To trigger another run, authenticate GitHub CLI and use `trigger`. ```bash Trigger another run theme={null} gh auth status remyxai outrider trigger --repo OWNER/REPO ``` If `gh auth status` shows that you are not signed in, run `gh auth login` first. Expected result: Each dispatched run appears in GitHub Actions. ## 5. Inspect the outcome Open the GitHub Actions log and any linked draft PR, discussion Issue, or branch. From the terminal, run `gh run list --repo OWNER/REPO` to find recent workflows. Outrider may also decide not to publish a change. Your team controls the merge decision. [Run modes](/platform/agents/outrider-run-modes) Already have a pull request to measure? [Evaluate a pull request](/platform/agents/validate). No Outrider setup is required. # Documentation for coding agents Source: https://docs.remyx.ai/resources/for-agents Find the current Remyx docs, API contracts, and command references as Markdown. The documentation is available as Markdown for coding assistants and other tools that read source text. ## Find a page Use the [documentation index](/llms.txt) to locate the relevant guide or reference. The index groups pages by topic and links to their current locations. ## Read the content Append `.md` to a documentation page URL to retrieve its Markdown form. Use [the complete documentation export](/llms-full.txt) when you need to process the whole collection rather than a single page. ## Check technical contracts Use the current API reference for endpoint schemas, the CLI reference for supported commands, and individual guides for prerequisites, side effects, limits, and example outputs. [API reference](/api-reference/api-reference) · [CLI & tools](/cli) · [Guides](/introduction) # MCP server Source: https://docs.remyx.ai/resources/mcp-server Configure the Remyx MCP server for supported agent and developer clients. Connect a compatible coding agent or developer client to Remyx through the [Model Context Protocol](https://modelcontextprotocol.io). The reference below includes setup instructions for Claude Code, Claude Desktop, and other MCP clients, followed by the server's registered tool names, arguments, responses, and permissions. A Studio feature is available through MCP only when a corresponding tool is registered. The same service supports the managed Slack agent and Studio's [Ask chat](#studio-ask-chat). ## Architecture The MCP server runs as a standalone service at `mcp.remyx.ai`. Connect using **Streamable HTTP** at `https://mcp.remyx.ai/mcp`, or **SSE** at `https://mcp.remyx.ai/sse` for clients that require it. Every tool calls the engine's REST API with your own key, so ownership checks and data match the studio exactly. The same tools are also callable over plain HTTP: `GET https://mcp.remyx.ai/api/tools` lists them, and `POST https://mcp.remyx.ai/api/tools/` invokes one with a JSON body of its parameters, for scripts that are not MCP clients. *** ## Connecting ### Claude Code Add to your Claude Code MCP configuration (`~/.claude/settings.json` or project-level): ```json theme={null} { "mcpServers": { "remyx": { "type": "http", "url": "https://mcp.remyx.ai/mcp", "headers": { "Authorization": "Bearer YOUR_REMYX_API_KEY" } } } } ``` ### Claude Desktop Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS): ```json theme={null} { "mcpServers": { "remyx": { "type": "http", "url": "https://mcp.remyx.ai/mcp", "headers": { "Authorization": "Bearer YOUR_REMYX_API_KEY" } } } } ``` ### Other MCP Clients Any client supporting the MCP Streamable HTTP transport can connect using the URL `https://mcp.remyx.ai/mcp` with a Bearer token in the Authorization header. > [!NOTE] > > Get your API key under **Account > API keys** at [studio.remyx.ai/account?tab=keys](https://studio.remyx.ai/account?tab=keys). > [!NOTE] > > MCP keys are **scoped**: a key only authorizes the tool categories its scopes grant, so a call to a tool outside the key's scopes is rejected. See [API Scopes](/api-reference/scopes) for the available scopes and how to mint a key with the right ones. > [!WARNING] > > **Only scoped API keys (`rmxu_…` / `rmxa_…`) authenticate.** Legacy JWT API tokens are rejected with `403 — "JWT API tokens have been replaced by API keys. Create one under Account → API keys and use it instead."` Mint a key under [Account > API keys](/platform/configure/account), and remember the `Bearer ` prefix; a missing prefix returns a 401 telling you so. *** ## Tools Parameter tables below are taken from the server's tool signatures. Optional parameters are marked; everything else is required. ### Outrider fleet tools The fleet tool plane mirrors what the studio shows in [Agents](/platform/agents/overview), [Reports](/platform/review/reports), and the [Inbox](/platform/review/inbox). | Tool | Parameters | What it does | | - | - | - | | `list_agents` | none | Your installations: repo, status, setup PR link, key state. Start here. | | `get_agent_activity` | `repo`; optional `since`, `until` (YYYY-MM-DD), `section` | One agent's runs, events, directions, and summary for a date window; `section` slims the payload | | `get_fleet_report` | optional `since`, `until`, `repos` (list) | The fleet report: headline, KPIs, per-agent states, findings, idea insights, pipeline, chronology | | `get_inbox` | none | The derived Inbox: drafts to review, held discussions, and health items | | `dismiss_inbox_item` | `item_id`; optional `undo` | Archive an inbox item, or restore it | | `get_artifact_digest` | `href` | The stored TLDR digest for one agent PR or Issue | | `get_artifact_diff` | `href` | Per-file diff of one agent-authored PR: filename, +/- counts, patch text, and a deep link to each file's diff | | `analyze_agent_run` | `repo`, `run_id` | Root-cause a run from its job log: cause, evidence, suggested next step. Cached per run | | `list_agent_runs` | optional `repo`, `limit` (20), `include_selection` | Run telemetry rows; `include_selection` adds the selection audit with per-candidate rejection reasons | | `trigger_agent_run` | `repo`; optional `pin_arxiv`, `lead_content`, `start_from_ref`, `ref` | **Dispatches a live run** on the operator's model budget; clients are told to confirm first | | `pause_agent` | `installation_id` | **Pauses an agent** by revoking its key; resuming requires re-provisioning | | `install_agent` | `interest_id`; optional `repo_url`, `branch` | Provisions an agent on a repo: key, workflow file, setup PR (async) | | `get_install_status` | `interest_id`; optional `task_id` | Poll an `install_agent` task, or read the interest's latest provisioning state | | `fork_repo` | `repo`; optional `branches` (list) | Forks a repo into *your* account (never the bot's), optionally seeding branches | ### Research and account | Tool | Parameters | What it does | | - | - | - | | `search_research` | `query`; optional `max_results` (10), `has_docker`, `categories` | Search the research index (papers and code) by natural-language query | | `get_research_asset` | `arxiv_id` | One asset: title, abstract, code repos, Docker availability | | `get_account_info` | none | Your profile plus credits and subscription state | | `list_connections` | none | Your connected integrations, which is the session's capability list | ### Discovery > [accordion] get\_digest > > Today's recommended resources across your Research Interests. > > **Parameters:** `period` (`today` or `week`, default `today`), `interests` (optional list of interest names or ids), `limit` (per interest, default 5), `include_github` (default `true`). > > **Returns:** Recommended resources grouped by Research Interest, with titles, abstracts, relevance scores, and links. > > **Example prompt:** *"Show me this week's research digest for my retrieval interest"* > [accordion] list\_interests > > List all of your configured Research Interests. > > **Parameters:** None. > > **Returns:** Array of interests with name, context, daily count, and `is_active`. > [accordion] get\_papers > > Resources for one Research Interest. > > **Parameters:** `interest` (name or id), `period` (default `today`), `limit` (default 10). > > **Returns:** Resources matched to the interest with metadata, scores, and Docker availability. > [accordion] refresh\_interests > > Trigger a fresh ranking for one or more interests. > > **Parameters:** `interests` (optional list of names or ids; all active interests if omitted). > > **Returns:** Confirmation that the refresh started. Results arrive via `get_digest` after processing. > [accordion] create\_interest > > Create a Research Interest. > > **Parameters:** `name`; optional `context` (the natural-language description Remyx ranks against), `daily_count` (default 2), `is_active` (default `true`), `source_repo_url`, `project_id` (a project-mode interest ranks against that project's experiments), `auto_update_from_experiments`, `included_experiment_ids`, and the repo-analysis fields `source_repo_metadata`, `generated_report`, `repo_analysis`. > > **Returns:** The created interest with its id and status. > [accordion] update\_interest > > Update an interest. > > **Parameters:** `interest_id`; optional `name`, `context`, `daily_count`, `is_active`, `source_repo_url`, `auto_update_from_experiments`, `included_experiment_ids`, and the repo-analysis fields. > > **Returns:** The updated interest. > > **Example prompt:** *"Pause my Long-context retrieval interest"* (sets `is_active: false`) > [accordion] delete\_interest > > Delete an interest and its recommendations. > > **Parameters:** `interest_id`. > [accordion] list\_project\_experiments\_for\_interest > > For a project-mode interest, the project's experiments with each row marked included or excluded from the curated subset that drives ranking. > > **Parameters:** `interest_id`. ### Repo analysis Generate or refine a Research Interest from a GitHub repository. Remyx profiles the repo and derives the interest context from what the code does. > [accordion] list\_github\_repos\_for\_interest > > Repos reachable through your connected GitHub integration. > > **Parameters:** None. > [accordion] analyze\_repo\_for\_interest > > Start a repo analysis. > > **Parameters:** `repo_url`. > > **Returns:** `task_id`; poll with `get_repo_analysis_status`. > [accordion] get\_repo\_analysis\_status > > Poll an analysis task. > > **Parameters:** `task_id`. > > **Returns:** Status, and the generated report plus structured analysis when complete. > [accordion] regenerate\_interest\_from\_repo > > Re-run analysis for an existing interest. > > **Parameters:** `interest_id`; optional `repo_url` to analyze a different repo than the stored source. > > **Returns:** `task_id`; poll with `get_repo_analysis_status`. ### Project context Set a default project once and omit `project_id` on later calls. Tools that take a `project_id` resolve it from this context when the parameter is omitted. | Tool | Parameters | What it does | | - | - | - | | `set_project_context` | `project_id` | Set the default project for this session | | `get_project_context` | none | The active project, or a note that none is set | | `clear_project_context` | none | Require an explicit `project_id` again | ### Projects and the quality gate | Tool | Parameters | What it does | | - | - | - | | `list_projects` | optional `team_id` | Non-archived projects you can operate on | | `create_project_from_repos` | `name`; optional `repo_urls` (list), `description` | Create a project linked to repos; seeds experiment-history extraction | | `get_project_config` | optional `project_id`, `section` | Project config: eval templates, decision policy, action dispatch | | `get_project_metrics` | `project_id` | Per-metric trends across the project's experiments | | `get_project_experiment_history` | `project_id` | The merged experiment history extracted from linked repos | | `refresh_project_extraction` | `project_id` | Re-run extraction over the linked repos (async) | | `configure_eval_template` | `template_name`, `template` (object); optional `project_id` | Create or update an eval template on the project config | | `set_decision_policy` | `policy` (object); optional `project_id` | Create or replace the project's decision policy (rules over your metrics for ship, reject, iterate) | | `get_decision_policy` | optional `project_id` | The current policy, or null | | `clear_decision_policy` | optional `project_id` | Remove the policy | | `get_insights` | optional `project_id` | Pattern clusters and recommendations across the project's experiments | | `score_candidate_experiments` | `candidates` (list of objects); optional `project_id`, `history_id`, `acquisition` (`mean`), `beta` (0.0), `include_explanation` | Score candidate experiments against the project's preference model | See [Evaluation & decision policy](/platform/review/evaluation) for what a policy contains. ### Experiments > [accordion] create\_experiment > > Create an experiment. > > | Parameter | Required | Description | > | - | - | - | > | `hypothesis` | Yes | One sentence: what you expect to change | > | `project_id` | No | Resolved from `set_project_context` if omitted | > | `name` | No | Defaults from the hypothesis | > | `resource_id` | No | Research asset (arxiv id) the experiment is sourced from | > | `source_type`, `source_ref` | No | `paper`, `hypothesis`, `incident`, or `recommendation`, with details | > | `target_metric` | No | The metric this experiment aims to move | > | `tags` | No | Free-form tags | > | `repo` | No | Target repo, `owner/name` | > | `ticket_project`, `ticket_source` | No | Create a linked ticket in Linear, Jira, or GitHub | > > **Example prompt:** *"Create an experiment in Customer Support AI: hybrid search with re-ranking will lift resolution rate by 2 points, target metric resolution\_rate, tags retrieval and re-ranking"* > [accordion] get\_experiments > > List experiments. > > **Parameters:** optional `status`, `resource_id`, `project_id`, `interest`, `limit` (20). > [accordion] update\_experiment > > Update an experiment's status, links, or results. > > **Parameters:** `experiment_id`; optional `status`, `outcome`, `ticket_url`, `pr_url`, `pr_status`, `external_ticket_id`, `ticket_source`, `statsig_experiment_id`, `notes`, `source_type`. > [accordion] log\_decision > > Record the team's decision on an experiment and the reasoning behind it: the *why* behind a call. > > **Parameters:** `experiment_id`, `decision` (text with reasoning); optional `outcome` (`ship`, `iterate`, `abandon`). > > **Example prompt:** *"Log decision on experiment 12: Ship to 100%. It combined retrieval with few-shot selection and produced our best single-experiment improvement."* > [accordion] Relationships: list\_experiment\_relationships, link\_experiment, update\_experiment\_relationship, unlink\_experiment > > Typed links between experiments. > > * `list_experiment_relationships(experiment_id, direction="both")`: `in`, `out`, or `both`. > * `link_experiment(experiment_id, to_experiment_id, kind, notes?)`: `kind` is `borrows_from`, `variant_of`, `replicates`, or `references`. > * `update_experiment_relationship(experiment_id, relationship_id, kind?, notes?)`. > * `unlink_experiment(experiment_id, relationship_id)`. ### Implementation > [accordion] get\_experiment\_context > > Full implementation context for a research-sourced experiment: resource metadata, Docker environment, repo structure, and implementation plan. > > **Parameters:** `experiment_id`. > [accordion] get\_reference\_implementation > > Reference sources for an experiment: GitHub repos and Docker images. > > **Parameters:** `experiment_id`. > [accordion] list\_active\_experiments > > Experiments ready to implement: they have context and no PR yet. > > **Parameters:** optional `project_id`, `limit` (20). > [accordion] run\_implement\_workflow > > Launch a sandboxed coding-agent container, using your connected provider key, to implement the experiment and open a PR. > > **Parameters:** `experiment_id`. > [accordion] run\_experiment\_implementation > > Trigger the server-side implementation pipeline; the fallback when no coding agent is available. > > **Parameters:** `experiment_id`. > [accordion] get\_implementation\_status > > Poll the pipeline: `pipeline_status`, PR info, error detail, `task_id`. > > **Parameters:** `experiment_id`. > [accordion] report\_implementation\_result > > Link a PR you opened yourself to the experiment. > > **Parameters:** `experiment_id`, `pr_url`; optional `branch_name`, `summary`. ### Experiment validation (Modal / W\&B Launch) These tools run the experiment-level evaluation path on connected compute ([Modal](/platform/configure/connectors#modal) or [Weights & Biases](/platform/configure/connectors#weights--biases) Launch) against a locked eval template. They are unrelated to [Validate](/platform/agents/validate), the PR validation agent, which has no MCP tools and is driven from the PR thread or `POST /validate/runs`. > [accordion] Eval templates: create\_eval\_template\_v2, lock\_eval\_template > > * `create_eval_template_v2(name, category, project_id?, runtime="lighteval", lighteval_task?, dataset_ref?, variants_schema?, seeds=3, metric_keys?, decision_criteria?, compute_hints?)`: a draft template with versioning. > * `lock_eval_template(template_id)`: one-way; a locked template is what runs execute against. > [accordion] Eval runs: launch\_eval\_run, get\_eval\_run, cancel\_eval\_run, get\_eval\_run\_results > > * `launch_eval_run(experiment_id, github_url, variants (list), template_id?, template_config?, seeds=3, pr_number?, pr_url?)`: builds the environment, runs the variants on Modal sandboxes, receives results by webhook, and computes a verdict. > * `get_eval_run(run_id)`: status, results received, verdict so far. > * `cancel_eval_run(run_id)`: terminates sandboxes and marks the run cancelled. > * `get_eval_run_results(run_id)`: final aggregated results with a Pass / Warn / Fail verdict. > [accordion] provision\_validation > > Provision compute and start a validation for an experiment against a locked template. > > **Parameters:** `experiment_id`; optional `template_name`. > > **Returns:** A job handle; poll with `check_validation_status`. > [accordion] check\_validation\_status > > **Parameters:** `experiment_id`. > > **Returns:** Job status and progress. > [accordion] cancel\_validation > > **Parameters:** `experiment_id`. > [accordion] get\_validation\_results > > Normalized baseline-vs-feature results for an experiment. > > **Parameters:** `experiment_id`; optional `include_raw`. > > > [!NOTE] > > > > Pulling live treatment vs. control results from Statsig or LaunchDarkly is still pending; those data arrive via `update_experiment` for now. > [accordion] close\_loop > > Composite: fetch the validation result, update the experiment's status, close its ticket, and notify a channel. > > **Parameters:** `experiment_id`; optional `disposition` (`shipped`, `iterating`, `rejected`, `abandoned`), `notify` (channel). ### HTTP-only tools `create_ticket` (create a Linear, Jira, or GitHub ticket for an experiment) is registered for the HTTP surface only: `POST https://mcp.remyx.ai/api/tools/create_ticket`. Every other tool above is available on both. *** ## Slack Agent The MCP server also powers the managed Slack agent. When you `@mention` the Remyx bot in Slack, it uses the same tools to respond: fetching digests, listing agents' output, logging decisions. See [Connectors > Slack](/platform/configure/connectors#slack) for setup. *** ## Studio Ask chat The **✦ Ask** button in the studio topbar opens a chat on every page; the Inbox's **Ask** and an agent's **Ask about this agent** open the same panel scoped to that agent. The panel docks to the right or the bottom, resizes, and minimizes to a pill. Each answer has **Copy** and, where it proposes work, **Start an agent run from this**, which hands the proposal to a confirm-gated dispatch. Two backends: * **Remyx AI, hosted** (the default). Works with nothing connected and answers from the server-assembled context for the agent, run, or Inbox item you opened it from. * **Bring your own provider.** Pick a connected Anthropic, Z.ai, or Moonshot AI key to upgrade the chat to a live agent session with the Remyx tool plane. The choice is remembered in your browser; unconnected providers stay visible but disabled. A live session can call a **read-only allowlist** of the tools above with your own authority (fleet, research, project, experiment, and eval reads, plus project context), and the CLI's web search and fetch tools for public pages such as papers and upstream repos. When you have GitHub or Hugging Face connected, their MCP servers are attached read-only with your own tokens. There is no shell or filesystem. Mutating tools (`trigger_agent_run`, `pause_agent`, `install_agent`, `fork_repo`) are deliberately absent: the chat proposes those actions in text, and the studio's confirm-gated buttons execute them through the engine. A turn that fails on a transient error is retried, up to three attempts. *** ## Implementation Pipeline For research-sourced experiments with a launch context, a connected coding agent can run the full implementation flow: 1. Fetch the experiment context (resource metadata, Docker environment, repo structure, implementation plan) 2. Read the target repository 3. Generate code changes implementing the technique 4. Push a branch and open a PR 5. Link the PR back to the experiment The PR status syncs via GitHub webhooks. When the PR is merged, the experiment status updates automatically. Scheduled [agent runs](/platform/agents/outrider-action) are the always-on version of this same loop. *** ## Registered tools Every tool the server registers, from the server source at the time of writing. | Tool | Arguments | What it does | | - | - | - | | `analyze_agent_run` | `repo`, `run_id` | Fetch and analyze one GitHub Actions run's job logs. | | `analyze_repo_for_interest` | `repo_url` | Kick off async repo analysis to seed a Research Interest. | | `cancel_eval_run` | `run_id` | Cancel an in-flight eval Run — terminates Modal sandboxes and marks the Run as canceled. | | `cancel_validation` | `experiment_id` | Cancel a running validation job for an experiment. | | `check_validation_status` | `experiment_id` | Poll the current validation job status for an experiment. | | `clear_decision_policy` | `project_id` | Remove the project's decision policy. | | `clear_project_context` | none | Unset the runtime-set project context. | | `close_loop` | `experiment_id`, `disposition`, `notify` | Close the experiment loop: fetch validation, update status, notify team. | | `configure_eval_template` | `template_name`, `template`, `project_id` | Create or update an eval template on a project. | | `create_eval_template_v2` | `name`, `category`, `project_id`, `runtime`, `lighteval_task`, `dataset_ref`, `variants_schema`, `seeds`, `metric_keys`, `decision_criteria`, `compute_hints` | Create a draft EvalTemplate with versioning + lock lifecycle. | | `create_experiment` | `hypothesis`, `project_id`, `name`, `resource_id`, `source_type`, `source_ref`, `target_metric`, `tags`, `repo`, `ticket_project`, `ticket_source`, `notes` | Create an experiment and start tracking it through the lifecycle. | | `create_interest` | `name`, `context`, `daily_count`, `is_active`, `source_repo_url`, `source_repo_metadata`, `generated_report`, `repo_analysis`, `project_id`, `auto_update_from_experiments`, `included_experiment_ids` | Create a Research Interest. | | `create_project_from_repos` | `name`, `repo_urls`, `description` | Create a project, optionally linked to GitHub repos — linking seeds the project's experiment-history extraction. | | `delete_interest` | `interest_id` | Delete a Research Interest and its associated recommendations. | | `dismiss_inbox_item` | `item_id`, `undo` | Mark one inbox item done, or restore it (undo=True). | | `fork_repo` | `repo`, `branches` | Fork a repo into the caller's own GitHub account (their identity, never the bot's). | | `get_account_info` | none | The caller's profile plus credits/subscription state. | | `get_agent_activity` | `repo`, `since`, `until`, `section` | One agent's full activity: runs (with telemetry: outcome, cost, selection shape), events trace, learned direction (what gets promoted or demoted and why), open artifacts, and summary findings. | | `get_artifact_diff` | `href` | Per-file diff for one agent-authored PR (href = the PR URL): filename, +/- counts, patch text, and a deep link to each file's diff on GitHub. | | `get_artifact_digest` | `href` | Stored TLDR + bullets for one agent-produced PR or Issue (by its GitHub URL) — what it proposes, why the agent chose it, what to verify. | | `get_decision_policy` | `project_id` | Read the project's current decision policy. | | `get_digest` | `period`, `interests`, `limit`, `include_github` | Get ranked research recommendations grouped by Research Interest. | | `get_eval_run` | `run_id` | Poll a Run for status, results received count, and verdict. | | `get_eval_run_results` | `run_id` | Get the final aggregated results + verdict for a completed eval Run. | | `get_experiment_context` | `experiment_id` | Get the full experiment context for implementation. | | `get_experiments` | `status`, `resource_id`, `project_id`, `interest`, `limit` | Query the experiment board. | | `get_fleet_report` | `since`, `until`, `repos` | Cross-agent fleet report: headline thesis, KPIs (dispatches, drafts, merged, spend), per-agent terminal states and outcome counts, aggregated findings, the deliverable pipeline, and a chronology. | | `get_implementation_status` | `experiment_id` | Poll the implementation pipeline status for an experiment. | | `get_inbox` | none | The operator inbox — what's genuinely waiting on a human: draft PRs to review, discussions the agent held for a call, and health items (setup PR unmerged, missing model key, blocked runs, saturated pool). | | `get_insights` | `project_id` | Get experiment insights: pattern clusters and recommendations. | | `get_install_status` | `interest_id`, `task_id` | Poll an install\_agent task by task\_id, or omit task\_id for the interest's latest provisioning status. | | `get_papers` | `interest`, `period`, `limit` | Get papers for a single Research Interest. | | `get_project_config` | `project_id`, `section` | Read project config including eval templates, decision policies, and action dispatch settings. | | `get_project_context` | none | Return the currently active project context, or `None` if unset. | | `get_project_experiment_history` | `project_id` | The project's merged experiment history extracted from its linked repos — what was tried before, by humans and agents alike. | | `get_project_metrics` | `project_id` | Per-project metric trends across its experiments. | | `get_reference_implementation` | `experiment_id` | Get reference implementation sources for an experiment. | | `get_repo_analysis_status` | `task_id` | Poll a repo-analysis task started by analyze\_repo\_for\_interest. | | `get_research_asset` | `arxiv_id` | Detail for one research asset by arxiv id: title, abstract, code repos, Docker availability. | | `get_validation_results` | `experiment_id`, `include_raw` | Get normalised validation results for an experiment. | | `install_agent` | `interest_id`, `repo_url`, `branch` | INSTALLS AN AGENT: provisions the Outrider action on a repo for a Research Interest — issues a key, writes the workflow file, and opens a setup PR on the repo. | | `launch_eval_run` | `experiment_id`, `github_url`, `variants`, `template_id`, `template_config`, `seeds`, `pr_number`, `pr_url` | Launch an eval run — the composite entry point for the full pipeline. | | `link_experiment` | `experiment_id`, `to_experiment_id`, `kind`, `notes` | Create a typed link from one experiment to another. | | `list_active_experiments` | `project_id`, `limit` | List experiments that are ready for implementation. | | `list_agent_runs` | `repo`, `limit`, `include_selection` | Recent run telemetry rows (status, outcome, artifacts, cost), newest first. | | `list_agents` | none | List your Outrider agent installations. | | `list_connections` | none | The caller's connected integrations (GitHub, Linear, model providers, ...) — your capability list for this session. | | `list_experiment_relationships` | `experiment_id`, `direction` | List typed cross-experiment relationships for an experiment. | | `list_github_repos_for_interest` | none | List GitHub repos the caller can pick from for repo-sourced interest creation. | | `list_interests` | none | List all Research Interests with their configuration. | | `list_project_experiments_for_interest` | `interest_id` | List experiments on a project-mode interest's linked project. | | `list_projects` | `team_id` | List non-archived projects the caller can operate on. | | `lock_eval_template` | `template_id` | Lock a draft EvalTemplate — one-way transition to the execution boundary. | | `log_decision` | `experiment_id`, `decision`, `outcome` | Record the team's interpretation and next-step decision on an experiment. | | `pause_agent` | `installation_id` | PAUSES AN AGENT: revokes the installation's key so its next scheduled run fails auth immediately. | | `provision_validation` | `experiment_id`, `template_name` | Provision compute resources and start validation for an experiment. | | `refresh_interests` | `interests` | Trigger a fresh re-ranking pass for Research Interests. | | `refresh_project_extraction` | `project_id` | Re-run the experiment-history extraction over a project's linked repos. | | `regenerate_interest_from_repo` | `interest_id`, `repo_url` | Re-run repo analysis for an existing Research Interest. | | `report_implementation_result` | `experiment_id`, `pr_url`, `branch_name`, `summary` | Report the result of an implementation back to Remyx. | | `run_experiment_implementation` | `experiment_id` | Trigger the server-side implementation pipeline. | | `run_implement_workflow` | `experiment_id` | Trigger the Claude Code agent pipeline for an experiment. | | `score_candidate_experiments` | `candidates`, `project_id`, `history_id`, `acquisition`, `beta`, `include_explanation` | Score candidate experiments by alignment with the team's trajectory. | | `search_research` | `query`, `max_results`, `has_docker`, `categories` | Search the research index (papers + code assets) by natural-language query. | | `set_decision_policy` | `policy`, `project_id` | Create or replace the project's decision policy. | | `set_project_context` | `project_id` | Set the active project for subsequent tool calls. | | `trigger_agent_run` | `repo`, `pin_arxiv`, `lead_content`, `start_from_ref`, `ref` | DISPATCHES A LIVE Outrider run on the repo's workflow. | | `unlink_experiment` | `experiment_id`, `relationship_id` | Remove a relationship row. | | `update_experiment` | `experiment_id`, `status`, `outcome`, `ticket_url`, `pr_url`, `pr_status`, `external_ticket_id`, `ticket_source`, `statsig_experiment_id`, `notes`, `source_type`, `source_ref`, `target_metric`, `observed_delta`, `delta_confidence`, `tags`, `ab_test_config`, `ab_test_results` | Update an experiment record. | | `update_experiment_relationship` | `experiment_id`, `relationship_id`, `kind`, `notes` | Update a relationship's kind and/or notes. | | `update_interest` | `interest_id`, `name`, `context`, `daily_count`, `is_active`, `source_repo_url`, `source_repo_metadata`, `generated_report`, `repo_analysis`, `auto_update_from_experiments`, `included_experiment_ids` | Update a Research Interest. | ## Next Steps > [card] [Agents](/platform/agents/overview) > > The fleet the tool plane drives > [card] [Connect Slack](/platform/configure/connectors#slack) > > Set up the managed Slack agent > [card] [Account > API keys](/platform/configure/account) > > Mint a scoped key for your MCP client > [card] [API Reference](/api-reference/api-reference) > > REST API documentation # Roadmap Source: https://docs.remyx.ai/roadmap What's shipped, what's in development, and what's planned for Remyx # Roadmap This page tracks the current state of Remyx's capabilities. We publish it so customers, collaborators, and researchers can see where we're going well before each capability ships. For the architectural through-line, see [Causal Intelligence](/concepts/causal-intelligence). For how these capabilities map to the customer journey, see [Maturity Progression](/concepts/maturity-progression). *** ## Status legend | Indicator | Status | Meaning | | - | - | - | | | **Shipped** | Available in production today | | | **In development** | Actively being built | | | **Planned** | Committed to the roadmap, not yet started | | | **Research** | Exploring feasibility, not yet committed | **Available now** — Pull request validation on Remyx compute, evidence linked to GitHub, prepaid credits, and run history in Studio. **Longer-term direction** — Use validated results and connected production signals across projects to improve recommendations and guide engineering priorities. *** ## Agents & ExperimentOps These capabilities are live in production today. They form the Stage 2 foundations: the agent fleet ([Agents](/platform/agents/overview), [Inbox](/platform/review/inbox), [Reports](/platform/review/reports)) plus the [ExperimentOps](/concepts/experimentops) machinery underneath it. | Status | Capability | Description | | - | - | - | | Shipped | Experiment capture | Full lifecycle from origin through decision, with target metric, hypothesis, and decision rationale captured directly on the experiment record | | Shipped | Cross-experiment patterns | Tag-based clustering identifies which directions consistently produce results | | Shipped | Resource discovery | Semantic search across papers, repos, models, and datasets, matched to your team's experiment history | | Shipped | History-aware recommendation ranking | The structured shipping history extracted from your repo feeds the recommendation ranker, so candidates that align with your team's actual trajectory rank higher | | Shipped | Preference-model ranking | A per-team preference model fit over past experiments scores candidate work and breaks ties behind relevance | | Shipped | Outrider agents | Outrider installed per repo: a scheduled GitHub Action that opens a reviewable draft PR — or a discussion Issue — for the next paper worth implementing; a person reviews and merges | | Shipped | Validate | `@remyx-ai validate` on a pull request proposes a test, runs it on Remyx compute after review, and returns the metrics, executed notebook and cost to the PR; one free completed validation per GitHub App installation, then prepaid credits | | Shipped | Multi-backend agents | Agent runs route at Anthropic (Claude), Z.ai (GLM), or Moonshot (Kimi), including the two-tier drafter/refiner fleet layout | | Shipped | Inbox | A derived triage queue of Review / Decide / Fix items that auto-resolves when GitHub does | | Shipped | Fleet reports | The fleet in one view: KPIs, deliverable pipeline, contribution grid, per-agent states, and cost per code artifact | | Shipped | Agent direction learning | Every merge, skip, and discussion re-weights what each agent proposes next — surfaced as evidence-backed Promoted / Demoted / Held directions | | Shipped | Connector framework | GitHub (GitHub App), model providers (Anthropic / Z.ai / Moonshot), Linear, Jira, Slack, W\&B, Hugging Face, Modal | | Shipped | MCP server | Programmatic access to the fleet and platform from Claude Code and other MCP clients, including the Outrider tool plane | | In development | Standalone Stage 1 product surface | Milestone-driven recommendations for early-dev teams without production traffic | *** ## Causal intelligence, evidence layer Stage 2. The evidence layer feeding the causal model. | Status | Capability | Description | | - | - | - | | Planned | Observational log ingestion | Pluggable adapters for Datadog, Honeycomb, structured JSON, OpenTelemetry. Customers connect existing telemetry, and Remyx populates the evidence layer without instrumentation changes | | Planned | Commit-correlated regime boundary detection | Extension of the existing repo integration to identify regime changes in the data-generating process from commit history | | Planned | Quasi-experiment identification | Combine observational logs with regime boundaries to produce identified causal effects via difference-in-differences, interrupted time series, or regression discontinuity | | Planned | Causal discovery | Bootstrap a partial causal graph from observational data, supplemented by regime-boundary structure. Discovered structure is human-validated before being treated as the working model | | Planned | Causal graph engine | Versioned causal graph as a top-level object, supporting interventional, counterfactual, and mediation-aware graphical models. Semi-Markovian formulation to handle latent confounders | | Planned | Causal data fusion | Combine evidence from multiple sources into one coherent posterior. Conflict resolution, graph refinement proposals, continuous incremental updates | *** ## Causal intelligence, query and interaction | Status | Capability | Description | | - | - | - | | Planned | Identification dispatcher | Given a question, classify it by required identification layer, route to evidence sources with appropriate identification, and dispatch to estimation logic | | Planned | Natural language query layer | Customer-facing interface that takes natural language questions, parses into formal estimands, and returns natural language answers with identification status and recommendations | *** ## Causal intelligence, Stage 3 | Status | Capability | Description | | - | - | - | | Planned | A/B test integration framework | Connectors for Statsig, Eppo, LaunchDarkly. A/B test results become an evidence source feeding the causal model | *** ## Causal intelligence, Stage 4 | Status | Capability | Description | | - | - | - | | Planned | Shadow-decision SDK (log-only mode) | Python SDK that wraps decision points in your AI system. The first version captures natural policy output without applying overrides | | Planned | Shadow-mode audit infrastructure | Dedicated product surface for the shadow-mode adoption phase. Audit trail viewer, override proposal review, and compliance reporting | | Planned | CTF-RAND override policies | Extension of the SDK with counterfactual randomization. Trajectory-consistent semantics by default. Per-decision-point semantics opt-in for mediation analysis | | Planned | ETT and NDE estimation | Counterfactual estimation procedures for effect-of-treatment-on-the-treated and natural direct effect | *** ## Hypothesis triage The triage layer matures with the customer — and it is what the agent fleet's ranking grows into. Stage 2 customers see quasi-experimental recommendations. Stage 3 customers see A/B test recommendations. Stage 4 customers see CTF-RAND recommendations. | Status | Capability | Description | | - | - | - | | Planned | Hypothesis ranking and evidence path recommendation | Rank hypotheses by expected information gain. Identify the cheapest evidence path to an answer for each | | Planned | Orchestration scheduler | Coordinate active interventions (CTF-RAND randomizations, A/B tests, quasi-experiment analyses) for maximum concurrency without compromising estimate validity | | Planned | Identification-enabling intervention proposals | Proactively propose CTF-RAND or A/B interventions that would make currently-unidentifiable hypotheses estimable | *** ## How we sequence Foundational evidence-layer capabilities ship first. The causal model and query layer come next. A/B integration follows. Counterfactual perturbations ship last. The dependency hot path through the architecture follows this order. 1. Evidence schema. 2. Observational log ingestion. 3. Quasi-experiment identification. 4. Causal graph engine and data fusion. 5. Identification dispatcher and natural language query. 6. A/B integration. 7. Shadow-decision SDK. 8. CTF-RAND override policies. 9. ETT and NDE estimation. This roadmap is shared publicly. We update it as priorities shift. # Explore changes for your repository Source: https://docs.remyx.ai/tutorials/discover-changes Review research and candidate changes connected to your codebase. Review candidate changes in Studio and decide which ones merit implementation. ## 1. Open your repository Select a connected repository in Studio. ## 2. Review candidates Inspect the research or method, why it may apply, and how it relates to the repository's code and contribution standards. ## 3. Request or review an implementation Use the supported Outrider flow to prepare a change. The available checks determine whether a draft is suitable for maintainer review. ## Next steps [Find and draft changes](/platform/agents/outrider-action) · [Run Outrider on a repository](/tutorials/outrider-repository) # Interpret evaluation results Source: https://docs.remyx.ai/tutorials/interpret-results Understand what a validation can and cannot establish. Read a validation result by connecting the question, measured criteria, and evidence. ## 1. Start with the question Identify what the test was designed to measure. Check whether it compares branches, tests a capability, or uses a reference threshold. ## 2. Read the measured values Compare each metric with its stated criterion. A diagnostic metric provides context but does not determine the verdict unless required by the spec. | Measure | Measured value | Required criterion | Scope of evidence | | - | - | - | - | | Explained variance | TopK 0.9666, AbsTopK 0.9677 | AbsTopK no more than 0.02 worse | Single-seed CPU run | | Negative selected latents | TopK 0%, AbsTopK 49.2% | AbsTopK at least 1% | Single-seed CPU run | [Read the PR result](https://github.com/salma-remyx/SAELens/pull/5) ## 3. Check the evidence Open the executed notebook and inspect the data, seed, compute tier, and any limitations recorded by the run. [Open the executed notebook](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/.remyx/validations/abstopk-vs-topk-tinystories.ipynb) ## 4. Decide what remains unknown A passing offline evaluation does not establish downstream production impact. An inconclusive or no-signal result means the chosen test did not establish a reliable answer. ## Next steps [Results and evidence](/platform/agents/validate-results) · [Validation spec](/platform/agents/validate-spec) · [Compute and limits](/platform/agents/validate-limits) # Run Outrider on a repository Source: https://docs.remyx.ai/tutorials/outrider-repository Set up Outrider, start a run, and review its GitHub output. Set up the GitHub Action and start a run against a repository. [Watch the Quickstart walkthrough](https://www.youtube.com/watch?v=hPdtk47tUBg) ## Requirements You need GitHub repository access and a supported model provider. The action uses your repository's GitHub Actions environment and configured credentials. ## 1. Connect Open Studio Quickstart to connect GitHub and your model provider. [Open Studio Quickstart](https://studio.remyx.ai/start) ## 2. Set up a repository Choose a prepared example or a repository you control. Studio installs the workflow. From the terminal, `init` defaults to `--mode auto`: after confirmation, it merges the setup PR and starts the first run. Choose `--mode review` to review the setup PR before merging. ```bash Configure Outrider for a repository theme={null} remyxai outrider init --repo OWNER/REPO --auto-interest ``` [CLI setup reference](/cli/outrider-setup) ## 3. Start or dispatch a run Use Studio's run control when the run has not started. A prepared example can start from an existing draft; your own repository runs against its available context. The default CLI setup already dispatches the first run. To start another run, sign in with GitHub CLI and trigger it manually. ```bash Trigger another run theme={null} gh auth status remyxai outrider trigger --repo OWNER/REPO ``` If GitHub CLI is not authenticated, run `gh auth login` before triggering. ## 4. Review the outcome Open GitHub Actions to inspect execution and checks. A run may leave a draft PR, Issue, branch, or skip publication. Follow the specific artifact to review the implementation. ## Next steps [Run modes](/platform/agents/outrider-run-modes) · [Configuration](/platform/agents/outrider-configuration) · [Model backends](/platform/agents/backends) # Tutorials Source: https://docs.remyx.ai/tutorials/overview Complete examples for starting runs, reviewing evidence, and configuring Remyx. Follow complete examples from setup to a result you can inspect. ## Choose a workflow Start with a pull request validation, an Outrider repository run, or an explanation of measured results. [Run your first validation](/tutorials/validate-first-run) · [Run Outrider on a repository](/tutorials/outrider-repository) · [Interpret evaluation results](/tutorials/interpret-results) [Explore changes for your repository](/tutorials/discover-changes) # Run your first validation Source: https://docs.remyx.ai/tutorials/validate-first-run Evaluate a PR with Validate and read the resulting evidence. Use Validate to run an evaluation on an open pull request. ## Requirements You need an open PR in a repository connected through the Remyx GitHub App, with Validations enabled. You need permission to comment as an owner, member, or collaborator. Confirm free-run eligibility or available credits in Studio. ## 1. Request a test Comment on the pull request: ```text theme={null} @remyx-ai validate ``` When the PR has no validation spec, Validate proposes a test and commits the proposal to the PR branch. ![GitHub PR showing a detailed @remyx-ai validate request and the bot acknowledging that it will prepare a proposed test.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/validate-pr-request.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=6143a1ef7ae742276e8143a3dd8ed3d2) A PR comment starts the test proposal. ## 2. Review the proposal Inspect the test and `.remyx/validation.yaml`. Check the comparison, metrics, and required criteria before running the evaluation. A committed spec can start immediately. Requests that explicitly ask for a revision and a run can also bypass a separate confirmation. ## 3. Run If Validate is waiting for confirmation, comment again: ```text theme={null} @remyx-ai validate ``` Follow the Remyx Validation GitHub Check as the run executes. ## 4. Read the evidence Review the metrics, specified criteria, executed notebook, run status, and cost on the PR. A passing result means the test met its criteria under that setup. It does not prove production impact. ![GitHub validation result for SAELens with a passing result and a table of metrics against the specified criteria.](https://mintcdn.com/remyxai/tTKg7dmpknqQ3oqG/images/docs/r4/validate-sae-result.png?fit=max\&auto=format\&n=tTKg7dmpknqQ3oqG\&q=85\&s=4d1db7eb14a421f1a38a4f786f47a203) The completed test reports criteria, measurements, and its verdict. [Read the PR result](https://github.com/salma-remyx/SAELens/pull/5) · [Open the executed notebook](https://github.com/salma-remyx/SAELens/blob/6989234239f49130fd1e77412f04a0513268fc87/.remyx/validations/abstopk-vs-topk-tinystories.ipynb) ## Next steps [Write a validation spec](/platform/agents/validate-spec) · [Understand results](/platform/agents/validate-results) · [Execution and safeguards](/platform/agents/validate-execution)