# API Reference
Source: https://docs.remyx.ai/api-reference/api-reference
REST API for the Remyx ExperimentOps platform
# API Reference
The Remyx REST API is available at `https://engine.remyx.ai/api/v1.0`. All endpoints require a Bearer token in the `Authorization` header.
```bash theme={null}
curl -H "Authorization: Bearer rmxu_xxxxxxxxxxxxxxxxxxxxxxxx" \
https://engine.remyx.ai/api/v1.0/interests
```
Create a named API key at **Account > API Access** in the Remyx app, or at [studio.remyx.ai/account](https://studio.remyx.ai/account). Each route is gated by [scopes](/api-reference/scopes); see [Authentication](#authentication) below.
***
## Experiments
Manage experiments, track outcomes, and log decisions.
### List Experiments
```
GET /api/v1.0/experiments
```
| Parameter | Type | Description |
| -------------------- | ------- | ------------------------------------------------------ |
| `project_id` | string | Filter by project UUID |
| `status` | string | Filter by status |
| `ticket_source` | string | Filter by ticket source (e.g., `linear`, `jira`) |
| `external_ticket_id` | string | Filter by external ticket ID |
| `created_via` | string | Filter by creation method (e.g., `mcp`, `web`, `seed`) |
| `limit` | integer | Max results to return |
**Response:** Array of experiment objects with all ExperimentOps fields.
### Get Experiment
```
GET /api/v1.0/experiments/
```
**Response:** Single experiment object with full detail including `source_type`, `source_ref`, `hypothesis`, `target_metric`, `observed_delta`, `delta_confidence`, `decision`, `tags`, `project_id`, `project_name`, `launch_context`, `ab_test_config`, `ab_test_results`, and linked resources.
### Update Experiment
```
PUT /api/v1.0/experiments/
```
| Field | Type | Description |
| -------------------- | ------ | ------------------------------------------------------------------------------------------------------------- |
| `name` | string | Experiment name |
| `status` | string | Lifecycle state: `draft`, `running`, `complete`, `analyzing`, `shipped`, `iterating`, `rejected`, `abandoned` |
| `hypothesis` | string | Hypothesis text |
| `outcome` | string | `ship`, `iterate`, `abandon` |
| `decision` | string | Decision text with reasoning |
| `observed_delta` | float | Measured metric change |
| `delta_confidence` | string | `significant`, `inconclusive`, `not_tested` |
| `target_metric` | string | Target business metric |
| `tags` | array | Free-form tags for grouping |
| `pr_url` | string | Link to implementation PR |
| `pr_status` | string | PR status |
| `external_ticket_id` | string | Linked external ticket ID |
| `ticket_source` | string | Ticket provider |
| `ticket_url` | string | Ticket URL |
***
## Implementation
Manage the experiment implementation pipeline — context assembly, implementation runs, and status tracking.
### Get Experiment Context
```
GET /api/v1.0/implement/experiments//context
```
Returns the full experiment context formatted for Claude Code: resource metadata, Docker environment, task description, and implementation plan.
### Get Reference Implementation
```
GET /api/v1.0/implement/experiments//reference
```
Returns reference implementation sources: GitHub repo, Docker image, key methods, and implementation plan extracted from the launch context.
### List Actionable Experiments
```
GET /api/v1.0/implement/experiments/actionable
```
| Parameter | Type | Description |
| ------------ | ------- | ---------------------- |
| `project_id` | string | Filter by project UUID |
| `limit` | integer | Max results |
Returns experiments that have a `launch_context` but no PR yet — ready for implementation.
### Run Implementation
```
POST /api/v1.0/implement/experiments//run
```
Triggers the server-side implementation pipeline. Returns `202 Accepted` with a task reference.
### Get Implementation Status
```
GET /api/v1.0/implement/experiments//status
```
Returns the current pipeline status: `pipeline_status`, `pr_url`, `pr_status`, and timing information.
### Report Implementation Result
```
POST /api/v1.0/implement/experiments//report
```
| Field | Type | Description |
| ------------- | ------ | ------------------------------ |
| `pr_url` | string | URL of the created PR |
| `branch_name` | string | Git branch name |
| `summary` | string | Implementation summary |
| `status` | string | Result status |
| `artifacts` | object | Additional artifact references |
***
## Research Interests
Manage your Research Interests — the context that drives personalized resource recommendations.
Two interests surfaces exist. The token/programmatic API uses **`/interests`** (documented here). The session/web app uses `/research-interests`. When calling with an API key, always use `/interests`.
### List Interests
```
GET /api/v1.0/interests
```
**Response:** Array of Research Interest objects with `name`, `context`, `daily_count`, `is_active`.
### Create Interest
```
POST /api/v1.0/interests
```
| Field | Type | Required | Description |
| ------------- | ------- | -------- | ------------------------------------------------------- |
| `name` | string | Yes | Interest label (e.g., "Retrieval-Augmented Generation") |
| `context` | string | Yes | URLs and descriptions defining the interest |
| `daily_count` | integer | No | Recommendations per day (default: 3) |
| `is_active` | boolean | No | Whether the interest is active (default: true) |
### Get Interest
```
GET /api/v1.0/interests/
```
**Response:** The full interest object. Includes an additive `experiment_history` field, a bullet list summarizing the experiments linked to this interest.
### Update Interest
```
PUT /api/v1.0/interests/
```
Accepts same fields as Create. Updating `context` invalidates the recommendation pool and triggers re-ranking.
### Delete Interest
```
DELETE /api/v1.0/interests/
```
Deletes the interest and cascading recommendations.
### Toggle Interest
```
POST /api/v1.0/interests//toggle
```
Toggles the `is_active` status without deleting the interest or its history.
### Provision an Interest
```
POST /api/v1.0/interests//provision
```
Kicks off provisioning of [automated discovery PRs](/platform/agents/outrider-action) (Outrider) for the interest: sets up the GitHub repo, injects an automation (`rmxa_`) key into repo secrets, and prepares the continuous-experimentation loop. Optional `agent` in the body (`claude` — the default — / `codex` / `backboard`) picks the [coding agent](/platform/agents/backends#the-agent-input) the rendered workflow runs; it's validated against the interest's model provider up front, so an impossible pair is a `400` rather than a `200` that renders the wrong agent. Requires the `provisioning:write` scope. Returns `202 Accepted` with a `task_id`.
```
GET /api/v1.0/interests//provision/
```
Polls a specific provisioning task started by the call above.
```
GET /api/v1.0/interests//provision-status
```
Returns the current provisioning state for the interest (independent of any single task).
```
POST /api/v1.0/interests//provision-cli-token
```
Mints a scoped CLI token for the provisioned environment. Requires the `provisioning:write` scope.
### Score Candidates
```
POST /api/v1.0/experiment-histories//score-candidates
```
Scores candidate experiments against the interest's preference model and returns ranked results.
### Refit Preference Model
```
POST /api/v1.0/experiment-histories//refit-preference-model
```
Re-fits the preference model for the experiment history from accumulated feedback. Returns `202 Accepted` with a task reference.
***
## Recommendations
Get personalized resource recommendations based on your Research Interests.
### List Recommendations
```
GET /api/v1.0/papers/recommended
```
| Parameter | Type | Description |
| ------------- | ------- | ----------------------- |
| `interest_id` | string | Filter by interest UUID |
| `limit` | integer | Max results |
| `period` | string | `today`, `week` |
**Response:** Array of recommended resources with titles, abstracts, relevance scores, and links.
### Get Digest
```
GET /api/v1.0/papers/recommended/digest
```
| Parameter | Type | Description |
| --------- | ------- | ------------------------ |
| `limit` | integer | Max results per interest |
| `period` | string | `today`, `week` |
**Response:** Structured digest grouped by Research Interest.
### Refresh Recommendations
```
POST /api/v1.0/papers/recommended/refresh
```
| Field | Type | Description |
| ------------- | ------- | ----------------------------------------------------------------- |
| `interest_id` | string | Refresh a specific interest (optional — refreshes all if omitted) |
| `num_results` | integer | Number of results to generate |
Returns `202 Accepted` with a task reference for polling.
### Poll Refresh Status
```
GET /api/v1.0/papers/recommended/refresh/
```
Returns the status of an async recommendation refresh task.
***
## Search
Search across research assets with hybrid retrieval.
### Search Assets
```
POST /api/v1.0/search/assets
```
| Field | Type | Description |
| ------------- | ------- | ------------------------------------------------------ |
| `query` | string | Natural language search query |
| `max_results` | integer | Max results to return |
| `has_docker` | boolean | Filter for resources with runnable Docker environments |
| `categories` | array | Filter by categories |
| `use_llm` | boolean | Enable LLM-enhanced search |
**Response:** Array of matching assets with titles, abstracts, categories, Docker availability, and relevance scores.
### Get Asset Details
```
GET /api/v1.0/search/assets/
```
Returns detailed information for a specific asset: full abstract, Docker status, Dockerfile, reference summary.
### List Recent Assets
```
GET /api/v1.0/search/assets/list
```
| Parameter | Type | Description |
| ------------ | ------- | ------------------------------ |
| `limit` | integer | Max results (default: 20) |
| `offset` | integer | Pagination offset |
| `category` | string | Filter by category |
| `has_docker` | boolean | Filter for Docker availability |
### Get Asset Statistics
```
GET /api/v1.0/search/stats
```
Returns aggregate statistics: total assets, counts by Docker availability, by category, and citation classification coverage.
***
## Integrations
Manage OAuth connections to external tools (GitHub, Linear, Jira).
### List Providers
```
GET /api/v1.0/integrations/providers
```
Returns available integration providers with configuration status.
### Get OAuth URL
```
GET /api/v1.0/integrations//auth-url
```
Returns the OAuth authorization URL to initiate the connection flow.
### Exchange OAuth Code
```
POST /api/v1.0/integrations//callback
```
| Field | Type | Description |
| -------------- | ------ | ---------------------------------- |
| `code` | string | OAuth authorization code |
| `redirect_uri` | string | Callback URI used in the auth flow |
### List Connections
```
GET /api/v1.0/integrations/connections
```
Returns all active integration connections for the current user.
### Check Connection Status
```
GET /api/v1.0/integrations//status
```
### Disconnect Provider
```
DELETE /api/v1.0/integrations/
```
Deactivates the integration connection for the specified provider.
### Get OAuth Token
```
GET /api/v1.0/integrations//token
```
Returns a valid OAuth token for the specified provider. Auto-refreshes expired tokens before returning. Accepts optional `user_id` query param for service tokens.
### Identity Resolution
```
GET /api/v1.0/integrations/identity/resolve
```
| Parameter | Type | Description |
| ---------------- | ------ | ------------------------------------------------- |
| `source` | string | External platform (e.g., `github`, `linear`) |
| `external_id` | string | External user ID |
| `external_email` | string | External user email (alternative to external\_id) |
```
POST /api/v1.0/integrations/identity/link
```
Creates an identity link between a Remyx user and an external platform identity.
```
GET /api/v1.0/integrations/identity/links
```
Lists all identity links for the current user.
***
## Outrider agents
The [agent](/platform/agents/overview) plane: run telemetry, the fleet report, the inbox, and dispatch. Base path `/api/v1.0/outrider`. These endpoints back the studio's Agents, Reports, and Inbox views and the MCP fleet tools, so what you read here matches the app exactly. Visibility is scoped to your own agents.
### Runs & telemetry
```
POST /api/v1.0/outrider/runs
GET /api/v1.0/outrider/runs
```
`POST /runs` is called by the [Outrider action](/platform/agents/outrider-action) itself (authenticated with the repo's provisioned agent key) to report per-run telemetry: cost identity (backend, model, tokens, cost basis), selection audit (pool sizes, per-candidate rejections, reasoning excerpt), routing and gate outcomes, chain phases, and file-touch lists. Idempotent on `run_id`.
`GET /runs` lists your runs; params `repo`, `limit` (default 20, max 100), and `include=selection` to add the selection audit trail.
```
POST /api/v1.0/outrider/agents/runs/analyze
```
Root-cause analysis of one run from its job log: `{repo, run_id}` → cause, evidence bullets, suggested next step. Cached per run.
### Fleet report, activity, and inbox
```
GET /api/v1.0/outrider/agents/report
GET /api/v1.0/outrider/agents/activity?repo=owner/name
GET /api/v1.0/outrider/agents/inbox
POST /api/v1.0/outrider/agents/inbox/dismiss
POST /api/v1.0/outrider/agents/inbox/digest
POST /api/v1.0/outrider/agents/artifact/evidence
POST /api/v1.0/outrider/agents/refresh
POST /api/v1.0/outrider/agents/chat
```
* `/agents/report` — the [fleet report](/platform/review/reports): headline, KPIs, per-agent timelines and terminal states, findings, deliverable pipeline, chronology. `since`/`until` (YYYY-MM-DD) and `repos` (intersected with your own footprint — it can only narrow) scope it.
* `/agents/activity` — one agent's runs, events, directions, and summary; backs [Agent detail](/platform/agents/overview#the-agent-report).
* `/agents/inbox` — the derived [Inbox](/platform/review/inbox); `inbox/dismiss` archives `{id}` or `{ids: […]}` (or `undo: true`), `inbox/digest` generates-once-and-persists the TLDR digest for a PR/Issue `{href}`, and `artifact/evidence` returns the deterministic gate/fidelity facts for one artifact.
* `/agents/refresh` — force a re-fetch of agent state from GitHub (`{full: false}` for a delta).
* `/agents/chat` — one stateless turn of run-grounded chat: `{repo, messages[], focus_run_id?}`; context is assembled server-side, and the transcript carries no privileged access.
### Installations & dispatch
```
GET /api/v1.0/outrider/installations
POST /api/v1.0/outrider/installations//revoke
POST /api/v1.0/outrider/trigger
GET /api/v1.0/outrider/run-status?repo=owner/name
```
* `/installations` — one row per provisioned agent, including paused (revoked) ones.
* `/installations//revoke` — **pause an agent**: marks the installation revoked and revokes its agent key, so the next run fails auth immediately.
* `/trigger` — the API twin of `remyxai outrider trigger`: `{repo, pin_arxiv?, interest_id?, ref?, agent?, provider?, model?, start_from_ref?, lead_content?, test_integration_policy?, fidelity_policy?}`. `agent` (`claude` / `codex` / `backboard`) is validated against `provider` before dispatch — an impossible pair is a `400`, and the credential the run is armed with follows the pair rather than whatever key the account has connected. Dispatches via the remyx\[bot] App token; returns `409 {pool_building: true}` while a fresh interest's candidate pool is still building.
* `/run-status` — the latest run for a repo, live from GitHub (no webhook lag), enriched with its outcome kind and artifact URL.
Outcome tracking (merge/close of the artifacts a run opened) is **webhook-driven** — it flows back through the GitHub webhooks rather than being polled here.
***
## GitHub App & Provisioning
Endpoints for installing the Remyx GitHub App and provisioning repos for [automated discovery PRs](/platform/agents/outrider-action) (Outrider). The per-interest provisioning endpoints live under [Research Interests](#research-interests).
### Get Install URL
```
GET /api/v1.0/github/app/install-url
```
Returns the URL to install the Remyx GitHub App on a user's account or org.
### Get Installation
```
GET /api/v1.0/github/app/installation?repo=owner/name
```
| Parameter | Type | Description |
| --------- | ------ | ------------------------------- |
| `repo` | string | Repository in `owner/name` form |
Returns the GitHub App installation that covers the given repo (if any).
### Get Installation Token
```
POST /api/v1.0/github/installation-token
```
Mints a short-lived GitHub App installation token for the target repo. Scope-gated; requires `github:write`.
| Field | Type | Description |
| ------ | ------ | ------------------------------- |
| `repo` | string | Repository in `owner/name` form |
### Fork a Repo
```
POST /api/v1.0/github/fork
```
Forks a repo into **your own account** (your identity, never the bot's), optionally seeding `branches` from the source network — this is what the Quickstart's set-example track uses to warm-start curated drafts. Idempotent.
| Field | Type | Description |
| ---------- | ------ | --------------------------------------------- |
| `repo` | string | Repository in `owner/name` form |
| `branches` | array | Branch names to seed into the fork (optional) |
***
## Eval Environments
Build and run containerized evaluation environments. Base path `/api/v1.0/eval-env`.
### Builds
```
POST /api/v1.0/eval-env/builds
GET /api/v1.0/eval-env/builds/
GET /api/v1.0/eval-env/builds//context
POST /api/v1.0/eval-env/builds//report
```
`POST /builds` starts a new environment build. `GET /builds/` returns build status; `/context` returns the assembled build context; `POST /builds//report` reports the build result back from a worker.
### Runs
```
GET /api/v1.0/eval-env/runs
POST /api/v1.0/eval-env/runs
GET /api/v1.0/eval-env/runs/
POST /api/v1.0/eval-env/runs//cancel
```
List or create eval runs, fetch a single run, or cancel an in-flight run.
### Templates
```
POST /api/v1.0/eval-env/templates
GET /api/v1.0/eval-env/templates
GET /api/v1.0/eval-env/templates/
PUT /api/v1.0/eval-env/templates/
POST /api/v1.0/eval-env/templates//lock
```
CRUD for reusable eval-environment templates. `POST /templates//lock` freezes a template so it can't be edited once it's in use.
### Result Webhooks
```
POST /api/v1.0/eval-env/webhooks/results
POST /api/v1.0/eval-env/webhooks/modal/results
```
Inbound webhooks that deliver eval results back to Remyx: `webhooks/results` for the generic path and `webhooks/modal/results` for Modal-hosted runs.
***
## Authentication
All API requests require an opaque API key passed as a Bearer token:
```bash theme={null}
curl -H "Authorization: Bearer rmxu_xxxxxxxxxxxxxxxxxxxxxxxx" \
https://engine.remyx.ai/api/v1.0/experiments
```
Keys come in two flavors, distinguished by prefix:
| Prefix | Type | Typical use |
| -------- | ---------- | ----------------------------------------------------- |
| `rmxu_…` | Personal | CLI, MCP, ad-hoc REST calls (no expiry) |
| `rmxa_…` | Automation | Outrider provisioning, CI, repo secrets (365-day TTL) |
Each key carries a set of **scopes** that gate which routes it can call (a route returning `403` means the key lacks the required scope). See [API Scopes](/api-reference/scopes) for the full catalog, presets, and which surface needs what.
Create and manage named keys at **Account > API keys** in the Remyx app, or programmatically:
```
GET /api/v1.0/account/api-keys # list keys (+ scope catalog & presets)
POST /api/v1.0/account/api-keys # create (plaintext returned once)
PATCH /api/v1.0/account/api-keys/ # rename / edit scopes (agent keys: rename only)
POST /api/v1.0/account/api-keys//revoke
DELETE /api/v1.0/account/api-keys/
POST /api/v1.0/account/delete # delete the account: revokes every key, disconnects integrations
```
Revoking an agent (`rmxa_`) key **pauses the agent that uses it**. See [Account & keys](/platform/configure/account).
The legacy single-token endpoints under `/account/tokens/*` are removed and now return `410 Gone`. Migrate to named keys via `/account/api-keys`.
### Response Codes
| Code | Meaning |
| ----- | -------------------------------------------------------------------- |
| `200` | Success |
| `201` | Created |
| `202` | Accepted (async task started) |
| `400` | Bad request (invalid parameters) |
| `401` | Unauthorized (missing or invalid token) |
| `403` | Forbidden (key lacks the required scope) |
| `404` | Not found |
| `409` | Conflict (e.g., duplicate resource) |
| `410` | Gone (legacy `/account/tokens/*` endpoints; use `/account/api-keys`) |
| `500` | Server error |
# CLI Reference
Source: https://docs.remyx.ai/api-reference/cli-reference
Complete command reference for the Remyx CLI
# CLI Reference
Quick reference for all `remyxai` commands. For usage examples and guides, see [Using the CLI](/cli).
***
## `remyxai interests`
Manage Research Interests — the context that drives personalized recommendations.
| Command | Description |
| ------------------------------------- | ------------------------------------------------- |
| `interests list` | List all Research Interests |
| `interests get -i ` | Get details for one interest |
| `interests create` | Create a new interest (interactive or with flags) |
| `interests from-repo ` | Create an interest from a GitHub repo profile |
| `interests from-project ` | Create an interest from a project's experiments |
| `interests update -i ` | Update an interest's fields |
| `interests toggle -i ` | Toggle active/inactive status |
| `interests delete -i ` | Delete an interest |
### `interests create`
```
remyxai interests create [OPTIONS]
```
| Flag | Type | Default | Description |
| ---------------------------- | ------ | ------------ | ----------------------------------------------------------- |
| `--name, -n` | string | *(prompted)* | Short label |
| `--context, -c` | string | *(prompted)* | Description or URLs (HuggingFace, GitHub, arXiv, free text) |
| `--daily-count, -d` | int | 2 | Recommendations per day (1-10) |
| `--inactive` | flag | — | Create as inactive |
| `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating |
| `--wait, -w` | flag | — | Block until the refresh completes |
| `--format, -f` | choice | `text` | `text` or `json` |
### `interests from-repo`
```
remyxai interests from-repo [OPTIONS]
```
| Argument | Type | Description |
| ---------- | ------ | ----------------------------- |
| `repo_url` | string | **Required.** GitHub repo URL |
| Flag | Type | Default | Description |
| ---------------------------- | ------ | ------------- | ------------------------------------------------------------------------------------------------ |
| `--name, -n` | string | *(from repo)* | Short label |
| `--daily-count, -d` | int | 2 | Recommendations per day (1-10) |
| `--inactive` | flag | — | Create as inactive |
| `--automate` | choice | `none` | `none`, `review`, or `auto`. `review`/`auto` provision Outrider on the repo like `outrider init` |
| `--timeout` | int | 300 | Seconds to wait on Outrider provisioning when automating |
| `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating |
| `--wait, -w` | flag | — | Block until the refresh completes |
| `--format, -f` | choice | `text` | `text` or `json` |
### `interests from-project`
```
remyxai interests from-project [OPTIONS]
```
| Argument | Type | Description |
| --------- | ------ | ---------------------------------- |
| `project` | string | **Required.** Project name or UUID |
| Flag | Type | Default | Description |
| ---------------------------- | ------ | ---------------- | --------------------------------------------------- |
| `--name, -n` | string | *(from project)* | Short label |
| `--daily-count, -d` | int | 2 | Recommendations per day (1-10) |
| `--inactive` | flag | — | Create as inactive |
| `--include-experiment, -e` | string | *(all)* | Limit context to a specific experiment (repeatable) |
| `--no-auto-update` | flag | — | Snapshot the project once instead of tracking it |
| `--refresh` / `--no-refresh` | flag | `--refresh` | Kick a recommendations refresh after creating |
| `--wait, -w` | flag | — | Block until the refresh completes |
| `--format, -f` | choice | `text` | `text` or `json` |
### `interests update`
```
remyxai interests update [OPTIONS]
```
| Flag | Type | Description |
| ------------------- | ------ | --------------------------------------------- |
| `--interest, -i` | string | **Required.** Name or UUID |
| `--name, -n` | string | New name |
| `--context, -c` | string | New context (invalidates recommendation pool) |
| `--daily-count, -d` | int | New daily count |
| `--activate` | flag | Set active |
| `--deactivate` | flag | Set inactive |
| `--format, -f` | choice | `text` or `json` |
### `interests delete`
```
remyxai interests delete [OPTIONS]
```
| Flag | Type | Description |
| ---------------- | ------ | -------------------------- |
| `--interest, -i` | string | **Required.** Name or UUID |
| `--yes, -y` | flag | Skip confirmation prompt |
| `--format, -f` | choice | `text` or `json` |
***
## `remyxai papers`
Get personalized recommendations from the Remyx resource index.
| Command | Description |
| --------------------------------- | ----------------------------------------------- |
| `papers digest` | Today's top recommendations grouped by interest |
| `papers list` | Flat list of recommendations with filtering |
| `papers refresh` | Trigger async recommendation refresh |
| `papers refresh-status ` | Poll a refresh task's status |
### `papers digest`
```
remyxai papers digest [OPTIONS]
```
| Flag | Type | Default | Description |
| -------------- | ------ | ------- | -------------------------------------- |
| `--limit, -n` | int | 5 | Max items per interest |
| `--period, -p` | choice | `today` | `today`, `week`, or `all` |
| `--format, -f` | choice | `text` | `text` or `json` |
| `--full` | flag | — | Show full reasoning without truncation |
### `papers list`
```
remyxai papers list [OPTIONS]
```
| Flag | Type | Default | Description |
| ------------------- | ------ | ------- | ---------------------------------------------------- |
| `--interest, -i` | string | *(all)* | Filter by interest name or UUID |
| `--limit, -n` | int | 20 | Max results (1-50) |
| `--period, -p` | choice | `all` | `today`, `week`, or `all` |
| `--source-type, -s` | string | *(all)* | Filter by source: `arxiv_paper`, `github_repo`, etc. |
| `--format, -f` | choice | `text` | `text` or `json` |
| `--full` | flag | — | Show full reasoning without truncation |
### `papers refresh`
```
remyxai papers refresh [OPTIONS]
```
| Flag | Type | Default | Description |
| ------------------- | ------ | ---------------- | ------------------------------------ |
| `--interest, -i` | string | *(all active)* | Refresh one interest by name or UUID |
| `--num-results, -n` | int | *(daily\_count)* | Items per interest |
| `--wait, -w` | flag | — | Block until all tasks complete |
| `--format, -f` | choice | `text` | `text` or `json` |
### `papers refresh-status`
```
remyxai papers refresh-status [OPTIONS]
```
| Argument | Type | Description |
| --------- | ------ | ----------------------------------------------- |
| `task_id` | string | **Required.** UUID from `papers refresh` output |
| Flag | Type | Default | Description |
| -------------- | ------ | ------- | ---------------- |
| `--format, -f` | choice | `text` | `text` or `json` |
***
## `remyxai search`
Search across the Remyx resource index.
| Command | Description |
| ------------------------ | -------------------------------------------- |
| `search query ` | Semantic search across papers, repos, models |
| `search info ` | Detailed info for a specific resource |
| `search list` | Browse recently added resources |
| `search stats` | Catalog statistics |
### `search query`
```
remyxai search query [OPTIONS]
```
| Argument | Type | Description |
| ------------ | ------ | ------------------------------------------- |
| `query_text` | string | **Required.** Natural language search query |
| Flag | Type | Default | Description |
| ------------------- | ------ | ------- | --------------------------------------- |
| `--max-results, -n` | int | 10 | Maximum results |
| `--category, -c` | string | *(all)* | arXiv category filter (repeatable) |
| `--docker` | flag | — | Only resources with Docker environments |
| `--no-docker` | flag | — | Only resources without Docker |
### `search info`
```
remyxai search info [OPTIONS]
```
| Argument | Type | Description |
| ---------- | ------ | ------------------------------------------- |
| `arxiv_id` | string | **Required.** arXiv ID (e.g., `2308.12345`) |
| Flag | Type | Default | Description |
| -------------- | ------ | ------- | ---------------- |
| `--format, -f` | choice | `text` | `text` or `json` |
### `search list`
```
remyxai search list [OPTIONS]
```
| Flag | Type | Default | Description |
| ---------------- | ------ | ------- | ---------------------------- |
| `--limit, -n` | int | 20 | Number of results |
| `--offset, -o` | int | 0 | Pagination offset |
| `--category, -c` | string | *(all)* | Category filter (repeatable) |
| `--docker` | flag | — | Only Docker-ready resources |
| `--no-docker` | flag | — | Only non-Docker resources |
### `search stats`
```
remyxai search stats
```
No options. Prints total assets, Docker availability counts, top categories, and citation coverage.
***
## `remyxai outrider`
Install [automated discovery PRs](/platform/agents/outrider-action) (the Outrider GitHub Action) on a repo.
| Command | Description |
| ------------------------------ | ---------------------------------------------------------------- |
| `outrider init` | Provision via the Remyx GitHub App (`remyx-ai[bot]`) |
| `outrider setup-local` | Provision via your own `gh` CLI, without the GitHub App |
| `outrider trigger` | Fire a one-shot `workflow_dispatch` run of an installed workflow |
| `outrider set-provider-secret` | Set a provider's API-key secret on a repo, safely |
### `outrider init`
```
remyxai outrider init [OPTIONS]
```
| Flag | Type | Default | Description |
| ----------------- | ------ | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL |
| `--interest, -i` | string | — | Research Interest UUID to wire in |
| `--auto-interest` | flag | — | Auto-create the interest from the repo (exclusive with `--interest`) |
| `--bulk-repos` | string | — | Path to a TSV of `owner/nameinterest_uuid` rows; loops with per-row error capture. Exclusive with `--repo` / `--interest` / `--auto-interest` |
| `--pace` | int | 3 | Seconds between repos in a `--bulk-repos` run |
| `--mode` | choice | `auto` | `auto`, `review`, or `off` |
| `--agent` | choice | `claude` | Coding-agent CLI to provision: `claude`, `codex`, `backboard`. Confirmed against the provisioned workflow |
| `--anthropic-key` | string | `$ANTHROPIC_API_KEY` | Model-provider key (used only if none is connected) |
| `--no-wait` | flag | — | Don't block on App install / provisioning |
| `--dry-run` | flag | — | Print the plan and exit |
| `--yes, -y` | flag | — | Skip the confirmation prompt |
### `outrider setup-local`
```
remyxai outrider setup-local [OPTIONS]
```
| Flag | Type | Default | Description |
| ----------------- | ------ | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL |
| `--interest, -i` | string | — | Research Interest UUID to wire in |
| `--auto-interest` | flag | — | Auto-create the interest from the repo |
| `--bulk-repos` | string | — | Path to a TSV of `owner/nameinterest_uuid` rows; loops with per-row error capture. Exclusive with `--repo` / `--interest` / `--auto-interest` |
| `--pace` | int | 3 | Seconds between repos in a `--bulk-repos` run |
| `--mode` | choice | `auto` | `auto` or `review` |
| `--no-cron` | flag | — | Render the workflow with the cron schedule commented out, keeping only `workflow_dispatch` |
| `--agent` | choice | `claude` | Coding-agent CLI the generated workflow runs by default: `claude`, `codex`, `backboard` |
| `--backend` | choice | *(the agent's own vendor)* | Default model backend: `anthropic`, `openai`, `zai`, `moonshot`, `openrouter`. Scoped to the single-file setup |
| `--model` | string | *(the agent's own default)* | Default model id for the install. Required with `--agent backboard` |
| `--anthropic-key` | string | `$ANTHROPIC_API_KEY` | Key set as the `ANTHROPIC_API_KEY` repo secret |
| `--dry-run` | flag | — | Print the plan + rendered workflow and exit |
| `--yes, -y` | flag | — | Skip the confirmation prompt |
Requires an authenticated `gh` CLI and admin on the target repo.
### `outrider trigger`
```
remyxai outrider trigger [OPTIONS]
```
| Flag | Type | Default | Description |
| ----------------- | ------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL |
| `--interest, -i` | string | *(configured interest)* | Research Interest to run against |
| `--pin-method` | string | — | A method query, or a literal arXiv ID, to pin for this run; bypasses the selection pass. Exclusive with `--pin-arxiv` |
| `--pin-arxiv` | string | — | An arXiv ID already in the candidate pool to pin. Exclusive with `--pin-method` |
| `--ref` | string | *(repo default branch)* | Git ref to dispatch the workflow on |
| `--agent` | string | *(workflow default)* | Coding-agent CLI for this dispatch (`claude`, `codex`, `backboard`); an invalid agent/provider pair is rejected locally. The workflow must declare an `agent` dispatch input |
| `--agent-timeout` | int | *(workflow default)* | Wall-clock seconds for each agent phase on this dispatch. `--claude-timeout` is the old name and still works |
| `--provider` | string | *(workflow default)* | Model backend for this dispatch (`anthropic`, `openai`, `zai`, `moonshot`, `openrouter`); the workflow must declare a `provider` dispatch input |
| `--model` | string | *(provider default)* | Model to request from the provider (e.g. `glm-5.3`, `gpt-5.4-mini`), forwarded as the selected agent's model variable |
Refuses if no Outrider workflow is registered on the repo. Requires an authenticated `gh` CLI.
### `outrider set-provider-secret`
```
remyxai outrider set-provider-secret [OPTIONS]
```
| Flag | Type | Default | Description |
| ------------ | ------ | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--repo` | string | *(git remote)* | Target repo `owner/name` or GitHub URL |
| `--provider` | string | **required** | Provider the key is for; selects the secret name (`anthropic` → `ANTHROPIC_API_KEY`, `zai` → `ZAI_API_KEY`, and the same convention for `openai`, `moonshot`, `openrouter`) |
| `--key-from` | path | **required** | File containing only the API key. Read from file to avoid the `gh secret set --body -` truncation trap |
***
## Global Options
All commands support `--help` for detailed usage:
```bash theme={null}
remyxai --help
remyxai interests --help
remyxai papers digest --help
```
# API Scopes
Source: https://docs.remyx.ai/api-reference/scopes
Key prefixes, the scope catalog, presets, and which surface needs what
Remyx API keys are opaque, named tokens that carry an explicit set of **scopes**. Each scope gates which routes a key can call: a request to a route the key isn't scoped for returns `403 Forbidden`.
Create and manage keys at **Account > API Access**. See [Account & API Keys](/platform/configure/account) for that, and the [API Reference](/api-reference/api-reference#authentication) for how scopes appear in REST calls.
***
## Key prefixes
Every key's prefix tells you what it's for and how long it lives:
| Prefix | Type | Use it for | Expiry |
| -------- | -------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------- |
| `rmxu_…` | **Personal** | Your CLI, MCP server, ad-hoc REST calls | Never expires |
| `rmxa_…` | **Automation** | Provisioning [automated discovery PRs](/platform/agents/outrider-action) (minted into repo secrets), CI, unattended scripts | 365-day TTL |
Both are opaque (prefix + base62 entropy + CRC32 checksum). Remyx stores only `sha256(key)`. The plaintext is shown **once** at creation, then rendered as `rmxu_…last4` for identification.
***
## Scope catalog
Scopes are `category:access`. Access is either `read` or `write`, and **`write` implies `read`** for the same category.
| Category | `read` grants | `write` grants (includes read) |
| ----------------- | ------------------------------------------ | ------------------------------------------------- |
| `account` | View your profile, teams, and key metadata | Update profile and team settings |
| `projects` | List and read projects | Create and modify projects |
| `interests` | List and read Research Interests | Create, update, toggle, and delete interests |
| `recommendations` | Read recommendations and digests | Refresh recommendation pools |
| `papers` | Read papers and asset search | *(read-only; no write scope)* |
| `experiments` | List and read experiments | Create, update, and decide experiments |
| `evals` | Read eval builds, runs, and templates | Create/run/cancel evals; manage templates |
| `workflows` | Read workflow definitions and state | Create and modify workflows |
| `integrations` | Read OAuth connection status | Connect/disconnect providers, manage tokens |
| `github` | Read GitHub App installation info | Mint installation tokens, push branches, open PRs |
| `provisioning` | Read provisioning status | Provision repos and mint CLI/automation tokens |
`papers` is read-only; there is no `papers:write`. The deprecated `models`/`datasets` surface is **not** in the scope catalog; those routes are reachable only with a legacy full-access key.
### Resource-qualified grants
A scope can be narrowed to a specific resource by appending its UUID:
```
interests:read:3f2a9c1e-… # read only this one interest
```
Use resource-qualified grants when a key should touch a single resource rather than the whole category. A digest bot scoped to one interest is a common case.
***
## Presets
When creating a key you can apply a one-click preset instead of picking scopes by hand:
| Preset | Scopes granted | Recommended key type |
| ------------------ | ----------------------------------------------------------------- | -------------------- |
| **Read-only** | `read` across the catalog | `rmxu_` |
| **Outrider setup** | `provisioning:write`, `interests:write`, `github:write`, and more | `rmxa_` |
| **Experiment CI** | `experiments:write`, `evals:write`, `projects:read` | `rmxa_` |
| **Digest bot** | `recommendations:read`, `papers:read`, `interests:read` | `rmxa_` or `rmxu_` |
You can always edit the scope set after applying a preset, before creating the key.
***
## Which surface needs what
| Surface | Key type | Recommended scopes |
| --------------------------------------------- | -------- | --------------------------------------------------------------------- |
| **CLI / MCP, personal use** | `rmxu_` | A broad personal key (or **Read-only** if you only consume data) |
| **Automated discovery PRs** (repo automation) | `rmxa_` | **Outrider setup** preset |
| **CI pipelines** | `rmxa_` | **Experiment CI** preset |
| **Digest / notification bot** | `rmxa_` | **Digest bot** preset (optionally resource-qualified to one interest) |
Prefer a narrowly-scoped key per use over one broad key everywhere. To rotate, create a new key and revoke the old one; keys are immutable once created.
***
## Related
Create, scope, and revoke named keys
REST endpoints and authentication
OAuth connections (GitHub, HuggingFace, and more)
Authenticate the CLI with a personal key
# Using the CLI
Source: https://docs.remyx.ai/cli
Install the Remyx CLI, authenticate, and manage Research Interests, recommendations, and search from your terminal.
The **Remyx CLI** is the command-line companion to Remyx Studio. Manage Research Interests, get personalized recommendations, search across papers, repos, and models, and install [Outrider agents](/platform/agents/overview) on your repos — all from your terminal. Source and latest command definitions live in the [official repo](https://github.com/remyxai/remyxai-cli).
## Requirements
* **Python 3** and `pip`
* A **Remyx API key** ([get one here](https://studio.remyx.ai/account))
## Installation
```bash theme={null}
pip install remyxai
```
Verify the executable is on your `PATH`:
```bash theme={null}
remyxai --help
```
## Authentication
Remyx authenticates API calls with a bearer token. Export it before running commands:
```bash macOS / Linux theme={null}
export REMYXAI_API_KEY=
```
```powershell Windows (PowerShell) theme={null}
$env:REMYXAI_API_KEY = ""
```
The client uses this variable when calling `https://engine.remyx.ai/api/v1.0`. Do not commit keys; use your shell profile, `.env` (with a loader), or your CI secret store.
Your CLI auth is always a single personal key (`rmxu_`) in `REMYXAI_API_KEY`. When Outrider provisions a repo (`outrider init` / `setup-local`, or `interests` with `--automate review`/`auto`), it writes a *scoped* automation key (`rmxa_`) into the repo's secrets as `REMYX_API_KEY`. Your personal key stays on your machine. See [API key scopes](/api-reference/scopes) for the difference.
***
## Command Overview
| Group | What it does |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `remyxai interests` | Manage Research Interests — the context that drives personalized recommendations |
| `remyxai papers` | Get daily recommendations, view digests, and trigger refreshes |
| `remyxai search` | Search across papers, repos, models, and datasets |
| `remyxai outrider` | Install and drive [Outrider agents](/platform/agents/overview) — the [Outrider action](/platform/agents/outrider-action) — on your repos |
Run `remyxai --help` for all options in a group.
***
## Research Interests
Research Interests define what you care about. They drive the recommendation engine — each interest has its own context and daily recommendation cadence.
### List Interests
```bash theme={null}
remyxai interests list
```
Shows all your Research Interests with name, context preview, daily count, and active status.
```bash Text output (default) theme={null}
remyxai interests list
```
```bash JSON output theme={null}
remyxai interests list --format json
```
### Create an Interest
```bash theme={null}
remyxai interests create \
--name "Retrieval-Augmented Generation" \
--context "https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B" \
--daily-count 3
```
| Flag | Description | Default |
| ---------------------------- | --------------------------------------------------------------------------------------- | ----------------------- |
| `--name, -n` | Short label for the interest | *(prompted if omitted)* |
| `--context, -c` | Natural language description, or URLs to HuggingFace models, GitHub repos, arXiv papers | *(prompted if omitted)* |
| `--daily-count, -d` | Recommendations per day (1-10) | 2 |
| `--inactive` | Create as inactive (excluded from daily digest until toggled) | — |
| `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` |
| `--wait, -w` | Block until the refresh completes | — |
| `--format, -f` | Output format: `text` or `json` | `text` |
Run `remyxai interests create` with no flags for an interactive prompt that walks you through each field.
### Create from a GitHub repo
```bash theme={null}
remyxai interests from-repo https://github.com/your-org/your-project
```
Builds an interest from a repo's profile (languages, topics, README, dependencies) instead of free text. Pass `--automate review` or `--automate auto` to also provision [Outrider](/platform/agents/outrider-action) on that repo in the same step, the same flow as `outrider init`.
| Flag | Description | Default |
| ---------------------------- | ------------------------------------------------------------------------------------------------ | ------------------------- |
| `--name, -n` | Short label for the interest | *(derived from the repo)* |
| `--daily-count, -d` | Recommendations per day (1-10) | 2 |
| `--inactive` | Create as inactive | — |
| `--automate` | `none`, `review`, or `auto`. `review`/`auto` provision Outrider on the repo like `outrider init` | `none` |
| `--timeout` | Seconds to wait on Outrider provisioning when automating | 300 |
| `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` |
| `--wait, -w` | Block until the refresh completes | — |
| `--format, -f` | Output format: `text` or `json` | `text` |
### Create from a project's experiments
```bash theme={null}
remyxai interests from-project "My RAG Project"
```
Builds an interest from a project's experiments. The `` argument accepts a project name or UUID. The interest tracks the project by default, so new experiments keep it current; pass `--no-auto-update` to take a one-time snapshot instead.
| Flag | Description | Default |
| ---------------------------- | --------------------------------------------------- | ---------------------------- |
| `--name, -n` | Short label for the interest | *(derived from the project)* |
| `--daily-count, -d` | Recommendations per day (1-10) | 2 |
| `--inactive` | Create as inactive | — |
| `--include-experiment, -e` | Limit context to a specific experiment (repeatable) | *(all experiments)* |
| `--no-auto-update` | Snapshot the project once instead of tracking it | — |
| `--refresh` / `--no-refresh` | Kick a recommendations refresh after creating | `--refresh` |
| `--wait, -w` | Block until the refresh completes | — |
| `--format, -f` | Output format: `text` or `json` | `text` |
### Get Interest Details
```bash theme={null}
remyxai interests get --interest "Retrieval-Augmented Generation"
```
The `--interest` flag accepts either a **name** (case-insensitive) or a **UUID**.
### Update an Interest
```bash theme={null}
remyxai interests update \
--interest "Retrieval-Augmented Generation" \
--context "https://github.com/your-org/your-rag-system" \
--daily-count 5
```
| Flag | Description |
| ------------------- | --------------------------------------------------------------------- |
| `--interest, -i` | Interest name or UUID (required) |
| `--name, -n` | New name |
| `--context, -c` | New context (invalidates recommendation pool and triggers re-ranking) |
| `--daily-count, -d` | New daily count |
| `--activate` | Set active |
| `--deactivate` | Set inactive |
Updating the `context` invalidates the pre-ranked recommendation pool. Run `remyxai papers refresh` afterward to get fresh results immediately.
### Toggle Active Status
```bash theme={null}
remyxai interests toggle --interest "Retrieval-Augmented Generation"
```
Toggles between active and inactive without deleting the interest or its recommendation history.
### Delete an Interest
```bash theme={null}
remyxai interests delete --interest "Retrieval-Augmented Generation"
```
Prompts for confirmation. Pass `--yes` to skip:
```bash theme={null}
remyxai interests delete --interest "Retrieval-Augmented Generation" --yes
```
***
## Recommendations
Get personalized resource recommendations based on your Research Interests.
### Daily Digest
```bash theme={null}
remyxai papers digest
```
Shows today's top recommendations grouped by Research Interest. Each entry includes the resource title, source type, relevance score, and why it was recommended.
| Flag | Description | Default |
| -------------- | ------------------------------------------- | ------- |
| `--limit, -n` | Max items per interest | 5 |
| `--period, -p` | `today`, `week`, or `all` | `today` |
| `--format, -f` | `text` or `json` | `text` |
| `--full` | Show full reasoning text without truncation | — |
```bash Today's digest theme={null}
remyxai papers digest
```
```bash This week, JSON output theme={null}
remyxai papers digest --period week --format json
```
```bash Full reasoning, 3 per interest theme={null}
remyxai papers digest --full --limit 3
```
### List Recommendations
```bash theme={null}
remyxai papers list
```
Flat list of all recommendations (not grouped by interest). Supports more granular filtering.
| Flag | Description | Default |
| ------------------- | ---------------------------------------------------------- | ----------------- |
| `--interest, -i` | Filter by interest name or UUID | *(all interests)* |
| `--limit, -n` | Max results (1-50) | 20 |
| `--period, -p` | `today`, `week`, or `all` | `all` |
| `--source-type, -s` | Filter by source type (`arxiv_paper`, `github_repo`, etc.) | *(all types)* |
| `--format, -f` | `text` or `json` | `text` |
| `--full` | Show full reasoning text | — |
```bash theme={null}
# Papers for a specific interest from this week
remyxai papers list --interest "Retrieval-Augmented Generation" --period week
# Only GitHub repos
remyxai papers list --source-type github_repo
```
### Refresh Recommendations
```bash theme={null}
remyxai papers refresh
```
Triggers an async recommendation refresh. By default, refreshes all active interests. Use `--wait` to block until complete.
| Flag | Description | Default |
| ------------------- | ------------------------------------------ | -------------------------------- |
| `--interest, -i` | Refresh a specific interest (name or UUID) | *(all active)* |
| `--num-results, -n` | Items per interest | *(uses interest's daily\_count)* |
| `--wait, -w` | Block until all tasks complete | — |
| `--format, -f` | `text` or `json` | `text` |
```bash Refresh all, wait for results theme={null}
remyxai papers refresh --wait
```
```bash Refresh one interest theme={null}
remyxai papers refresh --interest "Reinforcement Learning" --wait
```
### Check Refresh Status
```bash theme={null}
remyxai papers refresh-status
```
Poll the status of an async refresh task (from `papers refresh` without `--wait`).
***
## Search
Search across the Remyx resource index — papers, repos, models, and datasets with Docker availability filtering.
### Query
```bash theme={null}
remyxai search query "retrieval augmented generation for code"
```
| Flag | Description | Default |
| ------------------- | ----------------------------------------------------- | ------- |
| `--max-results, -n` | Maximum results | 10 |
| `--category, -c` | Filter by category (can be specified multiple times) | *(all)* |
| `--docker` | Only show resources with runnable Docker environments | — |
| `--no-docker` | Only show resources without Docker | — |
```bash theme={null}
# Search for Docker-ready resources only
remyxai search query "vision language models" --docker --max-results 5
# Filter by category
remyxai search query "reinforcement learning" --category cs.LG --category cs.AI
```
### Get Resource Details
```bash theme={null}
remyxai search info 2308.12345
```
Returns detailed information for a specific resource by arXiv ID, including full abstract, Docker status, Dockerfile contents, and reference summary.
| Flag | Description | Default |
| -------------- | ---------------- | ------- |
| `--format, -f` | `text` or `json` | `text` |
### List Recent Resources
```bash theme={null}
remyxai search list
```
Browse recently added resources with pagination.
| Flag | Description | Default |
| -------------------------- | ------------------------------------- | ------- |
| `--limit, -n` | Number of results | 20 |
| `--offset, -o` | Pagination offset | 0 |
| `--category, -c` | Filter by category (multiple allowed) | *(all)* |
| `--docker` / `--no-docker` | Filter by Docker availability | *(all)* |
### Catalog Statistics
```bash theme={null}
remyxai search stats
```
Shows aggregate stats: total assets, Docker availability counts, category breakdown, and citation classification coverage.
***
## Outrider
Install [automated discovery PRs](/platform/agents/outrider-action) on a GitHub repo without leaving the terminal. The CLI drives the same "set it up for me" flow as the web app — your local git is never touched.
### Install on a repo
```bash theme={null}
remyxai outrider init --repo owner/name --auto-interest
```
This drives the Remyx engine so the **Remyx GitHub App** (`remyx-ai[bot]`) sets the repo secrets, writes the workflow, opens a bot-authored setup PR, and — in `auto` mode — merges it and fires the first run. No personal `gh` token is needed; only your `REMYXAI_API_KEY`.
| Flag | Description | Default |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------- |
| `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from the current directory's git remote)* |
| `--interest, -i` | Research Interest UUID to wire in | *(prompted / see `--auto-interest`)* |
| `--auto-interest` | Auto-create a Research Interest from the repo (mutually exclusive with `--interest`) | — |
| `--bulk-repos` | Path to a TSV of `owner/nameinterest_uuid` rows; provisions each in turn with per-row error capture. Mutually exclusive with `--repo` / `--interest` / `--auto-interest` | — |
| `--pace` | Seconds to wait between repos in a `--bulk-repos` run | 3 |
| `--mode` | `auto` (provision + merge setup PR + first run), `review` (provision + open setup PR to review), `off` (create the interest only) | `auto` |
| `--agent` | Which coding-agent CLI runs the implementation: `claude`, `codex`, or `backboard`. A separate axis from the model provider. The CLI reads the provisioned workflow back and confirms the agent it was written with | `claude` |
| `--anthropic-key` | Anthropic API key to connect as the model provider when none is connected (use [Connectors](/platform/configure/connectors#model-providers) for any other provider). Falls back to `$ANTHROPIC_API_KEY` | *(only used if no provider is connected)* |
| `--no-wait` | Don't block polling for App install / provisioning; print next steps and return | — |
| `--dry-run` | Print the plan and exit without making changes | — |
| `--yes, -y` | Skip the confirmation prompt | — |
Connect the [GitHub](/platform/configure/connectors#github) and a [model provider](/platform/configure/connectors#model-providers) (Anthropic, Z.ai, or Moonshot) integration in [Connectors](/platform/configure/connectors) first; `outrider init` uses those connections. If the GitHub App isn't installed on the repo, the command surfaces the install link, and you can pass `--anthropic-key` to set the Anthropic key the first time.
```bash Auto-create the interest and set it all up theme={null}
remyxai outrider init --repo remyxai/RepoRanger --auto-interest
```
```bash Wire in an existing interest, review the setup PR theme={null}
remyxai outrider init --repo owner/name --interest --mode review
```
```bash Non-interactive (CI) theme={null}
REMYXAI_API_KEY=... ANTHROPIC_API_KEY=... \
remyxai outrider init --repo owner/name --interest --yes
```
### Install without the GitHub App
For organizations that can't grant a third-party GitHub App yet (e.g. a pending security review), `setup-local` does the same provisioning using **your own authenticated `gh` CLI** instead of the Remyx App — nothing new to security-review. The running Action opens PRs with the repo's built-in `GITHUB_TOKEN`.
```bash theme={null}
remyxai outrider setup-local --repo owner/name --auto-interest
```
| Flag | Description | Default |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- |
| `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* |
| `--interest, -i` | Research Interest UUID to wire in | — |
| `--auto-interest` | Auto-create a Research Interest from the repo | — |
| `--bulk-repos` | Path to a TSV of `owner/nameinterest_uuid` rows; provisions each in turn with per-row error capture. Mutually exclusive with `--repo` / `--interest` / `--auto-interest` | — |
| `--pace` | Seconds to wait between repos in a `--bulk-repos` run | 3 |
| `--mode` | `auto` (open + merge setup PR + dispatch first run) or `review` (open the setup PR for you to merge) | `auto` |
| `--no-cron` | Render the workflow with the cron schedule commented out, keeping only `workflow_dispatch` (manual / `outrider trigger` runs) | — |
| `--anthropic-key` | Anthropic key to set as the `ANTHROPIC_API_KEY` repo secret. Falls back to `$ANTHROPIC_API_KEY`, then prompts | — |
| `--agent` | Which coding-agent CLI the generated workflow runs by default: `claude`, `codex`, or `backboard`. Sets the `agent` dispatch input's default, so one install can still switch agents per run — provided that pair's secret is on the repo | `claude` |
| `--backend` | Default model backend for the single-file setup: `anthropic`, `openai`, `zai`, `moonshot`, or `openrouter` — on either API family, since the agent axis decides which one is spoken. Unset, it follows the agent to its own vendor (`codex` → `openai`). The rendered workflow declares `agent`/`provider`/`model` dispatch inputs either way, so any install can be re-routed per run | *(the agent's own vendor)* |
| `--model` | Model id the install runs by default, from the backend you picked (e.g. `gpt-5.4-mini` on `openai`, `glm-5.3` on `zai`). Becomes the `model` dispatch input's default. Required with `--agent backboard`, which addresses models as `/` and has no default of its own | *(the agent's own default)* |
| `--zai-key` | Z.ai GLM Coding Plan key, set as the `ZAI_API_KEY` repo secret when a GLM model is in play. Falls back to `$ZAI_API_KEY` | — |
| `--two-tier` | Write the [two-tier drafter/refiner](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) layout: a dispatch-capable main workflow, a daily branch-mode drafter, and a weekly refiner | — |
| `--drafter-model` / `--refiner-model` | Models for the two tiers — any tier from any supported provider, mixed freely (e.g. a `glm-5.2` drafter with a `claude-opus-4-8` refiner, or a `kimi-k3` drafter) | — |
| `--dry-run` | Print the plan + rendered workflow and exit | — |
| `--yes, -y` | Skip the confirmation prompt | — |
`setup-local` needs an authenticated `gh` (run `gh auth login`, or set `$GITHUB_TOKEN` with `repo` + `workflow` scopes) and admin on the target repo. The only runtime Remyx dependency is the `REMYX_API_KEY` the workflow uses.
It writes only the selected backend's secret; add another with [`set-provider-secret`](#set-a-providers-api-key-on-a-repo) to switch per dispatch. Claude Code and Codex read no credential of their own — the action derives theirs from the provider's key — so don't set an agent-named secret for them. Backboard is the exception: its own key *is* the credential (`gh secret set BACKBOARD_API_KEY`).
### Trigger a one-shot run
Once Outrider is installed on a repo, `trigger` fires a single `workflow_dispatch` run of the workflow through your local `gh`. Useful for testing or kicking a run between cron cycles without waiting for the schedule.
```bash theme={null}
remyxai outrider trigger --repo owner/name
```
| Flag | Description | Default |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
| `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* |
| `--interest, -i` | Research Interest to run against | *(the repo's configured interest)* |
| `--pin-method` | A method query, or a literal arXiv ID, to pin for this run; bypasses the candidate-selection pass. Mutually exclusive with `--pin-arxiv` | — |
| `--pin-arxiv` | An arXiv ID **already in the candidate pool** to pin for this run. Mutually exclusive with `--pin-method` | — |
| `--ref` | Git ref to dispatch the workflow on | *(repo default branch)* |
| `--agent` | Which coding-agent CLI runs the implementation on this dispatch (`claude`, `codex`, `backboard`) — a separate axis from `--provider`. An impossible agent/provider pair is rejected locally, before the dispatch is spent. The target workflow must declare an `agent` input; re-run `init --force` or `setup-local` on installs predating it | *(workflow default)* |
| `--agent-timeout` | Wall-clock seconds for each agent phase on this dispatch (preflight and implementation share the budget). Raise for very large monorepos or thinking-mode backends. Only Claude Code has a round cap, so on `codex` and `backboard` this is the only bound on spend. `--claude-timeout` is the old name and still works | *(workflow default)* |
| `--provider` | Route the coding agent at a specific model backend for this dispatch (`anthropic`, `openai`, `zai`, `moonshot`, `openrouter`). The target workflow must declare a `provider` dispatch input, which `setup-local` workflows do | *(workflow default)* |
| `--model` | Specific model to request from the provider (e.g. `claude-opus-4-8`, `glm-5.3`, `kimi-k3`, `gpt-5.4-mini`), forwarded as the selected agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, …) | *(provider's default)* |
`trigger` needs an authenticated `gh` (run `gh auth login`). It refuses up front if no Outrider workflow is registered on the repo; install one first with `outrider init` or `setup-local`.
### Route a run at a different model provider
Outrider's coding runtime speaks whichever model-API family its coding agent speaks, so runs can route at any compatible provider. A `setup-local` workflow declares `agent`, `provider`, and `model` as dispatch inputs — the action's [`provider` input](/platform/agents/backends#the-provider-input) resolves the credential, base URL, and auth style per run — so one workflow can A/B Anthropic against Z.ai's GLM or Moonshot's Kimi:
```bash theme={null}
# one run on z.ai's GLM; Anthropic stays the workflow default
remyxai outrider trigger --repo owner/name \
--pin-method "speculative decoding" --provider zai --model glm-5.3
# the same run on Moonshot's Kimi (raise the timeout for thinking-mode latency)
remyxai outrider trigger --repo owner/name \
--pin-method "speculative decoding" --provider moonshot --model kimi-k3 \
--agent-timeout 3600
```
`--provider` names the API endpoint; `--model` picks the model from it. Omit either to fall back to the workflow's (or the provider's) default. See [Model backends](/platform/agents/backends) for the full matrix — secrets, auth styles, recommended timeouts, and cost telemetry.
### Switch the coding agent for a run
`--agent` is the second axis: which coding-agent CLI does the implementing, independent of which model serves it.
```bash theme={null}
# Codex on OpenAI
remyxai outrider trigger --repo owner/name \
--pin-method "speculative decoding" --agent codex --provider openai --model gpt-5.4-mini
# Backboard's router — it addresses models as /, so name one
remyxai outrider trigger --repo owner/name \
--agent backboard --provider openrouter --model z-ai/glm-5.3
```
An agent speaks one API family and a provider serves one or more, so the CLI checks the pair locally before spending a dispatch — and names the agent that does serve your provider:
```
$ remyxai outrider trigger --agent codex --provider anthropic
Error: agent=codex speaks openai-responses, which Anthropic does not serve.
Use --agent claude for this provider.
```
Omit `--agent` and the check still runs, against the agent the installed workflow bakes rather than the CLI's default. The pair table is vendored from the action's [published matrix](https://github.com/remyxai/outrider/blob/main/docs/agent-matrix.json), so the CLI and the action can't disagree about what's supported.
To confirm a pairing actually reaches the vendor before spending a real run, dispatch the action's wiring check — it resolves the routing and makes one trivial call, no clone and no artifact, in about 30 seconds:
```bash theme={null}
remyxai outrider trigger --repo owner/name \
--mode smoke --agent codex --provider openai --model gpt-5.4-mini
```
### Set a provider's API key on a repo
`set-provider-secret` writes a provider's API key into the repo's secrets safely. It reads the key from a file, strips the trailing newline, and sidesteps the `gh secret set --body -` truncation trap that otherwise stores a literal `-` and breaks every run with a `401`.
```bash theme={null}
remyxai outrider set-provider-secret \
--repo owner/name --provider zai --key-from ~/zai-key
```
| Flag | Description | Default |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------- |
| `--repo` | Target repo as `owner/name` or a GitHub URL | *(detected from git remote)* |
| `--provider` | Which provider the key is for; selects the secret name by the `_API_KEY` convention (`anthropic` → `ANTHROPIC_API_KEY`, `openai` → `OPENAI_API_KEY`, `zai`, `moonshot`, `openrouter`). Covers every provider the action understands, which is wider than the set `init` can provision through the engine. **Required** | — |
| `--key-from` | Path to a file containing only the API key. **Required** | — |
See [Automated discovery PRs](/platform/agents/outrider-action) for what the action does once installed, its configuration inputs, and how to review the PRs it opens.
***
## Quick Reference
| Intent | Command |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Get today's digest | `remyxai papers digest` |
| Get this week's digest | `remyxai papers digest --period week` |
| Recommendations for one interest | `remyxai papers list --interest "RAG" --period today` |
| Refresh all interests | `remyxai papers refresh --wait` |
| Refresh one interest | `remyxai papers refresh --interest "RAG" --wait` |
| List interests | `remyxai interests list` |
| Create an interest | `remyxai interests create --name "RAG" --context "..." --daily-count 3` |
| Create an interest from a repo | `remyxai interests from-repo https://github.com/your-org/your-project` |
| Create an interest from a project | `remyxai interests from-project "My RAG Project"` |
| Toggle an interest | `remyxai interests toggle --interest "RAG"` |
| Delete an interest | `remyxai interests delete --interest "RAG" --yes` |
| Search for resources | `remyxai search query "your query" --max-results 10` |
| Search Docker-ready only | `remyxai search query "your query" --docker` |
| Get resource details | `remyxai search info 2308.12345` |
| Browse recent resources | `remyxai search list --limit 20` |
| Catalog stats | `remyxai search stats` |
| Install an agent | `remyxai outrider init --repo owner/name --auto-interest` |
| Install without the GitHub App | `remyxai outrider setup-local --repo owner/name --auto-interest` |
| Fire a one-shot Outrider run | `remyxai outrider trigger --repo owner/name` |
| Run a method via z.ai GLM | `remyxai outrider trigger --repo owner/name --pin-method "..." --provider zai --model glm-5.3` |
| Run a method via Moonshot Kimi | `remyxai outrider trigger --repo owner/name --pin-method "..." --provider moonshot --model kimi-k3 --agent-timeout 3600` |
| Run a method on Codex instead of Claude Code | `remyxai outrider trigger --repo owner/name --pin-method "..." --agent codex --provider openai` |
| Two-tier drafter/refiner install | `remyxai outrider setup-local --repo owner/name --auto-interest --two-tier` |
| Set a provider's API key on a repo | `remyxai outrider set-provider-secret --repo owner/name --provider zai --key-from ~/zai-key` |
***
## Troubleshooting
Export the key in the same shell session you use for `remyxai`. Verify with:
```bash theme={null}
echo $REMYXAI_API_KEY
```
If empty, set it:
```bash theme={null}
export REMYXAI_API_KEY=
```
The binary may not be on your `PATH`. Check where pip installed it:
```bash theme={null}
pip show remyxai | grep Location
```
On Linux, `pip install` without `sudo` puts the binary in `~/.local/bin`. Either add that to your `PATH` or install with `sudo pip install remyxai`.
Run `remyxai papers refresh --wait` to trigger a fresh ranking. If recommendations show for `--period week` but not `today`, they were generated on a previous day. Refreshing will surface the latest resources.
***
## Next Steps
* Set up an [agent](/platform/agents/overview) on a repo with `remyxai outrider init`
* Browse commands and source in the [Remyx CLI repo](https://github.com/remyxai/remyxai-cli)
* Connect the [MCP server](/resources/mcp-server) to drive your fleet from any MCP client
# Causal Intelligence
Source: https://docs.remyx.ai/concepts/causal-intelligence
The roadmap for causal inference as the through-line of Remyx's experiment intelligence
# Causal Intelligence
ExperimentOps captures decisions and surfaces patterns across the experiments your team has already run. Causal intelligence is what those captured experiments build toward: a maintained model of how your system actually works, updated as new evidence arrives from production logs, commit history, A/B tests, and instrumented decision points.
The capabilities described here ship in stages. See [Maturity Progression](/concepts/maturity-progression) for how they map to the customer journey, and the [Roadmap](/roadmap) for current shipping status.
> Most of what follows is forward-looking. We publish the architecture in full so customers, advisors, and researchers can engage with the direction, push back where they disagree, and see the shape of the system well before each piece ships.
## The case for broader evidence
When an AI team ships a change and resolution rate goes up, the question that matters is what *caused* the lift. Was it the prompt change, the retrieval upgrade, both, neither? Without a causal answer, the team's next decision is guesswork.
A/B testing answers this question well for the changes that warrant it. The limit is throughput. Each test carries fixed operational cost (instrumentation, traffic allocation, time to significance) that doesn't scale with the rate at which a modern AI team ships candidate changes.
Remyx's approach is to broaden the evidence base. Use A/B testing where it earns its operational cost. Get causal estimates from cheaper evidence everywhere else, and use perturbation-based evidence to answer the questions A/B testing structurally cannot.
## Three evidence sources
Remyx integrates three evidence sources into one causal model. Each has different strengths and different operational costs, and the system uses all three together.
Production logs windowed against commit boundaries. Each meaningful commit (a model swap, prompt change, retrieval update, routing rule) is a regime change in the data-generating process. The windows before and after a commit form a natural before-after comparison. Under the assumption that nothing else material changed in the window, this supports causal effect estimates from data your team is already collecting.
When your team runs randomized controlled trials through a platform like Statsig, Eppo, LaunchDarkly, or in-house, Remyx integrates with the platform and incorporates this evidence into the model. Randomization is the gold standard for causal identification, so these results carry the most weight.
Remyx's lightweight client SDK helps instrument your system's decision points, applying counterfactual perturbations that generate the evidence the causal inference engine needs to identify effects it otherwise cannot — which part of the pipeline is doing the work, the effect of a treatment specifically on the population that received it, effect estimates at low-traffic decision points. The SDK runs in shadow mode first to audit safety before any perturbation is applied to live traffic.
The causal model incorporates each new piece of evidence into its posterior, rather than treating sources separately or letting later evidence overwrite earlier results.
## How teams use the model
The causal model sits underneath the product. Your team interacts with three workflows on top of it.
### Answer questions in natural language
> "Did the prompt change last Tuesday cause the latency regression?"
The system parses the question, identifies what kind of evidence could answer it, routes to the relevant sources, and returns an estimate with a confidence interval and a clear note on whether the available data is strong enough to support a causal conclusion or only a correlational one.
### Triage hypotheses prospectively
When your team proposes ten things to try, the system ranks them by how much each would actually teach you given what you already know, and identifies the cheapest evidence path for each. Each hypothesis gets one of these paths.
* Already answerable from existing data.
* Answerable with quasi-experimental analysis.
* Requires an A/B test.
* Requires CTF-RAND at decision point X for two weeks.
* Unidentifiable even with full instrumentation.
### Surface what's not yet known
When you ask a question the current evidence cannot answer, the system tells you what data would answer it. That turns "we don't know" from a dead end into a planning surface.
## How this fits with A/B testing
Causal intelligence works alongside A/B testing. The three evidence layers each handle a different operational price point and answer a different class of question.
| | A/B testing | Causal data fusion *(quasi-experimental)* | Counterfactual randomization |
| :--------------- | :---------------------------------------- | :------------------------------------------------------------ | :----------------------------------------------------------------- |
| Evidence quality | Randomized | Quasi-experimental, with RCT evidence fused in when available | Counterfactual, from instrumented perturbations |
| Operational cost | High per change | Low marginal cost (reuses existing data) | Moderate (SDK instrumentation, shadow-mode audit before live) |
| Coverage | Changes worth gating with a traffic split | Every commit boundary in the log history | Decision points where the SDK is instrumented |
| Question types | Average effect of a randomized treatment | Average effects across changes the team already shipped | Attribution, mediation, effect on the treated, low-traffic effects |
| When to use | Decisions worth the cost of randomization | Continuous learning from the work the team is already doing | Questions A/B testing structurally cannot answer |
A/B tests sharpen the model where they earn their operational cost. Quasi-experimental evidence is the entry point for teams not yet platformed for A/B testing, producing causal estimates from observational data they already collect. Counterfactual randomization extends the model to questions that no amount of A/B testing can resolve.
## Theoretical foundation
The system makes only those causal claims that can be backed by evidence you could in principle collect. It avoids the layers of counterfactual reasoning that depend on assumptions which cannot be checked against data.
For readers familiar with Pearl's Causal Hierarchy (PCH), the evidence sources map to layers as follows. PCH organizes causal questions into three rungs of increasing strength: **L1** (associational, "what is"), **L2** (interventional, "what happens if I do"), and **L3** (counterfactual, "what would have happened if I had done"). Yang & Bareinboim (2025) introduce intermediate rungs L2.25 and L2.5 for the counterfactual evidence that CTF-RAND collects.
| Source | Layer | Notes |
| :--------------------------------- | :---- | :-------------------------------------------------------------- |
| Observational logs | L1 | Supports correlational queries and structural discovery |
| Quasi-experiments (logs + commits) | L2 | Local identification under "nothing else changed in the window" |
| A/B tests | L2 | Strong identification via randomization |
| CTF-RAND, default | L2.25 | Same overridden value applies across the trajectory |
| CTF-RAND, mediation opt-in | L2.5 | Per-child value assignment for mediation analysis |
Readers who want the formal account should start with the further reading below.
## Shipping status
For current status of each capability, see the [Roadmap](/roadmap). For how these capabilities map to the customer journey, see [Maturity Progression](/concepts/maturity-progression).
## Further reading
Yang, K., & Bareinboim, E. (2025). *A Hierarchy of Graphical Models for Counterfactual Inferences.* Background on graphical models used in causal and counterfactual inference, including the L2.25 and L2.5 layers referenced above.
Pearl, J., & Mackenzie, D. (2018). *The Book of Why.* Accessible introduction to the Causal Hierarchy and the difference between association, intervention, and counterfactual reasoning.
# Continuous Experimentation
Source: https://docs.remyx.ai/concepts/continuous-experimentation
The loop that keeps running on top of ExperimentOps: discovery, drafting, evaluation, and decision, with each turn sharpening the next. CI/CD-style cadence for AI experimentation, with a human at the merge.
[ExperimentOps](/concepts/experimentops) is your system of record — what your team tried, why, and what you decided. **Continuous Experimentation** puts that record to work: it runs the discovery-to-decision loop on a cadence, so keeping up with the field becomes a background process instead of a second job — the way CI/CD turned "we deploy when someone remembers to" into "every change runs the pipeline."
The bottleneck it removes is the legwork before a decision. New techniques ship across arXiv, Hugging Face, and GitHub every week; ideas are plentiful, and the cost is finding the few that fit your codebase and turning them into something you can actually evaluate. The loop does that work — surface, validate, draft — and leaves you the judgment, so effort flows to the most fruitful directions.
**What "continuous" means, precisely.** The loop runs on a schedule and does the watching, reading, and first-draft work automatically. It does **not** make the call — every cycle surfaces a reviewable artifact (a ranked recommendation, a draft PR, a scored variant) and a person decides. The CI/CD analogy, taken honestly: the pipeline runs on every change, but a human still approves the deploy.
***
## The loop
A single experiment in Remyx already has a lifecycle: it comes from somewhere, gets implemented, gets evaluated, and ends in a decision. Continuous Experimentation is that lifecycle run as a *standing loop*, where the output of each turn feeds the input of the next.
Your [Feed](/platform/discover/feed) surfaces new papers, repos, and models ranked against your team's actual shipping history. This runs daily without you asking.
Remyx's [automated discovery-PR agent](/platform/agents/outrider-action) (**Outrider**), a scheduled GitHub Action, picks the candidate most *implementable* against your codebase and opens a **draft PR** wiring it into a real call site, or a **discussion Issue** when a clean integration isn't possible. This is the step that used to require a human to sit down and start reading.
The variant is scored against the [eval template and decision policy](/platform/review/evaluation) your team [committed to ahead of time](/tutorials/get-started/define-how-progress-gets-measured), so the bar is fixed before the result is known.
A person reviews the evidence and logs the call — ship, iterate, or abandon — and why. This is the human gate, and it stays human.
The decision becomes part of the record. The next discovery pass is ranked against a history that now includes this experiment. The loop tightens.
The first two steps are where Continuous Experimentation does new work: it closes the gap between *a relevant paper exists* and *there's a reviewable proposal in front of me*. Steps three through five are the [ExperimentOps](/concepts/experimentops) loop you already run — now fed automatically.
***
## CI/CD for AI experimentation
The analogy is exact in the places that matter and worth stating plainly where it isn't.
| Software CI/CD | Remyx Continuous Experimentation |
| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| A commit triggers the pipeline | A schedule (or a context change) triggers a discovery + draft run |
| The pipeline builds and tests automatically | Outrider selects, drafts, validates, and self-reviews automatically |
| A failing build never reaches review | A recommendation that can't be cleanly integrated becomes a discussion Issue with the attempted diff attached |
| The team sets the gates (required checks, coverage) ahead of time | The team sets the eval template and decision policy ahead of time |
| **A human approves the deploy** | **A human reviews and merges the PR, and logs the decision** |
Where the analogy breaks — and where overclaiming would erode trust — is the last row. CI/CD ends in an automated deploy because the correctness bar is a passing test suite. AI experimentation ends in a *judgment* about whether a change actually moved a business outcome, which is not something to hand to a cron job. Remyx automates the toil up to that judgment and stops there.
This is also why the [Maturity Progression](/concepts/maturity-progression) stages stay **read-only and passive** through Stage 3. Continuous Experimentation reads your repo, ranks against your history, and proposes changes you approve. It does not touch production behavior. The first capability that does (Stage 4 counterfactual perturbations) ships only behind shadow-mode audits.
***
## How recommendations get sharper
A loop is only worth running continuously if each turn is better than the last. Two mechanisms make Remyx's recommendations improve as your history grows, rather than re-surfacing the same generic results:
The structured experiments Remyx extracts from your merge log feed the ranker as context, so a candidate aligned with the direction you've actually shipped ranks above a merely topical one. This shifts the top results meaningfully versus ranking from your interest description alone, and the reasoning cites specific past work instead of shallow keyword overlap.
Remyx fits a per-team preference model over your past experiments — learning from the order and lineage in which you shipped things — and scores new candidates with it as a tiebreaker behind relevance. It populates lazily and becomes meaningful past a few dozen experiments. It sharpens ranking only; it is deliberately *not* wired to auto-generate or auto-select experiments, which would put it in the decision seat the human holds.
Both converge on the same `ExperimentHistory` whether you reached it through a [Project](/platform/configure/projects) or a repo-driven [Research Interest](/platform/discover/feed#research-interests-from-a-repo), so the loop sees one coherent picture of your work. Beyond history, a [Deep Research](/platform/discover/briefs) brief feeds the ranker as a forward-looking axis: it captures where your team intends to go next, so candidates aligned with that direction rank up even before you've shipped against it. If a recommendation or draft is wrong, the cost is a PR you close — nothing reaches your default branch or your users without a person putting it there.
***
## Get started
Set up the scheduled draft-PR loop on a repo
Create the Research Interest that drives recommendations
The system of record this loop runs on top of
Why the loop stays passive and read-only
# ExperimentOps
Source: https://docs.remyx.ai/concepts/experimentops
The methodology behind systematic AI experimentation: capture decisions, surface patterns, compound what your team learns
# ExperimentOps
Every AI team experiments. New techniques ship weekly. Coding agents generate implementations in hours. But most of that effort doesn't compound because the decisions, context, and cross-experiment patterns never get captured in a system.
ExperimentOps is the practice of tracking the full lifecycle of every AI experiment your team runs, including the **decisions**: what the team tried, why they tried it, whether it worked, and what to do next. Over time, this builds institutional knowledge that persists through team changes and reveals strategic patterns that are invisible when experiments are tracked in isolation.
**Shipped today vs. on the roadmap.** The capabilities described on this page are live in production. ExperimentOps is the foundation. The architecture extends further over time, with a causal model that updates as evidence arrives from logs, commits, A/B tests, and eventually instrumented decision points. See [Causal Intelligence](/concepts/causal-intelligence) and the [Roadmap](/roadmap) for the forward-looking direction.
***
## Why This Matters Now
The #1 production blocker cited by AI practitioners in 2026 is **evaluation, testing, and measurement**: knowing whether the system is actually working and getting better over time. The bottleneck is not generating ideas or writing code. It's validating what works, at production depth, with real business outcomes.
Three structural problems prevent most teams from solving this:
A data scientist spends two months testing retrieval strategies. The reasoning behind the final choice (why hybrid search won, what alternatives were tested, what tradeoffs were considered) lives in their memory and a few Slack messages. MLflow logged the runs but not the interpretation. When that person leaves, the next engineer starts from scratch.
Every experiment without captured context is knowledge that walks out the door when someone changes teams or leaves the company.
A team runs 14 experiments in a quarter. Five explored retrieval improvements and all produced positive results. Three explored routing and none worked. But nobody sees this pattern because each experiment is tracked in a different tool: a Jira ticket, an MLflow run, a Notion page.
The strategic signal ("retrieval is our strongest direction, routing is not working") is invisible unless someone manually reviews all experiments and connects the dots. Nobody has time for that meta-analysis.
A CTO managing three AI projects needs to know: which are producing results? Which are stalled? Where should the team invest more? Today, getting that answer requires being in every room or scheduling status meetings with each team lead.
Without a portfolio view across AI projects, resource allocation is based on status reports instead of measured outcomes.
What was best practice weeks ago may already be suboptimal. New techniques, architectures, and tooling ship faster than any team can evaluate. The result: a persistent backlog of improvements that never gets touched because maintenance, firefighting, and stakeholder management consume all the bandwidth.
The result is a persistent backlog of improvements that never gets touched because maintenance and stakeholder work consume all the bandwidth.
***
## How ExperimentOps Works
ExperimentOps adds three layers that traditional experiment tracking doesn't capture:
### 1. The Decision Layer
Every experiment in Remyx records *what happened* (metrics, artifacts), *why it happened*, and *what the team decided to do about it*.
| Field | What it captures | Example |
| ----------------- | --------------------------------------------------- | --------------------------------------------------------------------------------- |
| **Source** | Where the idea came from | Paper, GitHub repo, HuggingFace model, hypothesis, incident, Remyx recommendation |
| **Hypothesis** | What the team expects | "Adding temporal decay to embeddings will increase 7-day retention by 1-2%" |
| **Target metric** | The business outcome that matters | Resolution rate, conversion rate, CSAT |
| **Decision** | What the team decided after seeing results, and why | "Ship to 100%. Explore temporal features in search ranking next." |
The decision field captures the reasoning that artifact-tracking tools miss. When a team logs "Ship to 100%. The re-ranker specifically helps with multi-topic tickets where the old retriever returned tangentially related articles," that reasoning persists through team changes, onboarding, and leadership transitions.
MLflow tracks what you did. Remyx tracks what it meant and what to do next.
### 2. The Pattern Layer
After enough experiments, Remyx groups them by direction and surfaces which directions are producing results:
```
Retrieval quality (5 experiments): 5 of 5 positive, avg +3.2% → HIGH SIGNAL
Prompt engineering (2 experiments): 2 of 2 positive, avg +1.2% → MODERATE
Routing / model selection (2 experiments): 0 of 2 significant → LOW SIGNAL
```
This meta-analysis is what no individual on the team can do on their own. It requires looking across all experiments, grouping them by theme, and computing which themes consistently produce results. For a team running 10+ experiments per quarter, this is the difference between iterating randomly and iterating strategically.
See how pattern detection works in the Insights view
### 3. The Recommendation Layer
For each high-signal direction, Remyx searches its resource index for techniques, models, and methods that align with the cluster's theme. Your team's own experiment history feeds the ranker directly: candidates that align with the direction you've actually shipped rank above ones that are merely topically related, and a learned **preference model** fit over your past experiments breaks ties. Past experiments inform future discovery, so recommendations improve as the team's data grows.
That same loop can run on a schedule — surfacing recommendations daily and opening reviewable draft PRs into your repo, with a person still approving every merge.
How Outrider runs the discovery-to-draft loop on a schedule, with the human at the merge
***
## The Experiment Lifecycle
An experiment starts from one of several sources:
* **Research**: A relevant paper, repo, or model surfaced by Remyx or found by the team
* **Hypothesis**: An engineer has an idea based on domain knowledge
* **Incident**: A production issue revealed a gap worth investigating
* **Recommendation**: Remyx detected a pattern and suggested a next step
For research-sourced experiments, Remyx assembles a **launch context** with resource metadata, Docker environment, target repo structure, and an AI-generated implementation plan grounded in actual file paths.
The team implements the experiment using the tools they already have:
* **GitHub**: PRs linked to experiments, status synced via webhooks
* **Linear / Jira**: Tickets created automatically, bidirectional status sync
* **Claude Code**: AI agent generates a draft PR via MCP, implementing the technique in the team's codebase
The experiment runs against the target metric. Remyx tracks A/B test configuration, traffic splits, duration, and results. The observed delta and statistical confidence are recorded and tied to the business outcome.
The team logs their interpretation: ship, iterate, or abandon, and why. A good decision captures reasoning, context, and next steps that help the next person (or the same person six months later) understand what happened and what it means.
As experiments accumulate, Remyx surfaces cross-experiment patterns. Tag-based clustering identifies which directions produce consistent results. Recommendations suggest what to try next, grounded in the team's own data.
***
## ExperimentOps vs. Traditional Experiment Tracking
| | MLflow / W\&B | Remyx ExperimentOps |
| ----------------------------- | ---------------------------------------------- | ------------------------------------------------------------------- |
| **Tracks** | Model runs, hyperparameters, artifacts | Full experiment lifecycle including decisions and business outcomes |
| **Entry point** | `mlflow.log_param()` in training code | Hypothesis, research resource, or production incident |
| **Decision capture** | Not supported | First-class `log_decision` with reasoning and next steps |
| **Cross-experiment patterns** | Manual analysis | Automatic tag-based clustering with hit rates |
| **Resource discovery** | None | Papers, repos, and models matched to experiment history |
| **Leadership view** | Dashboard of model metrics | Portfolio of projects with health indicators |
| **Integrations** | Model registry, artifact store | GitHub, Linear, Jira, Slack, Claude Code MCP |
| **Question answered** | "What hyperparameters produced the best loss?" | "Which directions are working, and what should we try next?" |
Remyx operates at a higher level than MLflow or W\&B. Link your MLflow runs as artifacts on a Remyx experiment to connect the training details with the business decision.
***
## Who Uses ExperimentOps
Track every experiment from hypothesis through results. Link PRs, tickets, and datasets. Log decisions so the next person understands why things were done this way. See which directions are producing results across the team's work.
Frees up time spent on maintenance triage and context reconstruction so engineers can focus on the experiments that actually improve the product.
See which projects are on track and which need attention. Understand hit rates across experiment directions. Know when a direction has been sufficiently explored or when it's worth doubling down.
Turns "which direction should we invest in?" from a debate into a data-informed decision.
**Get visibility without being the bottleneck.** The Portfolio view shows every project's experiment velocity, positive hit rate, and metric trends. Know which teams are producing results without reviewing every experiment or scheduling syncs. Let the system be the structured experimentation process so you don't have to be in every room.
Replaces status meetings with a live portfolio view that shows which projects are producing results and which need attention.
***
## Core Principles
**Decisions over metrics.** The most valuable artifact of an experiment is the team's interpretation: "This worked because X, and next we should try Y."
**Patterns over individual results.** A single experiment result is a data point. Ten experiments grouped by direction reveal a strategy.
**Compounding over starting fresh.** Every experiment builds institutional knowledge. That knowledge persists through team changes and informs future recommendations.
**Visibility over status meetings.** Leadership sees which projects are working on one screen, without booking a round of syncs.
***
## Get Started
Your first Outrider run — a draft PR in minutes
Explore the experiment timeline and detail views
Connect GitHub, Linear, Jira, Slack, and Claude Code
Use Remyx tools from Claude Code or Slack
# Maturity Progression
Source: https://docs.remyx.ai/concepts/maturity-progression
The four stages of experimentation maturity that Remyx supports, from early dev to counterfactual instrumentation
# Maturity Progression
AI teams progress through stages of experimentation maturity as their systems grow. Remyx is designed so each stage adds an evidence source and the queries it makes possible, without retiring what came before. Different teams enter at different stages. Mature teams progress through subsequent stages as their needs evolve.
The trust requirement increases gradually. Stages 1 through 3 are passive. Remyx reads commits, reads logs, reads A/B test results. Stage 4 is the first that touches production behavior, and only after shadow-mode audits.
For current shipping status of the capabilities at each stage, see the [Roadmap](/roadmap). For the architectural through-line, see [Causal Intelligence](/concepts/causal-intelligence).
***
## At a glance
| Stage | Evidence source | Trust requirement | Best fit |
| ----------------------------------- | ----------------------------------------------- | --------------------------- | ------------------------------------- |
| **1. Early dev** | Commit semantics | Read-only repo | Greenfield AI systems |
| **2. Production traffic** | Logs and commit-correlated quasi-experiments | Read-only repo and logs | Deployed AI systems at any scale |
| **3. A/B testing** | RCT results from Statsig, Eppo, LaunchDarkly | Read-only A/B test results | Mature experimentation discipline |
| **4. Counterfactual Perturbations** | Counterfactual randomization at decision points | SDK in production code path | A/B testing has become the bottleneck |
***
## Stage 1. Early dev
No production traffic.
**Where you are.** Building something new. The repo has commits but the deployed system has no traffic, or traffic too low to support quasi-experimental analysis.
**Remyx's role.** Parse the commit history to extract milestones, the meaningful structural changes that mark your team's evolving understanding. Use the semantic content of those milestones to recommend relevant new methods, models, evaluation strategies, and architectural patterns. Reusable scaffolding speeds up implementation.
There is no causal model yet because the system isn't gathering outcomes. The value comes from a meta-layer of recommendations grounded in commit semantics and matched against Remyx's resource index.
**Trust requirement.** Read-only access to your repo. Fully passive.
**Status.** Foundational capabilities live (commit ingestion, milestone extraction, recommendation engine). The standalone Stage 1 product surface is in active development.
***
## Stage 2. Production traffic and commits
Quasi-experiments from natural before-after windows.
**Where you are.** You have deployed something and traffic is flowing. You may already log production telemetry to Datadog, Honeycomb, structured JSON, or similar.
**Remyx's role.** Correlate observational data with commit boundaries to produce quasi-experimental causal evidence. Each meaningful commit becomes a regime boundary. The windows before and after are a natural before-after comparison. Accumulating these estimates across many commits builds a causal model of your system.
Recommendations from Stage 1 become evidence-grounded. Instead of *"teams using similar architectures have tried Y,"* you get *"your last three retrieval-related commits collectively shifted resolution rate by Z; the next intervention worth considering targets the prompt selection decision point."*
You can now ask questions in natural language. *"Did the model swap last Tuesday cause the latency regression?"* returns a causal answer with confidence intervals and identification status.
You connect your existing telemetry and your repo. The causal evidence comes from data you are already collecting.
**Trust requirement.** Read-only access to logs and repo. Still passive.
**Status.** Roadmap. Causal discovery, quasi-experiment identification, the graph engine, data fusion, and the natural language query layer are all on the development path. Foundational schema and ingestion pipelines are the priority for the next milestone.
***
## Stage 3. A/B testing in production
RCT evidence enters the model.
**Where you are.** Your experimentation practice is mature enough to run explicit A/B tests for important changes. You have an experimentation platform such as Statsig, Eppo, LaunchDarkly, or one built in-house.
**Remyx's role.** Integrate with your experimentation platform as another evidence source. A/B test results feed the causal model with high-confidence interventional updates. Estimates that were noisy from quasi-experimental analysis tighten when the same effect is measured through randomization. The model's quality improves as your team's RCT discipline matures.
This stage is additive. Quasi-experimental analysis continues for the long tail of changes that aren't worth A/B testing. RCT evidence sharpens estimates for the changes that justify the cost of a traffic split.
The hypothesis triage layer pays off here. The system surfaces recommendations like *"this hypothesis is borderline-identifiable from quasi-experiments; an A/B test would tighten the estimate by 3x"* so you can make informed decisions about which questions deserve A/B investment.
**Trust requirement.** Read-only access to A/B test results. Still passive. Remyx reads the randomization you are already doing and does not introduce new randomization.
**Status.** Roadmap. Connector framework will ship after Stage 2 quasi-experiment infrastructure is mature.
***
## Stage 4. Counterfactual Perturbations
A/B testing has become the bottleneck.
**Where you are.** Your team is doing enough A/B testing that the operational cost is real, and there are still important questions A/B testing cannot answer.
* **Mediation.** Did the prompt change cause the lift, or was it the retrieval change shipped the same week?
* **Effect of treatment on the treated.** The effect specifically on the subpopulation that receives a given policy.
* **Low-traffic decision points.** Effect estimates where there is not enough volume for a powered A/B test.
**Remyx's role.** Counterfactual randomization at instrumented decision points via the Remyx SDK. Adoption proceeds in two distinct steps with audits in between.
The SDK is integrated at decision points. The natural policy is captured. No overrides are applied. You see what data the system would collect under CTF-RAND and what questions become identifiable, without any risk to production behavior. You validate that the SDK is working correctly, audit the data being captured, and confirm proposed override policies.
Your team or compliance reviews the shadow-mode data and proposed override behavior. For some teams (regulated industries, high-stakes consumer products), this audit phase is non-negotiable and may take weeks.
After audits pass, overrides are enabled gradually, starting at 0% override rate, ramping to 1%, 5%, and so on. Each step is reversible via the kill switch. The first decision points to get live overrides are typically low-stakes (which retry strategy, which retrieval rerank) before higher-stakes decisions.
This stage produces the counterfactual evidence that makes ETT (Effect of Treatment on the Treated) and NDE (Natural Direct Effect) estimable. The questions that have been frustrating your team through earlier stages, mediation questions and policy-improvement questions, finally become answerable.
**Trust requirement.** SDK integrated into production code path. Natural-policy capture during shadow mode. Controlled override of production decisions after audit. The first stage that requires meaningful trust.
**Status.** Roadmap. SDK in log-only (shadow) mode is the first deliverable. CTF-RAND override policies and the counterfactual estimation layer ship after the causal model is mature.
***
## How to read this
If you are an early-dev team, Stage 1 is your entry point. The other stages describe what Remyx adds as your system grows.
If you have a mature experimentation practice with an in-house platform and are looking at how to handle the volume of AI changes that A/B testing alone cannot gate efficiently, Stage 3 and Stage 4 are the parts of the roadmap most relevant to you. The architectural commitment is to make causal inference operational alongside your existing A/B testing capacity.
The architectural through-line across stages
Shipped vs. in-development capabilities
# Introduction
Source: https://docs.remyx.ai/introduction
Decision intelligence for AI teams
# Decision intelligence for AI teams
**Know your next move.** Remyx is the decision layer between your AI coding agents and production: it identifies the next improvement worth making, filters out what doesn't apply, and learns from every result.
Every week adds another promising idea. Most will never justify the engineering time. The few that do are easy to miss. Remyx ranks candidate improvements against your codebase, constraints, and past results, then recommends the highest-confidence move — or explains why to hold.
Start free with Outrider and get your first recommendation in minutes
***
## Idea to production, systematically
Remyx closes the loop across your stack. Each cycle starts sharper than the last.
Candidate improvements — across prompts, retrieval, tools, routing — are ranked against your codebase, constraints, and past results. Every recommendation is matched to a call site in your code and gated on fit, reachability, and license.
An [agent](/platform/agents/overview) drafts the change as a reviewable PR — implemented, self-reviewed, and tested against your real codebase, authored by **remyx\[bot]**, never as you.
Your eval suite, offline and A/B, on your metrics. Results render against a decision policy you committed before the numbers landed: Ship, Reject, or Iterate.
A human gates every merge. The [Inbox](/platform/review/inbox) holds only the calls a human should make — and every result becomes evidence that sharpens the next recommendation.
***
## The studio
Three surfaces at [studio.remyx.ai](https://studio.remyx.ai), scoped to your active project:
* **Inbox** — what's waiting on you. Review a draft, decide a held discussion, fix a blocked agent. Nothing else.
* **Agents** — your fleet, one agent per repo: runs, learned direction, configuration.
* **Reports** — what the fleet produced, what it cost, where it needs attention.
Install, run, pause; per-agent runs, direction, and evidence.
Review / Decide / Fix cards that clear themselves when GitHub does.
Fleet KPIs, deliverable pipeline, cost per code artifact.
The research index your agents draw from — search is public, no sign-in.
The namespace each initiative runs in: team, agents, sources — plus the Pro quality gate.
GitHub, model providers, and the signal sources that feed each decision.
***
## Security that fits how you already work
Remyx runs server-side through a scoped GitHub App. Access is per repo and revocable, your keys never touch repo secrets unprotected, review mode is the default, and a human gates every merge. See [Connectors](/platform/configure/connectors#github) and [Account & keys](/platform/configure/account).
***
## Access Remyx
**The web app** — Inbox, Research, Agents, Projects, Reports, Connectors.
```bash theme={null}
pip install remyxai
remyxai outrider init --repo owner/name --auto-interest
```
**For Claude Code and other MCP clients** — drive the fleet, triage the inbox, and search research from your editor.
```json theme={null}
{
"mcpServers": {
"remyx": {
"type": "http",
"url": "https://mcp.remyx.ai/mcp",
"headers": {
"Authorization": "Bearer YOUR_REMYX_API_KEY"
}
}
}
}
```
**REST access to the engine** at `engine.remyx.ai/api/v1.0`.
***
## Learn more
Your first run, in minutes
What runs in your repo's GitHub Actions
The scientific method for AI teams
***
## Community
@remyxai
Email [contact@remyx.ai](mailto:contact@remyx.ai)
# Model backends
Source: https://docs.remyx.ai/platform/agents/backends
Two axes: which coding agent runs the implementation, and which model backend serves it, plus the two-tier drafter/refiner pattern
Outrider's coding runtime speaks whichever model-API family its [coding agent](#the-agent-input) speaks, so it can be pointed at any compatible endpoint. That makes the model backend a first-class choice per agent. The loop, guardrails, and refinement chain are identical on every backend; what changes is cost, latency, and the models on offer:
| Backend | `provider` value | Secret | Base URL | Default model | Recommended `agent-timeout` |
| ---------------------- | ---------------- | -------------------- | ----------------------------------------------------- | ------------------- | --------------------------- |
| **Anthropic (Claude)** | `anthropic` | `ANTHROPIC_API_KEY` | api.anthropic.com | `claude-opus-4-8` | action default (`1500`) |
| **Moonshot (Kimi)** | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/anthropic` (`/v1` for Codex) | `kimi-k3` | `3600` |
| **OpenAI** | `openai` | `OPENAI_API_KEY` | api.openai.com | *(the agent's own)* | action default (`1500`) |
| **OpenRouter** | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api` (`/v1` for Codex) | *(the agent's own)* | `3600` |
| **Z.ai (GLM)** | `zai` | `ZAI_API_KEY` | `https://api.z.ai/api/anthropic` | `glm-5.3` | `3600` |
| **Custom / on-prem** | `custom` | your choice | set `model-base-url` | varies | depends on backend |
An unset `provider` leaves the caller's own auth untouched, which is what installs written before the input existed rely on. Which providers a run can reach depends on the [coding agent](#the-agent-input) it runs: an agent speaks one model-API family and a provider serves one or more. OpenRouter serves every agent.
The higher timeout for Z.ai and Moonshot exists because their flagship models' thinking modes add per-turn latency; a GLM session routinely takes 15–20 minutes. The `provider` and `model` inputs are independent: `provider` names the endpoint, `model` picks the model from it (e.g. `glm-5.3` / `glm-5.2` / `glm-4.6` on Z.ai; `kimi-k3` / `kimi-k2.7-code` on Moonshot; `claude-opus-4-8` / `claude-sonnet-4-6` / `claude-haiku-4-5` on Anthropic).
In the app, the agent and its backend are set in the [agent setup wizard](/platform/agents/overview#set-up-an-agent) and editable later under the agent's [Configuration](/platform/agents/overview#configuration). Connect each key once under [Connectors](/platform/configure/connectors); Remyx pushes it to the repo as a secret at provisioning time.
***
## The `provider` input
In workflow YAML, one input wires the whole backend — auth env var, base URL, and mutual exclusion between the two auth styles:
```yaml theme={null}
- uses: remyxai/outrider@v1
with:
interest-id: ${{ vars.REMYX_INTEREST_ID }}
provider: ${{ inputs.provider }} # anthropic | openai | zai | moonshot | openrouter | custom
model: ${{ inputs.model }}
agent-timeout: 3600
env:
REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ZAI_API_KEY: ${{ secrets.ZAI_API_KEY }}
MOONSHOT_API_KEY: ${{ secrets.MOONSHOT_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
```
Pass every backend's secret in the step's `env:` block — the action reads only the one matching `provider`, so a single workflow can switch backends per dispatch. This input replaced the fork-side "Configure provider auth" case-switch step older workflows carried; an empty `provider` preserves the old behavior exactly, so existing installs need no migration. The action fails clean with a specific error if the selected provider's secret is missing (e.g. `provider=zai requires ZAI_API_KEY in the caller's env block`).
Don't set both `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` yourself: with both set the runtime sends *both* headers, so an unrelated Anthropic key travels to whichever gateway the run is pointed at. Naming a `provider` is what prevents it, and is most of why the input exists. The action builds the agent's launch environment explicitly and puts exactly one credential in it, the one that provider needs; nothing in a caller's workflow can add the other back.
Use `model-base-url` directly (instead of `provider`) only for custom or on-prem endpoints: Bedrock, Vertex, or a proxy of your own. Set alongside a named provider it still wins, since a gateway in front of that vendor is the usual reason to pass both. The run warns that it has overridden the registry endpoint and that the provider's credential is going there.
***
## The `agent` input
`provider` picks the model; `agent` picks the **coding-agent CLI** that drives the implementation. The two are independent axes. Each agent speaks one model-API family, so which providers it can reach follows from that:
| Agent | `agent` value | Speaks | Credential | Installed by the action |
| ------------------- | ------------- | ------------------ | --------------------------------------------------------------- | --------------------------------------------------- |
| **Backboard R-CLI** | `backboard` | its own router | `BACKBOARD_API_KEY`, which is also the model-routing credential | `curl -fsSL https://app.backboard.io/api/cli \| sh` |
| **Claude Code** | `claude` | Anthropic Messages | derived from the provider's key | `npm install -g @anthropic-ai/claude-code` |
| **OpenAI Codex** | `codex` | OpenAI Responses | derived from the provider's key | `npm install -g @openai/codex` |
Set the agent in the [setup wizard](/platform/agents/overview#set-up-an-agent), in the workflow file, from the [CLI](/cli#switch-the-coding-agent-for-a-run) (`--agent`), or per dispatch. An unset `agent` resolves to `claude`, so workflows written before the input existed keep running unchanged.
```yaml theme={null}
- uses: remyxai/outrider@v1
with:
interest-id: ${{ vars.REMYX_INTEREST_ID }}
agent: codex # claude | codex | backboard
provider: openai
model: gpt-5.4-mini
agent-timeout: 3600
env:
REMYX_API_KEY: ${{ secrets.REMYX_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
```
### Which pairs work
| `agent` | `provider` | Secret you set | Endpoint | Default model |
| ------------------ | ------------------------------- | -------------------- | ----------------------------------- | -------------------- |
| `backboard` | *(any, the router resolves it)* | `BACKBOARD_API_KEY` | | none; name a `model` |
| `claude` | `anthropic` | `ANTHROPIC_API_KEY` | vendor default | *(the agent's own)* |
| `claude` | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/anthropic` | `kimi-k3` |
| `claude` | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api` | *(the agent's own)* |
| `claude` | `zai` | `ZAI_API_KEY` | `https://api.z.ai/api/anthropic` | `glm-5.3` |
| `codex` | `moonshot` | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/v1` | `kimi-k3` |
| `codex` | `openai` | `OPENAI_API_KEY` | vendor default | *(the agent's own)* |
| `codex` | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/v1` | *(the agent's own)* |
| `claude` / `codex` | `custom` | your own | you set `model-base-url` | *(the agent's own)* |
Every pair above except `custom` has been verified end-to-end against the vendor. A pair whose two halves don't share an API family is refused before any work starts, naming the agent that does serve that provider:
```
agent=codex speaks openai-responses, which Anthropic does not serve.
Use agent=claude for this provider.
```
The pairs come from the action's own registry, published as [`agent-matrix.json`](https://github.com/remyxai/outrider/blob/main/docs/agent-matrix.json). The CLI and the engine read that contract instead of keeping their own copies, so the three stay in step.
OpenRouter reserves the requested `max_tokens` against your balance *before* calling the model, and both CLIs request a lot by default (Codex's is 131,072), so a thin balance answers HTTP 402 before the model is reached.
### What each agent can report
An agent can be partially capable and still usable: the affected telemetry degrades, the run doesn't fail.
| Capability | `backboard` | `claude` | `codex` |
| ----------------------------------------------------- | ----------- | -------- | -------------------- |
| `cost_usd`, the CLI's own dollar figure | ✓ | ✓ | |
| `turn_cap`, a round limit besides the timeout | | ✓ | |
| `guardrail_policy`, the injection-hardening tool gate | | ✓ | |
| `web_research`, the staged research pass | ✓ | ✓ | vendor endpoint only |
| `output_schema`, schema-constrained verdict passes | | | ✓ |
| `stream_transcript`, `token_usage`, `oneshot_json` | ✓ | ✓ | ✓ |
What a missing capability costs:
| Missing | Effect on the run |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `cost_usd` | Cost resolves from the per-model rate table instead; with no rate row the run reports `unavailable` instead of \$0.00. Token counts stay exact either way |
| `turn_cap` | `agent-timeout` becomes the only bound on spend, so keep it tight on cron-driven installs |
| `guardrail_policy` | The tool gate is not in effect and the run logs a warning; the post-hoc diff validators still apply |
| `web_research` | The staged research phase is skipped; the coding session runs without web context |
| `output_schema` | Verdict passes fall back to extracting JSON from the model's prose |
### Models, per agent
All three agents take the same `model` input; the action sets that agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, `BACKBOARD_MODEL`). Backboard addresses models as `/` and has no default of its own, so name a `model` with it. The action composes `provider: openai` plus `model: gpt-5.4-mini` into `openai/gpt-5.4-mini`, and leaves an already-qualified id alone.
Routing Codex off OpenAI's own endpoint pins two request fields for you, because Codex fills them from its own model catalog and an unrecognized model leaves them in a shape strict gateways reject: `web_search="disabled"` (the server-side tool is OpenAI's, not part of the protocol) and `model_reasoning_effort="medium"` (OpenRouter answers `Reasoning is mandatory for this endpoint` without it). Neither is applied on OpenAI, where Codex's per-model defaults win.
A Chat-Completions-only provider needs a translating gateway in front of it, since `codex exec` requires a **Responses** endpoint. Z.ai is the case people hit first: it serves Anthropic Messages, and its OpenAI-compatible surface is Chat Completions (`/v4/responses` answers 404), so `agent: codex` with `provider: zai` is refused rather than failing mid-run. GLM under Codex goes through a router that speaks Responses, `provider: openrouter` with `model: z-ai/glm-5.3`, or through your own gateway on `provider: custom`.
### Verify a pairing before you spend a run
`mode: smoke` resolves the routing, runs the agent's preflight, and makes one trivial call in about 30 seconds, without cloning the repo or opening anything. It reaches the vendor for real, so it catches a wrong secret, a bad endpoint, a model id from another vendor's vocabulary, or an exhausted quota. The run reports `smoke_ok` or `smoke_failed`:
```bash theme={null}
remyxai outrider trigger --repo owner/name \
--mode smoke --agent codex --provider openrouter --model openai/gpt-5.4-mini
```
Run it after changing `agent`, `provider`, `model`, or a secret, instead of waiting on a full dispatch to find out. A `setup-local` workflow declares `mode`, `agent`, `provider`, and `model` as dispatch inputs, so the same check runs from the Actions tab or `gh workflow run`.
Every key above can be connected once under [Connectors](/platform/configure/connectors), and Remyx pushes it to the repo at provisioning time. A [`setup-local`](/cli#install-without-the-github-app) install skips that path and sets the secret itself:
```bash theme={null}
remyxai outrider set-provider-secret --repo owner/name --provider openrouter --key-from ~/key
gh secret set BACKBOARD_API_KEY --repo owner/name < ~/backboard-key
```
***
## The two-tier drafter-refiner pattern
The recommended default for a repo you want worked continuously. It splits exploration and commitment into two workflows:
**Workflow 1: Drafter** — daily, budget tier, `publish: branch`
* Pushes draft branches only: no PRs, zero maintainer-facing noise. \~\$0.20–0.40/run.
* Each run accumulates repo intel — landing zones that worked, mechanism shapes that got rejected.
**Workflow 2: Refiner** — weekly, strong tier, `publish: pr`
* Picks the best drafter branch and generates a mechanism-level **gap analysis** against its source paper.
* Re-runs from that branch (`start-from-ref`) with the gap analysis as its brief (`lead-content`), through the full refinement chain, to a draft PR. \~\$5–10/week including the gap generation.
Why the split works: the drafter's branch anchors an extension point and test scaffolding, so the refiner commits to PR-shape on borderline cases; a budget tier makes daily exploration of the arXiv frontier (\~200 ML papers/day) rational; even failed branches contribute negative-space signal to the next pick.
Set it up from the wizard's **Advanced — models per phase → Two-tier** option (writes three workflows: the dispatch-capable main one, the drafter, the refiner), or from the CLI. Tier pairings are cross-provider and switchable — the app defaults to a GLM drafter with an Opus refiner:
```bash theme={null}
remyxai outrider setup-local --repo owner/name --auto-interest \
--two-tier --drafter-model glm-5.3 --refiner-model claude-opus-4-8
```
Runs report their `tier` (main / drafter / refiner) in fleet telemetry, so [Reports → Costs](/platform/review/reports#costs) prices each tier separately.
***
## Cost telemetry
Every run reports `cost_usd`, `input_tokens`, and `output_tokens` as action outputs and into fleet telemetry. How the dollar figure is computed depends on the backend:
* **Anthropic**: the runtime's own `total_cost_usd` figure is authoritative.
* **Z.ai / Moonshot**: cost is computed from a per-model rate table keyed by the model actually used (`glm-5.2`, `glm-4.6`, `kimi-k3`, `kimi-k2.7-code`, `kimi-k2.7-code-highspeed`, …), overriding the CLI's Anthropic-rate estimate. An unknown model on a known host is priced at that host's flagship rate and reported as approximated, which can be a 3–4× error across tier pairs, so set `model` explicitly. `glm-5.3` sits there today: it's the `zai` default and has no rate row yet.
* **Custom endpoints**: the reported figure is an estimate.
* **Agents differ in what they report.** One that reports its own dollar figure is used directly; one that reports none resolves through the rate table, and reports `unavailable` when the host has no rate row. Token counts stay exact in every case. The [capability table](#what-each-agent-can-report) says which is which.
Each run's step summary names the pair it ran, such as `Agent: Codex → OpenRouter`, alongside the cost basis it used.
The [Costs report](/platform/review/reports#costs) turns this into unit economics: runs per day by backend against **\$ per code artifact**.
***
## Choosing a backend
There's no house pick. Every backend runs the same loop behind the same guardrails; the practical variables are per-run cost, session latency, and the draft quality you observe on *your* repo:
* **Anthropic (Claude)** — Claude tiers over `x-api-key` auth; a full PR run with the refinement chain is roughly \$5–6.
* **Z.ai (GLM)** — GLM tiers at roughly 20× lower per-run cost; budget the longer timeout for thinking-mode latency.
* **Moonshot (Kimi)** — Kimi tiers, same Bearer-auth pattern and timeout guidance as Z.ai.
* **Custom / on-prem** — anything that speaks the API family your agent speaks, via `model-base-url`.
The cheap way to decide is evidence, not defaults: per-dispatch routing (`remyxai outrider trigger --provider … --model …`) makes A/B-ing providers on the same paper nearly free, and [Reports → Costs](/platform/review/reports#costs) shows each backend's cost per code artifact against what its runs actually produced. The [two-tier pattern](#the-two-tier-drafter-refiner-pattern) sidesteps the choice entirely: explore on a budget tier daily, promote on the tier your merges say you trust.
***
## Related
All configuration inputs, guardrails, and run outcomes
Connect Anthropic, Z.ai, and Moonshot keys
Per-dispatch provider/model overrides
The action's own backends reference
# Actions
Source: https://docs.remyx.ai/platform/agents/outrider-action
Remyx agents ship as actions that run in your own CI. Outrider, recent research into draft PRs, is the first.
**Set up from:** [Agents > + Set up an agent](/platform/agents/overview#set-up-an-agent) or the [`remyxai` CLI](/cli) | **Runs in:** your repo's GitHub Actions
Remyx agents ship as **actions** — workflows that run in your own repo's CI, on your own keys, reporting evidence back to the platform. **Outrider** is the first, one of more to come: a GitHub Action that discovers ideas that fit your codebase, drafts PRs for the most actionable changes, and files issues for ideas worth watching. This page documents that runtime — what a run does, its configuration inputs, guardrails, costs, and outcomes. The telemetry every run reports is what powers the [agent report](/platform/agents/overview#the-agent-report), [Reports](/platform/review/reports), and the [Inbox](/platform/review/inbox).
![A scheduled action opening a draft PR that wires a paper into an existing call site]()
The success of an AI application depends on continuously adopting the right improvements. Identifying what to build next requires constant monitoring and evaluation of new ideas found in papers, repositories, models, benchmarks, and community discussions. Outrider automates that process by discovering new ideas, validating them against your codebase, and proposing actionable pull requests and issues—so your team can focus on building instead of scouting.
On a schedule you set, Outrider:
* **Surfaces** new work relevant to your repo from the sources Remyx indexes, ranked against what your team has actually shipped.
* **Validates fit** against your real codebase — is there a call site, and is the change architecturally aligned (a refactor, a new technique, an upgrade)?
* **Drafts the most actionable candidate** into a reviewable PR, wired into an existing module with code and tests.
* **Files an Issue instead** for ideas worth watching that aren't a clean fit yet, or that need input from the team or the source.
Outrider helps push development in the most fruitful directions without paying the upfront cost of finding and vetting each idea manually. Spend your time reviewing concrete proposals.
**Human-reviewed by design.** Outrider proposes changes as draft PRs and Issues, but every decision to merge remains with your team.
***
## Quickstart
You'll need to connect Remyx to your GitHub account via the [Connectors](/platform/configure/connectors) view. The [**GitHub**](/platform/configure/connectors#github) connector will install the Remyx GitHub App in your account or organization. Make sure to include access to your desired repo(s).
Connecting a [**model provider**](/platform/configure/connectors#model-providers) is required to draft implemented PRs, billed to your provider account.
For the CLI, also generate a [Remyx API key](/platform/configure/account#api-access).
Either path provisions the workflow and fires the first run. A draft PR or Issue lands in minutes.
### From the CLI
```bash theme={null}
pip install remyxai
# authenticate the CLI with your Remyx API key
export REMYXAI_API_KEY=
# install Outrider on the repo and auto-create a Research Interest from it
remyxai outrider init --repo owner/name --auto-interest
```
`--auto-interest` creates a Research Interest from the repo; drop it and pass `--interest ` to use an existing one. If the GitHub App isn't installed yet, the command prints the install link.
**No GitHub App?** `setup-local` provisions with your own authenticated `gh` CLI instead; PRs then come from your account, not **remyx\[bot]**. ([details](/cli#install-without-the-github-app))
```bash theme={null}
remyxai outrider setup-local --repo owner/name --auto-interest
```
### From the UI
Use [**+ Set up an agent**](/platform/agents/overview#set-up-an-agent) on the Agents page (or create a [Research Interest from your repo](/platform/discover/feed#research-interests-from-a-repo) and choose **Set it up for me**) — no terminal required.
### Identity and install
Either path pins the marketplace action at `remyxai/outrider@v1`. A same-repo install needs no `github-token` — the action self-mints a **remyx\[bot]** App token at runtime. Commits land authored by the bot (GitHub identity remyx\[bot]) and show as **Verified**.
You can also converse with a run by mentioning `@remyx-ai` on the PR or Issue; replies are read-only — see [Deep Research](/platform/discover/briefs).
Outrider's artifacts carry [Open Knowledge Format](https://github.com/remyxai/outrider) frontmatter. A repo-root `CONTEXT.md` and agent-instruction files (`CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`) sharpen its picks.
***
## How it works
A run has two stages. The Remyx engine ranks a candidate pool server-side, then Outrider validates candidates against a clone of your repo and decides what to open.
**Ranking.** The pool is ranked from three inputs: your interest's context, your team's shipping history (extracted from the merge log), and a learned preference model. Candidates that match the direction your team has shipped rank above topically similar ones.
**Validation and decision.** A high relevance score does not mean a candidate is implementable; the top-ranked paper often has no call site in your repo. A selection pass checks each candidate against the codebase (code search and file reads) and picks the most implementable; if none fit, the run skips. Filters that do not call the model run first, so a run that cannot produce a useful result exits before any LLM cost. Each run ends in one of three outcomes: a draft PR, an Issue, or a skip.
![The Outrider pipeline: external sources and your repo feed the Remyx engine's ranking; the action fetches the candidate pool, gates it, selects the most implementable candidate, implements within guardrails, validates, and routes to a draft PR, an Issue, or a skip — with the refinement chain promoting PRs and run telemetry feeding back into ranking]()
The candidate integrates into an existing call site and passes the validators (tests, stub density, reachability). Outrider opens a draft PR with code and tests.
The candidate is relevant but not directly implementable (no call site, too many stubs, or unreachable code). Outrider opens an Issue with the attempted diff.
No candidate cleared the confidence, dedup, or fit gates. No artifact is created; the run records a [status](#run-outcomes).
***
## Refinement-pass chain
When `chain` is `true` (the default), opening a draft PR isn't the end of the run. Outrider keeps going in the same run and walks the draft through three phases before it asks for review:
Diffs the draft against the paper's reference implementation and emits a **coverage matrix** — what the diff *covers*, what it *defers*, and where it *deviates* from the source. When no public reference exists, it anchors the audit against the arXiv abstract instead; that's lower precision and the PR body says so.
Rewrites the PR body to the repo's canonical PR template, folds Outrider's own scaffolding into a collapsible **Discovery context** block, and applies contributor conventions learned from recent merged PRs. Runs `ruff` auto-fix and force-pushes the result as the bot.
Runs lint plus a focused `pytest` on the touched files. On a pass it flips the draft to **ready-for-review**; on failure the PR stays a draft for you to take over.
The Issue path gets a convention pass too: `mode: issue-convention` folds an Issue body to the repo's `ISSUE_TEMPLATE` shape.
Each phase reports its own Action output so you can gate downstream steps on them:
| Output | Set by |
| ------------------------------- | ---------------------------------------------------------- |
| `chain_fidelity_status` | Fidelity phase |
| `chain_convention_status` | Convention phase |
| `chain_test_status` | Test phase |
| `chain_issue_convention_status` | `issue-convention` pass on the Issue path |
| `chain_draft_dropped` | `true` if a phase dropped the draft rather than promote it |
Set `chain: false` to stop after the first draft — Outrider opens the PR exactly as it first wrote it and runs none of the phases above. That's the cheaper recommend-only mode.
***
## Configuration
The defaults are conservative; most repos only touch `interest-id` and the schedule.
| Input | Default | What it does |
| ------------------------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `interest-id` | *(required)* | The Research Interest UUID the run draws candidates from |
| `mode` | `recommend` | What the run produces: `recommend` (rank → validate → PR/Issue), `smoke` (verify the run's `agent` / `provider` / `model` wiring against the vendor in \~30s and exit, without cloning the repo or opening anything; see [Verify a pairing](/platform/agents/backends#verify-a-pairing-before-you-spend-a-run)), `weekly-summary` (post a digest, see [Weekly digest](#weekly-digest)), `fidelity` / `convention` / `test` (run a single [refinement phase](#refinement-pass-chain) standalone), or `issue-convention` (fold an Issue body to the repo's `ISSUE_TEMPLATE`) |
| `chain` | `true` | After opening a draft PR, run the [refinement chain](#refinement-pass-chain) inline in the same run (fidelity → convention → test). Set `false` for recommend-only — cheaper, leaves the draft as Outrider first wrote it |
| `min-confidence` | `moderate` | Skip recommendations below `high` / `moderate` / `low` |
| `draft-mode` | `always` | `always` (draft), `on_test_failure` (ready if tests pass), or `never` (skip on test failure) |
| `rate-limit-days` | `7` | Time-decayed cadence guard. Skip only if the most recently opened **open** Remyx artifact is younger than N days; older open artifacts age out and stop blocking. Merging or closing an artifact clears the guard immediately. `0` disables |
| `lookback` | `week` | Candidate pool window: `today` / `week` / `month` |
| `candidate-pool` | `25` | How many candidates the selection pass chooses from |
| `test-integration-policy` | `strict` | `strict` demotes to an Issue if new tests don't import an existing module; `soft` opens a draft with a warning; `off` skips the gate. Use `soft` for component/library repos where a standalone module *is* the contribution |
| `guardrails-blocklist` | `''` | Path globs the agent must **not** touch (e.g. `secrets/**`, `*.lock`), added on top of the fixed default block (`.github/workflows/**`). Blocklist matches always win. (`guardrails-allowlist` is deprecated as of v1.7.24 — path access is permissive-by-default and the old input is a no-op) |
| `publish` | `pr` | `pr` opens a draft PR and runs the inline chain; `branch` pushes the branch only — no PR, no chain, zero maintainer-facing artifact until the team opens the PR themselves. Branch mode is the drafter tier of the [two-tier pattern](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) |
| `agent` | `''` | Which coding-agent CLI drives the implementation: `backboard` (Backboard R-CLI), `claude` (Claude Code), or `codex` (OpenAI Codex); unset resolves to `claude`. A separate axis from `provider`, which picks the model. An agent speaks one API family, so an impossible pair is refused up front, naming the agent that does serve your provider. See [the `agent` input](/platform/agents/backends#the-agent-input) |
| `provider` | `''` | Route the coding agent at a named backend: `anthropic`, `openai`, `zai`, `moonshot`, `openrouter`, or `custom`. Resolves that vendor's credential, endpoint, and auth style, and hands the agent exactly one credential; pass each backend's secret in the step's `env:`. Empty preserves legacy behavior. See [Model backends](/platform/agents/backends#the-provider-input) |
| `model` | `''` | Model to request from the provider (e.g. `claude-opus-4-8`, `glm-5.3`, `kimi-k3`, `gpt-5.4-mini`), forwarded as the selected agent's own model variable (`ANTHROPIC_MODEL`, `CODEX_MODEL`, `BACKBOARD_MODEL`); also keys the per-model cost rate table. Required for `agent: backboard`, which has no default of its own |
| `start-from-ref` | `''` | Branch/tag/SHA on the target fork to start the coding session from — the agent sees that ref's diff already applied, enabling refinement runs on a prior draft. The result pushes to a `-refined` branch |
| `lead-content` | `''` | A scoped implementation spec (e.g. a gap analysis, or a LEAD Issue's content — Linear URLs resolve via `linear-api-key`) that replaces the paper's suggested experiment in the run's SPEC.md |
| `fidelity-policy` | `block` | Pre-PR fidelity gate: `block` (one scoped patch attempt on a needs-judgment verdict, then skip — keeps hallucinated implementations out of public PRs), `advisory` (audit attaches its coverage matrix but never blocks — best for curated warm-start runs), or `off` |
| `staged-synthesis` | `false` | Opt-in two-stage session: a bounded research pass first (target-repo state, prior attempts, conventions), then coding reads its findings. Adds \~\$0.15–0.50 and \~1–3 min |
| `maintain-state` | `false` | Opt-in per-fork cross-run learning via `.remyx/repo_intel.yaml`: confirmed landing zones, rejected mechanism shapes, exploration budget |
| `enable-cocoindex` | `true` | Installs AST-based semantic code search as a runner skill, grounding call-site claims on real paths (adds \~60–90s and \~1 GB of deps to the run; also powers the "Convention precedents" enrichment on Issue bodies). Disable to slim the runner |
| `agent-timeout` | *(falls back to `claude-timeout`)* | Wall-clock seconds for each coding-agent phase; every phase shares the ceiling. The generalized name for `claude-timeout`: pass either, and this one wins if both are set. For an agent with no `turn_cap` ([capability table](/platform/agents/backends#what-each-agent-can-report)) it is the **only** bound on spend, so keep it tight on cron-driven installs |
| `claude-timeout` | `1500` | The original name for the same ceiling, supported indefinitely. Raise to `3600` for backends whose thinking modes add per-turn latency, lower to cap cost |
| `pin-arxiv` | `''` | Implement one specific `arxiv_id` and skip the selection pass, for reproducible re-runs and warm starts. Falls back to fetching the paper directly from arxiv.org when it isn't in the Remyx catalog (`skipped_pin_arxiv_unresolvable` if neither resolves). Mutually exclusive with `search-method` |
| `search-method` | `''` | A free-text method query. Resolves the top match via engine search and implements it, bypassing the candidate pool and the selection pass. An explicit pin also overrides the don't-re-recommend throttle, so you can re-run a paper that already has an open artifact. Mutually exclusive with `pin-arxiv`. (The CLI exposes this as `remyxai outrider trigger --pin-method`) |
| `model-base-url` | `''` | Override the selected agent's base-URL variable (`ANTHROPIC_BASE_URL` or `CODEX_BASE_URL`) to reach a **custom** endpoint that speaks the agent's API family (AWS Bedrock, GCP Vertex, an on-prem proxy). For the known backends prefer the `provider` input; set alongside a named provider it still wins, with a warning that it has overridden the registry endpoint and that the provider's credential is going there. Ignored for a native router, whose base-URL variable addresses its own control plane. Cost telemetry becomes an estimate. See [Model backends](/platform/agents/backends) |
### Guardrails
Outrider constrains what the agent may change so a draft PR is reviewable:
* **Fixed block:** `.github/workflows/**` is always off-limits — it guards against agent-authored edits that would silently expand the run's own future agency. `guardrails-blocklist` adds your own globs on top (e.g. `secrets/**`, `*.lock`, `Dockerfile*`), and blocklist matches always win.
* **Integration checks** (post-session, pre-push): at most 3 new `.py` files per run, and at least one newly added function or class must be invoked from another changed file — an import alone doesn't count. Violations terminate the run before any push (`rejected_path_violations`); orphan code downgrades the run to an Issue.
* **Validators:** stub-density, the test-integration gate, a self-review orphan check, a diff-risk score gate, and an outbound secret scan — each with its own Issue-route or abort status.
These gates are why a run picks an Issue over a PR when an honest implementation would be larger or riskier than a reviewable draft.
***
## Costs
You bring your own model-provider key ([any backend](/platform/agents/backends)), so model usage is billed to you:
* **Model usage:** on Anthropic, a full PR-route run **with the inline [chain](#refinement-pass-chain)** is about **\$5–6**; recommend-only (`chain: false`) is **\$1–3**. On the [Z.ai or Moonshot backends](/platform/agents/backends) a run is roughly 20× cheaper. A run that skips after the selection pass costs less still: it reads the repo to verify fit, then exits.
* **GitHub Actions:** \~6–8 min on `ubuntu-latest` per run.
* **Remyx API:** included in your subscription.
With the default time-decayed cadence guard, typical engagement is **≈\$2–4/month** of model spend. Each run reports `cost_usd`, `input_tokens`, and `output_tokens` as Action outputs; on Z.ai and Moonshot the cost is computed from a per-model rate table. [Reports → Costs](/platform/review/reports#costs) turns this telemetry into fleet unit economics, and the [two-tier drafter/refiner pattern](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) is the structured way to spend less per artifact.
### Weekly digest
Run a second weekly cron with `mode: weekly-summary` and a `weekly-discussion-id` input to post a 7-day digest to a GitHub Discussion: outcomes, verified costs, license distribution, and open Issues with their next actions. It requires the `discussions: write` permission and costs about **\$0.10–0.20/week**.
***
## Run outcomes
Every run ends in a `status` in the Action log. The common ones:
| Status | Meaning |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `pr_opened` / `pr_opened_draft` | A PR was opened (ready or draft) |
| `issue_opened*` | Routed to an Issue — pre-flight, no integration found, too many stubs, or orphan code |
| `skipped_low_confidence` | Best candidate below `min-confidence` |
| `skipped_open_artifact` | An open Remyx artifact is still younger than `rate-limit-days` (the time-decayed cadence guard) |
| `skipped_by_selection_verification` | No candidate structurally fit the repo |
| `skipped_pr_exists` / `skipped_issue_exists` | Every candidate already has an open artifact |
| `skipped_issues_disabled` | The Issue route was needed but Issues are disabled (common on a fork) — enable Issues on the repo |
| `aborted_secret_in_payload` | The outbound secret scanner tripped on the draft payload; nothing was opened |
| `smoke_ok` / `smoke_failed` | Result of a `mode: smoke` wiring check: whether the configured agent, provider, and model reached the vendor |
| `fidelity_*` | Outcome of the [chain](#refinement-pass-chain)'s fidelity phase |
| `issue_convention_*` | Outcome of the `issue-convention` pass on the Issue path |
The full status-code set is in the [action's README](https://github.com/remyxai/outrider#status-codes).
***
## Reviewing an Outrider PR
Treat it like a first-pass proposal from a teammate:
1. **Read the self-review** in the PR body — what the integration covers and what it left out.
2. **Run the diff and tests** locally; the action runs `pytest` + an integration check, but your full suite is the real bar.
3. **Decide** — merge, push follow-ups, or close. Your call feeds the agent's [Direction](/platform/agents/overview#direction) either way, and the [Inbox](/platform/review/inbox) card clears itself when GitHub records the merge or close.
Merged and closed PRs feed back into the agent's [Direction](/platform/agents/overview#direction) — every decision re-weights what it proposes next.
***
## Related
Install and manage agents from the app
The agent and provider inputs, which pairs work, and the two-tier pattern
The Remyx GitHub App and the remyx\[bot] identity
Action source, inputs, and full status-code reference
# Agents
Source: https://docs.remyx.ai/platform/agents/overview
Autonomous workers on your repos: set up, run, pause, and drill into each agent's runs, direction, and evidence
**Nav:** Agents | **URL:** [`/agents`](https://studio.remyx.ai/agents)
An **agent** is an autonomous worker installed on one repo. It does a job on a schedule, reports evidence for everything it does, and holds the calls a human should make for your [Inbox](/platform/review/inbox). **Outrider**, a GitHub Action that turns recent research into draft PRs, is the first flagship agent; more agents with other functions are coming.
The Agents view is the fleet index: every repo you've installed on, its state, and its last outcome, with **+ Set up an agent** to add another. Click any agent to open its report.
***
## Set up an agent
Click **+ Set up an agent**. The setup wizard is the in-app twin of `remyxai outrider init`:
![The Set up an agent wizard: repository picker with the Remyx GitHub App confirmed installed, the research-interest dropdown on 'Create one from this repo automatically', Review vs Auto mode radios, the coding-agent and model-backend selectors with their connected providers marked, and the Advanced models-per-phase fold with Two-tier selected]()
Pick from your GitHub repos or paste a URL. The wizard preflights the **Remyx GitHub App** on that repo — the App is how agents author PRs and Issues as **remyx\[bot]** (without it they'd come from `github-actions[bot]`). If it's missing, the wizard shows the one-click install link and unlocks automatically once GitHub records it.
**Create one from this repo automatically** (the default — Remyx derives the interest from the repo's code and history) or pick an existing active interest.
* **Review** — the setup PR and every draft wait for you.
* **Auto** — the setup PR merges itself; drafts still wait for you.
Two independent choices. The **coding agent** is the CLI that writes the implementation: **Backboard R-CLI**, **Claude Code**, or **OpenAI Codex**. The **model backend** is the vendor serving it: **Anthropic (Claude)**, **Moonshot AI (Kimi)**, **OpenAI**, **OpenRouter**, or **Z.ai (GLM)**.
The wizard preselects a connected provider and flags one that isn't connected yet. Each agent speaks one model-API family, so the backend list narrows to the providers that agent can reach; the [pair table](/platform/agents/backends#which-pairs-work) has the full set. Picking Backboard resolves the backend inside your Backboard account instead, and needs an explicit model.
Under **Advanced — models per phase** you can choose the [two-tier drafter/refiner layout](/platform/agents/backends#the-two-tier-drafter-refiner-pattern), whose tiers can run different agents as well as different models.
Provisioning issues the repo a scoped **agent key** (`rmxa_`), writes the workflow file, and opens a bot-authored **setup PR** — merge it (or let Auto mode merge it) to activate the agent.
```bash From the terminal theme={null}
remyxai outrider init --repo owner/name --auto-interest --mode review
```
***
## Provisioning states
Each agent row shows its status chip, inline **Run now** / **Pause** actions, and its last outcome:
![Agent rows in the fleet index: each repo with its status chip (active, or setup PR open linking to the PR), Run now and Pause actions, and the last outcome — a discussion, a skip with its reason, or 'Provisioned · awaiting first run']()
The chip is derived the same way the CLI's preflights reason about it:
| State | Meaning | What to do |
| ---------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------- |
| **setup PR open** | The bot-authored setup PR hasn't been merged | Merge the PR (the chip links to it) |
| **needs model key** | No key is connected for its agent/backend pair | Connect the one that pair needs under [Connectors](/platform/configure/connectors) |
| **awaiting first run** | Provisioned, no run dispatched yet | **Run now**, or wait for the schedule |
| **active** | Running on its schedule | Nothing — review its drafts in the [Inbox](/platform/review/inbox) |
| **paused** | Its key is revoked; the next run stops immediately | **Resume** re-provisions with a fresh key |
## Run now / Pause
* **Run now** dispatches a one-shot run outside the schedule. Available once the setup PR is merged and the agent isn't paused.
* **Pause** revokes the agent's key: the next run fails auth immediately. The workflow file stays in the repo; nothing else is touched. Resuming is a fresh re-provisioning (new key, new setup PR). Pausing here, revoking the agent key under [Account](/platform/configure/account#agent-keys), and the MCP `pause_agent` tool all do the same thing.
```bash From the terminal theme={null}
remyxai outrider trigger --repo owner/name
```
***
## The agent report
Click an agent to open its report at `/agents/`. It leads with a KPI strip and **Findings** (each finding deep-links into the evidence it came from) over a shared **date window** (default last 7 days, presets up to All time) that rides the URL, so any view is shareable. Two header buttons: **Ask about this agent** and **Configure**. Anything blocking the agent is a banner up front; healthy agents get **Run now**.
![One agent's report: KPI strip (27 dispatches, 1 draft opened, 2 leads awaiting a call, $17.04 spend at $5.68 per artifact), findings such as '7 runs failed before reporting' and ''already tried / duplicate' dominates demotions' — each with an evidence link — and the Evidence list of Runs, Direction, and Activity trace sections]()
### Runs
Every run the agent has reported, in weekly rollups, filterable by outcome and searchable by title, arXiv id, or status:
| Outcome | Meaning |
| -------------- | ------------------------------------------------------------------------------ |
| **Draft PR** | The run opened a draft pull request |
| **Discussion** | The run opened an Issue — relevant, but safer to discuss first |
| **Branch** | The run pushed a draft branch; refinement promotes it to a PR |
| **Failed** | The run errored — see **Analyze** |
| **Skipped** | The run held publication (low confidence, cadence guard, no structural fit, …) |
Each row shows the artifacts, the run's **cost**, and an **Analyze ›** action that pulls the run's job log and returns a root-cause read with a suggested next step.
### Direction
What the agent has learned from your decisions. Every merge, skip, and discussion re-weights what it proposes next:
| Marker | Meaning |
| -------------- | ------------------------------------------------- |
| **↑ Promoted** | A preference your merges have strengthened |
| **↓ Demoted** | A direction your skips and closures have weakened |
| **→ Held** | Signal exists but isn't decisive yet |
Each direction is evidence-backed: expand it to see the specific PRs, Issues, and runs it was learned from.
### Activity trace
The agent's event stream, grouped by week: **Merged**, **Closed**, **Drafts**, **Discussed**, **Failed runs**, **Skipped** — each event linked to its artifact, with skip reasons attached.
### Configuration
The agent's operating setup, editable in place: repository, interest, workflow file, setup PR, **coding agent** and **model backend** (changing either re-provisions with a fresh setup PR), and status with **Pause** / **Resume**. A single run can override both without re-provisioning, via `remyxai outrider trigger --agent --provider`. The **Signal** panel lists what feeds this agent's decisions: GitHub merge history (the ranking baseline), the repo's [research brief](/platform/discover/briefs), validation runs, Linear tickets (interfaces under active work route risky candidates to discussion), and, when connected, A/B ship-rate feedback.
### Ask about this agent
A chat drawer grounded in the agent's real data (runs, telemetry, failing-run log tails, open artifacts, direction) via read-only tools. It explains; actions like triggering or pausing stay behind the app's confirm-gated buttons. [Inbox](/platform/review/inbox) **Ask** links land in this same drawer, focused on the relevant run.
***
## Where agents surface elsewhere
* The fleet rolls up in [Reports](/platform/review/reports); items needing you land in the [Inbox](/platform/review/inbox).
* An agent can belong to several [Projects](/platform/configure/projects) — link it under the project's **Agents** section.
* The full fleet is scriptable via the [MCP tool plane](/resources/mcp-server#outrider-fleet-tools) and the [REST API](/api-reference/api-reference#outrider-agents).
***
## Related
What actually runs in the repo's GitHub Actions
The coding agent and model backend axes, and the two-tier pattern
Where an agent's drafts and held calls wait for you
The fleet in one view
# Account & keys
Source: https://docs.remyx.ai/platform/configure/account
Profile, teams, API keys, agent keys, and where Remyx reaches you
**Nav:** Sidebar > Account (bottom left) | **URL:** [`/account`](https://studio.remyx.ai/account)
The Account page manages your profile, teams, API and agent keys, and email alerts.
***
## Profile
Your avatar, name, and email come from your sign-in; your **title** (e.g. "Staff ML Engineer") is editable and saved as you type.
***
## Teams
Projects belong to a team; owners and admins manage members and invitations. Expand a team to manage member roles (**admin**, **member**, **read-only**), invite by email, and see pending invites. A team's only admin must promote someone else before stepping down, and a team's projects must be deleted or re-assigned before the team can be deleted.
***
## API keys
Named keys authenticate the CLI, the MCP server, and GitHub Actions — passed as a Bearer token. Keys are shown **once** at creation (Remyx stores only a hash and cannot recover the plaintext) and can be scoped, revoked, or deleted at any time. Create one key per machine or workload so you can revoke them independently.
### Key types
| Prefix | Type | Use it for |
| -------- | ---------------------- | ----------------------------------------------------------- |
| `rmxu_…` | **Personal** | Your own CLI, MCP server, and ad-hoc REST calls |
| `rmxa_…` | **Agent (automation)** | Unattended automation — see [Agent keys](#agent-keys) below |
### Creating a key
1. Click **+ Create API key** and name it (e.g. `laptop-cli`, `ci-experiments`).
2. Pick an expiry: **Never expires**, 30 days, 90 days, or 1 year.
3. Choose access: **Full access**, or **Limited — pick per feature** with per-category read/write grants (GitHub-style; "write" includes the matching read). One-click presets — Read-only, Outrider setup, Experiment CI, Digest bot — fill the grid for common workloads. See [API Scopes](/api-reference/scopes) for the catalog.
4. Copy the key from the show-once modal and use it as `Authorization: Bearer `.
### Managing keys
Each row shows the key's access summary, created / last-used / expiry dates, and its display id (`rmxu_…last4`). **Revoke** disables a key immediately across the REST API, CLI, and MCP server — it stays listed for audit until you **Delete** it. To rotate, create a new key and revoke the old one; keys are immutable once created.
```bash theme={null}
# CLI authentication (personal rmxu_ key)
export REMYXAI_API_KEY=rmxu_xxxxxxxxxxxxxxxxxxxxxxxx
# REST
curl -H "Authorization: Bearer rmxu_xxxxxxxxxxxxxxxxxxxxxxxx" \
https://engine.remyx.ai/api/v1.0/interests
```
***
## Agent keys
Agent keys (`rmxa_`) are **created and rotated by the agent's Action for each repo** when you provision an agent — you don't mint them by hand. Their scopes are managed by provisioning and can't be edited (rename only).
**Revoking an agent key pauses that agent immediately** — its next run fails auth. This is the same operation as **Pause** on the [Agents](/platform/agents/overview#run-now--pause) page; resuming requires re-provisioning.
***
## Alerts
Your [Inbox](/platform/review/inbox) is the source of truth; alerts arrive by email, and each teammate sets their own:
| Alert | Cadence |
| ------------------------------- | ------------------------------------------------------------ |
| **Review — Draft PRs ready** | Email when a draft opens on one of your agents |
| **Decide — Discussions opened** | Daily digest — these can wait for your morning |
| **Run failures** | Immediately when an agent errors |
| **Weekly portfolio summary** | For leads: hit rates, direction signal, and stalled projects |
| **Monthly Mix newsletter** | Product updates and ML news |
***
## Danger zone
**Delete this account** — this deletes everything, immediately:
* Every API and agent key is revoked — agents stop mid-loop.
* All connected integrations are disconnected.
* Your interests, projects, and history are removed.
Type `DELETE` to confirm. It cannot be undone.
***
## Security best practices
* **One narrowly-scoped key per use.** Grant only the scopes a tool needs; don't reuse one broad key everywhere.
* **Rotate by replacement** — create a new key, revoke the old.
* **Never commit tokens.** Use environment variables, `.env` files (with a loader), or CI secret stores.
* Keys authenticate as you: any action taken with a key is attributed to your account, gated by that key's scopes.
***
## Related
Scope catalog, presets, and key prefixes
What provisions and uses agent keys
Model-provider keys live there, not here
Set up the CLI with a personal key
# Connectors
Source: https://docs.remyx.ai/platform/configure/connectors
Connect GitHub, model providers, and signal sources — every source feeds a specific part of the next decision
**Nav:** Manage > Connectors | **URL:** [`/connectors`](https://studio.remyx.ai/connectors)
Every source feeds a specific part of the next decision: GitHub sets the ranking baseline, a model provider powers the drafts, Linear and Jira flag interfaces under active work, W\&B feeds the gate. Connect each account once here; reference it per-project under [Projects → Sources](/platform/configure/projects#sources).
Each connector card shows a status chip — **connected**, **not connected**, **enterprise**, or **coming soon** — and connected cards offer **Test connection**, **Replace credentials** (for key-based connectors), and **Disconnect**.
***
## Catalog
| Category | Connectors |
| ------------------------ | ---------------------------------------------------------------------- |
| **Code** | GitHub |
| **LLM providers** | Anthropic (Claude), Moonshot AI (Kimi), OpenAI, OpenRouter, Z.ai (GLM) |
| **Coding agents** | Backboard R-CLI |
| **Project management** | Linear, Jira |
| **Communication** | Slack |
| **ML lifecycle** | Hugging Face |
| **Observability & eval** | Weights & Biases |
| **Compute** | Modal |
| **Coming soon** | Langfuse, Arize, LaunchDarkly, Statsig |
Enterprise connectors (MLflow, Databricks, Snowflake, AWS, GCP, Azure) are private-network setups done with the Remyx team — use the **Contact us** card.
***
## GitHub
**What it feeds:** the ranking baseline — every merge and revert re-weights the candidate pool — plus the write path for everything agents author.
**Setup:**
1. Click **Connect** on the GitHub card.
2. Install the **Remyx GitHub App** on your organization or account.
3. Select the repositories to grant access to.
The connector is a **GitHub App**, not a plain OAuth grant. Remyx authenticates as the App and mints short-lived, repo-scoped tokens on demand; the App's private key never leaves the Remyx engine. The base token covers `contents`, `pull-requests`, `issues`, and `workflow` (needed to install the agent workflow), and permissions like `discussions:write` are layered on only when your installation actually granted them.
### The `remyx[bot]` identity
Because the connector is a GitHub App, everything Remyx does on your repos is authored by **remyx\[bot]** — setup PRs, draft PRs, Issues — never as you, so it's clear at a glance which artifacts an agent originated. Commits show as **Verified**. The bot also answers `@remyx-ai` mentions on agent-opened PRs and Issues with grounded, **read-only** replies; see [Briefs](/platform/discover/briefs#the-remyx-bot).
The in-app [Quickstart](/quickstart) uses a repo-scoped OAuth connection (it needs fork rights for the demo repo); agent setup outside the quickstart runs on the App. If both exist, the App is preferred.
***
## Model providers and coding agents
**What they feed:** the agents themselves. Runs execute in your repo's GitHub Actions using **your own key**, pushed to the repo as a secret at provisioning time under the vendor's conventional name:
| Connector | Secret | API family it serves | Agents that can reach it |
| ---------------------- | -------------------- | ------------------------------------ | ------------------------ |
| **Anthropic (Claude)** | `ANTHROPIC_API_KEY` | Anthropic Messages | Claude Code |
| **Moonshot AI (Kimi)** | `MOONSHOT_API_KEY` | Anthropic Messages, OpenAI Responses | Claude Code, Codex |
| **OpenAI** | `OPENAI_API_KEY` | OpenAI Responses | Codex |
| **OpenRouter** | `OPENROUTER_API_KEY` | Anthropic Messages, OpenAI Responses | every agent |
| **Z.ai (GLM)** | `ZAI_API_KEY` | Anthropic Messages | Claude Code |
| **Backboard R-CLI** | `BACKBOARD_API_KEY` | its own router | Backboard |
The last column follows from the one before it: each coding agent speaks one model-API family, so it can reach exactly the providers serving that family. Backboard is the one entry that is an agent rather than a provider, and its key is both the agent credential and the model-routing credential.
A provider's own endpoint is not the only way to its models. Z.ai serves the Anthropic Messages API, and its OpenAI-compatible surface is Chat Completions, which `codex exec` no longer supports — so GLM under Codex goes through a router that speaks Responses, such as `--provider openrouter --model z-ai/glm-5.3`. The same applies to any model whose vendor doesn't serve your agent's family directly.
**Setup:** click **Connect** on the provider's card and paste an API key from that provider's console. The key is validated against the provider before it's stored and encrypted at rest; it's used only for your own runs, billed to your provider account.
Connect one to get started; connect several to choose an agent and backend per agent (or per dispatch), or to run the [two-tier drafter/refiner](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) with different tiers on different providers. See [Model backends](/platform/agents/backends) for the full comparison.
**Installing with your own `gh` instead.** A [`setup-local`](/cli#install-without-the-github-app) install doesn't go through Connectors, so its key goes on the repo directly. Same secret names:
```bash theme={null}
remyxai outrider set-provider-secret --repo owner/name --provider openrouter --key-from ~/key
gh secret set BACKBOARD_API_KEY --repo owner/name < ~/backboard-key
```
Either way the action reads the one key its `agent` / `provider` pair needs, and the run reports its backend to [Reports → Costs](/platform/review/reports#costs).
Each provider bills usage to its own account — make sure the account behind the key has API credit. (For Anthropic specifically, API credits are separate from a Claude Pro/Max subscription.)
**Prefer your own MCP client?** You can also point Claude Code, Claude Desktop, or Cursor at the Remyx MCP server with your Remyx API key — see the [MCP Server reference](/resources/mcp-server).
***
## Linear
**What it feeds:** draft-vs-discussion routing — open tickets flag interfaces under active work, which routes risky candidates to discussion instead of drafts. Decisions link back to issues, and `@remyx` mentions in comments get agent replies.
**Setup:** click **Connect**, authorize Remyx, then link Linear where you use it — as a project [source](/platform/configure/projects#sources), or as the durable store for two-tier gap analyses via the action's `linear-api-key` input.
## Jira
Same routing signal as Linear, for Jira Cloud teams. Sync uses the Jira Cloud v3 webhook API (`offline_access` + `manage:jira-webhook` scopes); inbound deliveries are verified against the Atlassian webhook identifier header. If auth errors appear after an extended period, disconnect and reconnect to refresh.
## Slack
Inbox items, daily digests, and weekly portfolio summaries in your channels, plus a managed agent: mention `@Remyx` to get your digest, list agents' output, or ask about a run. Install the app, pick a default channel, and `/invite @Remyx`.
***
## Weights & Biases
**What it feeds:** gate evidence — eval runs and metrics feed the [decision policy](/platform/review/evaluation) at the gate, and W\&B Launch is a compute backend for scored validation runs.
**Setup:** paste a W\&B API key from [wandb.ai/authorize](https://wandb.ai/authorize) (plus an optional entity and self-hosted URL). The key is validated against the W\&B GraphQL `viewer` query before it's saved.
## Hugging Face
Connects via OAuth so Remyx can read model and dataset metadata. Your Hugging Face token (for gated models in validation runs) lives on this connector card.
## Modal
**What it feeds:** validation compute — Remyx dispatches scored validation jobs to Modal Sandboxes (GPU-capable) on your account. Enter your Modal token ID and secret. See [Evaluation](/platform/review/evaluation#validation--compute).
***
## Security
* Credentials and OAuth tokens are encrypted at rest; keys are validated against their provider before being stored.
* Webhook payloads are verified for authenticity on every request.
* Disconnecting is immediate — agents and gates that reference the connection stop working until it's reconnected.
***
## Related
How the three LLM providers compare per run
Reference connected sources per project
What provisioning pushes to the repo
Use Remyx from your own MCP client
# Projects
Source: https://docs.remyx.ai/platform/configure/projects
The namespace each initiative runs in: its team, the agents feeding it, and the sources it references — plus the Pro quality gate
**Nav:** Manage > Projects | **URL:** [`/projects`](https://studio.remyx.ai/projects)
A **project** is the namespace an initiative runs in: its **team**, the **agents** feeding it, and the **sources** it references. The project switcher at the top of the sidebar scopes the Inbox, feeds, Agents, and Reports to the active project's repos and experiments.
The Projects index lists each project with its team, agent count, and source count. **+ New project** creates one inline; open a project to edit it.
***
## Scope
Free text, edited any time — e.g. *"Focus on the retrieval and prompting pipeline; ignore CI/infra changes."* Recommendations, agents, and gates all read this: saving it refreshes the project's interests, so it's the cheapest way to steer what the whole loop pays attention to.
## Agents
The [agents](/platform/agents/overview) linked to this project, as chips — add from your installed agents, remove with a click. An agent can belong to several projects.
## Sources
The connected accounts this project references, grouped by role:
| Group | Sources |
| ---------------------- | ------------------------------------- |
| **Code** | GitHub |
| **Project management** | Linear, Jira |
| **LLM providers** | Anthropic (Claude), Z.ai (GLM) |
| **Signal & data** | Weights & Biases, Hugging Face, Slack |
Linking references a connected account for this project; the account itself is connected once under [Connectors](/platform/configure/connectors).
## Experiments
The project's experiment history, extracted from its repos' merge logs — each row a change with its status, decision, and date. **Refresh extraction** re-reads the merge log; individual rows can be excluded (and restored) to curate what feeds ranking. Agent drafts file here as proposed experiments; merged work and extracted history join them.
This history is what agents rank candidates against — it's the "what your team has actually shipped" signal.
***
## The Pro quality gate
Three sections put a quality gate between agent drafts and your main branch. They carry a **Pro** badge — Remyx sets them up with you (self-service is coming; [contact us](mailto:contact@remyx.ai)):
What "better" means for this project: each metric with a **baseline** and a **target**. Metrics frame what agents recommend and what validation gates against.
Locked [eval templates](/platform/review/evaluation): the scoring harness is committed **before results exist, so nobody moves the bar after seeing them**. Future experiments inherit the most recently locked template.
Three cards — **Ship**, **Reject**, **Iterate** — rendering the promote bar at the gate. Ship and Reject are rules over your metrics (≥, improves-by, no-regression, …); Iterate is the automatic fallback when neither matches. Evidence is posted right on the PR.
See [Evaluation & decision policy](/platform/review/evaluation) for the full reference.
***
## Danger zone
**Delete this project** removes its configuration and unlinks its experiments. It cannot be undone.
***
## Related
The Pro gate in detail
The agents a project links
Connect the accounts projects reference
Fleet output, scoped by project
# Briefs
Source: https://docs.remyx.ai/platform/discover/briefs
Typed, cited research briefs that ground recommendation ranking and your agents' drafting context, plus the Remyx bot
**Nav:** Research > Briefs | **URL:** [`/research?mode=briefs`](https://studio.remyx.ai/research?mode=briefs)
Topical matching tells you what a paper is about. It doesn't tell you where an approach sits in the field, what's still unsolved, or which concrete change is worth trying in your repo. When you point Remyx at a repo, it runs a research pass that answers those questions and stores the result as a **research brief** that both ranking and drafting read from.
***
## Research briefs
A **research brief** is a typed, cited summary that Remyx produces and stores **section-grained** for a repo, so each section is addressable on its own. A brief contains:
| Section | What it holds |
| --------------------- | ------------------------------------------------------ |
| **Domain framing** | What problem space the repo sits in and why it matters |
| **Approaches survey** | The methods in use across the field, with tradeoffs |
| **Future directions** | Where the work could plausibly go next |
| **Candidate changes** | Concrete, implementable changes worth trying |
| **Citations** | The sources behind every claim above |
Because the brief is typed and cited, ranking and drafting can pull just the section they need, and you can trace any claim back to its source.
***
## How briefs are created and used
A brief is created automatically when you create a [Research Interest from a GitHub repo](/platform/discover/feed#research-interests-from-a-repo). Remyx runs the deep-research pass in the background; you don't trigger it separately.
Once stored, the brief is used in two places:
The brief's **future directions** become a forward-looking ranking axis alongside your shipping history, so the feed can weigh where the work is headed. The **candidate changes** seed recommendations directly.
When an [agent](/platform/agents/overview) drafts an implementation against your repo, the stored brief is part of the context it works from, alongside the recommendation and your repo's history. The agent's [Signal panel](/platform/agents/overview#configuration) shows the brief feeding it.
***
## The Remyx bot
You can ask about a brief and a run from the pull request or issue itself: any PR or Issue one of your [agents](/platform/agents/overview) opened.
In a comment on such a pull request or issue, mention **`@remyx-ai`** (aliases `@remyxai` and `@remyx` also work) and ask your question.
The bot posts a 👀 reaction so you know the mention was picked up.
It posts a single grounded reply as `remyx-ai[bot]`, retrieved over the stored research brief plus that run's telemetry: the recommendation runs, their outcomes, and rejected candidates.
Because it answers from the brief and the run's own telemetry, you can ask things like:
* "Why was this recommendation surfaced over the others?"
* "What did the brief say about alternatives to this approach?"
* "Which candidate changes were considered and rejected for this run?"
**The Remyx bot is read-only this phase.** It answers questions; it does not take write actions, edit code, push commits, or change PR state. A human stays at every decision.
It only responds when **all** of these hold:
* The comment **mentions the bot** (`@remyx-ai` / `@remyxai` / `@remyx`).
* The comment is on a pull request or issue an [agent](/platform/agents/overview) opened.
* The **permission check** passes.
If any condition fails, no reply is posted.
***
## Paper enrichment and PDF code-link recovery
A recommendation is only actionable if you can find the code behind it. Two background steps make more papers implementable. Both need nothing from you and run at ingest.
For a paper with no code link, an enrichment agent traverses the arXiv abstract and HuggingFace papers pages through to project pages, classifies the URLs it finds, resolves licenses, and recovers GitHub, HuggingFace, and SPDX links back into the paper record. Existing fields are left alone; it only fills empties.
Remyx fetches the paper's PDF and promotes a `Code:`, `Project page:`, or `Repository:` URL directly into the paper record at ingest. Downstream, that named reference repo is lifted into the agent's implementation context as the **source of truth**, so the agent translates from the reference and diffs against it before opening a PR.
More papers then arrive with a named reference repo already attached. That is the difference between a recommendation you can act on now and one you'd have to go find the code for first.
***
## Related
Repo-backed interests trigger a brief that feeds ranking
The agent runtime that drafts PRs from the brief
How discovery, briefs, and implementation form a loop
# Feed
Source: https://docs.remyx.ai/platform/discover/feed
Personalized daily recommendations grouped by your Research Interests
**Nav:** Research > Feed | **URL:** [`/research?mode=feed`](https://studio.remyx.ai/research?mode=feed)
Staying current with AI research is part of the job, but rarely the main job. You have models to ship, pipelines to maintain, and stakeholders asking if you've tried the latest technique they saw online. The result is a persistent backlog of ideas you'll get to "eventually" and relevant work you discover weeks after it would have been useful.
The Feed delivers a curated stream of resources (papers, repos, models) matched to what your team is building. You create focused **Research Interests** that each track a different area of your work, with their own recommendation cadence. There's no cap on how many interests you keep.
***
## Research Interests
Research Interests are the context that drives your recommendations. Each interest has its own name, context description, and recommendation cadence.
### Creating an Interest
Click **+ New Interest** at the top of the Feed view.
| Field | Description | Example |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| **Name** | Short label (appears as a pill tab) | "RAG & Retrieval" |
| **Context** | Natural language description, URLs to HuggingFace models / GitHub repos / arXiv papers / blog posts, a GitHub repo, or an existing [Project](/platform/configure/projects) | `https://huggingface.co/your-org/your-model` |
| **Daily Recommendations** | Resources per day (1-10) | 3 |
You can seed an interest from free text, from a GitHub repo (see [below](#research-interests-from-a-repo)), or **from an existing Project**. The Project source draws on that project's experiments, so the ranker scores candidates against the work you've actually run.
**Richer context = better recommendations.** Link GitHub repos with detailed READMEs, HuggingFace models with comprehensive cards, or specific arXiv papers that define your area. Resources that cite or benchmark against your work get boosted.
### Interest Pill Tabs
Your interests appear as **pill tabs** at the top of the Feed:
* **All** — Combined feed from all interests
* **\[Interest Name]** — Filtered to that interest only, with badge count showing how many new recommendations are available
### Managing Interests
Click the **gear icon** (⚙️) next to an interest name to:
* Edit the name, context, or daily count
* Toggle active/inactive (inactive interests stop generating recommendations but preserve history)
* Delete the interest
### Research Interests from a repo
Instead of a free-text context, you can create an interest **from a GitHub repo** (pick one from your connected GitHub, or paste a public URL). Remyx extracts the repo's merge history into a structured shipping record in the background, then ranks recommendations against the work your team has actually shipped. This is the same extracted history that powers a [Project](/platform/configure/projects); a repo-driven interest and a project that links the same repo converge on one history.
A repo-backed interest also triggers a typed, cited **[deep-research brief](/platform/discover/briefs)** for the repo. Its "future directions" become a forward-looking ranking axis alongside your shipping history, and its "candidate changes" seed recommendations, so the feed can weigh where the work is headed.
When you create an interest from a repo, the dialog also offers an **auto-PR** setting that can provision [automated discovery PRs](/platform/agents/outrider-action) for that repo:
| Choice | What happens |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Set it up for me** | Remyx installs the recommendation GitHub Action, opens and merges the setup PR, and fires the first run automatically once recommendations exist. |
| **Let me review the setup PR** | Same provisioning, but the setup PR is left open for you to review and merge first. |
| **Not now** | No provisioning — turn it on later from the interest's settings. |
Editing a repo-sourced interest lets you **curate its extracted history** (include/exclude individual changes), the same way you curate a project's experiment list. Keeping direction-setting work and dropping pure refactors sharpens what gets recommended.
***
## Recommendation Cards
Each recommendation shows:
| Element | Description |
| ------------------- | --------------------------------------------------- |
| **Title** | Resource title with link to detail viewer |
| **Source type** | Badge indicating `arxiv_paper`, `github_repo`, etc. |
| **Relevance score** | How well this matches your interest context |
| **Reasoning** | One-line explanation of why this was recommended |
| **Interest tag** | Which interest surfaced this resource |
Cards are sorted by recency by default. Use the **Sort** control to change ordering.
***
## Refreshing Recommendations
Each interest has a **Refresh** button in the toolbar. Click it to trigger an immediate re-ranking — don't wait until the next scheduled run.
Refresh runs asynchronously. A progress indicator shows the status. Navigate away and come back; the job continues in the background.
Scheduled recommendations refresh daily at **9:30 AM UTC** for all active interests. Use **Refresh** for instant updates, or update your context sources before the daily run.
### How recommendations are ranked
For interests backed by a shipping history (any [repo-sourced interest](#research-interests-from-a-repo) or a project), ranking goes beyond matching your context text:
* Your team's extracted shipping history is fed into the ranker, so candidates that align with the methodological direction you've actually shipped rank above ones that are merely topically related.
* A learned **preference model**, fit over your past experiments, scores candidates and breaks ties behind relevance.
Both make the feed sharper as your history grows.
***
## Resource Detail & Chat
Click **View** on any recommendation to open the detail page with:
* **Details tab** — Full abstract, resource links, citations, Docker availability
* **Chat tab** — Ask questions about the resource in natural language with streaming responses
See [Search > Resource Detail Viewer](/platform/discover/search#resource-detail-viewer) for full details.
***
## From Feed to agent
The feed is the same candidate stream your [agents](/platform/agents/overview) draw from: a repo-backed interest is the input to an agent, and the auto-PR setting above is how a feed becomes a running agent. Once installed, the agent ranks this stream against your shipping history, drafts the best fit, and your merges and skips sharpen what the feed surfaces next.
***
## CLI & API
You can also access your feed programmatically. By default the digest honors each interest's per-day limit; the API's **unlimited mode** (`limit=0`) returns every alerted recommendation instead, along with a `truncated` flag indicating whether anything was held back.
```bash CLI theme={null}
# Today's digest grouped by interest
remyxai papers digest
# Recommendations for one interest
remyxai papers list --interest "RAG & Retrieval" --period today
# Trigger a refresh
remyxai papers refresh --interest "RAG & Retrieval" --wait
```
```bash API theme={null}
# Get digest
curl -H "Authorization: Bearer $REMYXAI_API_KEY" \
"https://engine.remyx.ai/api/v1.0/papers/recommended/digest?period=today"
# Unlimited: every alerted recommendation across active interests
curl -H "Authorization: Bearer $REMYXAI_API_KEY" \
"https://engine.remyx.ai/api/v1.0/papers/recommended/digest?limit=0"
# Refresh an interest
curl -X POST -H "Authorization: Bearer $REMYXAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"interest_id": "uuid"}' \
"https://engine.remyx.ai/api/v1.0/papers/recommended/refresh"
```
***
## Related
Turn a repo-backed interest into a running agent
On-demand semantic search across all resources
Manage interests and recommendations from the terminal
# Search
Source: https://docs.remyx.ai/platform/discover/search
Semantic search across papers, repos, models, and datasets with runnable Docker environments
**Nav:** Research > Search | **URL:** [`/research?mode=search`](https://studio.remyx.ai/research?mode=search)
Search is also the app's **public surface**: signed out, [studio.remyx.ai](https://studio.remyx.ai) serves real search results (no fixtures) — the rest of the app asks you to sign in. Every paper has a shareable, anonymous-readable URL at `studio.remyx.ai/paper/`, and legacy engine paper links redirect there. Press `/` anywhere in the app to jump to search.
Every AI team has a backlog of techniques they want to try but never have time to evaluate. The pace of change means what was best practice weeks ago may already be suboptimal, but scanning arXiv, GitHub, and HuggingFace manually doesn't scale.
Search provides semantic search across research papers, GitHub repositories, HuggingFace models, and more, matched to what your team is building. Pre-built Docker environments let you test a technique in minutes instead of spending days on dependency setup.
***
## Search Interface
Type a natural-language query into the search bar. Remyx runs hybrid retrieval (BM25 + dense vector + re-ranking) across its resource index and returns ranked results.
```
retrieval augmented generation for customer support
```
`/search/assets` enforces a server-side deadline. If a query runs long, it returns a structured timeout (with a request id and the elapsed time) instead of hanging. Retry the request, and include the request id if you report a slow query.
Results appear in a card grid. Each card shows:
| Element | Description |
| --------------------- | ------------------------------------------------------------------------------------------- |
| **Title** | Resource title with link to detail viewer |
| **Authors** | Author list (for papers) |
| **Abstract** | Truncated summary |
| **Categories** | arXiv categories or source tags |
| **Runnable badge** | Blue "Runnable environment" badge with Docker icon if a pre-built Docker image is available |
| **Source indicators** | Icons for GitHub, HuggingFace, Docker availability |
Resources with the **Runnable environment** badge have pre-built Docker images with dependencies installed and example scripts ready to run. These are the fastest path from discovery to experiment.
***
## Filtering and Sorting
A toolbar appears above results after you search:
| Control | What it does |
| -------- | --------------------------------------------------- |
| **Tabs** | **All** (default), **Runnable** (Docker-ready only) |
| **Sort** | Relevance (default), Most Recent, Most Popular |
The **Runnable** tab filters to resources with verified Docker environments — useful when you want to test a technique immediately without environment setup.
***
## Personalized Suggestions
### Authenticated Users
When you're logged in and haven't searched yet, the page shows:
* **Suggestion cards** built from your experiment tags — topics you've already explored, surfaced as quick-search shortcuts
* **"Relevant to your work"** section with resources matched to your team's experiment history
### Anonymous Users
When not logged in, the page shows:
* **Trending suggestion cards** populated from live search queries
* **Recent resources** section with highlights from the index
***
## Resource Detail Viewer
Click **View** on any result to open the resource detail page:
* Full abstract / README
* Resource links (arXiv, GitHub, HuggingFace)
* Citation count and categories
* Docker image reference (if available)
* Dockerfile contents (if available)
Ask questions about the resource in natural language. Responses stream in real-time and conversation history is preserved.
Example questions:
* "What's the main contribution compared to prior work?"
* "How would I adapt this for a 3B parameter model?"
* "What datasets did they use for evaluation?"
Highlight passages and add notes directly on the resource. Annotations help your team collaborate on ideas that emerge during review and carry context forward into experiments.
* **Highlights**: Select text to highlight key methods, results, or claims
* **Notes**: Add written observations, questions, or hypotheses to any highlight
* **Personal / Team mode**: Annotations can be private or shared with your team. In team mode, you can filter between your own annotations and those from teammates.
* **Experiment context**: When you create an experiment from an annotated resource, your annotations are carried forward. The hypothesis field is pre-filled from your notes, so the reasoning captured during review becomes part of the experiment record.
***
## Briefs
When you create a repo-backed [Research Interest](/platform/discover/feed#research-interests-from-a-repo), Remyx runs a typed, cited deep-research pass and stores a section-grained **research brief** for that repo: domain framing, a survey of approaches, future directions, candidate changes, and citations. The brief feeds recommendation ranking and grounds your [agents'](/platform/agents/overview) drafting context. You can also ask questions about a brief and its runs by mentioning `@remyx-ai` on a pull request or issue an agent opened.
How research briefs are built and stored, the Remyx bot, and paper enrichment
***
## Related
Personalized daily recommendations
The agents that draw on this same index
# Evaluation & Decision Policy
Source: https://docs.remyx.ai/platform/review/evaluation
Commit how progress gets measured before a result exists: eval templates declare the harness, decision policy renders the bar against the evidence, and validation runs the eval on connected compute.
**Nav:** Manage > Projects > *project* (**Pro** sections) | **URL:** [`/projects`](https://studio.remyx.ai/projects)
An eval template and a decision policy are two of the three **Pro** gating artifacts on a [Project](/platform/configure/projects#the-pro-quality-gate) (the third is target metrics). Together they fix *how progress gets measured* before any variant exists: the template says **how to score**; the policy says **what counts as progress** — **Ship**, **Reject**, or **Iterate**. Both are committed ahead of time so agent drafts are graded against the same bar as human work, and so a result can't be graded against a goalpost that moved after the numbers landed.
This page documents the capability. For the step-by-step setup, see [Define how progress gets measured](/tutorials/get-started/define-how-progress-gets-measured).
***
## Eval templates
An eval template is a saved, project-level spec for scoring a variant against a baseline. It declares the runtime, the baseline, and the metrics once, so every evaluation under the project uses the same harness. Locking a template doesn't run anything; it captures the spec.
Two runtimes are supported:
Point Remyx at an evaluation script that already lives in your repo. You provide:
* **Eval script path**: the path in your repo Remyx invokes (e.g. `docker/eval_stage/`).
* **Baseline asset**: the model, dataset, or process variants are compared against (e.g. a HuggingFace model slug like `Qwen/Qwen2.5-VL-7B-Instruct`).
Your script defines what runs and which metrics it emits. Use this when your project already ships its own evaluation harness.
Pick a task from a whitelist of standard benchmarks (e.g. `gsm8k`, `mmlu`). The task's canonical metric (`exact_match`, `accuracy`) auto-populates, and you don't maintain an eval script in your repo.
Use this when the metric you care about is one of the standard tasks the runtime ships with.
The template draws its metrics from the project's **allowed metrics**, the shared vocabulary every experiment, template, and decision rule references. Once locked, the template autocompletes those metric names into the decision-policy editor, which keeps rule fields free of typos that wouldn't surface until a run.
Locking a template captures the spec; it does not launch an evaluation. To actually run the eval on compute, see [Validation & compute](#validation--compute) below.
***
## Decision policy
The decision policy is a per-disposition rule builder. For each disposition you define one rule, and a rule is a combinator (`all` or `any`) over a set of conditions:
| Disposition | What its rule defines |
| ----------- | ----------------------------------------------------------------------------------------------------------------- |
| `ship` | What the variant must achieve to count as progress worth shipping. |
| `reject` | What disqualifies the variant, typically a `no_regression` condition on each metric you'd be unhappy to see drop. |
`iterate` is the automatic fallback: when neither the `ship` rule nor the `reject` rule matches, the run classifies as iterate.
Conditions are predicates over the experiment's aggregated metrics and a few standard fields:
* **Metric conditions**: a metric from your locked template, an operator (`>=`, `>`, `improves_by`, and so on), a threshold, and an optional confidence floor.
* **Field conditions**: gates on fields like `delta_confidence`, `validation_status`, and `sample_size` (e.g. `delta_confidence >= 0.80`).
### Visualization only
The decision policy is a **read-side visualization aid**. When results are in, Remyx marks each predicate as met, missed, or not-yet-measured against the experiment's aggregated metrics, so you can see whether the evidence clears the bar you set. It does **not** auto-dispose the experiment. A human still reviews the evidence and logs the decision (ship, iterate, or abandon) and why.
This is deliberate. The policy tells you whether the evidence meets a pre-committed standard; it does not make the call about whether a change actually moved a business outcome. Automated disposition is explicitly deferred: the system automates the toil up to the judgment and stops there. See [Continuous Experimentation](/concepts/continuous-experimentation) for why this gate stays human.
***
## Validation & compute
When an eval template runs, Remyx can launch the run on a connected compute provider instead of asking you to run it by hand. This is the **validation** path: it executes the template's harness, then writes the results back onto the experiment.
Supported providers:
| Provider | Status |
| --------------------------- | --------------- |
| **Modal** | Supported |
| **Weights & Biases Launch** | Supported |
| MLflow | Adapter stubbed |
Connect a provider from [Connectors](/platform/configure/connectors). When a validation run finishes, results flow back onto the experiment via a **signed webhook**. Fields like `observed_delta` and `delta_confidence` land on the experiment and feed the decision-policy predicates described above.
Remyx submits the locked template's harness to your connected provider (Modal or W\&B Launch).
The eval executes on the provider against the template's baseline asset and metrics.
The provider posts results to Remyx over a signed webhook. `observed_delta`, `delta_confidence`, and related fields are written onto the experiment.
The decision policy re-renders against the new metrics, and the team logs the decision.
***
## Where to set these
Both artifacts are settable two ways:
The project's **Pro** sections: [**Target metrics**, **Evaluation** (the eval template), and **Decision policy**](/platform/configure/projects#the-pro-quality-gate). The rendered policy (Ship / Reject / Iterate) appears against the evidence at the gate — right on the PR and in the project's experiment history.
From a connected agent, the decision policy is managed with `set_decision_policy`, `get_decision_policy`, and `clear_decision_policy`. Validation runs are driven with `provision_validation`, `check_validation_status`, and `get_validation_results`.
***
## Related
Where eval templates and decision policies are declared (Pro)
Where Decide cards carry the evidence to the human gate
Why the decision stays a human gate
Step-by-step setup of the template and policy
# Inbox
Source: https://docs.remyx.ai/platform/review/inbox
Waiting on you: drafts to review, calls your agents held for you, and anything blocking the loop
**Nav:** Inbox | **URL:** [`/inbox`](https://studio.remyx.ai/inbox)
The Inbox is titled **"Waiting on you"**, and that's the whole contract: drafts to review, calls your agents held for you, and anything blocking the loop — in your own time. It's a **derived** queue, not a task tracker you maintain: items appear from live agent state, and anything resolved on GitHub clears itself.
![The Inbox: 'Waiting on you' with Review cards for draft PRs across repos, filter chips showing All 48 / Review 21 / Decide 27, a date window set to All time, and per-card Review the draft, Ask, Run log, Details, and Mark done actions]()
***
## Card types
Every card carries one of three tags, filterable with chips at the top:
| Tag | What it asks of you | Typical items |
| ---------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| **Review** | Read a diff and merge or close | A draft PR opened by an agent |
| **Decide** | Make a judgment call | A discussion Issue the agent opened instead of a PR; a saturated candidate pool (widen the interest or retire the agent?) |
| **Fix** | Unblock the loop | A blocked agent (failing runs), an unmerged setup PR, a missing model key |
***
## Auto-resolution
Item state follows GitHub. When a PR or Issue is merged or closed, its card auto-completes into the **Done** pile, Linear-style, labeled **Merged** / **Closed**. You never mark those done — the system does. Cards you clear by hand get **Done by you** and can be restored to the inbox anytime; auto-resolved ones can't (there's nothing left to do).
Three status views: **Todo** (open and waiting on you), **Done** (resolved on GitHub — merged or closed), **Archived** (marked done by you). A **Mark all as done** chip batch-archives.
The Inbox's date window defaults to **All time** — old waiting items must not hide. Health cards (setup PR, missing key) are never hidden by the window.
***
## Working a card
* **Details** expands a **TLDR digest** — a server-generated summary built from the run's telemetry on first expand — plus deterministic **run evidence**: gate check chips, a diff-risk band, flagged callouts, and scope, reported from the run's own passes rather than a recommendation.
* **Ask** opens a run-grounded chat scoped to that agent, seeded with the right question for the card kind (e.g. for a blocked agent: *"Why is this repo blocked — what is failing in the logs, and what should I change?"*). It's the same drawer as [Ask about this agent](/platform/agents/overview#ask-about-this-agent).
* **Analyze** (on Fix cards with a run) pulls the failing run's log and returns a root-cause read.
* The primary button does the obvious thing — open the PR, open the Issue, open Connectors — and **Run log ↗** / **View in agent** take you to the evidence.
![An expanded Inbox card: the TLDR digest summarizing what the draft proposes and what to verify, then 'From the run' evidence — Tests passed and Diff risk low chips, and an All-run-detail fold with fidelity and self-review statuses, stub density, diff-risk band, checks, and the selection rationale — footed by 'Reported from the run's own passes, not a recommendation']()
The sidebar badge counts all open items, so it lights up when a run lands. The footer links out to the [fleet report](/platform/review/reports) and [Agents](/platform/agents/overview).
When there's nothing left: *"You're all caught up."*
***
## Programmatic access
The inbox is also exposed on the [MCP server](/resources/mcp-server#outrider-fleet-tools) (`get_inbox`, `dismiss_inbox_item`, `get_artifact_digest`) and the REST API (`GET /outrider/agents/inbox`, `POST /outrider/agents/inbox/dismiss`), so an agent or script can triage the same queue you see.
***
## Related
The fleet view behind the queue
Where evidence links land
Fix provisioning states surfaced by Fix cards
Pre-committed bars for the Decide calls (Pro)
# Reports
Source: https://docs.remyx.ai/platform/review/reports
The fleet in one view: what your agents produced, where pools are depleting, and what's waiting on you
**Nav:** Reports | **URL:** [`/reports`](https://studio.remyx.ai/reports)
Reports is the fleet dashboard: what your agents produced, where pools are depleting, and what's waiting on you. It opens with a written **headline** — a one-line thesis about the window — then a **KPI strip** and **findings**, each finding deep-linking into the agent evidence it came from.
![The Reports dashboard: headline '15 drafts opened and 6 merged and 6 leads routed to discussion', a KPI strip with dispatches, drafts opened, merged, leads, needs-attention, and spend with cost per artifact, findings that deep-link to agent evidence, and the fleet-activity chart stacked by outcome kind per day]()
Two controls scope everything and ride the URL (so any view is shareable):
* **Agent picker** — all agents or a subset; the scope carries into every chart and drill-down.
* **Date window** — the same window control as agent views, from 7 days to All time.
A **↻ Refresh** button pulls the latest agent state from GitHub and rebuilds the report; the "Updated N minutes ago" stamp tells you how fresh the data is.
***
## Fleet activity
A daily stacked-bar chart of run volume by outcome kind — Draft PR / Discussion / Branch / Skipped / Failed — with spend overlaid. Legend chips toggle outcome kinds; zoom presets narrow to the last 90/30/14/7 days.
***
## Deliverable pipeline
Everything currently in flight, as cards: drafts and discussions with title, repo, age, and a direct **Open ↗** link. This is the maintainer-facing work the fleet has put in front of you, in one place.
***
## Per-agent breakout
Each agent classified into one of six states, filterable by chips (the summary counts how many **need attention** — blocked plus saturated):
| State | Meaning |
| ------------- | ------------------------------------------------------------------------ |
| **blocked** | Runs are failing — something needs fixing |
| **saturated** | The candidate pool is exhausted — widen the interest or retire the agent |
| **shipped** | Work merged in the window |
| **active** | Running normally |
| **new** | Recently provisioned |
| **retired** | Deliberately wound down |
Two views of the same data:
* **Outcomes** — proportional stacked bars per agent, scaled either absolute (bar length by busiest agent) or as share (100% rows).
* **Timeline** — a **contribution grid**: one cell per day (per week on long windows), colored by the highest-signal outcome that day and shaded by run count. A season of agent work reads at a glance, like a GitHub contribution graph for your fleet.
![The per-agent breakout in Outcomes view, share scale: one 100% stacked bar per agent splitting its runs across draft PRs, discussions, and branches, with state filter chips (All 64 / shipped 12 / active 17 / new 35) and each agent's state labeled at the row's end]()
Sortable by status, name, run count, draft count, draft share, or failure share. A **Chronology** fold lists the discrete events — Enabled, Retired, Merged, Draft, Discussion — each linked to its artifact.
***
## Costs
The **Spend** KPI drills into `/reports/costs` (carrying your window and agent scope): unit economics for the fleet. Runs per day stacked by model backend — toggle between one series per provider or per model — against a \*\*$ per code artifact** line, plus per-backend "$ / artifact" KPIs. Runs carry their coding agent, backend, model, and tier (main / drafter / refiner), so a [two-tier setup](/platform/agents/backends#the-two-tier-drafter-refiner-pattern) shows exactly what exploration costs versus promotion. A run whose [agent reports no dollar figure](/platform/agents/backends#what-each-agent-can-report) is left out of spend rather than counted as \$0, and the number of those runs rides along — so a partial total says so instead of quietly understating.
***
## Programmatic access
The same report is available from the [MCP server](/resources/mcp-server#outrider-fleet-tools) (`get_fleet_report`) and the REST API (`GET /outrider/agents/report`), scoped to your own agents.
***
## Related
The action items behind "needs attention"
Drill into one agent's runs, direction, and trace
What drives the cost-per-artifact differences
Scope the fleet to an initiative
# Quickstart
Source: https://docs.remyx.ai/quickstart
Your first Outrider run: connect GitHub and a model provider, install an agent, and watch a draft PR land
This guide mirrors the in-app Quickstart at [studio.remyx.ai/start](https://studio.remyx.ai/start): connect two accounts, choose where the agent runs, fire a run, and watch a draft PR open under **remyx\[bot]**. Every run is live.
Each step's CLI equivalent is called out below. If you prefer the terminal, install and authenticate once:
```bash theme={null}
pip install remyxai
# create a personal (rmxu_) key under Account > API keys, then:
export REMYXAI_API_KEY=
```
***
## 1. Connect your GitHub account
Open the [Quickstart](https://studio.remyx.ai/start) and click **Connect GitHub**. The connection is used to:
* **Fork the demo repo** — created under your account, yours to delete
* **Install the agent workflow** — one workflow file + repo secrets, via a setup PR
* **Open draft PRs & Issues** — authored as remyx\[bot], never as you
* **Revoke anytime** — from GitHub settings or the Connectors page
Authorize in the tab that opens; the step completes itself when GitHub confirms (**✓ GitHub connected**). From the terminal, `remyxai outrider init` prints the same authorization link when a repo needs the App.
***
## 2. Connect your model provider
Runs use your own model key — connect one provider now; add or switch later under [Connectors](/platform/configure/connectors). Pick one of:
| Provider | Key | Agents that can reach it |
| ---------------------- | ------------------------------ | --------------------------------------- |
| **Anthropic (Claude)** | Anthropic API key (`sk-ant-…`) | Claude Code |
| **Moonshot AI (Kimi)** | Moonshot AI API key | Claude Code, Codex |
| **OpenAI** | OpenAI API key | Codex |
| **OpenRouter** | OpenRouter API key | every agent |
| **Z.ai (GLM)** | Z.ai API key | Claude Code |
| **Backboard R-CLI** | Backboard API key | Backboard, which resolves models itself |
Each coding agent speaks one model-API family, so it reaches the providers serving that family; a router like OpenRouter reaches any model from any agent.
The key is validated against the provider before it's stored, encrypted at rest, and used only for your own runs. See [Model backends](/platform/agents/backends) for the full provider list, how each compares on cost and speed, and the coding agents they pair with.
```bash CLI equivalent theme={null}
# any provider the action supports: anthropic, openai, zai, moonshot, openrouter
remyxai outrider set-provider-secret --repo --provider --key-from ~/key
```
***
## 3. Choose where the agent runs
Two tracks, both real runs:
**Fork remyxai/peft** — Remyx's fork of the Hugging Face fine-tuning library, a codebase you may already know, so the diff speaks for itself. A set of papers is pre-matched, each with a **curated draft branch** already seeded into your fork. A run warm-starts from that branch (via `start-from-ref`) and refines it into a draft PR **in minutes**.
Click **Fork it & install the agent**. Remyx forks the repo under your account, installs the Outrider workflow through a setup PR, and pushes your model key as a repo secret.
**Run on your own code.** Pick any repo you own. Outrider reads its code and history, builds the candidate pool, and your first run returns live results: pool audit, selection, implementation from scratch, self-review. Expect **15–20 minutes**.
Click **Set up on my repo** and pick the repository.
```bash CLI equivalent theme={null}
remyxai outrider init --repo --auto-interest
```
***
## 4. Run it
On the **set example** track, pick one of the pre-matched papers — for instance *Riemannian Preconditioned LoRA* (arXiv 2402.02347) or *AdaMoLE: adaptive mixture of LoRA experts* (arXiv 2405.00361) — each shown with its curated branch's diffstat so you can inspect the change on GitHub before running. Or click **Or let Outrider choose**: exactly like a scheduled run, it audits the pool, selects the best-fit deliverable paper, and implements it from scratch (\~15–20 min).
On the **your own repo** track there's nothing to pick — Outrider dispatches automatically once the agent is ready, selects the strongest anchored paper itself, and implements it. Everything it decides is in the run log.
```bash Your own repo theme={null}
remyxai outrider trigger --repo
```
```bash Set example (warm start) theme={null}
remyxai outrider trigger --repo --pin-arxiv # + start-from-ref warm start
```
If the candidate pool is still building for a fresh repo, the run starts automatically the moment it's ready (the app retries for you).
***
## 5. Watch it land — then it's always on
A warm-start run takes a few minutes: checkout of the curated draft, a review-and-refine pass, then the draft PR opens on your fork under **remyx\[bot]**. An own-repo run takes 15–20 minutes and ends in one of four real outcomes:
| Outcome | What it means |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| **Draft PR** | Your first draft PR is ready — review the diff and the self-review in the PR body |
| **Discussion** | The agent opened an Issue instead, judging this one safer to discuss first |
| **Branch** | The run pushed a draft branch — refinement promotes it to a PR |
| **Skipped** | The run held publication: its validators judged the result not ready to put in front of maintainers, and the log has the full audit trail |
Runs execute in your repo's GitHub Actions, so the full log lives there; `gh run watch` gives you a terminal view.
The repo is now a full **agent**: it shows up in [Agents](/platform/agents/overview), the fleet [Report](/platform/review/reports), and your [Inbox](/platform/review/inbox), and future runs happen on its schedule with you at every merge.
***
## What you just set up
Discovery grounded in the repo (the interest reads its code and history), ranked candidates with the evidence behind them, drafts that arrive as reviewable PRs, and an Inbox that only ever asks for decisions a human should make.
Where drafts and decisions wait for you
Runs, Direction, Activity trace, and Configuration for your new agent
Coding agents, model backends, and the two-tier drafter/refiner pattern
# MCP Server
Source: https://docs.remyx.ai/resources/mcp-server
Drive your Outrider fleet, research, and projects from Claude Code, Slack, or any MCP client
# MCP Server
The Remyx MCP server exposes the platform via the [Model Context Protocol](https://modelcontextprotocol.io). Connect from Claude Code, Claude Desktop, Cursor, or any MCP-compatible client to drive your [agent fleet](/platform/agents/overview), triage the [Inbox](/platform/review/inbox), and search research using natural language.
The server also powers the managed Slack agent and the studio's [Ask-about-this-agent chat](#studio-agent-chat).
***
## Architecture
The MCP server runs as a standalone service at `mcp.remyx.ai`. Connect using **Streamable HTTP** at `https://mcp.remyx.ai/mcp`, or **SSE** at `https://mcp.remyx.ai/sse` for clients that require it.
Every tool call is logged and powers the activity feed in the experiment detail view.
***
## Connecting
### Claude Code
Add to your Claude Code MCP configuration (`~/.claude/settings.json` or project-level):
```json theme={null}
{
"mcpServers": {
"remyx": {
"type": "http",
"url": "https://mcp.remyx.ai/mcp",
"headers": {
"Authorization": "Bearer YOUR_REMYX_API_KEY"
}
}
}
}
```
### Claude Desktop
Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS):
```json theme={null}
{
"mcpServers": {
"remyx": {
"type": "http",
"url": "https://mcp.remyx.ai/mcp",
"headers": {
"Authorization": "Bearer YOUR_REMYX_API_KEY"
}
}
}
}
```
### Other MCP Clients
Any client supporting the MCP Streamable HTTP transport can connect using the URL `https://mcp.remyx.ai/mcp` with a Bearer token in the Authorization header.
Get your API key at **Account > API Access** in the Remyx app, or at [studio.remyx.ai/account](https://studio.remyx.ai/account).
MCP keys are **scoped**: a key only authorizes the tool categories its scopes grant, so a call to a tool outside the key's scopes is rejected. See [API Scopes](/api-reference/scopes) for the available scopes and how to mint a key with the right ones.
**Only scoped API keys (`rmxu_…` / `rmxa_…`) authenticate.** Legacy JWT API tokens are rejected with `403 — "JWT API tokens have been replaced by API keys. Create one under Account → API keys and use it instead."` Mint a key under [Account > API keys](/platform/configure/account#api-access), and remember the `Bearer ` prefix — a missing prefix returns a 401 telling you so.
***
## Tools
### Outrider Fleet Tools
The fleet tool plane mirrors what the studio shows in [Agents](/platform/agents/overview), [Reports](/platform/review/reports), and the [Inbox](/platform/review/inbox) — the engine stays the single source of truth, so ownership checks and data match the app exactly.
| Tool | What it does |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `list_agents` | Your installations: repo, status, setup PR link, key state. Start here. |
| `get_agent_activity` | One agent's runs, events, directions, and summary for a date window (a `section` param slims the payload) |
| `get_fleet_report` | The fleet report: headline, KPIs (dispatches, drafts, merged, spend), per-agent states, findings, pipeline, chronology |
| `get_inbox` | The derived Inbox: drafts to review, held discussions, and health items (unmerged setup PR, missing model key, blocked runs, saturated pool) |
| `dismiss_inbox_item` | Archive an inbox item (or undo) |
| `get_artifact_digest` | The stored TLDR digest for one agent PR or Issue |
| `analyze_agent_run` | Root-cause a run from its job log: cause, evidence, suggested next step |
| `list_agent_runs` | Run telemetry rows; `include_selection` adds the selection audit — what was considered and why it was set aside |
| `trigger_agent_run` | **Dispatches a live run** (spends the operator's model budget — clients are told to confirm first) |
| `pause_agent` | **Pauses an agent** by revoking its key; resuming requires re-provisioning |
| `install_agent` | Provisions an agent on a repo: issues a key, writes the workflow, opens the setup PR (async; poll `get_install_status`) |
| `fork_repo` | Forks a repo into *your* account (never the bot's), optionally seeding branches — what the Quickstart's set-example track uses |
Alongside the fleet plane, the same revamp added `search_research` / `get_research_asset` (search the index and read one resource), `get_account_info` / `list_connections` (your profile and which integrations this session can use), and project-configuration tools (`create_project_from_repos`, `get_project_metrics`, `get_project_experiment_history`, `refresh_project_extraction`).
### Discovery Tools
Get today's recommended resources across all your Research Interests.
**Parameters:** None required. Optionally pass `period` (`today`, `week`) and `limit`.
**Returns:** Recommended resources grouped by Research Interest, with titles, abstracts, relevance scores, and links.
**Example prompt:** *"Show me today's research digest"*
List all of your configured Research Interests.
**Parameters:** None.
**Returns:** Array of interests with name, context, daily paper count, and status (active/paused).
**Example prompt:** *"What Research Interests do I have set up?"*
Trigger a fresh resource ranking for one or all Research Interests.
**Parameters:** `interest_name` (optional; refresh all if omitted).
**Returns:** Confirmation that the refresh job was started. Results are available via `get_digest` after processing.
**Example prompt:** *"Refresh my Retrieval Systems interest"*
Get resources for a specific Research Interest.
**Parameters:** `interest_name` (required), `period` (optional), `limit` (optional).
**Returns:** Resources matched to the specified interest with metadata, scores, and Docker availability.
**Example prompt:** *"Show me resources about spatial reasoning from this week"*
List the projects in your workspace.
**Parameters:** None.
**Returns:** Array of projects with ID, name, and description. Use a project ID with `set_project_context` or any experiment tool.
**Example prompt:** *"What projects do I have?"*
Create a new Research Interest.
**Parameters:** `name` (required), `context` (optional; the natural-language description Remyx uses to rank resources).
**Returns:** The created interest with its ID and status.
**Example prompt:** *"Create a Research Interest called 'Long-context retrieval' focused on RAG over million-token corpora"*
Update an existing Research Interest's name, context, or status.
**Parameters:** `interest_name` (or `interest_id`) required; optional `name`, `context`, `status` (`active`/`paused`).
**Returns:** The updated interest.
**Example prompt:** *"Pause my Long-context retrieval interest"*
Delete a Research Interest.
**Parameters:** `interest_name` (or `interest_id`) required.
**Returns:** Confirmation of deletion.
**Example prompt:** *"Delete the Long-context retrieval interest"*
### Repo Analysis Tools
Generate or refine a Research Interest from a GitHub repository. Remyx reads the repo and derives the interest context from what the code actually does.
List the GitHub repositories available to analyze for an interest (those reachable through your connected GitHub App installation).
**Parameters:** None required.
**Returns:** Array of repos with owner, name, and default branch.
**Example prompt:** *"Which repos can I analyze for a Research Interest?"*
Kick off analysis of a GitHub repo to generate a Research Interest from it.
**Parameters:** `repo` (required, `owner/name`), `interest_name` (optional; target interest to attach the analysis to).
**Returns:** A job handle; poll with `get_repo_analysis_status`.
**Example prompt:** *"Analyze acme/inference-server and turn it into a Research Interest"*
Check the status of a repo-analysis job.
**Parameters:** `job_id` (or `repo`) required.
**Returns:** Status (`pending`/`running`/`complete`/`failed`) and the derived interest when complete.
**Example prompt:** *"Is the analysis of acme/inference-server done yet?"*
Re-run analysis on a repo to refresh the interest context after the codebase has changed.
**Parameters:** `interest_name` (or `interest_id`) required; optional `repo` override.
**Returns:** A job handle; poll with `get_repo_analysis_status`.
**Example prompt:** *"Regenerate my inference-server interest from the latest code"*
### Project Context Tools
Set a default project once and omit `project_id` on subsequent calls (gcloud-style context resolution). Experiment tools (and others that take a `project_id`) resolve it from this context when the parameter is omitted.
Set the default project for this session.
**Parameters:** `project_id` (required).
**Returns:** Confirmation with the resolved project name.
**Example prompt:** *"Set my project context to Customer Support AI"*
Show the currently active default project.
**Parameters:** None.
**Returns:** The active `project_id` and name, or a note that no context is set.
**Example prompt:** *"What project context am I in?"*
Clear the default project so calls require an explicit `project_id` again.
**Parameters:** None.
**Returns:** Confirmation the context was cleared.
**Example prompt:** *"Clear my project context"*
### Experiment Tools
Create a new experiment in Remyx.
**Parameters:**
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | ----------------------------------------------------------- |
| `name` | string | Yes | Experiment name |
| `source_type` | string | No | `paper`, `hypothesis`, `incident`, or `recommendation` |
| `source_ref` | object | No | Source details (paper ID, hypothesis text, etc.) |
| `target_metric` | string | No | Business metric this experiment aims to move |
| `tags` | array | No | Free-form tags for grouping |
| `project_id` | string | No | Project ID (resolved from `set_project_context` if omitted) |
| `resource_id` | string | No | Resource ID (for research-sourced experiments) |
**Returns:** Created experiment with ID and all fields.
**Example prompt:** *"Create an experiment called 'Hybrid search with re-ranking' targeting resolution-rate, tagged retrieval and re-ranking, in the Customer Support AI project"*
List experiments with optional filters.
**Parameters:** `project_id` (optional), `status` (optional), `tags` (optional).
**Returns:** Array of experiments with all ExperimentOps fields.
**Example prompt:** *"Show me all experiments in Customer Support AI"*
Update fields on an existing experiment.
**Parameters:**
| Parameter | Type | Description |
| ------------------ | ------ | ------------------------------------------- |
| `experiment_id` | string | Required |
| `observed_delta` | float | Measured result |
| `delta_confidence` | string | `significant`, `inconclusive`, `not_tested` |
| `tags` | array | Updated tag list |
| `pr_url` | string | Link to implementation PR |
| `pr_status` | string | PR status |
| `ab_test_config` | object | A/B test configuration |
| `ab_test_results` | object | Test results |
| `status` | string | Experiment status |
**Returns:** Updated experiment.
**Example prompt:** *"Update the hybrid search experiment with observed delta +3.1%, statistically significant"*
Record the team's decision on an experiment and the reasoning behind it. This is the key ExperimentOps primitive: capturing the *why* behind a call.
**Parameters:**
| Parameter | Type | Required | Description |
| --------------- | ------ | -------- | -------------------------------- |
| `experiment_id` | string | Yes | Which experiment |
| `decision` | string | Yes | The decision text with reasoning |
| `outcome` | string | No | `ship`, `iterate`, `abandon` |
**Returns:** Updated experiment with decision, timestamp, and author.
**Example prompt:** *"Log decision on experiment 12: Ship to 100%. This is the synthesis experiment — it combined retrieval with few-shot selection and produced our best single-experiment improvement."*
Get the full implementation context for a research-sourced experiment, including resource metadata, Docker environment, repo structure, and implementation plan.
**Parameters:** `experiment_id` (required).
**Returns:** Launch context object with resource details, file tree, and implementation plan.
**Example prompt:** *"Get the implementation context for experiment 7"*
Trigger the coding-agent implementation pipeline for a research-sourced experiment. This generates a PR in the target repository.
**Parameters:** `experiment_id` (required).
**Returns:** Implementation status and PR link when complete.
**Example prompt:** *"Run the implementation for experiment 3"*
Poll the implementation pipeline for a running implementation.
**Parameters:** `experiment_id` (required).
**Returns:** Pipeline stage (`queued`/`generating`/`opening_pr`/`complete`/`failed`) and the PR link once the pipeline opens it.
**Example prompt:** *"What's the implementation status for experiment 3?"*
### Decision Policy Tools
Define the rules Remyx uses to recommend ship / iterate / reject decisions on an experiment's outcome: the thresholds and confidence requirements that gate a terminal transition. See [Evaluation](/platform/review/evaluation).
Set the decision policy for a project (or experiment).
**Parameters:** `project_id` (optional; resolved from `set_project_context`), `policy` (object: thresholds, required confidence level, and outcome rules).
**Returns:** The stored policy.
**Example prompt:** *"Set a decision policy: ship only when the observed delta is positive and statistically significant, otherwise iterate"*
Show the active decision policy.
**Parameters:** `project_id` (optional; resolved from context).
**Returns:** The current policy, or a note that none is set (defaults apply).
**Example prompt:** *"What's my decision policy for Customer Support AI?"*
Remove the decision policy so default decision behavior applies.
**Parameters:** `project_id` (optional; resolved from context).
**Returns:** Confirmation the policy was cleared.
**Example prompt:** *"Clear the decision policy for this project"*
### Validation Tools
These tools run **real validation jobs** on connected compute ([Modal](/platform/configure/connectors#modal) or [Weights & Biases](/platform/configure/connectors#wandb) Launch) and return scored results.
Provision and start a validation run for an experiment on connected compute (Modal or W\&B Launch).
**Parameters:** `experiment_id` (required), `provider` (optional; `modal` or `wandb`, defaults to the connected provider), `config` (optional; run configuration).
**Returns:** A validation job handle; poll with `check_validation_status`.
**Example prompt:** *"Provision a validation run for experiment 7 on Modal"*
Check the status of a provisioned validation run.
**Parameters:** `experiment_id` (required), or `job_id`.
**Returns:** Job status (`queued`/`running`/`complete`/`failed`) and progress.
**Example prompt:** *"Is the validation for experiment 7 finished?"*
Cancel an in-flight validation run.
**Parameters:** `experiment_id` (required), or `job_id`.
**Returns:** Confirmation the job was cancelled.
**Example prompt:** *"Cancel the validation run for experiment 7"*
Get the scored results of a completed validation run.
**Parameters:** `experiment_id` (required).
**Returns:** Scored validation data: metrics from the run, plus any control vs. treatment comparison recorded for the experiment.
Scored validation runs on Modal / W\&B Launch are live. The full **external A/B close-loop** (pulling live treatment vs. control results from Statsig / LaunchDarkly) is still pending; those data still arrive via `update_experiment` for now.
Close the experiment loop: record final results and move the experiment to a terminal state (`shipped`, `iterating`, `rejected`, or `abandoned`).
**Parameters:** `experiment_id` (required), `final_results` (object).
**Returns:** Updated experiment in its terminal state with final results recorded.
Scored validation results from Modal / W\&B Launch can be closed in directly. Automatically pulling results from external A/B platforms (Statsig / LaunchDarkly) is still pending.
***
## Slack Agent
The MCP server also powers the managed Slack agent. When you `@mention` the Remyx bot in Slack, it uses the same MCP tools to respond: fetching digests, listing agents' output, logging decisions, etc. See [Connectors > Slack](/platform/configure/connectors#slack) for setup.
***
## Studio agent chat
The **Ask about this agent** drawer in the studio (and the Inbox's **Ask** button) runs on a hosted chat runner attached to this server. Each turn runs a headless agent session whose only capabilities are a **read-only allowlist** of these MCP tools, called with your own authority — no shell, no filesystem, no network beyond the MCP surface. Mutating verbs (`trigger_agent_run`, `pause_agent`, `install_agent`, `fork_repo`) are deliberately absent from the chat: it proposes those actions in text, and the studio's confirm-gated buttons execute them through the engine.
The session is powered by your own connected model provider — selectable per thread when several are connected, otherwise the first available one — your key, your spend, mirroring how your agent runs are billed.
***
## Implementation Pipeline
For research-sourced experiments with a launch context, a connected coding agent can run the full implementation flow:
1. Fetch the experiment context (resource metadata, Docker environment, repo structure, implementation plan)
2. Read the target repository
3. Generate code changes implementing the technique
4. Push a branch and open a PR
5. Link the PR back to the experiment
The PR status syncs via GitHub webhooks. When the PR is merged, the experiment status updates automatically. Scheduled [agent runs](/platform/agents/outrider-action) are the always-on version of this same loop.
***
## Next Steps
The fleet the tool plane drives
Set up the managed Slack agent
Mint a scoped key for your MCP client
REST API documentation
# Roadmap
Source: https://docs.remyx.ai/roadmap
What's shipped, what's in development, and what's planned for Remyx
# Roadmap
This page tracks the current state of Remyx's capabilities. We publish it so customers, collaborators, and researchers can see where we're going well before each capability ships.
For the architectural through-line, see [Causal Intelligence](/concepts/causal-intelligence). For how these capabilities map to the customer journey, see [Maturity Progression](/concepts/maturity-progression).
***
## Status legend
| Indicator | Status | Meaning |
| ------------------------------ | ------------------ | ----------------------------------------- |
| | **Shipped** | Available in production today |
| | **In development** | Actively being built |
| | **Planned** | Committed to the roadmap, not yet started |
| | **Research** | Exploring feasibility, not yet committed |
***
## Agents & ExperimentOps
These capabilities are live in production today. They form the Stage 2 foundations: the agent fleet ([Agents](/platform/agents/overview), [Inbox](/platform/review/inbox), [Reports](/platform/review/reports)) plus the [ExperimentOps](/concepts/experimentops) machinery underneath it.
| Status | Capability | Description |
| ------------------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Shipped | Experiment capture | Full lifecycle from origin through decision, with target metric, hypothesis, and decision rationale captured directly on the experiment record |
| Shipped | Cross-experiment patterns | Tag-based clustering identifies which directions consistently produce results |
| Shipped | Resource discovery | Semantic search across papers, repos, models, and datasets, matched to your team's experiment history |
| Shipped | History-aware recommendation ranking | The structured shipping history extracted from your repo feeds the recommendation ranker, so candidates that align with your team's actual trajectory rank higher |
| Shipped | Preference-model ranking | A per-team preference model fit over past experiments scores candidate work and breaks ties behind relevance |
| Shipped | Outrider agents | Outrider installed per repo: a scheduled GitHub Action that opens a reviewable draft PR — or a discussion Issue — for the next paper worth implementing; a person reviews and merges |
| Shipped | Multi-backend agents | Agent runs route at Anthropic (Claude), Z.ai (GLM), or Moonshot (Kimi), including the two-tier drafter/refiner fleet layout |
| Shipped | Inbox | A derived triage queue of Review / Decide / Fix items that auto-resolves when GitHub does |
| Shipped | Fleet reports | The fleet in one view: KPIs, deliverable pipeline, contribution grid, per-agent states, and cost per code artifact |
| Shipped | Agent direction learning | Every merge, skip, and discussion re-weights what each agent proposes next — surfaced as evidence-backed Promoted / Demoted / Held directions |
| Shipped | Connector framework | GitHub (GitHub App), model providers (Anthropic / Z.ai / Moonshot), Linear, Jira, Slack, W\&B, Hugging Face, Modal |
| Shipped | MCP server | Programmatic access to the fleet and platform from Claude Code and other MCP clients, including the Outrider tool plane |
| In development | Standalone Stage 1 product surface | Milestone-driven recommendations for early-dev teams without production traffic |
***
## Causal intelligence, evidence layer
Stage 2. The evidence layer feeding the causal model.
| Status | Capability | Description |
| -------------------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Planned | Observational log ingestion | Pluggable adapters for Datadog, Honeycomb, structured JSON, OpenTelemetry. Customers connect existing telemetry, and Remyx populates the evidence layer without instrumentation changes |
| Planned | Commit-correlated regime boundary detection | Extension of the existing repo integration to identify regime changes in the data-generating process from commit history |
| Planned | Quasi-experiment identification | Combine observational logs with regime boundaries to produce identified causal effects via difference-in-differences, interrupted time series, or regression discontinuity |
| Planned | Causal discovery | Bootstrap a partial causal graph from observational data, supplemented by regime-boundary structure. Discovered structure is human-validated before being treated as the working model |
| Planned | Causal graph engine | Versioned causal graph as a top-level object, supporting interventional, counterfactual, and mediation-aware graphical models. Semi-Markovian formulation to handle latent confounders |
| Planned | Causal data fusion | Combine evidence from multiple sources into one coherent posterior. Conflict resolution, graph refinement proposals, continuous incremental updates |
***
## Causal intelligence, query and interaction
| Status | Capability | Description |
| -------------------------------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Planned | Identification dispatcher | Given a question, classify it by required identification layer, route to evidence sources with appropriate identification, and dispatch to estimation logic |
| Planned | Natural language query layer | Customer-facing interface that takes natural language questions, parses into formal estimands, and returns natural language answers with identification status and recommendations |
***
## Causal intelligence, Stage 3
| Status | Capability | Description |
| -------------------------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------------------- |
| Planned | A/B test integration framework | Connectors for Statsig, Eppo, LaunchDarkly. A/B test results become an evidence source feeding the causal model |
***
## Causal intelligence, Stage 4
| Status | Capability | Description |
| -------------------------------------- | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Planned | Shadow-decision SDK (log-only mode) | Python SDK that wraps decision points in your AI system. The first version captures natural policy output without applying overrides |
| Planned | Shadow-mode audit infrastructure | Dedicated product surface for the shadow-mode adoption phase. Audit trail viewer, override proposal review, and compliance reporting |
| Planned | CTF-RAND override policies | Extension of the SDK with counterfactual randomization. Trajectory-consistent semantics by default. Per-decision-point semantics opt-in for mediation analysis |
| Planned | ETT and NDE estimation | Counterfactual estimation procedures for effect-of-treatment-on-the-treated and natural direct effect |
***
## Hypothesis triage
The triage layer matures with the customer — and it is what the agent fleet's ranking grows into. Stage 2 customers see quasi-experimental recommendations. Stage 3 customers see A/B test recommendations. Stage 4 customers see CTF-RAND recommendations.
| Status | Capability | Description |
| -------------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Planned | Hypothesis ranking and evidence path recommendation | Rank hypotheses by expected information gain. Identify the cheapest evidence path to an answer for each |
| Planned | Orchestration scheduler | Coordinate active interventions (CTF-RAND randomizations, A/B tests, quasi-experiment analyses) for maximum concurrency without compromising estimate validity |
| Planned | Identification-enabling intervention proposals | Proactively propose CTF-RAND or A/B interventions that would make currently-unidentifiable hypotheses estimable |
***
## How we sequence
Foundational evidence-layer capabilities ship first. The causal model and query layer come next. A/B integration follows. Counterfactual perturbations ship last.
The dependency hot path through the architecture follows this order.
1. Evidence schema.
2. Observational log ingestion.
3. Quasi-experiment identification.
4. Causal graph engine and data fusion.
5. Identification dispatcher and natural language query.
6. A/B integration.
7. Shadow-decision SDK.
8. CTF-RAND override policies.
9. ETT and NDE estimation.
This roadmap is shared publicly. We update it as priorities shift.
# Connect Tools to Your Experiments
Source: https://docs.remyx.ai/tutorials/connect-tools-to-experiments
Set up GitHub, Linear, and Slack so your experiments automatically link to PRs, tickets, and team conversations.
Every experiment generates artifacts across multiple tools — a PR in GitHub, a ticket in Linear, a thread in Slack. Without linking them, you're context-switching between tabs and copying URLs manually. This tutorial connects your tools so that creating an experiment in Remyx automatically creates a linked ticket, and changes in either direction sync in real time.
## What this sets up
By the end of this tutorial, you'll have:
* **GitHub** connected for PR linking and repo file tree access
* **Linear** connected for automatic ticket creation and bidirectional status sync
* **Slack** connected for the managed AI agent and daily digests
**Starting point:** You have a Remyx account and at least one experiment created. You have admin access to a GitHub org, Linear workspace, and Slack workspace.
***
## Step 1: Connect GitHub
GitHub integration gives Remyx access to your repos for launch context generation and PR linking.
1. Navigate to **Connectors** in the sidebar
2. Click **Connect** on the GitHub card
3. You'll be redirected to GitHub to authorize the Remyx app
4. Select the organization and repositories to grant access to
5. Click **Authorize** — you'll be redirected back to Remyx
### Verify
Go to any experiment detail page and check the **Implement** section. You should see the Claude Code command pre-configured with your connected repo.
***
## Step 2: Connect Linear
Linear integration creates tickets automatically when you create experiments.
1. Navigate to **Connectors** in the sidebar
2. Click **Connect** on the Linear card
3. Authorize Remyx when prompted
4. Go to **Project Settings** for your project (click the gear icon from any experiment page)
5. Under **Linear Project**, select which Linear project should receive experiment tickets
### Verify
Create a new experiment. After creation, check:
* The experiment detail page shows a **Linear ticket** link in the sidebar
* Opening the Linear ticket shows the experiment name, hypothesis, and a link back to Remyx
### Bidirectional Sync
Changes in Linear automatically update the experiment in Remyx:
| Action in Linear | Effect in Remyx |
| --------------------- | ------------------------------ |
| Change issue status | Experiment status updates |
| Add `ship` label | Decision state updated |
| Add `abandon` label | Decision state updated |
| Comment with `@remyx` | AI agent responds with context |
***
## Step 3: Connect Slack
The Slack integration adds a managed AI agent to your workspace.
1. Navigate to **Connectors** in the sidebar
2. Click **Connect** on the Slack card
3. Install the Remyx AI app to your workspace
4. Select a default channel for digest delivery (e.g., `#ml-experiments`)
5. In Slack, invite the bot: `/invite @Remyx` in your channel
### Verify
In your Slack channel, type:
```
@remyx show me my experiments
```
The bot should respond with a list of your recent experiments.
### Test the Digest
In your Slack channel, ask:
```
@remyx show me today's digest
```
The bot should respond with your latest recommendations grouped by Research Interest.
***
## Step 4: Test the Full Flow
Now let's test everything end-to-end:
Create an experiment with a hypothesis, target metric, and tags (agent drafts file into your project as proposed experiments automatically; this manual path exists for work an agent didn't originate).
A ticket should appear in your linked Linear project within seconds. Open it — the description includes the hypothesis and a link back to Remyx.
Change the ticket status in Linear (e.g., move to "In Progress"). Switch back to the Remyx experiment detail page — the status should be synced.
In your channel, ask: `@remyx what's the status of my latest experiment?` The bot responds with the current experiment state.
***
## Troubleshooting
Check that your project has a linked Linear project in **Project Settings**. Without this, Remyx doesn't know which project to create tickets in.
Verify the webhook exists: In Linear, go to **Settings > API > Webhooks**. You should see a Remyx webhook. If it's missing, disconnect and reconnect Linear at **Connectors**.
Make sure the bot is invited to the channel (`/invite @Remyx`). Check that the Slack connection shows as active at **Connectors**.
***
## Next Steps
Use the MCP server to implement experiments with AI
Full integration documentation
# Create your project
Source: https://docs.remyx.ai/tutorials/get-started/create-your-project
Connect a repo and watch Remyx extract structured experiment history from your merge log.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Setup phase · \~3 minutes*
A **project** in Remyx is a workspace that scopes a set of related experiments. It carries shared context (the eval template, the decision policy, the discovery feed, integration credentials) so individual experiments don't have to re-establish it.
The fastest way to get a project off the ground is to connect a repo and let Remyx extract structured experiment records from the merge history. That's what this tutorial does.
The whole series uses [remyxai/VQASynth](https://github.com/remyxai/VQASynth) as a stand-in example. Substitute your own repo URL anywhere it appears. The flow is identical.
## Connect the repo
1. Open the project switcher at the top of the sidebar and click **+ New project**. (You can also reach the same wizard from the **Projects** page under Manage.)
2. The wizard opens with a tabbed repo picker. Pick your repo from the **From GitHub** tab if it's connected, or use **Paste URL** to provide a public URL. To follow along, paste `https://github.com/remyxai/VQASynth`.
3. Give the project a name (defaults to the repo name) and write a one-paragraph **scope context** describing what you care about. The scope context shapes both the cold-start extraction and downstream recommendations.
For VQASynth, scope context might read:
> Synthetic data pipeline for training spatial-reasoning VLMs. We care about pipeline-stage swaps (depth estimator, segmentation, captioner) and their downstream effect on benchmark accuracy. Not focused on hyperparameter sweeps.
For an LLM application:
> Customer support chatbot built on Claude. We care about retrieval quality, especially for ambiguous tickets, and routing rules that decide when to escalate to a human agent.
For a retrieval system:
> Hybrid retrieval over technical documentation. Prioritizing recall for multi-hop queries and reducing latency on the rerank step.
4. Check **Create a research interest from this project**. The next tutorial covers the digest in detail; checking this here saves a step.
5. Click **Create**.
![New Project wizard with repo URL, scope context, and research interest checkbox filled in]()
A background extraction job kicks off as soon as the project is created. It walks the recent merged PRs in your repo and turns each meaningful one into a structured **experiment** (a single tracked change with a hypothesis, a target metric when one is implied, and a link back to the PR).
## Watch the history extract
The Experiments tab fills in as the extraction runs. Records carry a **Backfilled** badge, meaning Remyx auto-extracted them from a merged PR (as opposed to experiments you create manually later).
![Experiments tab showing extracted records with Backfilled badges]()
This extracted history becomes part of your project's standing context. Every experiment you create from here can reference it. Recommendations and decisions get sharper as the context grows.
## See the timeline
Click the **Outcomes** tab. Every experiment under the currently selected project lives here, grouped by date. The chart-mode selector lets you flip between three views (Trend, Velocity, Impact).
The project switcher at the top of the sidebar scopes Outcomes, Insights, and any other project-level view. Switching projects updates all of them at once.
![Outcomes timeline with Velocity chart mode active]()
Backfilled experiments often don't have measured deltas. Remyx has the merge but not the metrics, so they appear under a **Baseline only** badge and contribute to velocity but not impact. As you fill in deltas on individual records, the Impact chart populates.
## Recap
You now have:
* A Remyx project linked to your repo
* An experiment record extracted from your repo's merge history
* An Outcomes timeline showing what's shipped under this project
The extraction is the seed of the project's shared context. Subsequent tutorials build on it.
## Next
Turn the extracted history into a daily digest of relevant new work.
See the full arc
Wire up GitHub, Linear, Slack, and Jira
# Define how progress gets measured
Source: https://docs.remyx.ai/tutorials/get-started/define-how-progress-gets-measured
Lock in the eval template and decision policy that future variants will be scored against, before any of them exist.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Setup phase · \~10 minutes*
Evaluating AI variants reliably needs the same discipline as any reproducible test: same setup, same baseline, same metrics, same threshold for what counts as progress. Without that, results from one experiment aren't comparable to the next, and improvements get harder to attribute to specific changes.
This tutorial creates two project-level artifacts that lock in that consistency:
* An **eval template**. A saved config that says how to score a variant against a baseline. What script to run, which baseline to compare against, which metrics matter.
* A **decision policy**. A saved set of rules that says when a variant counts as shipping, when it counts as rejecting, and when the team should iterate.
Both are written before any variant exists. When results land in [Run an evaluation](/tutorials/get-started/run-an-evaluation), the team's bar is already on the page next to the numbers.
**Prerequisites.** You've completed [Create your project](/tutorials/get-started/create-your-project). The project exists and you can open Project Settings.
## Define your allowed metrics
Before configuring the eval, declare the metrics this project tracks. Allowed metrics are the project-level vocabulary every experiment, eval template, and decision rule under this project draws from. They appear as dropdown options when creating experiments and as suggestions in the eval template's metric picker.
1. Open **Project Settings** → **Allowed Metrics**.
2. Click **+ Add a metric**.
3. Fill in:
* **Key**: machine identifier (lowercase, e.g., `spatialscore`). This is what the eval script emits and what decision rules reference.
* **Label**: how the metric is displayed in the UI (e.g., "SpatialScore").
* **Unit**: what the value represents (e.g., `accuracy`, `%`, `seconds`).
* **Baseline**: the current value to compare future variants against.
4. Repeat for every metric this project will track.
For VQASynth, define one row per spatial benchmark:
| Key | Label | Unit | Baseline |
| -------------- | ------------ | -------- | -------- |
| `spatialscore` | SpatialScore | accuracy | `0.42` |
| `omnispatial` | OmniSpatial | accuracy | `0.31` |
| `space10` | SpaCE-10 | accuracy | `0.55` |
| `mindcube` | MindCube | accuracy | `0.38` |
The baseline values are what `Qwen/Qwen2.5-VL-7B-Instruct` scores on each benchmark today. Future variants will be measured as a delta against these numbers.
![Allowed Metrics section with four spatial benchmark rows]()
Keep metric labels business-oriented and short. They show up on charts, in dropdowns, and in decision-policy rule descriptions. "SpatialScore" reads better than "Spatial Score Accuracy %"; `spatialscore` as a key reads better than `spatial_score_metric`.
***
## Configure the eval template
The eval template tells Remyx **how to measure**.
1. Scroll down to **Project Settings** → **Evaluation**.
2. Click **+ Configure eval**.
3. Pick a runtime.
* **Custom.** Point Remyx at an evaluation script that already lives in your repo. Provide the script path and a **baseline asset** (the model, dataset, or process variants will be compared against).
* **lighteval.** Pick a task from the supported whitelist (e.g., `gsm8k`, `mmlu`). Metric defaults populate automatically.
4. Add the metrics you want this template to track. The listbox suggests the allowed metrics you just defined; select from the suggestions or type your own.
5. Click **Save and lock**.
For VQASynth, that looks like:
* Runtime: **Custom**
* Eval script path: `docker/eval_stage/`
* Baseline asset: `Qwen/Qwen2.5-VL-7B-Instruct` (a HuggingFace model slug)
* Metrics: `spatialscore`, `omnispatial`, `space10`, `mindcube` (the four allowed metrics from the previous step)
![Eval template form with Custom runtime, script path, baseline asset, and metrics]()
The template appears with a green **locked** badge. Locked templates are what [Run an evaluation](/tutorials/get-started/run-an-evaluation) runs against, and they autocomplete metric names into the decision policy below.
![Eval template saved with green locked badge]()
Locking the template doesn't run the eval. Locking captures the spec so the policy editor below can autocomplete metric names from your template, and so future runs use the same harness every time.
**Use lighteval** when the metric you care about is one of the standard benchmarks the runtime ships with (`gsm8k`, `mmlu`, etc.). The metric chip auto-populates with the task's canonical metric (`exact_match`, `accuracy`), and you don't need to maintain an eval script in your repo.
**Use Custom** when your repo already ships its own eval script. Your script defines what runs, what metrics come out, and what the baseline asset is.
You can create additional templates later if you have multiple evaluation strategies you'd like to deploy.
A Baseline asset is whatever your project's variants will be compared against. The exact form depends on your eval script and what your project produces.
## Lock in the decision policy
The decision policy tells Remyx **what counts as progress**. The eval template gives you measurements; the policy gives you a verdict against pre-committed rules.
1. Open **Project Settings** → **Decision Policy**.
2. Build a **ship** rule. This defines what the variant must achieve to count as progress worth shipping.
* Pick the combinator (`all` or `any`).
* Add metric conditions: pick a metric (autocomplete from your locked template), pick an operator (`>=`, `>`, `improves_by`, etc.), set a threshold, optionally set a confidence floor.
* Add field conditions if you want to gate on confidence or sample size (e.g., `delta_confidence >= 0.80`).
3. Build a **reject** rule. This defines what disqualifies the variant from shipping.
* Same shape as ship. Common pattern: a `no_regression` condition on every metric you'd be unhappy to see drop.
4. Optional: add **notes** with free-form context the team should remember about why these criteria.
5. Click **Save policy**. A "Changes saved" flash confirms.
`iterate` is the automatic fallback. When neither ship nor reject matches, Remyx classifies the run as iterate and prompts the team to keep working.
![Decision Policy editor with ship and reject rules and iterate fallback]()
For a project comparing two VLMs on four spatial benchmarks, a defensible policy might read:
**Ship** (combinator: `all`)
* `spatialscore.delta` `>=` `0.02`, confidence floor `0.80`
* `omnispatial.delta` `>=` `0.02`, confidence floor `0.80`
* `delta_confidence` `>=` `0.80` (field condition)
**Reject** (combinator: `any`)
* `spatialscore.delta` `<` `-0.01`
* `omnispatial.delta` `<` `-0.01`
* `space10.delta` `<` `-0.01`
* `mindcube.delta` `<` `-0.01`
**Notes**: *"3-of-4 across the suite at 80% confidence to ship. Any single benchmark regression worse than 1% rejects."*
If the variant improves on some benchmarks but doesn't clear the ship bar, and nothing regresses badly enough to trigger reject, the team iterates.
## Recap
You now have:
* A locked eval template defining how variants will be scored
* A locked decision policy defining what counts as ship, reject, or iterate
* A team-agreed bar that lives in the project alongside its history
When you run an evaluation in [Run an evaluation](/tutorials/get-started/run-an-evaluation), the policy renders alongside results with a per-predicate ✓ / ✗ / · indicator.
## Tips
The decision-policy editor autocompletes metric names from the locked eval template. Writing a policy without a locked template means typing metric names from memory, which leads to typos that won't be caught until evaluation runs.
A `reject` rule with `no_regression` on every metric you wouldn't want to see drop is cheap insurance. Without it, a variant that improves the headline metric but tanks a secondary one falls through to iterate, which can be misleading.
Put context there that future-you (or a future teammate) will need to remember why this policy is what it is. The structured rules are what get evaluated against results.
## Next
Turn a paper into a structured experiment with hypothesis, target metric, and tags.
Full series arc
Why pre-committed criteria matter
# Implement an Experiment
Source: https://docs.remyx.ai/tutorials/get-started/implement-an-experiment
Generate a PR that implements your scoped experiment.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Build phase · \~10 minutes*
The experiment from [Scope an experiment from a recommendation](/tutorials/get-started/scope-an-experiment) is scoped but unimplemented. There's no code yet. This tutorial connects Claude Code to your Remyx project over MCP and uses the experiment's launch context to generate a PR that puts the variant into your repo.
**Prerequisites.** You've completed [Scope an experiment from a recommendation](/tutorials/get-started/scope-an-experiment). There's a scoped experiment with a generated launch context. You also need [Claude Code installed](https://docs.claude.com/en/docs/claude-code/setup) on your machine.
## Get your Remyx API key
1. Open [studio.remyx.ai/account](https://studio.remyx.ai/account).
2. Go to **API Access**.
3. Create a new key (or copy an existing one).
Keep the key handy. You'll paste it into Claude Code's MCP config in the next step.
## Connect Claude Code to Remyx
Add the Remyx MCP server to Claude Code's settings. Two scopes are available. Pick whichever fits your workflow.
Edit `~/.claude/settings.json`:
```json theme={null}
{
"mcpServers": {
"remyx": {
"type": "http",
"url": "https://mcp.remyx.ai/mcp",
"headers": {
"Authorization": "Bearer YOUR_REMYX_API_KEY"
}
}
}
}
```
Available across every Claude Code session.
Create or edit `.claude/settings.json` in your repo's root:
```json theme={null}
{
"mcpServers": {
"remyx": {
"type": "http",
"url": "https://mcp.remyx.ai/mcp",
"headers": {
"Authorization": "Bearer YOUR_REMYX_API_KEY"
}
}
}
}
```
Scoped to this repo only.
### Verify the connection
Start Claude Code in your repo, then ask:
```
What Remyx tools do you have available?
```
The response should list the available MCP tools, including `get_digest`, `get_experiments`, `create_experiment`, `update_experiment`, `get_experiment_context`, `log_decision`, and `set_project_context`.
## Set the project context
So that subsequent tool calls operate against the right Remyx project:
```
> Set my project context to the VQASynth project I just created.
[Claude calls set_project_context]
```
Project-scoped tool calls (`get_experiments`, `create_experiment`, etc.) use this context automatically. Override on individual calls to switch projects mid-session.
## Pull the experiment context
Get the launch context for the experiment scoped in the previous tutorial:
```
> Show me the experiment we scoped for the depth-estimator swap,
with its full implementation context.
[Claude calls get_experiment_context]
```
The response includes:
* **Resource metadata.** Title, abstract excerpt, key methods, your annotations.
* **Docker environment.** Reference to a pre-built environment if the resource ships one.
* **Target repo structure.** File tree of the repo so the agent knows where things live.
* **Implementation plan.** AI-generated plan referencing actual file paths in your repo.
This is what the agent uses to ground the implementation. The more you edited the implementation plan during scoping, the cleaner the result.
## Generate the implementation
Ask the agent to implement the technique:
```
Implement the technique from this experiment in my repo,
following the implementation plan. Open a PR when done.
```
The agent will:
1. Read the implementation plan.
2. Navigate the repo structure.
3. Generate the code changes.
4. Commit on a new branch.
5. Push and open a PR.
For an automated end-to-end run that bundles the steps:
```
Run the implementation pipeline for the depth-estimator-swap experiment.
```
This calls `run_experiment_implementation`, which triggers the full clone → implement → push → PR sequence in one shot.
## Link the PR back to the experiment
Once the PR is open, link it to the experiment so status syncs and you can find it later:
```
Update the depth-estimator-swap experiment with the PR link
https://github.com/myorg/myrepo/pull/123.
```
The agent calls `update_experiment` to record the PR.
If your repo has the GitHub integration connected (see [Connectors](/platform/configure/connectors)), the PR's status (open, merged, closed) syncs automatically via webhooks. You don't need to update the experiment manually as the PR moves through review.
## Review and iterate
The implementation produced is a starting point. Treat the PR like any other: review the diff, run local tests, adjust where the agent missed nuance, push commits.
The experiment's status updates as the PR moves. It transitions from `draft` to `running` once the variant is built and the eval can pick it up in the next tutorial.
## Recap
You now have:
* Claude Code connected to Remyx via MCP
* A PR in your target repo that implements the experiment
* The PR linked back to the Remyx experiment, with status syncing automatically
The variant is real code that will be evaluated next.
## Full session example
End-to-end Claude Code session:
```
> Set my project context to the VQASynth project.
[Claude calls set_project_context]
> Show me today's discovery digest.
[Claude calls get_digest, lists recommendations]
> Create an experiment from the VGGT depth-estimator paper.
Hypothesis: VGGT will lift spatialscore on the suite by ≥2%
without regressing latency. Target metric: spatialscore.
Tags: depth-estimation. Project: VQASynth.
[Claude calls create_experiment]
> Get the implementation context for that experiment.
[Claude calls get_experiment_context]
> Implement the technique following the plan, open a PR.
[Claude reads plan, edits files, commits, pushes, opens PR]
> Update the experiment with the PR link.
[Claude calls update_experiment]
```
After this, [Run an evaluation](/tutorials/get-started/run-an-evaluation) runs the actual eval against the locked template.
## Tips
The agent matches against your experiment list. Specific names avoid ambiguity when several experiments share keywords.
Open the experiment in the Remyx UI and edit the AI-generated implementation plan before asking for the implementation. Specific guidance produces better PRs than letting the agent infer from the launch context alone.
With GitHub connected, the PR auto-links to the experiment. Status changes (merged, closed) sync via webhooks. You don't need to manually update the experiment as the PR moves through review.
## Next
Score the variant against the locked eval template on Modal, log the decision.
Full Remyx tool list for agents
Full series arc
# Build Your First Project
Source: https://docs.remyx.ai/tutorials/get-started/overview
An end-to-end walkthrough of the experimentation cycle in Remyx. Connect a repo, set up shared context, run an experiment, and review the result.
This series walks through the full cycle on a real open-source AI project. You'll connect a codebase, set up the shared context that grounds future work, scope a change from a recommendation, implement it, evaluate it against a pre-committed standard, and track the result back into your project's history.
By the end, you'll have closed one loop by hand and have a working pattern you can repeat on your own repos.
**The fastest first step is the [Quickstart](/quickstart)** — install an Outrider agent and let it land a draft PR for you. This series is the manual, understand-every-step version of what agents then do on a schedule: the drafts they open arrive in your [Inbox](/platform/review/inbox) as proposed experiments, and the measurement machinery you set up here (metrics, eval template, decision policy) is exactly what grades them.
**Recorded on the previous UI.** The videos and screenshots in this series were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai); some navigation labels have changed.
## What you'll learn
The series has seven tutorials grouped into five phases. Each tutorial is short and focused on a single user-facing question.
| Phase | Tutorial | Time | Outcome |
| ------------- | --------------------------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Setup** | [Create your project](/tutorials/get-started/create-your-project) | \~3 min | A Remyx project linked to your repo, with experiment history extracted from merged PRs |
| **Setup** | [Set up your discovery feed](/tutorials/get-started/set-up-your-discovery-feed) | \~3 min | A daily discovery digest ranked against your team's actual work |
| **Setup** | [Define how progress gets measured](/tutorials/get-started/define-how-progress-gets-measured) | \~10 min | A locked eval template and decision policy. Your team's bar for shipping is written down before any variant runs |
| **Discovery** | [Scope an experiment from a recommendation](/tutorials/get-started/scope-an-experiment) | \~5 min | A scoped experiment with hypothesis, target metric, and tags, ready to be implemented |
| **Build** | [Implement an Experiment](/tutorials/get-started/implement-an-experiment) | \~10 min | A PR that implements the technique, generated by Claude Code reading your project context via MCP |
| **Decide** | [Run an evaluation](/tutorials/get-started/run-an-evaluation) | \~15 min | A variant scored against the locked eval template on Modal, with the decision logged |
| **Reflect** | [Stay in the loop](/tutorials/get-started/stay-in-the-loop) | \~5 min | A read on the project's velocity, the signals emerging across directions, the rationale behind every decision, and the lineage between related experiments |
Each tutorial is self-contained, but they're designed to compose. The artifact you produce in one tutorial is the input to the next. Following the whole series end-to-end leaves you with a real experiment closed in Remyx.
## The example used throughout
The series follows along on [remyxai/VQASynth](https://github.com/remyxai/VQASynth), an open-source synthetic-data pipeline for spatial-reasoning vision-language models. It's a public repo you can clone, fork, or just read alongside the tutorials.
You don't need to use VQASynth. It's a stand-in for whatever AI repo you actually care about. Wherever you see VQASynth mentioned, you can substitute your own repo URL. The flow works the same way on a retrieval stack, an LLM application, a multi-stage agent, a model-training pipeline, or anything in between.
Using a single example throughout means each tutorial's artifact carries forward to the next without re-establishing context.
## Concepts you'll meet along the way
A few terms recur across the series. Here's what they mean upfront:
* **Project.** A workspace that scopes a set of related experiments. Carries shared context (history, eval template, decision policy, integrations) so individual experiments don't have to re-establish it.
* **Experiment.** A single tracked change with a hypothesis, a target metric, and an outcome. Can be backfilled (auto-extracted from a merged PR) or created manually.
* **Eval template.** A saved configuration that says how to score a variant against a baseline.
* **Decision policy.** A saved set of rules that says when a variant counts as shipping, when it counts as rejecting, and when the team should iterate.
* **Variant.** A candidate alternative to the baseline that gets evaluated.
* **Baseline.** Whatever the variant gets compared against. The form depends on what your project produces (a model checkpoint, a dataset, a system version).
Each tutorial defines these inline as they come up. This list is here for orientation.
## Prerequisites
**Required for the full series.**
* A Remyx account
* A connected GitHub integration (see [Connectors](/platform/configure/connectors))
* Claude Code installed (for the implementation tutorial)
**Required for the evaluation tutorial.**
* A Modal account, with billing enabled
* A Hugging Face token, set as a project secret
## Start the series
Connect a repo and watch Remyx extract structured experiment history from your merge log. About three minutes.
## Or skip ahead
Daily discovery digests ranked against your team's work
Lock in your eval template and decision policy
Turn a paper into a structured experiment
Generate a PR via the MCP integration
Score the variant on Modal, log the decision
Velocity, signal patterns, decision rationale, and lineage views
# Run an evaluation
Source: https://docs.remyx.ai/tutorials/get-started/run-an-evaluation
Score your variant against the locked eval template on Modal infrastructure, see the side-by-side, and log the decision.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Decide phase · \~15 minutes*
You've configured the harness in [Define how progress gets measured](/tutorials/get-started/define-how-progress-gets-measured), scoped an experiment in [Scope an experiment from a recommendation](/tutorials/get-started/scope-an-experiment), and implemented it in [Implement an Experiment](/tutorials/get-started/implement-an-experiment). Now you find out whether the variant is worth shipping.
This tutorial triggers the eval against the locked template, reads the side-by-side results, and logs the decision against the policy you wrote in advance.
**Prerequisites.**
* You've completed the previous tutorials in the series. The project, eval template, decision policy, scoped experiment, and implementation PR all exist.
* A Modal account, connected via [Integrations](/tutorials/connect-tools-to-experiments). Modal sandboxes run under your workspace, and your account pays for the compute.
* A Hugging Face account connected via Integrations, so the eval can pull the baseline and variant model weights.
## Trigger the head-to-head
The eval runs as a two-arm Modal job. One sandbox runs the **baseline** asset (defined in your eval template), one runs the **variant** asset (what your PR produced). Each runs the locked eval script and emits the metrics defined in the template.
From the experiment detail page, click **Run evaluation**. A dialog asks for the baseline and variant references. The baseline defaults to the asset on the eval template; the variant is the model produced by the implementation PR.
Or from Claude Code:
```
> Run an evaluation on the depth-estimator-swap experiment,
baseline Qwen/Qwen2.5-VL-7B-Instruct,
variant myorg/Qwen2.5-VL-VQASynth-FT-v1.
[Claude calls provision_validation]
```
Either path calls `provision_validation`, which builds an eval-environment Dockerfile (using gitingest plus Gemini-driven generation against your repo), submits it as two parallel sandbox jobs to Modal, and registers a webhook to receive results. Runs go in parallel, so wall-clock time is whichever arm takes longer (typically the bigger model, or whichever has slower benchmarks).
While it runs, you can step away. The experiment detail page surfaces the results when both arms complete.
## Read the results
Once both arms complete, the experiment's detail page populates with the side-by-side. Each metric in the template appears with the baseline value, the variant value, the delta, and a directional confidence indicator.
Below the metrics, the **Decision policy** block from the locked template renders next to the actuals. Each predicate from the policy is evaluated against the actual deltas, with a ✓ / ✗ / · indicator next to each.
The classification is one of three things:
* **Ship.** Every predicate of the ship rule passes. The variant cleared the bar set in advance.
* **Reject.** Any predicate of the reject rule fires. The variant tripped a guard rail (e.g., regressed a benchmark by more than the threshold).
* **Iterate.** Neither ship nor reject matched. The variant moved some metrics but didn't decisively earn a ship-or-stop verdict.
These match the three terminal states of an experiment in Remyx.
## Log the decision
The classification is the engine's recommendation. The decision is yours. Sometimes you'll override (the policy said reject, but the regression was on a metric you've already deprecated; ship anyway with a note). Sometimes you'll defer (the engine said ship, but you want a second eval on a different dataset before committing).
Log what you decided and why.
From the UI: open the experiment, click the disposition button (Ship / Iterate / Reject), and write the rationale in the modal that opens.
Or from Claude Code:
```
> Log a decision on the depth-estimator-swap experiment:
Ship to 100%. Three of four benchmarks improved at >= 2%
with confidence above 0.85. The MindCube regression was
inside the noise floor.
[Claude calls log_decision]
```
A good rationale captures three things: **what** (the disposition), **why** (the reasoning grounded in the actual results), and **what's next** (any follow-up work it implies). Future-you reading this in six months will need all three.
## What just happened
You closed a loop. The experiment moves from `running` to its terminal state (`shipped`, `rejected`, or `iterating`). The rationale is captured alongside the metrics. The policy that governed the decision is recorded with the experiment, so anyone reading it later can replay the logic.
This is also when the project's outcomes timeline updates. A new node lands on the chart, the project's hit rate adjusts, and the decision becomes part of the project's standing context. Future discovery recommendations and future cluster patterns reason over this result automatically.
## Recap
You now have:
* A variant scored against the locked eval template on Modal infrastructure
* A side-by-side comparison with deltas and confidence per metric
* The decision policy evaluated against the actuals, predicate by predicate
* A logged decision with rationale
* A closed-loop experiment that's now part of the project's history
## Tips
A 3% improvement at 0.4 confidence is weaker evidence than a 1% improvement at 0.95 confidence. The policy locked in the previous setup tutorial should already gate on confidence. If you find yourself wanting to ship despite low confidence, that's a sign to re-run with a larger evaluation set rather than override the policy.
The discipline that makes pre-committed criteria valuable is *not changing them after the fact*. If a result makes you want to adjust the policy, file that as a separate change. Let the current experiment ship or reject under the policy that was in place when it was scoped, then update the policy for the next round.
Sometimes the right call differs from the engine's classification. Write down why in the rationale field. Six months from now, an override without explanation looks like an unexplained anomaly in the project's outcomes record.
## Next
Read the patterns across your experiments, the decisions behind them, and the directions paying off.
Loop back to scoping with a new recommendation
Full series arc
# Scope an experiment from a recommendation
Source: https://docs.remyx.ai/tutorials/get-started/scope-an-experiment
Turn a paper, repo, or insight from your daily digest into a structured experiment your team can implement.
*Discovery phase · \~5 minutes*
A research paper isn't an experiment. To act on a recommendation, you need a hypothesis, a target metric, a target repo, and enough context that whoever implements it doesn't have to re-read the paper to remember why.
This tutorial walks through turning a recommendation into a scoped experiment ready to implement.
**Prerequisites.** You've completed the previous tutorials in the series, so the project, discovery feed, eval template, and decision policy are all in place. Recommendations have populated, or you can use Search instead.
## Where recommendations come from
Three surfaces produce candidate experiments. Pick whichever fits your moment.
Your project's research interest delivers a daily digest of papers, repos, models, and datasets ranked against the experiments your team has shipped.
1. Open the [Feed](https://studio.remyx.ai/papers).
2. Browse the recommendations. Each card shows why it was surfaced (which experiments it's adjacent to in your project's history).
3. Click **View** on the one you want to act on.
1. Go to [Search](https://studio.remyx.ai/resources) and enter a query (e.g., *"multi-hop retrieval for complex queries"*, *"depth estimation for indoor scenes"*).
2. Browse results. Resources tagged **Runnable environment** ship with a pre-built Docker image.
3. Click **View** on a resource.
1. Go to [Insights](https://studio.remyx.ai/experiments/insights) to see which experiment directions in your project are producing results.
2. Expand a high-signal cluster. The **Recommended resources** column shows what to try next on that direction.
3. Click **Start Experiment** on a recommendation. The cluster's tag and the active project are pre-filled.
The flow that follows is the same regardless of which surface you came from.
## Annotate before you scope
On the resource detail page, two tabs help you build the context you'll need before the experiment exists:
* **Chat.** Ask questions about the method. Useful for clarifying details the abstract glossed over.
* **Annotations.** Highlight key passages and add notes. Annotations carry into the experiment context, so you can come back to "the part that mattered" without re-reading.
Spend two minutes here. The annotations you write are what makes the experiment scoped to your work rather than a generic implementation of the paper.
## Click Create Experiment
From the resource detail page, click **Create Experiment**. A scoping form opens with the resource pre-linked.
Fill in:
* **Name.** Short, descriptive. *"Swap DepthPro for VGGT in the depth-estimation stage"* is better than *"Try VGGT."*
* **Hypothesis.** One sentence. *"VGGT will improve spatial accuracy on small objects without requiring retraining."* The hypothesis is what the eval and decision policy from the previous setup tutorial will test against.
* **Target metric.** Pick from the project's allowed-metrics list. The metric should be one of the metrics in your locked eval template, so the decision policy can score against it.
* **Tags.** Use existing project tags where possible. Tags drive cluster patterns in Insights.
* **Target repository.** `owner/repo` format. Defaults to the project's primary linked repo.
Click **Create**.
After creation, the experiment detail page loads with an **Origin** section that generates a launch context within a few seconds:
* Resource metadata (title, abstract excerpt, key methods)
* Reference to a pre-built Docker environment (if available)
* Target repo file tree
* An AI-generated implementation plan grounded in actual file paths from your repo
The launch context is what the implementation tutorial uses next.
## Recap
You now have:
* A scoped experiment with a hypothesis, target metric, tags, and target repo
* A launch context with the implementation plan ready to feed into implementation
* An annotated resource that captures *why this is worth trying*
The experiment is at status `draft`. No code has been written, no eval has run.
## Tips
The decision policy from the previous tutorial only evaluates rules against metrics in the locked eval template. If your experiment targets a metric that isn't in the template, the policy can't score the variant. Match the names.
A good hypothesis says what you expect to change *and* by roughly how much. "VGGT will improve spatial accuracy on small objects" is fine; "VGGT will lift spatialscore by ≥2% on the suite without regressing latency" is better. The decision policy already encodes the threshold, so the hypothesis can match it.
The launch context's implementation plan is AI-generated. Skim it and edit it before you hand it off. Five minutes of editing here saves an hour of debugging later.
## Next
Connect Claude Code via MCP and generate the PR that implements the experiment.
Full series arc
All Remyx tools available to agents
# Set up your discovery feed
Source: https://docs.remyx.ai/tutorials/get-started/set-up-your-discovery-feed
Turn your project's extracted history into a daily discovery digest of papers and repos ranked against your team's actual work.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Setup phase · \~3 minutes*
A generic AI discovery feed surfaces what's hot. A discovery feed grounded in your project surfaces what's relevant to the directions your team has actually shipped, ranked against real history.
In this tutorial, you'll wire up that feed using the experiment history from the previous tutorial so the recommendations Remyx delivers are scoped to your work.
**Prerequisites.** You've completed [Create your project](/tutorials/get-started/create-your-project). The project has finished cold-start extraction.
## What a discovery feed gives you
A **research interest** is a saved configuration that tells Remyx what kinds of papers, repos, models, and datasets you want to hear about. The system runs daily and surfaces the top results that match, ranked against the experiments your project has already shipped.
In practice, this means:
* A paper about a new depth estimator gets ranked higher if your project has been swapping depth estimators.
* A retrieval rerank technique gets ranked higher if your project's history shows retrieval-quality experiments.
* Off-direction work (an unrelated benchmark, a refactor framework) gets pushed down or filtered out.
Setting this up takes one minute.
## Create the interest from your project
If you checked **Create a research interest from this project** during the wizard in the previous tutorial, an interest already exists for this project. Skip to the curation step below.
Otherwise:
1. Open **Feed** in the sidebar and click **+ New Interest** in the page header.
2. Pick the **From project** tab (it's the default).
3. Select the project you created in the previous tutorial.
4. Save.
The interest pre-fills with every experiment in the project. A "Preparing recommendations..." overlay runs while Remyx assembles the context, then a refresh kicks off automatically and the first set of recommendations populates as soon as the build completes.
## Curate the experiment list
The pre-filled list is a checklist. The experiments you keep checked are the signal Remyx uses to rank papers. The experiments you uncheck become noise to ignore.
Keep:
* Model swaps and architecture changes
* Evaluation method changes (new benchmarks, new metrics)
* Routing or policy changes
* Prompt-engineering experiments
* Data-pipeline-stage upgrades
Uncheck:
* Pure refactors (file moves, type-hint cleanups)
* Dependency bumps
* Security patches
* Infrastructure-only changes (CI, build, packaging)
These don't reflect direction-setting decisions, so leaving them in the context dilutes the signal.
The experiments you keep don't just filter topics — they train the ranking. Remyx feeds the curated history into the recommendation ranker and fits a preference model over it, so the feed leans toward the methodological directions you've actually shipped. See [How recommendations are ranked](/platform/discover/feed#how-recommendations-are-ranked).
![Research interest editor with From project tab and curated experiment checklist]()
Save the curated list. The refresh re-fires.
## Set the daily count
In the interest's settings, choose how many recommendations you want delivered each day. You can update this value as needed.
## Optional: let the feed open PRs for you
If this interest is backed by a GitHub repo, you can go a step further than a digest and have Remyx open a **reviewable draft PR** for the most implementable recommendation on a schedule — the cold-start reading and scaffolding done for you, with you still reviewing and merging. This is covered in [Automated discovery PRs](/platform/agents/outrider-action).
Turn the feed into scheduled draft PRs you review (Outrider)
## Recap
You now have:
* A research interest that knows your project's direction
* A daily count for how much research Remyx surfaces
* A feedback loop where curating the experiment list sharpens future recommendations
Once the first batch of recommendations lands, [Scope an experiment from a recommendation](/tutorials/get-started/scope-an-experiment) shows how to turn one of them into a structured experiment.
## Next
Lock in the eval template and decision policy that future variants will be judged against.
Turn recommendations into scheduled draft PRs you review
Full series arc
# Stay in the loop
Source: https://docs.remyx.ai/tutorials/get-started/stay-in-the-loop
See velocity across your project, the signal patterns emerging across directions, and the decisions behind every closed loop.
**Recorded on the previous UI.** The videos and screenshots in this tutorial were captured on the old engine.remyx.ai interface and are pending a re-shoot on [studio.remyx.ai](https://studio.remyx.ai). The flow they narrate still works, but navigation labels have changed — see the [current app tour](/introduction#your-journey-through-the-app).
*Reflect phase · \~5 minutes*
You've now closed at least one experiment end-to-end. As a project accumulates experiments, the value shifts from each individual experiment's result to the pattern across them. Which directions are paying off? Which aren't? Where are decisions defensible vs. ad-hoc? What can a teammate (or a leader, or future-you) read on this screen and pick up the project's state, without scheduling a sync?
This tutorial walks through the views that surface those answers: the outcomes timeline, the cluster signals across directions, the decision rationale on individual experiments, and the typed lineage between related ones. Useful for an IC staying close to a project, a senior IC briefing leadership, or an engineering leader opening Remyx for the first time.
## Open the project's Outcomes timeline
Pick your project from the switcher at the top of the sidebar, then click **Outcomes** under Experiments. This is the project's experiment timeline — every experiment *within the selected project*, grouped by date, with three chart modes.
Outcomes is scoped to one project. For the cross-project **portfolio** — every project on one screen with health indicators and hit rates — open **Projects** under Manage. This tutorial stays inside a single project; see [Projects](/platform/configure/projects) for the portfolio view.
* **Trend.** Accuracy of the team's bets over time. How often do experiments ship?
* **Velocity.** Experiment throughput, with shipped experiments highlighted. How fast is the team running?
* **Impact.** Observed deltas above and below zero. How big are the wins, and how often does anything regress?
The chart-mode selector lets a reader flip the same data through three lenses without leaving the page. A leader reading this screen for the first time gets a velocity check, a trend check, and an impact check in 30 seconds.
## Drill into a decision
Click any shipped experiment open. The detail page shows the disposition, the rationale captured at decision time, the metrics that were used, and the eval template + decision policy that governed the call.
This is where a project's reasoning lives. A rationale like *"Shipped because three of four benchmarks improved at ≥2% with confidence above 0.85; the MindCube regression was inside the noise floor"* (written when the decision was made, against the policy that was in place) is what makes the project's history defensible months later.
Browse a few decisions in a row and you can usually feel which directions the team has clarity on and which are still ambiguous. Repeated overrides on the same metric, missing rationales, or reject classifications followed by an override-to-ship are all signals worth tracking.
## Read the cluster patterns
Open **Insights**. This view groups experiments by tag, computes hit rates and average deltas per cluster, and classifies each direction as HIGH, MIXED, or LOW signal.
A typical pattern looks like:
```
Retrieval (5 experiments): 5 of 5 positive, avg +3.2% → HIGH SIGNAL
Prompt engineering (2 experiments): 2 of 2 positive → MODERATE
Routing / model selection (2 experiments): 0 of 2 sig → LOW SIGNAL
```
This meta-analysis is what nobody on a team can produce on their own. It requires looking across every experiment, grouping by direction, and computing hit rates. A job that gets ignored at most teams because nobody has time. With it, the question of which direction to double down on moves from debate to data.
The **Recommended resources** column for each high-signal cluster surfaces what to try next on that direction. That closes the loop: past results inform future picks.
## Trace lineage across experiments
Each experiment's sidebar lets you link related experiments by type:
* **`borrows_from`**. This variant uses ideas from an earlier experiment.
* **`variant_of`**. A fork of an earlier experiment with one parameter changed.
* **`replicates`**. Re-running an earlier experiment to confirm a result.
* **`references`**. A softer relationship; useful as a breadcrumb for context.
The sidebar has a typeahead picker for finding existing experiments, an inline kind dropdown for changing the relationship, and tag-based suggestions for experiments you might want to link. Linking flows in both directions automatically: linking experiment A to B with `variant_of` shows up on B's sidebar as well.
To trace a chain across experiments, click through the sidebar one hop at a time. *"This shipped variant `borrows_from` an earlier embedding-swap experiment, which `replicates` a paper from the digest two months ago."* Three clicks, no syncs.
A graph-based lineage view sits on the roadmap as an additional way to read the same data. The relationships you create today will populate it when it lands.
## Use it for a leadership brief
Three views, one screen, in roughly this order:
1. **Outcomes (Velocity / Impact).** Shipping rate, hit rate, magnitude of impact.
2. **Insights (cluster patterns).** Which directions are paying off, which aren't.
3. **A specific experiment's rationale.** Why a particular call was made.
Most leadership conversations boil down to one of those three questions. With a populated project, the answers are on the screen rather than assembled in a deck the night before.
## Recap
You've completed the full series. Your project now has:
* An extracted history that bootstraps the record
* A discovery feed grounded in your team's work
* A locked eval template and decision policy
* At least one closed-loop experiment with a logged decision
* A portfolio view that reads at three levels (velocity, cluster, individual decision)
* Typed relationships across related experiments
Run the loop again on the next experiment and the project's history compounds.
## Tips
Experiments extracted by cold-start in [Create your project](/tutorials/get-started/create-your-project) appear in Outcomes and Insights alongside experiments created manually since. Where you remember the deltas, fill them in with the click-to-edit pill on each record. The impact chart and cluster patterns get sharper.
Don't assemble a deck. Open the project's Outcomes tab in the meeting. Flip through the three chart modes. Click into the experiment a leader asks about. The screen is the artifact.
The sidebar's typed relationships are most useful when added at the time you create the new experiment. Adding them retroactively works, but it's a job nobody actually does. Make it part of the scoping habit instead.
## Next
Loop back to scoping with the next idea from your digest
Let automated discovery PRs initiate the next experiment on a schedule
Wire integrations into your project (Linear, Slack, Jira)
The methodology behind what you just built
Full series arc
# Tutorials
Source: https://docs.remyx.ai/tutorials/overview
Hands-on walkthroughs of the Remyx workflow, from connecting a repo to closing the loop on an experiment.
Tutorials are organized into focused series. Each series covers a specific workflow end-to-end with a real example you can follow along on. Standalone tutorials cover smaller topics: integrations, individual product surfaces, single-task walkthroughs.
If you're new to Remyx, run the [Quickstart](/quickstart) first — it installs your first agent and lands a real draft PR in minutes. Then the [Get Started series](/tutorials/get-started/overview) goes deeper on the project machinery around the loop: shared context, measurement, and decisions.
## Series
A series is a multi-part walkthrough where the artifact you produce in one tutorial carries into the next. Read the series in order for the full arc, or skip directly to the part you need.
Seven tutorials covering the full cycle on a real open-source AI project. Connect a repo, set up shared context, implement a change, and review the result against a pre-committed bar. About 50 minutes for the full series.
More series are in development. As Remyx adds product surfaces (observability hooks, custom integrations, scaled experiment orchestration), each will get its own series in this tab.
## Standalone tutorials
Single-topic walkthroughs that don't fit a longer series.
Set up GitHub, Linear, Jira, and Slack integrations so experiments link to PRs, tickets, and channels.
## Looking for something else?
Install an agent and land your first draft PR
The methodology behind the workflow
All Remyx tools available to agents
Endpoint-level reference
What's shipped vs. on the roadmap