> ## Documentation Index
> Fetch the complete documentation index at: https://docs.remyx.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Results and evidence

> Interpret validation criteria, metrics, artifacts, passing and inconclusive outcomes.

A validation result connects the question, executed test, observed metrics, criteria, artifacts, and cost.

Read the PR comment for the summary. Use the GitHub Check for execution status and the notebook and run artifacts for the underlying work. Results also appear in Studio under Reports → Validations.

## How to read a result

Read the question, comparison, measured metrics, acceptance criteria and supporting artifacts together. A passing result means the required criteria were met in that run. It does not prove production improvement.

An inconclusive result, a no-signal result, an execution error and a timeout have different meanings. Use the detailed status definitions below before deciding what to change or rerun.

[Interpret evaluation results](https://docs.remyx.ai/tutorials/interpret-results.md)

| Measure | Measured value | Required criterion | Scope of evidence |
| - | - | - | - |
| Explained variance | TopK 0.9666, AbsTopK 0.9677 | AbsTopK no more than 0.02 worse | Single-seed CPU run |
| Negative selected latents | TopK 0%, AbsTopK 49.2% | AbsTopK at least 1% | Single-seed CPU run |

A missed criterion means the test did not meet its specified target. A diagnostic metric supplies context but does not determine the outcome unless the spec makes it required.

An inconclusive or no-signal result means the run did not establish a reliable answer under its chosen test and data. Review missing metrics, comparability, uncertainty, or execution limitations before treating the result as a decision input.

## Comparisons

Some validations compare base and PR branches. Others test a capability against an absolute threshold or reference value. The result identifies which comparison ran.

## Artifacts

Executed notebooks are committed to the PR branch by default. If the notebook suite sets `commit: false`, Validate provides time-limited artifacts instead. Review artifacts before relying on the summary.

## GitHub checks and merging

A passing or failing result is not an automatic merge decision. The Remyx Validation check affects merge protection only when your repository requires it.

## Result fields and status reference

The tables below define exact result fields, outcome labels, and status semantics.

| Status | Meaning |
| - | - |
| ✅ **Met your criteria** | The target passed and no guardrail crossed. |
| ⚠️ **Met your criteria, with a warning** | The target passed; something else needs a look. |
| ❌ **Missed your criteria** | The target fell short of your bar. |
| ❌ **Missed your criteria, guardrail crossed** | A guardrail moved past its line. Flagged for review. |
| ⚠️ **Inconclusive, `<metric>` was not measured** | The test did not produce a metric the result needs. |
| 📊 **Measured** | The target's bar is a floor or a sanity check, not a goal. Read the numbers. |
| 🔍 **No signal** | The run cannot answer the question. See [No signal](#no-signal) below. |
| ⏱️ **Ran out of time** | The budget ended before a result. Says nothing about your code. |
| ✅ / ❌ **Your tests pass / do not pass on this PR** | A `checks:` only spec. |

Every result comment follows one layout: the status line, your ask, a one-sentence answer, what was found, the metric table, then the details.

| Part | What it tells you |
| - | - |
| **Status** | The outcome in one line, from the table above. |
| **You asked** | Your own words, when this thread is where the ask came from. An inherited spec reports what it ran instead. |
| **Answer** | The target metric, the comparison, and whether your bar was met. A crossed guardrail or a metric the test never produced is named here. |
| **What we found** | Findings from the run. A finding whose metric was not emitted is dropped and listed under **Not measured**. |
| **Table** | One row per metric: direction, baseline, this PR, delta, your bar, pass mark. Guardrail, cost and diagnostic rows carry a legend. |
| **Links** | The executed notebook on GitHub, Colab and nbviewer; W\&B dashboards when the run logged there. |
| **Footer** | What the run cost and what it was charged to. See [the cost footer](#the-cost-footer). |

When a fix loop ran, a **What I tried** table lists each attempt, what failed, and the commit that tried to fix it. When the loop stopped, the first line says why: the test itself failed, a file is missing from the branch, or the circuit breaker tripped.

> [accordion] Illustrative example of a result comment
>
> Illustrative example. The numbers are invented to show the layout. For a real one, open the [completed validation](https://docs.remyx.ai/platform/agents/validate.md#see-a-completed-result) above.
>
> ```markdown theme={null}
> ## ✅ Met your criteria
>
> **You asked:** does spectral targeting halve trainable params without losing fit vs standard LoRA?
>
> **Answer:** `gsm8k_accuracy` came in at **0.412**, vs **0.407** on `main` (+0.005). Your bar was ≥ 0.40 — met.
>
> **What we found**
> - Trainable parameters fell from 6.8M to 3.4M with the same rank.
> - Eval loss moved inside seed noise across 3 seeds.
>
> | Metric | Goal | main | this PR | Δ | Your bar | |
> |---|---|---|---|---|---|---|
> | `gsm8k_accuracy` | ↑ | 0.407 | 0.412 | +0.005 | ≥ 0.40 | ✅ |
> | `trainable_params` | ↓ | 6.8M | 3.4M | -3.4M | ≤ 3.5M | ✅ |
> | `eval_loss` 🛡️ | ↓ | 1.21 | 1.22 | +0.01 | ≤ 1.30 | ✅ |
>
> 🛡️ guardrail — a metric that must not regress, even if the target improves.
>
> Notebook: [GitHub](…) · [Colab](…) · [nbviewer](…)
>
> ---
> 💳 **$1.84 debited from your Remyx balance — $18.16 left.**
> ```

### Evidence strength

Tier A is the default answer: a scaled workload at a matched budget, labelled directional. Tier B is the repo's own harness at its own regime, what a maintainer asks for. Declare `compute.rigor: A|B`; otherwise a reused harness is Tier B and everything else is Tier A. A smoke run is neither.

### No signal

Before any result, deterministic checks compare what each arm declared with what it reported. Two arms that declare different settings and report identical provenance, a factor that drifted, or a difference inside the noise produce **No signal, this run can't answer the question** instead of a confident number.

### Targets, guardrails, and diagnostics

| Role | Gates the result | Shown as |
| - | - | - |
| `target` | Yes (warn) | The headline row |
| `guardrail` | Yes (fail, with `policy.guardrail_veto`) | 🛡️ must not regress even if the target improves |
| `cost`, `latency` | No | ℹ️ measured and reported |
| `diagnostic` | No | 📊 measured to inform the reading |

A metric's `bar` is a `goal` (the default), a `floor` (a bound the measurement cannot fall below; missing it points at the harness) or a `sanity` check on the harness.

### The cost footer

| Basis | Footer |
| - | - |
| Free validation | 🎁 That was this installation's free validation. What it would have cost, and the two ways on: add credits, or a team pilot. |
| Balance | 💳 What was debited and what is left. **Add credits** appears when the balance is below the larger of that debit and \$5. |
| Counted grant | How many validations are left on the grant. |
| Unlimited grant | No footer. |

## Next steps

[Run a validation tutorial](https://docs.remyx.ai/tutorials/validate-first-run.md) · [Read the validation spec](https://docs.remyx.ai/platform/agents/validate-spec.md) · [Use the API](https://docs.remyx.ai/api-reference/validate.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.