Version 1.0 · November 2026

Our commitments

A cream paper document with a grid of green, mustard and terracotta squares and a terracotta paper seal

Why we make these

A verifier is only useful if it can be trusted itself. When we test an agent, we are given access to a real system, often one that holds a company’s data. When we publish a result, people make decisions on it: whether to connect an agent, list a server, or give either one access to their own data. Both of those carry responsibility.

These commitments set out how we meet it. They apply to every test we run and every report we sign, whoever the customer is and whatever they pay. They are written to be specific enough that anyone can tell when we have broken one.

To the agents we test, and their owners

When an owner gives us access to their agent or server, these are the limits we work within.

1

We only change what we can prove we created

Every object our tests create carries a marker. Updates and deletes only ever run on marked objects. If we can’t confirm that an object is ours, the case stops there, and the report says why it stopped.

2

Nothing leaves the test account

No email, message or invite we trigger reaches a real person. Tools that send things outward run only against a recipient the owner gives us and controls, or they don’t run at all and are listed as untested.

3

A person approves the plan before anything runs

We draft the test plan; the owner reads and signs it off. Nothing touches their system before that. If the agent’s tools change while a test is running, we stop rather than carry on against something no one approved.

4

Credentials never appear in our records

Not in transcripts, reports or logs, including when an agent echoes a credential back in its own answer. We redact before anything is stored.

To everyone who reads our results

A report is only worth something if the people reading it can rely on how it was made.

5

Results aren’t for sale

Paying for a test doesn’t change its verdict, and no customer can edit a signed report. An owner can add a response, which is shown alongside the report and clearly marked as theirs.

6

Inconclusive is never a pass, and gaps are always shown

A result our judges couldn’t agree on is reported on its own, never counted as a pass. Anything we didn’t test is listed by name, so a report can’t imply more coverage than it has.

7

Every verdict can be checked

Each verdict links to the exact request and response it was based on, and the whole report is signed, so anyone holding it can verify it and re-run the same tests against a later version.

8

Owners hear about security issues first

If a test uncovers a vulnerability, we tell the owner privately and publish nothing about it until it is fixed or 90 days have passed, whichever comes first.

To teams who rely on a benchmark

A monthly number is only useful if this month’s can be compared with last month’s, and if everyone can see what it rests on.

9

Scenarios and the standard are fixed before a run

Which tasks we run, and what counts as done, are agreed and recorded before the first run. We never change either after seeing results. A change starts a new, dated version, and every report says which version it used.

10

We always show the N

Every pass rate comes with the number of runs behind it, and a likely range when the sample is small. We never publish a rate without the count it rests on.

11

Month to month means like for like

We only compare months that ran the same scenarios under the same standard. When either changes, the comparison starts again, and the report says so.

How this page changes

We will add to these commitments as Suncly tests more kinds of tools, and we expect to tighten some of them as we learn. Every change is dated and the previous version is kept, so anyone can see what we promised and when.

If you think we have broken one of these commitments, write to us. We will reply, and if we got it wrong, we will say so here.