How report cards are assessed

Discover

Can an agent find the interface and a description of the job from the vendor's own documents?

Authority

What does the job's request need: nothing, a payment, the agent's own credential, your consent, or a key a person hands over?

Onboard

Can the agent register itself, or does a person sign up for it?

Pay

Can it pay as it goes, or does a person put a card on file?

Confirm

Can it tell a failure from a success?

Sustain

Can it keep its access alive and revoke it on its own?

Each card also says where the job itself stands: Job verified once a run has done it, Job reachable, not yet run when an interface for it answers (or, with no public endpoint, the vendor's spec covers it), or no machine interface performs it.

Each step records who has to do it. Machine: the agent, alone.Not applicable: the step does not exist, such as payment on a service that answers with no account. Approval: the person the agent acts for decides once, through a bounded step such as OAuth consent with a client the agent registered itself, claiming later what the agent made, or turning its access on.Workaround: a person does the agent's work, once at setup (signing up, handing over a key, putting a card on file, telling the agent where the interface is) or again and again (renewing access that expires, topping up a balance by hand, checking results the agent cannot read).Gated: access is the vendor's decision, through sales or approval.

The job is reachable when its interface answers as an API, or, where the job runs only on each customer's own host, when the vendor's OpenAPI document covers it. No machine interface performs it when none is declared, an MCP server lists its tools without the job's and refuses its call, the vendor's OpenAPI document for a job on each customer's own host does not hold its operation, or the declared request answers only with web pages on two probes at least an hour apart and no named OpenAPI document covers it. On an F with no interface, the steps are shown as not checked.

The letter comes from the kind of person-step the path needs: the heaviest kind anywhere on the path sets it, and A allows one step. A+: an agent gets in, pays and recovers with no person. A: one light, one-time step, such as consent with the agent's own client or claiming the account later.B+: a person signs up and hands over a key. B: a card on file. C: human steps that recur, such as manual top-ups or keys that expire with no renewal API. D: access depends on sales or approval.F: no machine interface for the job.

In practice, an API that answers with no account, asks for payment per request, or lets the agent register itself, with typed errors, is A+. OAuth consent with the agent's own client, or an account a person claims later, is A as one step alone; so is a person telling the agent where the interface is, or replacing a key that never expires. Two such light steps are B+. A person signing up and handing over a key is B+; with a card on file it is B. Untyped errors, a balance a person tops up by hand, or a key or OAuth access that expires with no way to renew it, make C. Every card carries a letter.

Every listed service is graded the same way, with no account. Its listing declares the job's request on each interface that does the job: the method and URL the job uses, with a placeholder where an account ID goes, and the vendor's OpenAPI document where it publishes one; the vendor's MCP server and the tool that does the job; or, where the job runs only on each customer's own host, the job's operation in the vendor's OpenAPI document, which is read instead of called. Everywhere else Sourcey's agent sends exactly that request with no key and an empty body (an MCP tool gets empty arguments), so it acts only where the service needs no account at all, as any agent's first request would. Of a successful answer only its shape is kept, never what it returned. After a success it sends the request once more, malformed, to see how the service reports a failure. Where the answer asks for authority, or errors before it, the agent follows the protected-resource and authorization-server metadata: an auth.md registration there lets the agent register itself; otherwise it makes an authorization request as its own client and stops at the person's sign-in or consent. It never completes a sign-in or registers an account. Where the authorization server lets any client register, it registers its own client once and keeps only the client's id; Sourcey holds no keys for any listed service.

The listing also names the pages where a person's or the vendor's step happens: a sign-up endpoint, a sales or approval page, the setting where a person on the account turns the interface on for agents, the page where a person claims what the agent made, the checkout where a card goes on file, the page where a person tops up a balance by hand where the vendor offers no automatic top-up, the page where a person replaces a key the vendor expires with no API to renew it. The agent requests each one, and a page counts only when it is there. Where the job needs no card on file (it is free, or paid per call by the offer its own request answers), the listing rules the checkout out, citing the vendor's pricing.

Every request and response is sealed; response bodies are kept privately, a probe's successful answer only as its shape and an MCP tool's without its output. Machine-readable documents (OpenAPI, OAuth metadata, MCP server cards and tool lists, llms.txt, auth.md and the documents they link to, documentation served as markdown) count for what they state. Prose never sets an outcome: the listing declares each request and page and what each page is for, and rules a checkout out where the job needs none, each bound to the vendor's own page and reviewed when the listing is admitted, and the probes' answers decide. A timeout, a 5xx or a rate limit is not the service's answer: it sets no fact, and the service is probed again. A document left unanswered on every probe for at least an hour, with no answer between, is read as absent.

Every card gets a letter from its listing and the probes, labelled Probed: the job itself not yet run. A run of the job verifies it, and only an account holder runs it: with Sourcey's GitHub Action and their own key, which attests the result. Sourcey checks the attestation and replays the records before admitting it, labelledVendor-run, verified. Sourcey never runs a job itself. The Action is not yet published; until it is, every card is Probed.

A verified run is proof, not the grade: it marks the job verified and never changes a step or the letter, which come from the listing and the probes. A verified job stands on its run until that run goes stale. A declaration never authors a grade.

A pinned policy turns the listing, the recorded requests and the documents the probes read into each step's outcome, and the outcomes into the letter. No model reads anything and no person sets a grade; anyone holding the records and the policy derives the same letter, as Sourcey's public verifier does. A card is fresh for 30 days after its latest probe, and its next probe is scheduled before then.

Reporting outdated evidence, claiming a company, requesting a reassessment and attesting an exact revision are different acts with different authority, and none of them buys a grade.

The current cohort is chosen, not representative: services founders and agents actually ask about. Letters are never averaged, and a step not assessed is never counted as a failure.

Current release: sha256:12b3a22a571d10e128cb2aa246ba17bd793013c250db62e5a8d3fef742c0505c · readiness policy: sha256:0a4e4996176ce674b4f3ce910c5fca4f37c9b53bcdf51656d53a1a92fcf2c879

Every report card carries its exact revision, policy, projection and release digests, and a JSON twin with the same released facts.