Reporting issues

How coding agents in other projects file a GitHub issue when a mock diverges from its oracle, lacks a feature they call, or breaks, or when they need a service mocked that the catalog does not have yet. Agents in this repository pick the issues up and resolve them.

This guide is written for coding agents. Read it top to bottom once, then use Quick reference as the checklist every time you file.

A mock's oracle is the thing it must behave like: the vendor's sandbox or test API for an HTTP mock (Stripe test mode, the Junction sandbox, …), the real engine for the database mocks (SQLite, PostgreSQL), and a real Medplum server for medplum. When your tests show that the mock and its oracle disagree, or the mock lacks something you need, file an issue. Do not quietly work around the mock in your own project. The issue is how the mock gets fixed for everyone.

#Quick reference

<service> is the package suffix: stripe for @crvouga/mockingbird-service-stripe. The catalog is listed in llms.txt.

1. Search for an existing issue. Comment on it rather than opening a duplicate.

2. Redact keys, tokens and personal data.

3. Copy the kind's template, fill in every section, and save it to a file.

4. File it:

   gh issue create --repo crvouga/mockingbird \
     --title "[stripe] parity: POST /v1/customers accepts an invalid email" \
     --label agent-reported,parity \
     --body-file issue.md

5. Give your human the issue URL.

File one issue per divergence, missing feature, or service, so each one can be fixed, tested and closed independently. The title prefix alone is enough: a workflow applies the agent-reported and kind labels from it, including when your account cannot set labels.

#When to file, and when not to

File when:

• parity: the status, body shape, field value, error code, ordering, pagination, webhook payload or signature, or side effect differs from the oracle's for the same requests.
• feature: your code calls something the mock answers with 404, not implemented, or by silently ignoring it. Check the package's SUPPORT.md / COMPATIBILITY.md first. A feature can be a whole operation, one parameter or filter, a webhook event, a lifecycle transition, an error case, or an admin control (a fault preset, an outbox, a clock advance) your tests need.
• bug: the mock crashes, leaks state between namespaces, contradicts its own README, or fails to install, build, or type-check.
• new-service: you depend on a vendor API that no package mocks, and you would use a mock in your tests if one existed.

Do not file when:

• The package README (under Deliberately not modelled) or SUPPORT.md documents the divergence as intentional. If you think the decision is wrong, file a feature issue that explains what the omission breaks for you.
• You only suspect how the vendor behaves. State what the oracle actually returned and how you know (a live response, the vendor docs, the SDK source). "Stripe probably returns 400" is a guess, not a report.
• The cause is your own code or configuration: a wrong base URL, a missing x-mockingbird-namespace, or state left over from an earlier test.

#Never include

These packages mock health, payments, and identity APIs, and issues are public. Before you file, remove:

• API keys, tokens, webhook secrets, passwords, and signed URLs. Replace each one with <redacted>.
• Real personal data or PHI: names, emails, phone numbers, addresses, dates of birth, lab results. Use obviously fake values (jane@example.com, +15005550006).
• Your project's proprietary source. Describe the behavior at the wire, as requests and responses, instead of pasting your code.

When you cannot show the problem without sensitive data, stop and ask your human instead of filing.

#1. Search first

gh issue list --repo crvouga/mockingbird --state all --search "<service or vendor> <operation or field> in:title,body"

If an open issue covers the same thing, add a comment with your evidence (package version, requests, responses, or the behaviors you need) instead of opening a duplicate. A 👍 reaction on a new-service or feature issue also counts as demand. If a closed issue covers it, check that you are on a version at or after the fix before you reopen it.

#2. Parity and bug reports: reduce to a reproduction

Reduce the failure to the smallest sequence of requests against a fresh mock that shows it. Prefer a self-contained script that runs the mock in-process:

import { createRuntime } from "@crvouga/mockingbird-service-<service>"

const mock = createRuntime()
const res = await mock.fetch(
  new Request("http://mock/v1/customers", {
    method: "POST",
    headers: { authorization: "Bearer sk_test_<redacted>", "content-type": "application/x-www-form-urlencoded" },
    body: "email=not-an-email",
  }),
)
console.log(res.status, await res.text())

Record the oracle's response to the same requests. If you captured it live, paste the status and the redacted body. If it comes from vendor documentation, link the page. If it comes from a parity run, include the provider, the FC_SEED, and the command that reproduces it.

#3. Features and new services: describe the behavior

A feature or new-service issue is a specification. The agent that picks it up turns each behavior into an acceptance test and then builds the mock until the tests pass, so write what the mock must do, observed from outside, rather than how to build it. Everything below is what that agent needs and cannot guess.

How you will use it. Say whether your code calls the API through the vendor's official SDK (name and exact version) or through raw fetch, how you point it at a different base URL (an SDK option or environment variable), and whether you will run the mock in-process (createRuntime().fetch) or as a server (npx mockingbird-<service> serve). The SDK version decides the wire format the mock must speak.

Surface. List only the operations you call. For each one, give the method and path, what your code sends (the parameters and body fields you actually set), and which response fields your code reads. Link the vendor's reference page and its OpenAPI or Postman file, if one is published.

Auth. The scheme (bearer, basic, HMAC request signing, AWS SigV4, OAuth client credentials), the header names, and what the vendor answers when credentials are missing or wrong.

State. The resources and how they relate: which ids the API returns, which ids a request references, the lifecycle states a resource moves through, and what moves it (a request, the passage of time, an action in the vendor's dashboard, a webhook).

Behaviors. Numbered Given / When / Then statements. Each one must be checkable through the API alone, and together they must cover the failures your code handles as well as the happy path:

B1. Given no orders exist, when I POST /v1/orders with a valid body, then the response is 201
    with an `id` starting `ord_` and `status: "pending"`, and GET /v1/orders/{id} returns the
    same order.
B2. Given an order in `pending`, when I POST /v1/orders/{id}/cancel, then its status is
    `cancelled` and an `order.cancelled` webhook is delivered.
B3. When I POST /v1/orders without `patient_id`, then the response is 422 with
    `{ "errors": { "patient_id": ["is required"] } }`.
B4. Given 101 orders, when I GET /v1/orders?limit=100, then 100 are returned with a `next_cursor`,
    and following the cursor returns the last one.

Webhooks. For each event your code receives: when it fires, a redacted payload, the signature scheme (header name, algorithm, what is signed, timestamp tolerance), and the retry behavior you depend on.

Test controls. What your tests need to force that a real sandbox cannot easily do, such as advancing an order to shipped, making the next call return 429 or time out, reading the messages a comms API "sent", or returning a lab result. Each control becomes an /__admin route or a fault preset.

Out of scope. Operations and behaviors you do not need, so the mock stays small.

Oracle. Whether the vendor offers a sandbox or test mode, whether it is free, and how a maintainer gets access. Never include the credentials themselves.

A feature request on an existing mock uses the same sections, trimmed to the one feature. A new-service request with a clear surface and concrete behaviors is ready to build. One that only names the vendor waits until someone supplies them.

#4. File the issue

Copy the template for the kind (links below), fill in every section, delete the HTML comments, and save it to a file. Then run:

gh issue create --repo crvouga/mockingbird \
  --title "<title in the format from the quick reference>" \
  --label agent-reported,<kind> \
  --body-file issue.md

Each template begins with a --- front-matter block; leave it out of the body file.

If gh is not installed or not authenticated, do not install it or log in on your own. Give your human the finished title and body and the link to the matching form, for example https://github.com/crvouga/mockingbird/issues/new?template=new-service.md.

#What happens next

Agents in this repository run /resolve-issues on the agent-reported queue.

• parity and bug: the agent checks the claim against the oracle, adds a failing regression test, fixes the mock where the behavior is defined (implementation, OpenAPI contract, or generator), and opens a PR that closes the issue. If the oracle does not behave as reported, the agent comments with what it observed and closes the issue.
• feature: the agent confirms the behavior against the oracle or vendor documentation, adds it to the contract, and ships it with an acceptance test for each behavior you listed.
• new-service: the agent builds a new package by following AUTHORING_A_SERVICE.md. Your behaviors become its acceptance suite and your SDK version its drop-in test. It is released as wip until it is verified.

An issue labelled needs-info is waiting on you: answer the question in the comments. Fixes ship in the next release of the package; see RELEASING.md.