Tasks and runs

### Faults

Real APIs fail. A test can make the apps fail the way the real services do, and pass only an agent that copes: one that retries, checks before it writes twice, and doesn't give up on a 401 it didn't earn.

### Kinds of fault

| Kind | What happens |
| --- | --- |
| outage | Every request fails for a while. |
| transient\_auth | One call answers 401 for a valid token. |
| rate\_limited | Rate limited for a while. |
| hang | The answer comes a minute late. |
| error\_after\_commit | A write goes through but answers an error. |
| hang\_after\_commit | A write goes through, then its answer hangs. |
| drop | The answer breaks off midway. |
| search\_lag | A search leaves out what was just written. |
| update\_conflict | An update answers 409 conflict. |

Each fault answers in the app's own words: a Slack 401 is Slack's invalid\_auth , a Salesforce outage is the bare 404 its edge sends. Only the kinds that can happen at an endpoint are offered there: a read never answers error\_after\_commit , and Salesforce has no rate\_limited .

### Seen live, or carried over

Every preset says where it comes from. Seen live presets were observed on the real service, such as a Slack history page breaking off mid-response or a Shopify create that succeeds but answers an error. Carried over presets take a failure seen on one service to another where it was not observed. They are useful for robustness, and marked 未验证 ("not verified") so you can weigh them.

### Faults in a task

A task names its faults in task.toml , under \[metadata\] , in one of three ways (one way per task). The faults act on the apps the task uses ( stateweave\_apps ).

| Key | What acts |
| --- | --- |
| stateweave\_faults | Presets, by id ( sw task faults SB lists them), each once. A preset fires where it did on the real service; nth , seconds , after\_bytes , lag or match move it, where the preset has them. |
| stateweave\_random\_faults | Faults drawn at random on a share of each app's calls, from what each endpoint can meet: rate (0.005 to 1, default 0.05), kinds (default: all), max , max\_hang , max\_outage per app and run, and seed to fix the draw. |
| stateweave\_pinned\_faults | A fault of one kind at one endpoint ( fault\_targets() lists both), from the nth call to it (default 1): app , endpoint , kind , and seconds (outages, hangs, search lag) or cut (drop, 0.05 to 0.95) where they apply. |

task.toml

\[metadata\] stateweave\_sandbox = "p-041ba8" \# presets: by id, or with where they fire stateweave\_faults = \["slack.dropped\_page", \{ preset = "slack.outage", seconds = 5 \}\]

task.toml

\[metadata\] stateweave\_random\_faults = \{ rate = 0.1, kinds = \["outage", "transient\_auth"\] \}

task.toml

\[metadata\] stateweave\_pinned\_faults = \[ \{ app = "slack", endpoint = "chat.postMessage", kind = "error\_after\_commit" \}, \{ app = "zendesk", endpoint = "support/UpdateTicket", kind = "update\_conflict", nth = 2 \}, \]

A fault that cannot be (an unknown preset, a kind that cannot happen at that endpoint, two ways at once) is refused when the task is read, with what is wrong.

### How a task with faults is proven

On a hosted workspace it is proven like any other task: doing nothing must fail. Nothing shows beforehand that it can be passed under its faults, and a twin run has none, so try it with your agent once it is uploaded: the run's page lists each fault that acted.

A task with solution/solve.py , which hosted workspaces don't take yet, proves more: the reference must also pass with the faults, at least one of them must act on it (random ones get other seeds until one does, and the task keeps that seed), and it must pass without them too.

A test's listing says how many faults it may inject ( faults in tests() and GET /api/v1/tests ), never which; its page lists them.

### List what can fail

The fault presets of a sandbox's apps, and where each comes from:

sw

task faults p-041ba8

for

preset

in

client.

faults

(

"p-041ba8"

):

print

(preset\[

"id"

\], preset\[

"title"

\],

"seen live"

if

preset\[

"observed"

\]

else

"carried over"

)

Output

slack.dropped\_page A history page breaks off after 4 KB seen live slack.transient\_auth One call answers invalid\_auth for a valid token carried over (未验证) shopify.error\_after\_commit A create goes through but answers INTERNAL\_SERVER\_ERROR seen live shopify.search\_lag Product search lags 15 s behind new products seen live

Where faults can act, per app and endpoint:

Python

targets = client.

fault\_targets

(

"p-041ba8"

)

print

(targets\[

"apps"

\]\[

"zendesk"

\]\[

"endpoints"

\]\[:

3

\])

fault\_targets() answers \{"apps": \{app: \{"title", "kinds", "endpoints"\}\}, "kinds", "noise"\} : per app the kinds it has and each endpoint's id , label (such as PUT /api/v2/tickets/\{ticket\_id\} ) and kinds , then what each kind does and random injection's defaults and limits. Endpoints are mapped for Slack, Gmail, Salesforce, Shopify and Zendesk; Google Calendar, Sheets, Drive, WhatsApp, Microsoft Teams, Stripe, HubSpot and GitHub have presets only.

From a shell, sw task faults p-041ba8 --targets prints both, per app and endpoint. Over HTTP, see Fault presets and Fault targets .

### In a run

Faults act during your agent's test run as they did when the test was checked. The run's page lists each fault that acted, on which call, next to your agent's calls, so you can see what it did next. In Python, run.trace() returns them as faults , and run.download() writes them to faults.json . A twin run has no faults.

[Fault presets](https://stateweave.net/docs/api#fault-presets)

[Fault targets](https://stateweave.net/docs/api#fault-targets)