Tasks | StateWeave Docs
Tasks and runs
Tasks
A task is what your agent is asked, plus how a run of it is judged. Write it as a folder in Harbor's layout, judge your agent's runs with it while you work, and upload it when you want it kept as a test.
Judged only through what you can see. A task's checks use each app's public API (the vendor's own paths and payloads), the agent's answer, and the requests the agent sent. Nothing about how StateWeave stores the apps' data is part of a task.
The folder
Layout
my-task/ ├── instruction.md what the agent is asked, word for word ├── task.toml which sandbox, what kind of task, timeouts └── tests/checks.json declarative checks
A hosted workspace needs these three. Harbor's layout also has places for code, tests/verify.py and solution/solve.py , which hosted workspaces refuse for now (see Code in a task ). A folder made for Harbor may also hold environment/ , tests/test.sh and solution/solve.sh . StateWeave neither sends nor runs these, because the sandbox is the environment.
task.toml
task.toml
[task] name = "acme/refund-digest" description = "Post this week's refunds to #support" # the test's title [metadata] stateweave_sandbox = "p-6a129c" stateweave_kind = "action" stateweave_answer_format = "" [agent] timeout_sec = 600 [verifier] timeout_sec = 120
| Key | Description |
|---|---|
| stateweave_sandbox | The sandbox's id. Required unless you give it when importing. |
| stateweave_kind | action (the agent changes data, the default) or answer (it answers something). |
| stateweave_answer_format | Required for an answer task: the answer's shape with placeholder values, e.g. '{"orders": ["#0000"], "total_usd": 0.0}' . The agent sees it, so it must hold no real value. |
| stateweave_persona | Whose credentials the agent gets, by handle. Default: the sandbox's own person ( owner in a blank sandbox). |
| stateweave_apps | The apps a run opens. Default: all of the sandbox's. |
| stateweave_version | Optional. Runs start from the sandbox's latest version, so it may only be that one. |
| stateweave_faults , stateweave_random_faults , stateweave_pinned_faults | Optional, one of them: the faults a run injects (presets, random, or pinned to endpoints). See Faults in a task . |
| [verifier] timeout_sec | How long checks may take: 120 s by default, 300 s at most. |
StateWeave's keys sit under [metadata] , which Harbor leaves to each task, so the same folder stays valid for Harbor.
Learn the data through the APIs
Write a task against the sandbox's latest version. Open fresh copies of it and read them the way your checks will: open_twins() in Python (or a twin run over the REST API) gives each app's address and tokens. Note the ids you need, such as a channel, a ticket or a record: every copy of a version has the same ids, so checks may name them. Records your agent creates get new ids in each run; find those through the API.
Python
import
stateweave
with
stateweave.
Client
().
open_twins
(
"p-7d21c4"
)
as
twins: channels = twins.
call
(
"slack"
,
"GET"
,
"/api/conversations.list"
, credential=
"user"
)[
"json"
][
"channels"
]
({c[
"name"
]: c[
"id"
]
for
c
in
channels})
# {"general": "C0A1B2C3D4E", ...}
Checks
tests/checks.json is a list. A run passes when every check holds. There are three kinds.
tests/checks.json
[ {
"name"
:
"posted to #support"
,
"app"
:
"slack"
,
"request"
: {
"method"
:
"GET"
,
"path"
:
"/api/conversations.history"
,
"query"
: {
"channel"
:
"C0SUPPORT"
}},
"expect"
: [{
"path"
:
"messages[*].text"
,
"op"
:
"contains"
,
"value"
:
"#1042"
}]}, {
"name"
:
"posted once"
,
"calls"
: {
"app"
:
"slack"
,
"http_method"
:
"POST"
,
"path"
:
"/api/chat.postMessage"
},
"expect"
: [{
"op"
:
"count_eq"
,
"value"
:
1
}, {
"path"
:
"request.channel"
,
"op"
:
"equals"
,
"value"
:
"C0SUPPORT"
}]}, {
"name"
:
"the total"
,
"answer"
: {
"path"
:
"total_usd"
},
"expect"
: [{
"op"
:
"equals"
,
"value"
:
180.5
}]} ]
| Kind | What it looks at |
|---|---|
| request | An app's API after the agent worked. path is the vendor's path on the app's host (see below ); query , body and headers are sent as the API takes them. "at": "start" asks the apps as the version holds them, before the agent. "as": "grace" asks with that person's credential (by default the owner's, else the agent's). |
| calls | The requests the agent sent that match app , http_method , path (exact, or a glob such as /api/v2/tickets/* ), path_matches (a regular expression) and status . Each one is {app, http_method, path, query, status, ok, request, response} , with bodies as JSON (a form becomes an object). |
| answer | The agent's answer as JSON, or the part of it at path . |
Each expect item is {"path"?, "op", "value"?, "all"?} . The ops are equals , not_equals , contains , not_contains , matches , exists , not_exists , gt , gte , lt , lte , in , count_eq , count_gte and count_lte . Paths look like a.b[0].c , with [*] for every item. On a response you can also use $status , $body and $headers.<name> . With several values, an assertion holds when one does, or when every one does with "all": true . A count op without a path counts the matched requests or values.
Only the APIs: what that means
The APIs behave as the real ones. Searches can miss items, lists page, and fields may need fields= . Read by id and page through whole lists. The upside is that the same checks work on a real account.
A credential sees what its person sees. Another person's messages or mailbox need "as" . Every person in the sandbox can be used.
Some facts no API reads. For example, WhatsApp has no endpoint for reading messages: what customers write arrives only by webhook. Judge those by the agent's requests and its answer.
Ids of records in the version are the same in every run , so checks may name them. Records the agent creates get new ids: find them through the API.
Judge your agent with it
You don't need to upload a task to use it. On fresh copies of the sandbox, run your agent, then hand the task's checks to check with the agent's answer: it judges them exactly as a graded run would.
Python
import
json
import
stateweave checks = json.
load
(
open
(
"refund-digest/tests/checks.json"
))
with
stateweave.
Client
().
open_twins
(
"p-6a129c"
)
as
twins: answer =
my_agent
(
open
(
"refund-digest/instruction.md"
).
read
(), twins.apps)
(twins.
check
(checks, answer))
See Run your agent . To check the task itself (doing nothing must fail) without keeping it, use sw task import DIR --check : see Upload a task .
Each app's API
| App | Paths on its host | Token header |
|---|---|---|
| Slack | /api/<method> | Authorization: Bearer |
| Gmail | /gmail/v1/users/me/… | Authorization: Bearer |
| Google Calendar | /calendar/v3/… | Authorization: Bearer |
| Google Sheets | /v4/spreadsheets/… , Drive's /drive/v3/… | Authorization: Bearer |
| Google Drive | /drive/v3/… , uploads /upload/drive/v3/files | Authorization: Bearer |
| Salesforce | /services/data/v68.0/… | Authorization: Bearer |
| Shopify | /admin/api/2026-10/graphql.json | X-Shopify-Access-Token |
| Zendesk | /api/v2/… | Authorization: Bearer |
| /v26.0/… | Authorization: Bearer | |
| Microsoft Teams | /v1.0/… (Microsoft Graph) | Authorization: Bearer |
| Stripe | /v1/… | Authorization: Bearer |
| HubSpot | /crm/v3/… , /crm/v4/… | Authorization: Bearer |
| GitHub | /repos/… , /user , /orgs/… , /search/… ; GraphQL POST /graphql | Authorization: Bearer |
Checks set the header for you. Limits: 1 MB of files in all, 50 files. Uploads count toward the daily test limit.
Code in a task
Not on hosted workspaces yet. A hosted workspace does not run uploaded code: an upload with tests/verify.py or solution/solve.py is refused there, so for now judge with checks. This section is how code works once it is accepted.
verify.py
For what checks cannot say, such as paging through a list, comparing before and after, or arithmetic:
tests/verify.py
def
verify
(run): slack = run.
api
(
"slack"
)
# after the agent, as the owner
before = run.
api
(
"slack"
, at=
"start"
)
# as the version holds it
def
texts
(api): r = api.
get
(
"/api/conversations.history"
, params={
"channel"
:
"C0SUPPORT"
}).
json
()
return
[m[
"text"
]
for
m
in
r[
"messages"
]] added =
len
(
texts
(slack)) -
len
(
texts
(before))
return
{
"passed"
: added ==
1
,
"reasons"
: [
f"{added} new message(s) in #support"
]}
run.answer and run.calls are as above. run.api(app, as_=None, at="end") is an httpx client on the app's host with the token already set. It offers .get , .post , .put , .patch , .delete and .request , plus .endpoints and .token . Return True , False or {"passed", "reasons"} . It can import httpx and the standard library only, and it reaches only this run's apps.
solve.py
A reference solution, which proves the task can be done through the apps' public APIs, the way your agent works.
solution/solve.py
import
httpx
def
solve
(env): slack = env[
"slack"
] token =
next
(v
for
k, v
in
slack[
"credentials"
].
items
()
if
k.
startswith
(
"user:"
)) httpx.
post
(slack[
"endpoints"
][
"web_api"
] +
"chat.postMessage"
, headers={
"Authorization"
:
"Bearer "
+ token}, data={
"channel"
:
"C0SUPPORT"
,
"text"
:
"Refunds this week: #1042"
})
env is {app: {"base_url", "endpoints", "credentials"}} with the agent's own credentials, as the agent gets them. Return the answer for an answer task, or None .
Each code file can be up to 64 KB.