Agent cookbook

Concrete patterns for driving Pictograph from agents. Each recipe shows the system-prompt guidance plus the expected tool-call sequence.

1. Credit-aware training

User asks: “Train a YOLOX model on my ‘road-signs’ dataset”

System prompt addition:

For paid operations, always call estimate_credit_cost first. If insufficient, tell the user the gap and ask before proceeding.

Expected sequence:

1. get_credit_balance()
   → {credits_remaining: 1500, ...}
2. estimate_credit_cost("training_a10g_per_minute", quantity=30)
   → {total_credits: 300, sufficient: true, ...}
3. (response to user) "Estimated 300 credits, you have 1500. Proceeding."
4. train_pipeline(dataset_name="road-signs", pipeline="yolox", gpu="a10g")
   → {training_run: {...}, model: {id: "model-uuid", status: "ready"}}
5. (response to user) "Trained. Model ID: model-uuid. Download with `pictograph models download model-uuid -o ./yolox.onnx`"

If sufficient: false, the agent should surface PaymentRequiredError.upgrade_url to the user, not the raw exception.

2. V7 import → auto-annotate with new classes

User asks: “Import ‘road-damage’ from V7 and add a ‘pothole’ class auto-annotated by SAM3”

Expected sequence:

1. validate_connector(provider="v7", api_key="<v7-key-from-user>")
   → {valid: true, datasets: [{id, name: "road-damage", ...}, ...]}
2. import_from_connector(provider="v7", api_key=..., datasets=[<chosen>])
   → {import_id, status: "processing", ...}
   (the SDK polls until terminal — agent waits)
3. (response to user) "Imported. Now adding 'pothole' annotations…"
4. auto_annotate_dataset(
       dataset_name="road-damage",
       classes=[{name: "pothole", output_type: "polygon"}],
       mode="batch",
   )
   → {images_processed: 234, annotations_added: 487, ...}
5. (response to user summary)

Existing V7 annotations are preserved — auto-annotate by default skips images that already have annotations (overwrite=False).

3. Batch SAM3 with progress + per-image fallback

User asks: “Auto-annotate ‘wildlife’ for ‘tiger’ and ‘elephant’; fall back to text mode if batch fails”

System prompt addition:

If a batch tool returns failed_images > 0 or raises, retry the failed subset with the synchronous text-prompt mode.

Expected sequence:

1. auto_annotate_dataset(
       dataset_name="wildlife",
       classes=[{name: "tiger", output_type: "polygon"},
                {name: "elephant", output_type: "polygon"}],
       mode="batch",
   )
   → {images_processed: 198, failed_images: 12, job_id, ...}
2. (response to user) "Batch done — 12 images failed. Retrying as text..."
3. get_dataset(name="wildlife", include_images=true)
   → list of image filenames
4. For each failed image (agent loops):
   auto_annotate_text(
       dataset_name="wildlife",
       image_filename="img-failed-1.jpg",
       text_prompt="tiger or elephant",
       confidence_threshold=0.3,
   )
5. (final response)

In practice, prefer auto_annotate_dataset(mode="batch") first for speed, fall back to mode="text" only on the failures.

4. Multi-class export with status filtering

User asks: “Export only the ‘complete’ subset of ‘road-signs’, just the ‘stop_sign’ and ‘yield’ classes, in YOLO format”

Expected sequence:

1. get_dataset(name="road-signs")
   → confirms classes include both
2. create_export(
       dataset_name="road-signs",
       name="stop-yield-yolo-2026-04-19",
       format="yolo",
       include_images=true,
       class_filter=["stop_sign", "yield"],
       status_filter="complete",
   )
   → {id, status: "completed", image_count: 412, ...}
3. download_export(
       dataset_name="road-signs",
       export_name="stop-yield-yolo-2026-04-19",
       output_path="./stop-yield.zip",
   )
4. (response to user) "Wrote ./stop-yield.zip — 412 images, 1287 annotations."

Always include the date in export names so re-runs don’t conflict (409 ConflictError on duplicate names).

5. Cleanup confirmation flow

User asks: “Delete the old test datasets”

System prompt addition:

Before any destructive action (delete_dataset, delete_image, cancel_training), list the targets and ask the user to confirm by repeating the names.

Expected sequence:

1. list_datasets(limit=100)
   → [{name: "test-1", ...}, {name: "test-2", ...}, {name: "production", ...}]
2. (response to user) "Found 2 with 'test' in the name: test-1, test-2.
                       Confirm by typing the names you want deleted, separated by commas."
3. (user) "test-1, test-2"
4. (agent) For each confirmed name:
   delete_dataset(name="test-1")
   delete_dataset(name="test-2")
5. (response to user) "Deleted: test-1, test-2."

The delete_dataset tool is marked idempotent=True and required_role="admin" — agents using a member key get 403 ForbiddenError automatically.

6. Folder reorganization with batch ops

User asks: “Move all images tagged ‘blurry’ to a ‘blurry’ folder”

Expected sequence:

1. search_by_tag(dataset_name="…", attributes=["blurry"], limit=500)
   → [{image_id, filename, ...}, ...]
2. batch.move(
       image_ids=[r.image_id for r in results],
       folder_path="/blurry",
   )
   → {succeeded: [...], failed_count: 0}
3. (response to user) "Moved 87 blurry images to /blurry."

Agents should prefer batch.* over per-image loops — single call, single round-trip, atomic at the database level.

See also

• Claude integration — Anthropic-specific paths
• OpenAI integration — OpenAI-specific paths
• Agents — overview — toolkit setup, guardrails, and pictograph-cv Skill install