Auto-annotate

SAM3 point, box, and text prompts plus async batch jobs for AI-generated annotations.

SAM3 auto-annotation: three single-image prompt modes, an async batch endpoint for many images at once, and a quote that prices a batch before you run it.

Scenario	Mode
User clicks one spot	`point`
User drags a rough box	`box`
One image, many objects	`text`
Many images, known classes	`batch`

Prompt results are returned, not saved: call client.annotations.save to persist them.

point

Click here, segment that. Returns one polygon annotation per prompt. The first prompt on an image generates its GPU embedding (1 to 2 seconds on a warm container); follow-ups on the same image are sub-second.

Source: AutoAnnotate.point

Arg	Type	Default	Notes
`dataset_name`	`str`	required	Dataset name.
`image_filename`	`str`	required	Image filename, not the UUID
`x`, `y`	`int`	required	Anchor point in absolute pixels
`name`	`str`	`"object"`	Class name for the annotation
`positive_points`	`list[(x, y)] \| None`	`None`	Extra positive anchors
`negative_points`	`list[(x, y)] \| None`	`None`	Excluded regions
`score_threshold`	`float`	`0.75`	Minimum SAM3 score, 0 to 1

result = client.auto_annotate.point(
    dataset_name="road-signs",
    image_filename="img-1.jpg",
    x=320,
    y=240,
    name="car",
    positive_points=[(310, 250)],
    negative_points=[(100, 100)],
)

pictograph auto-annotate point road-signs img-1.jpg --x 320 --y 240 --name car

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/sam3/point" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "image_filename": "img-1.jpg", "x": 320, "y": 240, "name": "car", "positive_points": [[310, 250]], "negative_points": [[100, 100]]}'

Returns PromptResult

PromptResult · 4 fields

class PromptResult(BaseModel):
    """Outcome of a single SAM3 prompt (point / box / text)."""
    status: Literal['success', 'no_detection', 'below_threshold']
    annotations: list[Annotation] = []
    score: float | None = None
    inference_time: float | None = None

On success, annotations[0] is a PolygonAnnotation.

box

Segment everything inside this box. return_polygon=False returns only the refined bounding box.

Source: AutoAnnotate.box

Arg	Type	Default	Notes
`dataset_name`	`str`	required	Dataset name.
`image_filename`	`str`	required	Image filename
`box`	`{x, y, w, h}`	required	Bounding box in absolute pixels
`name`	`str`	required	Class name
`confidence_threshold`	`float`	`0.5`	Minimum SAM3 confidence, 0 to 1
`return_polygon`	`bool`	`True`	Include a polygon as well as the bbox
`negative_boxes`	`list[{x, y, w, h}] \| None`	`None`	Exclusion zones

result = client.auto_annotate.box(
    dataset_name="road-signs",
    image_filename="img-1.jpg",
    box={"x": 100, "y": 200, "w": 200, "h": 150},
    name="car",
    return_polygon=True,
    negative_boxes=[{"x": 50, "y": 50, "w": 30, "h": 30}],
)

pictograph auto-annotate box road-signs img-1.jpg --box 100,200,200,150 --name car

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/sam3/box" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "image_filename": "img-1.jpg", "box": {"x": 100, "y": 200, "w": 200, "h": 150}, "name": "car", "return_polygon": true, "negative_boxes": [{"x": 50, "y": 50, "w": 30, "h": 30}]}'

Returns PromptResult

PromptResult · 4 fields

class PromptResult(BaseModel):
    """Outcome of a single SAM3 prompt (point / box / text)."""
    status: Literal['success', 'no_detection', 'below_threshold']
    annotations: list[Annotation] = []
    score: float | None = None
    inference_time: float | None = None

text

Find all of a thing. Open-vocabulary phrase grounding, best for many objects in one image. Results come back sorted by confidence.

Source: AutoAnnotate.text

Arg	Type	Default	Notes
`dataset_name`	`str`	required	Dataset name.
`image_filename`	`str`	required	Image filename
`text_prompt`	`str`	required	Natural-language description
`output_type`	`str`	`"polygon"`	`"polygon"` or `"bbox"`
`confidence_threshold`	`float`	`0.3`	Minimum confidence, 0 to 1
`max_detections`	`int`	`50`	Cap on result count, 1 to 100

result = client.auto_annotate.text(
    dataset_name="road-signs",
    image_filename="img-1.jpg",
    text_prompt="red cars",
    output_type="polygon",
    max_detections=50,
)

pictograph auto-annotate text road-signs img-1.jpg --prompt "red cars"

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/sam3/text" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "image_filename": "img-1.jpg", "text_prompt": "red cars", "output_type": "polygon", "max_detections": 50}'

Returns PromptResult

PromptResult · 4 fields

class PromptResult(BaseModel):
    """Outcome of a single SAM3 prompt (point / box / text)."""
    status: Literal['success', 'no_detection', 'below_threshold']
    annotations: list[Annotation] = []
    score: float | None = None
    inference_time: float | None = None

quote

Ask what a batch would cost without running it. The quote and the charge come from one server-side function, so the number here is what batch deducts.

projected prices images that do not exist yet. That is what lets you decide before you spend, most importantly for video: a video is one file but hundreds of frames, and the frames are what you pay for.

Source: AutoAnnotate.quote

Arg	Type	Default	Notes
`dataset_name`	`str \| None`	`None`	Required to price images that already exist
`image_filenames`	`list[str]`	`()`	Existing images; their stored dimensions drive SAHI pricing
`projected`	`list[{count, width, height}]`	`()`	Images that do not exist yet
`classes`	`list[BatchClass]`	`()`	Each class adds a grounding pass, so it changes the price
`model`	`str \| None`	`None`	Trained model by name; `None` routes to SAM3
`sahi` / `sahi_slice_size`	`bool` / `int`	`False` / `640`	See SAHI

from pictograph import BatchClass

uploaded = client.video.upload(
    local_path="drive.mp4",
)
meta = client.video.probe(
    gcs_path=uploaded.gcs_path,
)
frames = int(meta.duration_seconds * 5)   # sampling at 5 fps
quote = client.auto_annotate.quote(
    projected=[{"count": frames, "width": meta.width, "height": meta.height}],
    classes=[BatchClass(name="car", output_type="bbox")],
)
print(quote.estimated_credits / 1e6, "USD", quote.sufficient)

pictograph auto-annotate quote --frames 900 --width 1920 --height 1080 --classes car:bbox

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/batch/quote" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"projected": [{"count": 900, "width": 1920, "height": 1080}], "classes": [{"name": "car", "output_type": "bbox"}]}'

Returns BatchQuote

BatchQuote · 8 fields

class BatchQuote(BaseModel):
    """What a batch job WOULD cost - the same deposit `batch()` would take."""
    total_images: int
    estimated_credits: int
    sahi_tiles: int = 0
    containers: int = 0
    remaining_credits: int = 0
    sufficient: bool = True
    max_images: int = 5000
    exceeds_max_images: bool = False

batch

Async, many images. Use it above roughly 10 images. Kicks off one job and, by default, polls until it reaches a terminal status.

SAM3 jobs (model omitted) need at least one class. Trained-model jobs (model="Road Sign Detector") accept empty classes for classification models. On the REST wire the trained model is addressed as model_id with a UUID, and SAHI is opted in with sahi_enabled / sahi_slice_size.

Source: AutoAnnotate.batch

Arg	Type	Default	Notes
`dataset_name`	`str`	required	Dataset name.
`image_filenames`	`list[str]`	required	Filenames to process, 1 to 5000
`classes`	`list[BatchClass]`	required for SAM3	`{name, output_type}` per class
`confidence_threshold`	`float`	`0.5`	Minimum confidence
`model`	`str \| None`	`None`	Trained model by name (a UUID works); `None` routes to SAM3
`top_k`	`int`	`1`	Classifier only: tags per image
`wait`	`bool`	`True`	Poll until terminal
`poll_interval` / `timeout`	`float`	`5.0` / `1800.0`	Polling cadence and deadline, in seconds

from pictograph import BatchClass

job = client.auto_annotate.batch(
    dataset_name="road-signs",
    image_filenames=["img-1.jpg", "img-2.jpg", "img-3.jpg"],
    classes=[
        BatchClass(name="car", output_type="polygon"),
        BatchClass(name="person", output_type="bbox"),
    ],
    confidence_threshold=0.5,
    wait=True,
)
print(job.status, job.processed_images, job.total_annotations_added)

pictograph auto-annotate batch road-signs \
  --images img-1.jpg,img-2.jpg,img-3.jpg \
  --classes car:polygon,person:bbox

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/batch" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "image_filenames": ["img-1.jpg", "img-2.jpg"], "classes": [{"name": "car", "output_type": "polygon"}, {"name": "person", "output_type": "bbox"}], "confidence_threshold": 0.5}'

Returns BatchJob

BatchJob · 10 fields

class BatchJob(BaseModel):
    """Snapshot of an auto-annotate batch job's progress."""
    job_id: str
    status: Literal['pending', 'running', 'completed', 'failed', 'cancelled']
    progress: int = 0
    total_images: int = 0
    processed_images: int = 0
    total_annotations_added: int = 0
    failed_images: int = 0
    error_message: str | None = None
    estimated_credits: int | None = None
    completed_at: datetime | None = None

SAHI sliced inference

For high-resolution images with small objects (drone shots, wide industrial scenes, litter detection), enable SAHI. Each image is sliced into overlapping tiles, every tile runs at near-native resolution alongside one full-image pass, and tile fragments are merged back into whole instances server-side.

job = client.auto_annotate.batch(
    dataset_name="road-signs",
    image_filenames=["site-4k.jpg"],
    classes=[BatchClass(name="person", output_type="polygon")],
    sahi=True,
    sahi_slice_size=640,
)

pictograph auto-annotate batch road-signs \
  --images site-4k.jpg --classes person:polygon --sahi --sahi-slice-size 640

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/batch" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "image_filenames": ["site-4k.jpg"], "classes": [{"name": "person", "output_type": "polygon"}], "sahi_enabled": true, "sahi_slice_size": 640}'

SAHI is SAM3 only; a trained-model job with SAHI enabled is rejected with a 400. Smaller slices (256 to 1024 pixels) find smaller objects but run more passes, and cost scales with each image’s tile count. quote prices it exactly.

dataset

Pages the dataset’s image list, holds back images that already have annotations unless overwrite=True, runs batch or text mode, saves the results, and returns a per-image report. No CLI command; use pictograph auto-annotate batch with an explicit --images list.

Source: AutoAnnotate.dataset

Arg	Type	Default	Notes
`dataset_name`	`str`	required	Dataset name.
`classes`	`Sequence[BatchClass \| tuple[str, str] \| dict[str, str]]`	required	Class configs to detect. Accepts: - `BatchClass` instances (canonical), - `(name, output_type)` tuples (shorthand), - `{"name": ..., "output_type": ...}` dicts. `output_type` defaults to `"polygon"` for tuples without one.
`mode`	`AnnotateMode`	`'batch'`	`"batch"` (default) - async batch job; `"text"` - synchronous per-image text prompt.
`confidence_threshold`	`float`	`0.5`	SAM3 confidence cutoff (0-1).
`overwrite`	`bool`	`False`	When `False` (default), skip images that already have at least one annotation. When `True`, re-annotate every image.
`max_images`	`int \| None`	`None`	Cap the number of images processed (useful for dry-runs). `None` means “all”.
`poll_interval`	`float`	`5.0`	`"batch"` mode only - seconds between polls.
`timeout`	`float`	`1800.0`	`"batch"` mode only - max seconds to wait.

report = client.auto_annotate.dataset(
    dataset_name="road-signs",
    classes=[("car", "polygon"), ("person", "bbox")],
    mode="batch",
    overwrite=False,
    max_images=None,
)
print(report.images_processed, report.annotations_added, len(report.failures))

# Batch auto-annotation across a dataset.
pictograph auto-annotate batch road-signs \
  --images "img-001.jpg,img-002.jpg" \
  --classes "stop sign:bbox"

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/batch" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" -H "Content-Type: application/json" \
  -d '{"dataset_name": "road-signs", "classes": ["stop sign"]}'

Returns AnnotateReport

AnnotateReport · 8 fields

class AnnotateReport(BaseModel):
    """Outcome of an AutoAnnotate.dataset call."""
    dataset_name: str
    images_attempted: int = 0
    images_processed: int = 0
    images_skipped: int = 0
    images_capped: int = 0
    annotations_added: int = 0
    failures: list[AnnotationFailure] = []
    job_id: str | None = None

cancel_batch

Cancel a pending or running batch job.

Source: AutoAnnotate.cancel_batch

Arg	Type	Default	Notes
`job_id`	`str`	required	Job id, as returned when the job was created.

job = client.auto_annotate.cancel_batch(
    job_id="1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94",
)
print(job.status, job.processed_images, "of", job.total_images, "already done")

pictograph auto-annotate cancel-batch 1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94

curl -s -X POST "https://api.pictograph.io/api/v1/developer/auto-annotate/batch/$JOB_ID/cancel" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY"

Returns BatchJob

BatchJob · 10 fields

class BatchJob(BaseModel):
    """Snapshot of an auto-annotate batch job's progress."""
    job_id: str
    status: Literal['pending', 'running', 'completed', 'failed', 'cancelled']
    progress: int = 0
    total_images: int = 0
    processed_images: int = 0
    total_annotations_added: int = 0
    failed_images: int = 0
    error_message: str | None = None
    estimated_credits: int | None = None
    completed_at: datetime | None = None

get_batch

Fetch the current status of a batch job.

Source: AutoAnnotate.get_batch

Arg	Type	Default	Notes
`job_id`	`str`	required	Job id, as returned when the job was created.

job = client.auto_annotate.get_batch(
    job_id="1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94",
)
print(job.status, job.progress, job.total_annotations_added)

# No `get-batch` command; use the SDK or REST.
pictograph auto-annotate get 1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94

curl -s "https://api.pictograph.io/api/v1/developer/auto-annotate/batch/$JOB_ID" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY"

Returns BatchJob

BatchJob · 10 fields

class BatchJob(BaseModel):
    """Snapshot of an auto-annotate batch job's progress."""
    job_id: str
    status: Literal['pending', 'running', 'completed', 'failed', 'cancelled']
    progress: int = 0
    total_images: int = 0
    processed_images: int = 0
    total_annotations_added: int = 0
    failed_images: int = 0
    error_message: str | None = None
    estimated_credits: int | None = None
    completed_at: datetime | None = None

wait_for_batch

Poll a batch job until terminal status or timeout.

Source: AutoAnnotate.wait_for_batch

Arg	Type	Default	Notes
`job_id`	`str`	required	Job id, as returned when the job was created.
`poll_interval`	`float`	`5.0`	Seconds between polls.
`timeout`	`float`	`1800.0`	Max seconds to wait before raising `PollTimeoutError`. The job keeps running.

job = client.auto_annotate.wait_for_batch(
    job_id="1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94",
    timeout=1800.0,
)
print(job.status, job.total_annotations_added, "annotations across", job.total_images, "images")

# No `wait-for-batch` command; use the SDK or REST.
pictograph auto-annotate get 1a6e4f28-9b30-4c57-8d21-6f3b0a5e7c94

# No wait endpoint - `wait_for_batch` polls this one until `status` is terminal:
curl -s "https://api.pictograph.io/api/v1/developer/auto-annotate/batch/$JOB_ID" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY"

Returns BatchJob

BatchJob · 10 fields

class BatchJob(BaseModel):
    """Snapshot of an auto-annotate batch job's progress."""
    job_id: str
    status: Literal['pending', 'running', 'completed', 'failed', 'cancelled']
    progress: int = 0
    total_images: int = 0
    processed_images: int = 0
    total_annotations_added: int = 0
    failed_images: int = 0
    error_message: str | None = None
    estimated_credits: int | None = None
    completed_at: datetime | None = None

Cost

SAM3 auto-annotation draws on your USD compute credit and is available on every plan. Point and box prompts share one image embedding per session, so follow-ups on the same image are near-free. Batch is priced per image processed, times the class count, times the SAHI tile count: quote computes it exactly, and PaymentRequiredError.credit_cost carries the ask in micro-USD on rejection.

Common errors

Status	Exception	Cause
402	`PaymentRequiredError`	Out of credits
404	`NotFoundError`	Dataset or image missing
408	`PollTimeoutError`	Batch did not finish within `timeout`. The job keeps running server-side.
422	`ValidationError`	SAM3 with no classes, SAHI with a trained model, or `sahi_slice_size` out of range

Auto-annotate

point

box

text

quote

batch

SAHI sliced inference

dataset

cancel_batch

get_batch

wait_for_batch

Cost

Common errors

See also