Auto-annotate

SAM3 single-prompt (point, box, text) and async batch endpoints for AI-generated annotations.

Pictograph runs SAM3 (Segment Anything Model 3) on T4 GPUs for auto-annotation. Three single-image prompt modes plus an async batch endpoint for many images at once.

from pictograph import Client
client = Client()

point — “click here, segment that”

Best when the user knows the object’s location.

result = client.auto_annotate.point(
    dataset_name="my-dataset",
    image_filename="img-1.jpg",
    x=320, y=240,
    name="car",
    positive_points=[(310, 250)],   # optional extra positives
    negative_points=[(100, 100)],   # exclude regions
    score_threshold=0.75,
)
# result.annotations[0] is a polygon — call client.annotations.save() to persist.

Returns PromptResult with status ∈ {"success", "no_detection", "below_threshold"}. On success, annotations[0] is a PolygonAnnotation.

box — “segment everything in this box”

Best when the user has drawn a rough bounding box.

result = client.auto_annotate.box(
    dataset_name="my-dataset",
    image_filename="img-1.jpg",
    box={"x": 100, "y": 200, "w": 200, "h": 150},
    name="car",
    return_polygon=True,           # also include polygon (not just bbox)
    confidence_threshold=0.5,
    negative_boxes=[{"x": 50, "y": 50, "w": 30, "h": 30}],
)

return_polygon=False returns only the refined bbox.

text — “find all ”

Open-vocabulary text prompt. Best for many objects in one image.

result = client.auto_annotate.text(
    dataset_name="my-dataset",
    image_filename="img-1.jpg",
    text_prompt="red cars",
    output_type="polygon",         # or "bbox"
    confidence_threshold=0.3,
    max_detections=50,
)

batch — async, many images

Use for >10 images. Kicks off one job; polls until terminal status.

from pictograph import BatchClass

job = client.auto_annotate.batch(
    dataset_name="my-dataset",
    image_filenames=["img-1.jpg", "img-2.jpg", "..."],
    classes=[
        BatchClass(name="car", output_type="polygon"),
        BatchClass(name="person", output_type="bbox"),
    ],
    confidence_threshold=0.5,
    wait=True,
    poll_interval=5.0,
    timeout=1800.0,                # 30 min default
)
print(job.status, job.processed_images, job.total_annotations_added)

wait=False returns the job immediately — poll later via:

job = client.auto_annotate.get_batch(job.job_id)
job = client.auto_annotate.wait_for_batch(job.job_id, timeout=600.0)
client.auto_annotate.cancel_batch(job.job_id)

auto_annotate_dataset workflow

Higher-level helper that paginates a dataset’s image list, runs batch or text mode, and returns a per-image report:

from pictograph.workflows import auto_annotate_dataset

report = auto_annotate_dataset(
    client,
    "my-dataset",
    classes=[("car", "polygon"), ("person", "bbox")],
    mode="batch",                  # "batch" | "text"
    confidence_threshold=0.5,
    overwrite=False,               # skip already-annotated images
    max_images=None,               # None = all
)
print(report.images_processed, report.annotations_added, len(report.failures))

Choosing a mode

Scenario	Mode
User clicks one spot	`point`
User drags a rough box	`box`
Many images, known classes	`batch` (or `text` per image for small datasets)
Single image, multiple objects	`text`

Cost

SAM3 is paid:

3-credit minimum per session (image embedding generation).
~1 credit per additional prompt on the same image (sub-second).
Batch is charged per image processed.

client.credits.estimate("sam3_per_minute", quantity=N) for an A10G-time estimate — pre-charge is fixed per call. Read PaymentRequiredError.required on rejection for the exact ask.

Common errors

Status	Exception	Cause
402	`PaymentRequiredError`	Out of credits
404	`NotFoundError`	Dataset or image missing
408	`PollTimeoutError`	Batch job didn’t finish within `timeout` (job keeps running on the backend)