SAM3 Auto-Annotation

SAM3 turns a prompt into a mask. Give it a point, a box, or a text phrase and it returns a pixel-perfect polygon you can edit, export, or train on. You can run it on a single image in the editor, or as an async batch over a whole dataset from the SDK or CLI. Manual annotation is always free.

from pictograph import Client

client = Client()

result = client.auto_annotate.text(
    dataset_name="Road Signs",
    image_filename="img-001.jpg",
    text="stop sign",
    name="stop_sign",
)
client.annotations.save("img-001", result.annotations)

How do point, box, and text prompts work?

There are three prompt modes, and all three return the same kind of result: editable polygons (and bounding boxes where you ask for them).

• Point. Click to add positive points on the object and negative points to exclude regions. Best for one specific instance.
• Box. Drag a box around an object. SAM3 segments everything that matches inside it, which is the fastest way to label many similar objects.
• Text. Type a phrase such as “stop sign”. SAM3 grounds the concept and labels every match, with no training required.

# Point prompt
client.auto_annotate.point(
    dataset_name="Road Signs",
    image_filename="img-001.jpg",
    points=[{"x": 240, "y": 180, "label": 1}],   # label 1 = positive, 0 = negative
    name="stop_sign",
)

# Box prompt
client.auto_annotate.box(
    dataset_name="Road Signs",
    image_filename="img-001.jpg",
    box={"x": 120, "y": 90, "w": 200, "h": 200},
    name="stop_sign",
)

How do I auto-annotate a whole dataset?

Use a batch job. It runs N images across M classes in a single async job, off the request path, so it scales to thousands of images.

job = client.batch.auto_annotate(
    dataset_name="Road Signs",
    classes=["stop sign", "yield sign", "speed limit"],
)
job = client.batch.wait(job.id)
print(job.annotated, "images labeled")

How do masks become editable annotations?

Every result is a standard Pictograph annotation: a polygon with paths, or a bounding box with x, y, w, h. They are saved to the image like any manual annotation, so you can refine vertices in the editor, filter by class, export to COCO or YOLO, or train a model directly on them.

Next steps

• Auto-annotation overview
• Text-prompt segmentation
• Train a model on your labeled data

Last updated June 2026.