SAM3

SAM3 is Pictograph’s built-in segmentation model: it needs no training and knows any class you can name. Send it an image with one prompt and it returns the objects it finds, as polygons or boxes.

The image is sent with the call. It does not have to be in a dataset, and nothing is written: no dataset, no image, no annotation. For an image that is already in a dataset, the auto-annotate prompts run on it in place, and batch saves draft labels for many images.

predict

Run SAM3 on one image with one prompt. Give exactly one of text, box or points. The first call after a quiet spell starts a GPU and can take a minute or two; more prompts on the same image after that are fast.

Source: Sam3.predict

result = client.sam3.predict(
    image="shelf.jpg",
    text="price tag",
)
for annotation in result.annotations:
    print(annotation.name, annotation.type, annotation.confidence)

pictograph sam3 predict shelf.jpg --text "price tag" --json

curl -s -X POST "https://api.pictograph.io/api/v1/developer/sam3/predict" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" \
  -F "file=@shelf.jpg" \
  -F "text=price tag"

Returns Sam3Prediction

class Sam3Prediction(BaseModel):
    """SAM3's answer to one prompt on an image you sent. Nothing was saved."""
    status: Literal['success', 'no_detection', 'below_threshold']
    annotations: list[Annotation] = []
    score: float | None = None
    inference_time: float | None = None
    prompt: Literal['text', 'box', 'point']
    width: int
    height: int
    image_sha256: str | None = None
    confidence_threshold: float | None = None
    degraded: bool = False

status is "success" when annotations holds at least one object, "no_detection" when SAM3 found nothing for the prompt, and "below_threshold" when it found objects but none scored above confidence_threshold. degraded is true when the image held more matches than fit in GPU memory, so only the most confident came back: confidence_threshold says where the cut fell. The REST response wraps the result as {"data": {...}}.

Prompts

A box prompt with a region carved out, from a URL:

result = client.sam3.predict(
    image="https://example.com/street.jpg",
    box=(120, 80, 200, 160),
    negative_boxes=[(150, 100, 20, 20)],
    name="car",
    output="bbox",
)

pictograph sam3 predict https://example.com/street.jpg --box 120,80,200,160 --negative-box 150,100,20,20 --name car --output bbox

curl -s https://example.com/street.jpg -o street.jpg
curl -s -X POST "https://api.pictograph.io/api/v1/developer/sam3/predict" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" \
  -F "file=@street.jpg" \
  -F "box=120,80,200,160" -F "negative_box=150,100,20,20" \
  -F "name=car" -F "output=bbox"

Points on one object, with one point that excludes a region:

result = client.sam3.predict(
    image=frame_bytes,
    points=[(320, 240), (330, 260)],
    negative_points=[(300, 400)],
    name="forklift",
)

cat frame.jpg | pictograph sam3 predict - --point 320,240 --point 330,260 --point 300,400,neg --name forklift

curl -s -X POST "https://api.pictograph.io/api/v1/developer/sam3/predict" \
  -H "X-API-Key: $PICTOGRAPH_API_KEY" \
  -F "file=@frame.jpg" \
  -F "point=320,240" -F "point=330,260" -F "point=300,400,neg" \
  -F "name=forklift"

Over HTTP the image is the multipart file, and every prompt field rides beside it: text, box=x,y,w,h, point=x,y (repeat it; x,y,neg excludes a region), negative_box=x,y,w,h (repeatable), name, output, confidence_threshold and max_detections. A field the route does not read is refused, so a typo never runs with a default instead.

Many images, a video, or a dataset

The CLI runs the same prompt on several inputs and prints one JSON line per image or frame: {"input", "result"}, or {"input", "error"} for one that failed.

pictograph sam3 predict 'shots/*.jpg' --text forklift --jsonl
pictograph sam3 predict --video clip.mp4 --sample-fps 1 --text person
pictograph sam3 predict --dataset my-dataset --directory /val --text car --limit 20

A video is decoded on your machine (pip install "pictograph[inference]" brings OpenCV) and sampled with the same rule a workflow uses, so each line carries the frame’s index and time. Each frame is one call and one image charged; for a long video, a workflow reads it on Pictograph instead. --dataset reads one of your datasets’ images and prints the results; it writes nothing. A batch stops at an error that would fail every remaining image the same way, such as an exhausted credit balance.

Coordinates

Every coordinate, in the prompt you send and in the annotations that come back, is an absolute pixel of the image as it is displayed, with the origin at the top-left. SAM3 applies the EXIF orientation first, so a phone photo taken sideways is the upright picture. width and height on the result are that upright size. A prompt outside the image is refused with the size it was checked against.

Cost

Each image is charged the SAM3 session fee once, when a prompt on it completes: about a cent per image, from your compute credits. The image is identified by its bytes (image_sha256), so more prompts on the same image are free, and a prompt that fails is free. pictograph tools describe sam3 shows the current price, and Pricing lists every operation’s cost. An organization whose balance cannot cover one session gets a 402 before anything runs.

Common errors

See also

• Auto-annotate: the same prompts on an image already in a dataset, and batch draft labels
• SAM3 auto-annotation: drafting labels for a dataset, then reviewing them
• Vision tools: pictograph tools describe sam3 returns this call’s card