Pictograph annotation format

Every annotation in Pictograph follows the same schema. Snake-case field names, no shorthand: bounding boxes are objects {x, y, w, h}, polygons are multi-ring paths, polylines are ordered point lists, keypoints are single points.

The class-label field is name (not class). Do not improvise.

Discriminator

type	Geometry container	Notes
bbox	bounding_box: {x, y, w, h}	Axis-aligned rectangle.
polygon	polygon: {paths: [[{x, y}, ...], ...]}	Multi-ring (holes via even-odd).
polyline	polyline: {path: [{x, y}, ...]}	Open path, doesn’t close.
keypoint	keypoint: {x, y}	Single landmark.

Required fields

Field	Type	Notes
id	non-blank string	Unique within the image. UUIDs preferred.
name	non-blank string	Class label. Must match a class in project_config.classes (case-sensitive).
type	one of bbox/polygon/polyline/keypoint	Discriminator.
<geometry>	see table above	Field name is determined by type.

Optional fields

Field	Default	Notes
confidence	1.0	Range [0, 1]. SAM3 sets this; manual annotations get 1.0.
created_by	null	UUID of the creator. Backend fills this for SDK uploads.
attributes	[]	User-defined metadata. Backend stores opaque.
bounding_box (polygon/polyline)	computed	Backend auto-computes the enclosing rectangle if omitted.

Examples

Bounding box

{
  "id": "ann-1",
  "name": "person",
  "type": "bbox",
  "bounding_box": {"x": 100, "y": 200, "w": 50, "h": 80}
}

Polygon

{
  "id": "ann-2",
  "name": "car",
  "type": "polygon",
  "polygon": {
    "paths": [[
      {"x": 10, "y": 20}, {"x": 110, "y": 20},
      {"x": 110, "y": 80}, {"x": 10, "y": 80}
    ]]
  }
}

Polygon with hole

{
  "id": "ann-3",
  "name": "donut",
  "type": "polygon",
  "polygon": {
    "paths": [
      [{"x": 0, "y": 0}, {"x": 100, "y": 0}, {"x": 100, "y": 100}, {"x": 0, "y": 100}],
      [{"x": 30, "y": 30}, {"x": 70, "y": 30}, {"x": 70, "y": 70}, {"x": 30, "y": 70}]
    ]
  }
}

Polyline

{
  "id": "ann-4",
  "name": "lane_centerline",
  "type": "polyline",
  "polyline": {
    "path": [
      {"x": 0, "y": 100}, {"x": 50, "y": 100}, {"x": 100, "y": 100}
    ]
  }
}

Keypoint

{
  "id": "ann-5",
  "name": "left_eye",
  "type": "keypoint",
  "keypoint": {"x": 250, "y": 180}
}

Storage

Annotations are stored in project_images.annotations_json as a plain array — no wrapper:

[
  {"id": "ann-1", "name": "person", "type": "bbox", "bounding_box": {…}},
  {"id": "ann-2", "name": "car",    "type": "polygon", "polygon": {…}}
]

Updating an image’s annotations is a full overwrite: pass the complete list every time. There is no partial-update endpoint.

Common mistakes

• ❌ "class": "person" — must be "name".
• ❌ "polygon": [[10, 20, 30, 40]] — flat array. Must be [{"x": …, "y": …}].
• ❌ "bbox": [x, y, w, h] — array. Must be "bounding_box": {x, y, w, h} object.
• ❌ Class label not in project_config.classes — backend rejects with 400.
• ❌ Polygon ring with < 3 points — Pydantic rejects on save.

SDK helpers

from pictograph import BBoxAnnotation, BoundingBox, PolygonAnnotation, PolygonGeometry, Point

bbox = BBoxAnnotation(
    id="ann-1",
    name="person",
    bounding_box=BoundingBox(x=100, y=200, w=50, h=80),
)

polygon = PolygonAnnotation(
    id="ann-2",
    name="car",
    polygon=PolygonGeometry(paths=[
        [Point(x=10, y=20), Point(x=110, y=20), Point(x=110, y=80)],
    ]),
)

client.annotations.save(image_id, [bbox, polygon])

The SDK Pydantic models are the source of truth — they generate the JSON Schema this page describes. If a backend rejects your payload, diff your dump (.model_dump(mode="json", exclude_none=True)) against the rejection message.