Local inference

Your trained models are published in every executable form your edge actually runs. Load one by name and predict: the weights download and cache on first use, and the result is a per-task type whose predictions are already narrowed to the annotations that task can produce.

Source: inference/__init__.py

from pictograph import get_model, DetectionModel, DetectionResult

model: DetectionModel = get_model(name="Woody Whirling Grouse", task="object_detection")
result: DetectionResult = model.predict(
    image="photo.jpg",
)

for p in result.predictions:
    print(p.name, round(p.confidence, 2), p.bounding_box)

pip install 'pictograph[inference]'

task= is what lets the annotation typecheck without a cast, and it is verified: the loader reads the model’s real task and raises if they disagree, so it can never silently lie. Omit it and you get the AnyModel union to narrow with isinstance.

Weight formats and their runtimes

The same weights, five formats, four runtimes. Every one returns the same task class and the same typed result, so switching is a one-word change.

You pick a weight format; the runtime follows from it. There is no runtime= argument - the format determines which engine can execute it, so asking for both would let you ask for a contradiction. (pytorch and safetensors are the same tensors in different containers, which is why five formats resolve to four runtimes.)

format=	Artifact	Runtime	Install	Best for
onnx (default)	.onnx	onnxruntime	pip install 'pictograph[inference]'	The default. Runs anywhere, CPU or GPU
pytorch	.pth	pytorch	pip install 'pictograph[inference]'	Research, fine-tuning, reaching the live nn.Module
safetensors	model.safetensors	pytorch	pip install 'pictograph[inference]'	The same weights in a safer container
pytorch_engine	.pte	executorch	pip install 'pictograph[inference,executorch]'	Portable edge - phones, ARM boards, a Jetson’s CPU
tensorrt_engine	.engine	tensorrt	pip install 'pictograph[inference,tensorrt]'	NVIDIA, lowest latency

pictograph[inference] is the whole install for the first three rows - .onnx, .pth and .safetensors - across every model family. It carries torch, torchvision, safetensors and the pinned segmentation-models-pytorch alongside the ONNX stack, and the YOLOX and RF-DETR architectures are vendored into the wheel, so there is never a second pip install to run a model you trained here. The two engine runtimes need their own extra only because they cannot be folded in: ExecuTorch pins one exact torch minor, and tensorrt has no non-CUDA distribution at all.

from pictograph import get_model

onnx = get_model(name="Shelf Detector", task="object_detection")
pte = get_model(name="Shelf Detector", task="object_detection", format="pytorch_engine")
torch_model = get_model(name="Shelf Detector", task="object_detection", format="pytorch")

for m in (onnx, pte, torch_model):
    result = m.predict(image="photo.jpg")  # same type, same fields, every time
    print(m.backend, m.device, len(result.predictions))

onnxruntime coreml 7
executorch cpu 7
pytorch mps 7

What actually ran

Provenance is reported as measured, never as requested - a CUDA session that fell back to CPU says cpu.

Attribute	Meaning
model.backend	pytorch | executorch | onnxruntime | tensorrt
model.device	cpu | cuda | mps | coreml - the device that ran it
model.providers	ONNX Runtime providers in resolution order; the .pte’s delegate backends; the plan’s TensorRT version and GPU target. Empty on pytorch
model.classes	Class names in the model’s label order

Every result carries the same three, so a prediction can be traced back to the runtime that produced it:

model = get_model(name="Shelf Detector", task="object_detection")
result = model.predict(
    image="photo.jpg",
)
print(result.backend, result.device, result.inference_ms)

Offline loading

load_model needs no API key and makes no network call. Point it at the weights and the config.json from the model’s Files tab. The runtime comes from the weights suffix, so the call shape never changes:

from pictograph import load_model, ClassificationModel

# Artifacts are named after the MODEL, so a directory of downloads stays legible.
for weights in (
    "shelf-detector.onnx",
    "shelf-detector-xnnpack.pte",
    "shelf-detector-sm75-trt10.13.3.9.engine",
):
    model: ClassificationModel = load_model(
        weights=weights,
        config="shelf-detector.config.json",
        task="classification",
    )
    print(model.backend, model.device)

Precision

fp16 means fp16 weights with fp32 inputs and outputs - a drop-in for the fp32 serving path, not a different interface.

from pictograph import get_model, DetectionModel

model: DetectionModel = get_model(
    name="Shelf Detector",
    task="object_detection",
    format="onnx",
    precision="fp16",
)
print(model.backend, model.device)

.onnx, .pte and .engine are all published per precision, so precision= selects which artifact is fetched. A native PyTorch checkpoint is the one artifact with no derived form - it is the raw trained tensors - so asking for a precision the model was not trained at raises and tells you what to do instead.

Choosing a device

device= names the hardware. The default "auto" looks at what your chosen format= can run on and picks the best available. "cpu" is reproducible and instant to load, which is what you want in CI and for numerical-parity work.

from pictograph import get_model

model = get_model(name="My Classifier", task="classification", device="cpu")

device=	Meaning
auto (default)	Best available for the chosen format
cpu	CPU only
mps	Apple acceleration
cuda / cuda:N	NVIDIA, optionally a specific GPU

The execution provider follows from the (format, device) pair, so you never name one. mps is the Apple accelerator family, not just PyTorch’s backend: it means CoreML for an .onnx graph and torch-MPS for a checkpoint. cuda means the CUDA provider for .onnx and TensorRT for an .engine.

Not every device fits every format: a .engine is CUDA-only, a .pte never runs on CUDA. Asking for a pair that cannot exist raises and names what that format can run on, rather than quietly falling back to something slower than you asked for.

model.device reports what actually ran, which can be more specific than what you asked for. Request mps with an .onnx and it reports coreml.

TensorRT engines are not portable

A TensorRT engine is not a model file. It is a compiled, hardware-specific execution plan. It is bound to all four of:

1. the GPU architecture it was built on - T4 is sm75, A100 sm80, A10G sm86, L4 sm89, H100 sm90;
2. the exact TensorRT version - a minor bump invalidates every previously serialized plan;
3. the precision it was built for - fp32 and fp16 are different plans;
4. its build-time shape profile.

An engine built on an A100 does not run on a T4. It fails at load, not at accuracy. The filename records the binding (sm80-trt10.13.3.9-fp16.engine) and the loader checks it against your device before deserializing, so a mismatch is an explanation rather than a crash:

RuntimeError: This TensorRT engine was built for trt-10.13.3.9 on sm80; this
device is sm75. A TensorRT plan is compiled for one GPU architecture and one
TensorRT version and cannot be loaded anywhere else - rebuild the engine for
your device.

Because of that, get_model(format="tensorrt_engine") defaults to fetching the engine for your GPU. Pass target= to fetch a different one deliberately:

from pictograph import get_model

model = get_model(
    name="Shelf Detector",
    task="object_detection",
    format="tensorrt_engine",
    target="sm80",  # build for an A100 from a machine that is not one
)

If you rename an engine, keep its manifest row beside it as <name>.json so the check still works.

An ExecuTorch .pte has none of this: it is portable across devices for its lowering backend, and the portable-CPU (XNNPACK) build is the default published artifact precisely because it runs everywhere.

Switching runtimes does not change your results. Every runtime shares the same preprocessing and postprocessing and differs only in the forward pass, so the same weights predict the same classes at the same coordinates.

The five task types

One model class and one result class per task, whichever runtime is behind it.

task=	Model class	Result class	predictions
object_detection	DetectionModel	DetectionResult	BBoxAnnotation
instance_segmentation	InstanceSegmentationModel	InstanceSegmentationResult	PolygonAnnotation | BBoxAnnotation
semantic_segmentation	SemanticSegmentationModel	SemanticSegmentationResult	PolygonAnnotation
keypoint_detection	KeypointModel	KeypointResult	KeypointAnnotation
classification	ClassificationModel	ClassificationResult	(ranked classes)

Classifiers return ranked classes instead of geometry, and top is never None:

from pictograph import get_model, ClassificationModel, ClassificationResult

model: ClassificationModel = get_model(name="My Classifier", task="classification")
result: ClassificationResult = model.predict(
    image="cat.jpg",
    top_k=3,
)

print(result.top.name, round(result.top.confidence, 2))
for score in result.classes:
    print(score.name, round(score.confidence, 3))

Batches and cleanup

from pictograph import get_model

with get_model(name="Shelf Detector", task="object_detection") as model:
    for result in model.predict_batch(images=["a.jpg", "b.jpg", "c.jpg"]):
        print(len(result.predictions))

Every model is a context manager, and you should close it when you are done: an ONNX session, a CUDA allocator and an ExecuTorch memory arena all hold memory the garbage collector will not promptly return.

Accepted image inputs

predict takes a file path, an http(s) URL, raw bytes, a BGR numpy array (what cv2.imread returns), or a PIL image.

import cv2
from pictograph import get_model

model = get_model(name="Shelf Detector", task="object_detection")
result = model.predict(
    image=cv2.imread("photo.jpg"),
)

Score a model against labelled data

Source: metrics/_detection.py

pictograph.metrics runs entirely locally. evaluate_detections matches predictions to ground truth by IoU and returns per-class and overall precision, recall, F1 and average precision, including mAP (result.mean_average_precision).

from pictograph import Client
from pictograph.metrics import evaluate_detections

client = Client()
filenames = ["img-001.jpg", "img-002.jpg"]
ground_truth = {
    name: client.annotations.get(dataset_name="road-signs", image=name)
    for name in filenames
}
predictions = {name: run_my_model(name) for name in filenames}

result = evaluate_detections(predictions, ground_truth, iou_threshold=0.5)
print(result.precision, result.recall, result.f1)
for name, m in result.per_class.items():
    print(name, m.precision, m.recall, m.support)

confusion_matrix() in the same module surfaces cross-class confusion. To have Pictograph run the inference and store the result instead, use client.model_evaluations - the metric math is identical, so a server run and a local run on the same data agree.

Feed a PyTorch DataLoader

Source: _torch_dataset.py

client.datasets.as_pytorch(name) returns a map-style torch.utils.data.Dataset. Each item is an (image, target) pair following the torchvision detection convention (boxes in xyxy, integer labels, area, iscrowd, image_id, and the raw annotations). Images download lazily on first access and are cached.

from torch.utils.data import DataLoader
from pictograph import Client

client = Client()
dataset = client.datasets.as_pytorch(
    name="road-signs",
)
loader = DataLoader(dataset, batch_size=8, collate_fn=lambda batch: tuple(zip(*batch)))

for images, targets in loader:
    ...  # your training step

Draw annotations

Source: viz.py

draw_annotations renders any Pictograph annotations onto an image using only Pillow, a base dependency. All four annotation types render, and each class gets a stable, distinct colour.

from pictograph import Client, draw_annotations

client = Client()
annotations = client.annotations.get(
    dataset_name="road-signs",
    image="img-001.jpg",
)
annotated = draw_annotations(
    image="photo.jpg",
    annotations=annotations,
)
annotated.save("photo.annotated.png")