Tile & augment

Reshape a dataset before training - slice images into a grid for small-object detection, or generate flipped, rotated and colour-jittered variants. Annotation geometry follows.

View as Markdown

Two ways to turn a dataset into a better training set. Tiling slices each image into a grid so small objects occupy more of their frame. Augmentation generates randomized variants to expand the effective set.

Both read the source dataset, remap every annotation’s geometry, and upload the results through the standard ingest path, so generated images get embeddings, content tags and thumbnails exactly like uploads and are searchable immediately. Neither needs an extra dependency.

Tiling is preprocessing: deterministic, driven by object size. Augmentation is version generation: randomized, driven by wanting more data. They compose - tile first for small-object detection, then augment the tiled dataset.

Source: resources/images.py

Tile for small objects

client.images.tile() slices every image into a rows × cols grid. It is the standard fix for aerial, satellite and microscopy data: annotations are translated into each tile’s local coordinates and clipped to the tile frame, so a shape straddling a boundary is split correctly across neighbours.

from pictograph import Client, TileReport

client = Client()

report: TileReport = client.images.tile(
    source="aerial",
    rows=2,
    cols=2,
    overlap=0.1,
    into="aerial-tiled",
)
print(f"{report.tiles_created} tiles generated from {report.source_images} images")

client.images.tile(
    source: str,
    *,
    rows: int = 2,
    cols: int = 2,
    overlap: float = 0.0,
    min_visibility: float = 0.1,
    include_empty: bool = True,
    into: str | None = None,
    directory_path: str = "/tiles",
    max_source_images: int | None = None,
    jpeg_quality: int = 95,
    on_progress: Callable[[int, int], None] | None = None,
) -> TileReport

rows / cols - grid dimensions (2 × 2 = four tiles per image).
overlap - a fraction (0.0–0.9) each tile extends past its edge, so an object sitting on a boundary appears whole in at least one neighbour.
min_visibility - an annotation is dropped from a tile when less than this fraction of its area survives the clip.
include_empty - set False to skip tiles left with no annotations; the default keeps them as useful background negatives.

Augment to expand the set

client.images.augment() produces N variants of every source image - flipped, rotated, cropped, colour-jittered. A flip moves each box, a rotation rotates each polygon’s points, a crop clips and drops out-of-frame objects.

from pictograph import Client, AugmentReport
from pictograph.augment import Brightness, HorizontalFlip, Rotate

client = Client()

report: AugmentReport = client.images.augment(
    source="road-signs",
    ops=[HorizontalFlip(), Rotate((-15, 15)), Brightness((0.8, 1.2))],
    multiplier=3,
    into="road-signs-aug",
)
print(f"{report.variants_created} images generated across {report.source_images} originals")

client.images.augment(
    source: str,
    ops: Sequence[Augmentation],
    *,
    multiplier: int = 3,
    into: str | None = None,
    include_original: bool = True,
    directory_path: str = "/augmented",
    seed: int | None = None,
    max_source_images: int | None = None,
    jpeg_quality: int = 95,
    drop_classes: Iterable[str] | None = None,
    skip_empty: bool = False,
    on_progress: Callable[[int, int], None] | None = None,
) -> AugmentReport

Two arguments run before augmentation, mirroring “generate a version”: drop_classes removes annotations of the named classes (and drops them from a new target’s class config), and skip_empty=True skips a source image left with no annotations, counted in report.skipped_empty.

With a seed, the sequence of variants is deterministic - re-running yields identical output - while each variant still differs from the last.

Available ops

Magnitudes accept a fixed value or a (low, high) range sampled per application (Rotate((-15, 15)), Brightness((0.8, 1.2))). p is the probability an op fires.

Geometric - transform the image and remap annotation geometry:

Op	What it does
`HorizontalFlip(p=0.5)`	Mirror left↔right.
`VerticalFlip(p=0.5)`	Mirror top↔bottom.
`Rotate90(k=1)`	Lossless 90/180/270° rotation (`k=None` picks one at random).
`Rotate(degrees=(-15, 15))`	Arbitrary rotation; the canvas expands so nothing is cropped. Boxes grow to the axis-aligned enclosure of the rotated box.
`Resize(width, height)`	Resize to a fixed size; scales geometry.
`Crop(scale=(0.8, 1.0))`	Random crop keeping a fraction of each side; clips geometry and drops objects below `min_visibility`.
`Shear(degrees=(-10, 10))`	Horizontal shear; keeps the canvas and clips geometry.

Photometric - change pixels only, geometry unchanged:

Op	What it does
`Brightness(factor=(0.8, 1.2))`	Scale brightness.
`Contrast(factor=(0.8, 1.2))`	Scale contrast.
`Saturation(factor=(0.8, 1.2))`	Scale colour saturation.
`HueShift(degrees=(-20, 20))`	Rotate the hue channel.
`Grayscale(p=1.0)`	Convert to grayscale.
`Blur(radius=(0.0, 2.0))`	Gaussian blur.
`Noise(amount=(0.0, 0.08))`	Additive luminance noise.
`CutOut(size=(0.1, 0.3), count=1)`	Erase random rectangles (random-erasing regularization).

Where the output goes

into names a new dataset, created if missing, with the source’s class config copied across. Pass into=None (or the source’s own name) to append into the source itself, under the /tiles or /augmented virtual directory. Either way, every generated image counts toward your organization’s image quota exactly like a normal upload.

Both methods return a report rather than raising on one bad image - each failure carries image_id, filename and reason, so you can retry just the affected sources.

report = client.images.tile(
    source="aerial",
    rows=3,
    cols=3,
    into="aerial-tiled",
)
print(report.source_images, report.tiles_created, report.annotations_written)
for f in report.failures:
    print(f.filename, f.reason)

The local engines

Source: tile/_tiler.py · augment/_engine.py

Both sit on a standalone engine you can drive on any local (image, annotations) pair, with no API call.

from pictograph import Client
from pictograph.tile import tile_image

client = Client()
annotations = client.annotations.get(
    dataset_name="aerial",
    image="aerial.jpg",
)

tiles = tile_image("aerial.jpg", annotations, rows=2, cols=2, overlap=0.1)
for t in tiles:
    t.image.save(f"tile_r{t.row}_c{t.col}.jpg")
    print(len(t.annotations), "annotations in this tile", t.origin)

Each Tile carries the cropped image, the geometry-remapped annotations, its grid row/col, and its origin (top-left corner in source pixels).

from pictograph import Client
from pictograph.augment import Augmenter, Brightness, HorizontalFlip, Rotate

client = Client()
annotations = client.annotations.get(
    dataset_name="road-signs",
    image="photo.jpg",
)

aug = Augmenter([HorizontalFlip(), Rotate((-15, 15)), Brightness((0.8, 1.2))], seed=42)

image, moved = aug("photo.jpg", annotations)            # one variant
variants = aug.generate("photo.jpg", annotations, n=3)  # three distinct variants
print(len(moved), "annotations moved;", len(variants), "variants")

Augmenter accepts a file path or an open Pillow image, and annotations as typed models or raw dicts.

CLI

# slice every image into a 2×2 grid → a new dataset
pictograph tile dataset aerial --into aerial-tiled --rows 2 --cols 2

# a 3×3 grid with 10% overlap, dropping empty tiles
pictograph tile dataset aerial --into aerial-tiled \
    --rows 3 --cols 3 --overlap 0.1 --exclude-empty

# generate a 3× augmented copy into a new dataset
pictograph augment dataset road-signs --into road-signs-aug \
    --multiplier 3 --flip --rotate 15 --brightness 0.2

# list every op and its flag
pictograph augment ops

Each augment flag maps to an op at a sensible strength (--rotate 15 → Rotate((-15, 15)), --brightness 0.2 → Brightness((0.8, 1.2))). Preprocessing flags work too: --drop-class person --drop-class bike --skip-empty. Run pictograph augment ops for the full list.

Geometry, precisely

Boxes are clipped to the tile or crop; polygons are Sutherland–Hodgman-clipped; keypoints falling outside are dropped.
An annotation whose visible area falls below min_visibility is removed from that tile or variant.
Flips and 90° rotations are lossless. Arbitrary rotations expand the canvas so no object is lost.
Tiling is deterministic; augmentation is deterministic under a fixed seed. Both re-run to byte-identical output.