Dataset formats and the exact round trip

Three things “format support” can mean

These are genuinely different, and conflating them is where data quietly goes missing. AnnotateIt keeps them separate:

• The standard layer — the files a third-party COCO / YOLO / VOC / Datumaro / Supervisely / MOT / KITTI / MOTS tool will read. Fully valid on its own.
• External import — the variants of someone else’s archive AnnotateIt can validate and convert into its own model.
• The exact AnnotateIt round trip — re-importing an archive AnnotateIt itself wrote. The annotateit-metadata.json sidecar restores the app features the standard cannot express, without breaking the standard files.

Original media and safe video timing

Dataset formats such as COCO and YOLO describe annotations, not image or video codecs. Downloads, dataset exports and project backups preserve untouched original media bytes; exporting does not alter the stored originals. Archive entry names may follow format or collision rules — for example, MOTS numbers its video sequences — while output extensions and media MIME types match the encoding. The PNG still used to display a TIFF, HEIC/HEIF, AVIF or GIF does not replace that original on export. An explicit image-editor save is different: it replaces the edited media, and the archive then contains those saved bytes with consistent filenames and annotation references.

A third-party training tool must still be able to decode the exported media. A valid COCO or YOLO archive does not make HEIF, TIFF or an uncommon video codec readable in that tool. Prepare a compatible copy when needed; export does not silently convert the originals.

New video imports record verified frame timing, and annotated exports carry its version marker in the sidecar. Re-import checks the saved frame rate and frame count against the actual video before saving its annotations; a mismatch fails the import. Legacy annotations without that marker are preserved but remain blocked from video editing. They are not silently upgraded, renumbered or moved onto a newly measured timeline. Re-importing an original as a new video does not automatically migrate its old annotations; keep the old data and review any transfer separately.

Format matrix

COCO, YOLO, Pascal VOC and KITTI hold no native video; Supervisely Video, MOT and MOTS hold no still images; plain ZIP holds no annotations. The Export dialog only offers a format the current dataset can actually fill, and it lists everything a chosen format leaves behind before you export — nothing is dropped silently.

Every shape in every format

The table above says what a format is for; this one says exactly what happens to each shape you can draw. Read down a column to see what a format costs you. Every cell is the behaviour of the exporter itself — which combinations you can actually reach is the next table.

Which of those cells you can actually reach depends on the project type, because the Export dialog only lists formats that suit it:

Pascal VOC, MOT and KITTI are detection-only, which is why no polygon, polyline or mask reaches them from the interface; MOTS is the mirror case — instance-segmentation video only. Rotated boxes remain supported when imported, and Datumaro is the only standard layer that exports their angle intact.

The exact round trip: annotateit-metadata.json

The sidecar does not replace the standard files — it sits beside them. So one archive is read by a third-party tool within the chosen standard and, at the same time, restored completely by AnnotateIt. Re-importing a YOLO archive AnnotateIt wrote gives back the attributes and prompts YOLO itself has nowhere to store.

New sidecars use schema v2. The reader also accepts production schema v1 only when its legacy tasks array contains exactly one task; it normalises that record in memory and applies every current validation below. Zero or more than one legacy task is rejected. The task domain must match the project domain and its label ids must exactly equal the exported label set. Root, task and keypoint records accept only the keys defined for their schema. What the sidecar restores:

• Labels — names, colours, groups, hierarchy, behaviour and empty-label semantics.
• Attributes — the full definitions (type, options, range, default, mutable, required) and the values on each annotation and track, including foreign namespaces from other formats.
• Text prompts on every label.
• The project task — its title, domain and label membership — plus the classification subtype.
• Exact annotation DTOs — ids are re-minted on import, but labels, revisit state, probability, source, geometry, visibility, z-order and attributes are kept.
• The keypoint skeleton — joint labels, edges, template x/y and per-joint visibility.
• Video — stored frame annotations, and native tracks with their sorted keyframes, outside flags and attributes.

The sidecar is fail-closed. A missing or unknown schema version, an unsupported field, a duplicate or unknown id, incomplete task membership, a label cycle, an invalid keypoint template, an out-of-bounds shape, a keyframe outside the video, or an attribute value with no matching definition all reject the import before a single row is written — it never restores half a dataset and calls it success.

What to know per format

COCO

Detection writes bbox [x, y, width, height]; instance segmentation writes polygon rings with a shoelace area, and a pixel mask writes native run-length encoding; a keypoint project writes joint names, a 1-based skeleton and per-point visibility. iscrowd is a first-class field and other attributes go in a generic attributes block. On import, several COCO JSONs (split files) are merged with a category-id consistency check; an RLE mask is imported as a mask, so holes and thin bridges survive instead of being traced away; and every keypoint annotation in the file becomes its own skeleton, so a multi-person image arrives as several independent poses rather than being refused.

YOLO

Detection writes normalised class cx cy w h; segmentation writes class x1 y1 x2 y2 …; split mode writes the Ultralytics images/<subset> + labels/<subset> layout with a data.yaml. Single-label image classification instead writes the Ultralytics ImageFolder layout train|val|test/<class>/<image>; an unlabeled or multiply labeled image, hierarchy, unsafe class directory, or split with no training image stops the export rather than producing a partial dataset. Multi-label, hierarchical, anomaly and video classification are not offered as YOLO. On import, object-row YOLO reads the class list from obj.names or a .yaml (inline, block or index:name map), and validates every row — class index, coordinate count, finite values, normalised bounds and non-zero area; ImageFolder classification import is not claimed. The standard has no place for attributes, prompts or hierarchy; those survive only in the sidecar of an archive AnnotateIt wrote.

Pascal VOC

Detection only, honestly: the exporter writes VOC XML with standard 1-based inclusive coordinates and does not pretend to do segmentation, which would need indexed PNG masks. VOC’s occluded, truncated, difficult and pose become editable attributes on import, and ImageSets/Main/train|val|test.txt restores the split as locked manual assignments.

Datumaro

The fidelity-first choice for classification, anomaly, rich label metadata, or a dataset that mixes images and video. The standard layer covers label annotations, bbox, rotated bbox (rotation), polygons and generic attributes; external bitmap masks and point-only keypoint projects are deliberately not claimed. Original video files stay in the archive, and their exact frame annotations and tracks travel in the sidecar.

Supervisely Video and MOT

Both write native video: Supervisely keeps object identity with a figure per frame; MOT writes gt/gt.txt with a stable id per track plus sequence metadata, for detection boxes only. Third-party import of these two is not claimed, but an archive AnnotateIt wrote restores its annotations and tracks through the sidecar, subject to the video-timing checks above. Media-only detection finds the videos; legacy annotations are retained without silently enabling an unverified timeline.

KITTI

KITTI 2D object detection, images only. Each image gets a label_2/<stem>.txt with one fifteen-field row per object: type, truncated, occluded, alpha, the box as left/top/right/bottom, and the 3D fields. AnnotateIt fills what a 2D annotation actually has — the class name and the box, plus truncated (a float 0–1) and occluded (0–3) taken from those attributes when you declared them — and uses KITTI’s own devkit sentinels for everything it cannot know rather than inventing measurements: -10 for the observation angle and rotation, -1 for the 3D size, -1000 for the 3D location, and -1 for truncated/occluded when the annotation never recorded them. It is the one export that encodes truncated and occluded natively, but its encoding is deliberately not VOC’s: the same attribute names mean a float and a 0–3 enum here, not booleans. A custom attribute has no field in KITTI and travels only in the sidecar. Every non-box shape is written as its axis-aligned box, exactly as Pascal VOC does.

MOTS

MOTS (MOT plus segmentation), video only, for instance-segmentation projects. It writes what the official mots_tools evaluator loads: one text file per sequence at instances_txt/<id>.txt (sequences are numbered 0000, 0001, … because the seqmap parses the id as an integer), a four-column seqmap.txt (id, placeholder, first frame, last frame), the videos and per-sequence seqinfo. Each line is one object per frame — time_frame, id, class_id, image height, image width, and the mask as COCO run-length encoding over the whole frame. Because MOTS recognises only car (1), pedestrian (2) and ignore (10), you map each project label to one of those (or exclude it) in the export dialog; the id folds in the class as class_id × 1000 + instance_id, with instance_id stable across frames for one object — at most 999 per class, and an export that needs more fails with an error rather than clamping. A mask is written as run-length encoding re-based to the frame; a polygon or box is rasterized into a mask first; masks are made non-overlapping per frame (a shared pixel goes to the later object); a polyline or keypoint has no pixels and is skipped. Every skip/exclude/overlap-drop and the numeric-id-to-video mapping is reported in the README. Like MOT, the extracted frames (img1/) are not written — the exporter has no frame decoder in the browser — and the README gives the ffmpeg command that fills them.

Plain ZIP

Original image and video bytes, no annotations, labels, prompts or attributes. If the dataset has a saved split, an annotateit-splits.json manifest is added so the train/validation/test membership can be restored on re-import into a new dataset.

Train / validation / test splits

A saved split records its ratios, seed, stratification flag and per-image assignments (subset, manual/automatic, locked). It is dataset state, so it travels two ways:

• Export the entire dataset and an annotateit-splits.json manifest is added, so the exact split can be restored later even though the standard files carry no split of their own.
• Export in split mode and the format additionally lays out its native structure — COCO instances_train/val/test.json with images/<subset>, the YOLO Ultralytics folders with data.yaml, Datumaro per-subset annotation files, or VOC and KITTI ImageSets/Main lists.
• On import, an AnnotateIt manifest wins over guessing from folders; external images/train, train/images and VOC ImageSets/Main layouts are recognised and imported as locked manual assignments.

Splits are image-only — videos are never placed in a train/validation/test set. Importing into an existing dataset does not replace its current split; import as a new dataset or project to restore a saved one, and the dialog says so.

Dataset versions freeze all of it

A dataset version is an immutable snapshot — media, image and video-frame annotations, tracks, the label schema, attributes, prompts, behaviour, hierarchy, the project task, subtype, keypoint structure and the split. Its content hash reflects every one of those, a download is built only from the frozen rows (never the current project), and a missing blob or inconsistent reference blocks the export rather than shipping a partial archive. Restore never edits the project task or labels: frozen task membership must exactly match the frozen labels, and the current title, domain, subtype, label ids and full keypoint template must still match before the safety snapshot and again inside the replacement transaction. A mismatch is refused; missing labels are not recreated. A valid restore re-creates the frozen split too. Versions are free on every tier — but they live on the device, so they are a working history, not a backup.

Import into a new project, or an existing one

Import as a new project and the full schema is recreated — attribute definitions, prompts, label hierarchy and behaviour, the project task and the split all come back exactly. Import into an existing project and the incoming labels are mapped onto the ones already there: the target project’s schema wins, and the dialog warns that definitions, prompts, hierarchy, task schema and split are not replaced. Annotation values still land on the objects, and any attribute definitions the target label was missing are added so those values are visible and editable rather than stored and invisible.

What the importer refuses

The contract is fail-closed: a supported, well-formed archive imports completely, and anything ambiguous or corrupt is rejected with an error before anything is saved, rather than half-imported. Refused up front:

• Unsafe archive paths (absolute, drive, .., NUL), and duplicate or case-colliding entries.
• A zip bomb or an archive larger than the device can hold — checked against both the declared uncompressed size and the platform ceiling before anything is expanded.
• Empty or undecodable images and videos; macOS __MACOSX and ._* resource forks are not mistaken for media.
• Unreadable frame timing, or a marked video sidecar whose cadence type, average frame rate or frame count disagrees with the media.
• A file name that matches more than one image, so an annotation cannot be attached unambiguously.
• Geometry outside the image, an attribute value with no declaring definition, or a split reference that resolves to no image.

If a write does begin and something later fails — a bad video, a split that will not resolve — the media already written to the target dataset are rolled back, so a failed import does not leave a half-populated dataset behind.

The “Recommended” badge

The Export dialog marks one format Recommended and says why, chosen only from the formats currently selectable:

A later background statistics refresh never overrides a format you picked by hand; the recommendation is recomputed fresh each time the dialog opens.