Documentation

Documentation

Plain HTML pages, each answering one question completely. Start with the operational guides for projects, annotation, media and recovery, then use the format, model and platform references for exact technical behaviour. The in-app Help remains the shortest context-aware answer while you work.

• Know where your dataset came from

  Captured media carries a source snapshot and capture reason so you can trace an image after collection, filter the dataset and understand why a frame was selected. Source configuration and the history attached to a media item have different lifetimes.

• Evaluate a model against your own ground truth

  Model evaluation runs an available prediction engine on annotated images in the saved test split, scores its predictions locally and keeps the result with the scope that produced it. It helps you choose a model or find weak labels before another annotation pass.

• Connect a runner for fine-tuning and inference

  Pipelines is an experimental integration with a separately operated ML runner. The current recipe targets NVIDIA TAO RT-DETR for axis-aligned image object detection. AnnotateIt prepares a frozen dataset and manages job requests; Docker and GPU execution happen in the runner environment.

• Platform capabilities reviewed on September 19, 2026

  This overview extends the September 5 review with changes in the current source through September 19, 2026, including the September 12–19 weekly review. It describes current source capabilities and their limits, not a claim that every web or store build was released on these dates.

• 3D point-cloud segmentation (experimental)

  Assign semantic labels to individual points in PLY and PCD files on your device. This experimental workspace runs in a desktop browser or desktop app with WebGL2. It labels points, not 3D cuboids, and is unavailable in native mobile apps and the hosted API service.

• Importing a YOLO dataset

  Zip your YOLO dataset — images, .txt label files and a class list (obj.names, or names in a .yaml/.yml) — then create a project in AnnotateIt and import the archive. Detection rows become bounding boxes, longer even-length rows become polygons, and the format is detected automatically; nothing is uploaded at any point.

• Exporting COCO

  A COCO export is a .zip containing annotations/instances_default.json and images/default/. Category ids come from the full label list and start at 1, boxes are absolute [x, y, width, height], polygons are written to segmentation, and turning on dataset splits produces three annotation files with matching image folders.

• Dataset formats and the exact round trip

  A dataset export is a .zip in a standard computer-vision format, chosen in the Export dialog. Every annotated export also carries an annotateit-metadata.json sidecar: the standard files stay fully valid for third-party tools, and the sidecar preserves what the standard has no field for — exact shapes, attribute definitions and values, text prompts, label colours and hierarchy, the project task schema, the keypoint template, and native video tracks. Restoring video editing also requires verified frame timing, as explained below. Nothing is uploaded at any stage; reading, decoding and writing all happen on your device.

• Segment Anything, SAM 2.1 and SAM 3

  Click an object and get an editable polygon by default, or keep the model’s exact pixel mask. MobileSAM is built in on every platform; optional SAM 2.1 and SAM 3 variants appear on web and desktop only when their runtime is supported. Every available engine runs on your own hardware — the image is never sent anywhere.

• Text-prompt detection with Grounding DINO

  Type a description — "forklift", "helmet", "red car" — and an open-vocabulary detector proposes boxes for review. Grounding DINO Tiny is a local WebGPU-only download offered in Windows, macOS and compatible desktop browsers with a working adapter; it is absent on mobile.

• Auto-annotation: curated detection and segmentation models

  Auto-annotation models are ready-to-use ONNX detectors and 2 instance-segmentation families that AnnotateIt exports, verifies and hosts, grouped into branded families with a Recommended variant and higher-capacity alternatives where available. You download one from the Models page and set it up inside a matching project; every variant predicts the 80 COCO classes, is Apache-2.0, and runs locally on your CPU. They are distinct from importing your own ONNX, and any predictions they make in a batch run wait in a Pending AI Review queue until you accept them.

• Importing a custom ONNX model

  On web and desktop, you can pre-label with your own trained weights. A five-step wizard takes an ONNX file up to 512 MiB, inspects it, maps classes to project labels, requires a successful test on one of your images and only then saves it. Supported architectures are YOLOv8 detection, segmentation and pose, RT-DETR, DETR, RF-DETR-style and EdgeCrafter-style segmenters, and plain image classifiers; the feature is experimental and absent on mobile.

• Project backup and portability

  Use "Export project" to produce a .zip containing the complete project — every dataset, all media, every annotation, the label and task configuration and the per-project settings — and "Import project" to restore it on any machine. This archive is AnnotateIt’s own format, not COCO or YOLO; it exists to move a whole project, while a dataset export exists to feed a training pipeline.

• Platform differences

  It is the same product everywhere, but the runtimes genuinely differ. Native desktop builds bundle the built-in AI engines and add desktop-only surfaces; the web build downloads engines on demand; iPhone and iPad run only the built-in engines because of a hard WebView memory ceiling. Everything below is derived from one capability module in the application, not from marketing.

• Labels, text prompts and attributes

  A label is a name and a colour. Two optional fields let you say more, and they answer different questions: a text prompt describes what the class looks like and is written once, while an attribute records a property that differs between individual objects and is filled in per object. If the answer is the same for every car in your dataset it belongs in the prompt; if it changes from car to car it belongs in an attribute.

• Annotating video with tracks

  A track is one object followed across a video: you set its shape on keyframes, then boxes and compatible polygons interpolate between them. Raster masks stay exact by holding the previous mask until the next keyframe, so every changed mask needs its own keyframe. Object Detection uses identity-bearing box tracks, while Instance Segmentation uses object-mask tracks. Keypoint Detection stays frame-by-frame, and the three Classification variants use frame ranges instead. Nothing is uploaded: all track math stays on your machine.

• Versions, quality, splits and history

  Each dataset has four tabs beyond its media: Quality scans it for the problems that ruin a training run, Splits assigns images to train, validation and test deterministically, Versions freezes the whole state as an immutable snapshot you can diff and restore, and History is an append-only record of what happened to it. None of this needs a server, and all of these tools are free.

• Semantic search and batch pre-labelling

  On web and desktop, Semantic search ranks still images against a plain-language description instead of matching file names. Its local index powers ranked matches, a coarse heatmap and a reviewed batch pre-labelling run. Nothing is uploaded. iPhone and iPad do not expose this feature.

• The Models page

  On web and desktop, AnnotateIt organises AI around capabilities backed by interchangeable engines. The Models page shows the engines relevant to the current capability, marks incompatible choices as unavailable, explains why, and lists licence, size, runtime, speed and accuracy. Available engines can be downloaded, switched and removed locally. Mobile bundles its supported small engines and has no Models page.

• Keypoints, skeletons and pose assistance

  A keypoint project is built around one skeleton template, and each subject is an independent pose. Manual keypoints work on every platform. Web and desktop add Auto-pose for the exact COCO-17 human template through a downloadable on-device RTMPose engine; mobile omits it. COCO Keypoints export and import carry the whole structure both ways.

• Annotate with ChatGPT or Claude

  AI Assistant can discuss the current image, generate editable annotations and manage project labels. Choose an assistant in Settings → AI Assistant, then open its annotation button in the annotator, describe what you want and review the result on the canvas. This is an optional online workflow, separate from the local Human and AI modes.

• Getting media in, and preparing it

  A dataset accepts image and video files, existing dataset archives, photos and silent clips recorded with Camera, and stills extracted from videos already in the project. Once media is in, the built-in editor and media grid prepare, filter, sort and search it on-device.

• The local REST API

  Windows and macOS can run an HTTP API bound to 127.0.0.1, so a script on the same machine can create projects and datasets, upload media, read and write annotations and video tracks, and drive dataset versions, splits and quality scans. It is off by default, needs a personal access token you create yourself, and never leaves the loopback interface — it is local automation, not a server anybody else can reach.

• YOLO format compatibility matrix

  AnnotateIt reads and writes YOLO detection and instance-segmentation datasets, and does not read or write YOLO pose or OBB. Because YOLOv5, YOLOv8 and YOLOv11 share one label-row grammar, there is no per-version importer: what decides whether an archive works is the row shape and the class list, not the version number on the box. This page is the full matrix, including what is refused and why.

• COCO ↔ YOLO round trip, measured

  Converting between COCO and YOLO preserves geometry almost exactly — a full cycle cannot move a box corner by more than 9 × 10⁻⁴ of a pixel, and a 128-shape sweep gets nowhere past 6 × 10⁻⁴ — and loses everything that is not geometry, because a YOLO row has no field for it. This page separates the three outcomes: preserved, converted into something else, and lost. Every line comes from a test that runs the real exporters and parsers in both directions.

• MobileSAM vs SAM 2.1 vs SAM 3: a local benchmark

  Every interactive Segment Anything engine the app runs locally — the built-in MobileSAM, the three SAM 2.1 sizes, and the SAM 3 Tracker — measured on two fully specified machines: an Apple M4 Max laptop and an Intel desktop with an RTX 3090. The built-in MobileSAM encodes an image in 72 ms on the Mac’s integrated GPU; SAM 2.1 Tiny takes 150 ms there, almost exactly what the discrete RTX 3090 needs (154 ms); SAM 3 takes 3.3 s and SAM 2.1 Large 10.3 s in that Mac WebGPU run. The click that follows stays between 12 and 46 ms wherever threads or a GPU are available. The script that produced every number is in the repository, so you can get your own device’s rows.

• Getting started with AnnotateIt

  Create a project, add images or videos, annotate them, review the dataset and export it — all on the device where AnnotateIt is running. There is no account, no project limit and no automatic sync, so the first workflow includes making a backup as well as producing a training archive.

• Project lifecycle and task-type conversion

  A project card is the control point for rename, conversion, duplication, project export, cross-dataset Semantic search and deletion. Duplication creates an independent project with selectable label and annotation copying while reusing media bytes. Task-type conversion edits the existing project in place and can discard geometry, so inspect its calculated preview and export the original first.

• Annotator tools, filters and shortcuts

  The annotator combines a canvas, a task-specific tool bar, the current tool’s options, an annotation list and the dataset navigator. Manual shapes are committed immediately; most smart-tool results stay pending until confirmed. Ordinary edits save when you submit or navigate, while Pending AI Review remains a separate review state.

• Capture sources, selection rules and video recording

  Capture collects stills from a camera, supported screen sharing, a playable local video file or, on desktop, a network camera into a staging gallery. Photo mode offers Manual, Timelapse, Motion and Smart capture. Camera, screen and network-camera sources can record silent clips when supported; watched folders are not implemented.

• Batch auto-annotation and review

  Auto-annotate runs one available prediction source over the whole dataset, selected files, the current filter or Semantic search results. It writes isolated drafts marked Pending AI Review: your existing annotations are never replaced, and drafts stay out of exports, versions and ground-truth statistics until you accept them.

• Multiple datasets, copying and combined export

  A project can contain several independent datasets that share its task, labels and attributes. Each dataset keeps its own media, split, versions and history. Copy creates an independent working set while reusing media bytes; Export all datasets as one merges selected image datasets under the shared schema and an explicit split strategy.

• Settings, local storage and recovery

  AnnotateIt stores projects and media on the current device. Application settings control appearance, local AI, backup and platform-specific administration; project settings control annotator behaviour for one project. No setting enables cloud sync, so recovery depends on profile backups and project archives made before the device or browser storage is lost.

• Video ranges, frames and propagation

  A video can carry ordinary annotations on individual frames, object tracks across keyframes or whole-frame classification labels over a range. These are separate data models: propagation copies one frame’s ordinary annotations forward, frame extraction creates still images, ranges classify spans, and tracks preserve object identity through interpolation.

Looking for help with the interface itself?In-app documentation lives inside the application.