← All tutorials
YOLO & Label Studio

How Ultralytics YOLO & Label Studio work

These two tools are the two halves of building a custom object detector: Label Studio is where you create the labeled examples, and Ultralytics YOLO is what learns from them and detects. This guide opens up how each one works internally, then shows the loop that connects them.

Label Studioβ€” produces labels β†’ Ultralytics YOLOβ€” produces detections β†’ your app
Where you've seen them. The licence-recognizer lab uses both end to end. This tutorial is the "how do they actually work?" companion.

Ultralytics YOLO  What YOLO is

YOLO β€” "You Only Look Once" β€” is a family of single-stage object detectors. Where older detectors scanned an image in two stages (propose regions, then classify each), YOLO predicts all the bounding boxes and their classes in one forward pass of a neural network. That's what makes it fast enough for real-time.

Ultralytics is the company/library that maintains the modern YOLO versions (YOLOv8, YOLO11, …) and wraps them in one friendly Python + CLI API β€” so you train, evaluate, predict and export without touching the network internals.

One look, many boxes. The model divides the image into a grid and, for every location and scale, predicts "is there an object here, where exactly, and what is it?" β€” then a cleanup step removes duplicates. No sliding windows, no region proposals.

Ultralytics YOLO  How YOLO works

Inside, a YOLO model is three parts plus a post-processing step:

The YOLO detection pipeline An input image passes through a backbone that extracts features, a neck that fuses them across scales, and a head that predicts boxes and classes on a grid; then non-max suppression removes overlapping duplicates to give final detections. Input image e.g. 640Γ—640 Backbone extract features Neck fuse scales (FPN/PAN) Head boxes + classes / grid NMS remove overlaps one forward pass predicts everything Β· NMS then keeps the best, non-overlapping boxes

🧱 Backbone

A convolutional network that turns pixels into feature maps β€” edges β†’ shapes β†’ objects, at several resolutions.

πŸ”— Neck

Fuses features across scales (FPN/PAN) so the model detects both tiny and large objects well.

🎯 Head

For each grid cell & scale, predicts a box (x,y,w,h), an objectness score, and class probabilities. Modern YOLO is anchor-free.

🧹 NMS

Non-Max Suppression: many cells fire for one object, so keep the highest-confidence box and drop others that overlap it (by IoU).

How it learns: the loss & training

During training the model compares its predictions to your labeled boxes and computes a loss with three parts: box loss (how far off the coordinates are), classification loss (right class?), and objectness/DFL loss (is something here?). Back-propagation nudges the weights to reduce that loss over many epochs. Starting from pretrained weights (transfer learning) means it already knows general vision, so it converges on your classes with little data.

Ultralytics YOLO  The Ultralytics library

One class, four verbs β€” the whole lifecycle:

the entire API surface
from ultralytics import YOLO

model = YOLO("yolo11n.pt")                 # load a pretrained model
model.train(data="data.yaml", epochs=100, imgsz=640)   # train on your data
model.val()                                # evaluate (precision, recall, mAP)
results = model("image.jpg")               # predict on new images
model.export(format="onnx")                # export for deployment

# read the predictions
for box, cls, conf in zip(results[0].boxes.xyxy,
                          results[0].boxes.cls,
                          results[0].boxes.conf):
    print(model.names[int(cls)], float(conf), box.tolist())
ChooseOptions
Taskdetect (boxes) Β· segment (masks) Β· classify Β· pose (keypoints) Β· obb (rotated boxes)
Model sizen nano β†’ s β†’ m β†’ l β†’ x extra-large: bigger = more accurate, slower
Weightsyolo11n.pt (pretrained, transfer learning) or yolo11n.yaml (from scratch)

Your data is described by a data.yaml (train/val folders + class names), and labels are one .txt per image with normalized class x_center y_center w h lines. Ultralytics also auto-augments (mosaic, flips, HSV) during training β€” free robustness.

Label Studio  What Label Studio is

Label Studio is an open-source data-labeling platform. It gives you a browser UI to draw boxes (or polygons, keypoints, transcriptions, spans…) on your data and save them as structured annotations β€” the ground-truth examples a model like YOLO trains on. It handles images, text, audio, video and more, supports teams, and exports to the formats trainers expect (including YOLO).

Why a dedicated tool. Good labels are the highest-leverage part of a model. Label Studio gives you consistent classes, a fast keyboard-driven UI, review workflows, and clean exports β€” far better than home-grown scripts.

Label Studio  How it works

Four concepts:

πŸ“ Project

A labeling job: its data, its label schema (config), and its annotations.

🧾 Labeling config

An XML-like template that defines the UI & the labels β€” what you can draw and tag.

πŸ—‚οΈ Tasks

Each data item (an image) is a "task" to be annotated.

✏️ Annotations

The saved answers β€” boxes with class labels β€” for each task.

The labeling config is the heart of it. For bounding-box detection you declare an image and the rectangle-labels:

labeling config β€” bounding boxes
<View>
  <Image name="img" value="$image"/>
  <RectangleLabels name="label" toName="img">
    <Label value="card_nz"      background="#22c55e"/>
    <Label value="f_dob"        background="#3b82f6"/>
    <Label value="f_licence_no" background="#a855f7"/>
  </RectangleLabels>
</View>

Then: import your images β†’ annotate (draw a box, press the class hotkey, repeat) β†’ export. Choose the YOLO export and you get the exact images/ + labels/*.txt + classes layout Ultralytics trains on.

Label type (config tag)For
RectangleLabelsObject detection (bounding boxes) β†’ YOLO
PolygonLabels / BrushLabelsSegmentation masks
KeyPointLabelsPose / landmarks
ChoicesWhole-image classification
Labels (on Text)NER / span tagging

Label Studio  ML backend & pre-labeling

Label Studio can connect to a model (an ML backend) that pre-annotates tasks. Instead of drawing every box from scratch, labelers correct the model's predictions β€” much faster. Connect your trained YOLO as the backend and each new image arrives already boxed, needing only fixes.

new image→ ML backend (YOLO) pre-labels→ human corrects→ annotation
Active learning. Route the model's least-confident predictions to humans first β€” you label the examples that teach the model the most, so accuracy climbs fastest per hour of labeling.

Together  The label ↔ train loop

The two tools form a flywheel that keeps improving your detector:

raw images→ Label Studio: annotate→ export YOLO→ Ultralytics: train→ weights .pt↺ back as ML backend
StepToolWhat happens
1. AnnotateLabel StudioDraw boxes + classes on your images
2. ExportLabel StudioEmit YOLO-format images/ + labels/ + data.yaml
3. TrainUltralyticsmodel.train(...) β†’ best.pt weights
4. Pre-labelbothConnect best.pt as the ML backend to pre-annotate new data
5. Correct β†’ retrainbothHumans fix the hard cases; re-export & re-train. Repeat.
The payoff. Early on you label everything by hand; once YOLO is decent, it does most of the drawing and you just correct β€” so each round is faster and raises accuracy. That's the whole game of building a custom detector.

Knowledge check

Six questions on YOLO & Label Studio. Pick an answer for each, then check your score.

1. What does "You Only Look Once" mean?

2. What are the three main parts of a YOLO model?

3. What does Non-Max Suppression (NMS) do?

4. In Ultralytics, what does the model size letter (n/s/m/l/x) trade off?

5. What is the Label Studio "labeling config"?

6. How do the two tools connect into a loop?

Next steps

πŸͺͺ The licence lab

Use both end to end β€” train a real detector that reads driver-licence fields.

πŸ”₯ PyTorch

What's underneath β€” YOLO is built on PyTorch; see the framework that runs it.

🧬 Embedding models

Vectors from data β€” the idea behind end-to-end document-AI models.

πŸ““ All tutorials

The hub β€” ML, LLMs, agents, cloud & more.