How Ultralytics YOLO & Label Studio work
These two tools are the two halves of building a custom object detector: Label Studio is where you create the labeled examples, and Ultralytics YOLO is what learns from them and detects. This guide opens up how each one works internally, then shows the loop that connects them.
Ultralytics YOLO What YOLO is
YOLO β "You Only Look Once" β is a family of single-stage object detectors. Where older detectors scanned an image in two stages (propose regions, then classify each), YOLO predicts all the bounding boxes and their classes in one forward pass of a neural network. That's what makes it fast enough for real-time.
Ultralytics is the company/library that maintains the modern YOLO versions (YOLOv8, YOLO11, β¦) and wraps them in one friendly Python + CLI API β so you train, evaluate, predict and export without touching the network internals.
Ultralytics YOLO How YOLO works
Inside, a YOLO model is three parts plus a post-processing step:
π§± Backbone
A convolutional network that turns pixels into feature maps β edges β shapes β objects, at several resolutions.
π Neck
Fuses features across scales (FPN/PAN) so the model detects both tiny and large objects well.
π― Head
For each grid cell & scale, predicts a box (x,y,w,h), an objectness score, and class probabilities. Modern YOLO is anchor-free.
π§Ή NMS
Non-Max Suppression: many cells fire for one object, so keep the highest-confidence box and drop others that overlap it (by IoU).
How it learns: the loss & training
During training the model compares its predictions to your labeled boxes and computes a loss with three parts: box loss (how far off the coordinates are), classification loss (right class?), and objectness/DFL loss (is something here?). Back-propagation nudges the weights to reduce that loss over many epochs. Starting from pretrained weights (transfer learning) means it already knows general vision, so it converges on your classes with little data.
Ultralytics YOLO The Ultralytics library
One class, four verbs β the whole lifecycle:
from ultralytics import YOLO
model = YOLO("yolo11n.pt") # load a pretrained model
model.train(data="data.yaml", epochs=100, imgsz=640) # train on your data
model.val() # evaluate (precision, recall, mAP)
results = model("image.jpg") # predict on new images
model.export(format="onnx") # export for deployment
# read the predictions
for box, cls, conf in zip(results[0].boxes.xyxy,
results[0].boxes.cls,
results[0].boxes.conf):
print(model.names[int(cls)], float(conf), box.tolist())
| Choose | Options |
|---|---|
| Task | detect (boxes) Β· segment (masks) Β· classify Β· pose (keypoints) Β· obb (rotated boxes) |
| Model size | n nano β s β m β l β x extra-large: bigger = more accurate, slower |
| Weights | yolo11n.pt (pretrained, transfer learning) or yolo11n.yaml (from scratch) |
Your data is described by a data.yaml (train/val folders + class names), and labels are one .txt per image with normalized class x_center y_center w h lines. Ultralytics also auto-augments (mosaic, flips, HSV) during training β free robustness.
Label Studio What Label Studio is
Label Studio is an open-source data-labeling platform. It gives you a browser UI to draw boxes (or polygons, keypoints, transcriptions, spansβ¦) on your data and save them as structured annotations β the ground-truth examples a model like YOLO trains on. It handles images, text, audio, video and more, supports teams, and exports to the formats trainers expect (including YOLO).
Label Studio How it works
Four concepts:
π Project
A labeling job: its data, its label schema (config), and its annotations.
π§Ύ Labeling config
An XML-like template that defines the UI & the labels β what you can draw and tag.
ποΈ Tasks
Each data item (an image) is a "task" to be annotated.
βοΈ Annotations
The saved answers β boxes with class labels β for each task.
The labeling config is the heart of it. For bounding-box detection you declare an image and the rectangle-labels:
<View>
<Image name="img" value="$image"/>
<RectangleLabels name="label" toName="img">
<Label value="card_nz" background="#22c55e"/>
<Label value="f_dob" background="#3b82f6"/>
<Label value="f_licence_no" background="#a855f7"/>
</RectangleLabels>
</View>
Then: import your images β annotate (draw a box, press the class hotkey, repeat) β export. Choose the YOLO export and you get the exact images/ + labels/*.txt + classes layout Ultralytics trains on.
| Label type (config tag) | For |
|---|---|
RectangleLabels | Object detection (bounding boxes) β YOLO |
PolygonLabels / BrushLabels | Segmentation masks |
KeyPointLabels | Pose / landmarks |
Choices | Whole-image classification |
Labels (on Text) | NER / span tagging |
Label Studio ML backend & pre-labeling
Label Studio can connect to a model (an ML backend) that pre-annotates tasks. Instead of drawing every box from scratch, labelers correct the model's predictions β much faster. Connect your trained YOLO as the backend and each new image arrives already boxed, needing only fixes.
Together The label β train loop
The two tools form a flywheel that keeps improving your detector:
| Step | Tool | What happens |
|---|---|---|
| 1. Annotate | Label Studio | Draw boxes + classes on your images |
| 2. Export | Label Studio | Emit YOLO-format images/ + labels/ + data.yaml |
| 3. Train | Ultralytics | model.train(...) β best.pt weights |
| 4. Pre-label | both | Connect best.pt as the ML backend to pre-annotate new data |
| 5. Correct β retrain | both | Humans fix the hard cases; re-export & re-train. Repeat. |
Knowledge check
Six questions on YOLO & Label Studio. Pick an answer for each, then check your score.
Next steps
πͺͺ The licence lab
Use both end to end β train a real detector that reads driver-licence fields.
π₯ PyTorch
What's underneath β YOLO is built on PyTorch; see the framework that runs it.
𧬠Embedding models
Vectors from data β the idea behind end-to-end document-AI models.
π All tutorials
The hub β ML, LLMs, agents, cloud & more.