← All tutorials
PyTorch

PyTorch β€” deep learning that feels like Python

PyTorch is an open-source deep-learning framework (from Meta, now under the PyTorch Foundation) built to make building and training neural networks intuitive and flexible. Its define-by-run dynamic graphs mean models behave like ordinary Python you can debug step by step β€” which is why it dominates AI research and the LLM ecosystem.

tensors→ build (nn.Module)→ autograd→ train loop
One line. PyTorch makes experimenting with neural networks feel like writing normal Python β€” dynamic, debuggable, flexible β€” so you iterate fast, which is exactly what research and rapid prototyping need.

Why  The problem it solves

Early deep-learning frameworks used static graphs: you defined the whole computation up front, then ran it β€” fast, but rigid and painful to debug (you couldn't just drop in a print or a breakpoint). PyTorch popularized dynamic graphs: the graph is built on the fly as your code runs, so a model is just Python β€” you inspect tensors, step through with a debugger, and use normal control flow.

🐍 Pythonic & intuitive

Models are regular Python objects; no separate graph-definition language to learn.

πŸ”¬ Fast iteration

Dynamic graphs + easy debugging make experimenting and prototyping quick.

🧠 Autograd

Automatic differentiation computes gradients for you β€” call .backward() and go.

Mechanics  How it works

The core object is a tensor (an n-dimensional array that can live on a GPU). As you operate on tensors that require_grad, PyTorch records a dynamic graph; calling loss.backward() walks it backward to fill in gradients (autograd). You define a model by subclassing nn.Module, then write an explicit training loop β€” forward, loss, backward, step.

forward pass→ loss→ loss.backward()→ optimizer.step()↺ repeat
train.py β€” model + explicit loop
# pip install torch
import torch
import torch.nn as nn

model   = nn.Sequential(nn.Linear(784, 64), nn.ReLU(), nn.Linear(64, 10))
loss_fn = nn.CrossEntropyLoss()
opt     = torch.optim.Adam(model.parameters())

for epoch in range(5):
    for x, y in loader:                 # a DataLoader yields batches
        opt.zero_grad()                 # reset gradients
        preds = model(x)                # forward pass (builds the graph)
        loss  = loss_fn(preds, y)
        loss.backward()                 # autograd fills in gradients
        opt.step()                      # update the weights

That explicit loop is PyTorch's signature: more lines than Keras's fit(), but every step is visible and hackable. Higher-level wrappers (PyTorch Lightning, fastai) hide the boilerplate when you want them to, and torch.compile() (PyTorch 2.x) JIT-compiles the model for a big speed-up without giving up the dynamic feel.

Reach  The ecosystem

πŸ€— Hugging Face

The vast majority of pretrained models & LLMs are PyTorch-first β€” the modern NLP hub.

⚑ Lightning / fastai

Higher-level training frameworks that remove loop boilerplate.

πŸš€ torch.compile

PyTorch 2.x JIT compilation for major speed-ups.

πŸ“¦ TorchServe / ExecuTorch

Serving and on-device/edge deployment β€” the production story, much improved.

🌐 Distributed

DDP / FSDP for multi-GPU and large-scale training.

πŸ” ONNX export

Export models to run in other runtimes (including TF-adjacent tooling).

The trade-offs  Limitations

What to weigh

Deployment Historically PyTorch's production/deployment story lagged TensorFlow's. It has caught up a lot (TorchServe, ExecuTorch, ONNX), but mobile/edge/browser paths are still generally less mature than TF Lite / TensorFlow.js.

Boilerplate The explicit training loop means more code than model.fit() β€” great for control, but you write (and can bug) the loop yourself unless you adopt Lightning/fastai.

Overhead Dynamic graphs add some runtime overhead versus a fully static compiled graph (largely mitigated by torch.compile).

Mobile/browser On-device and in-browser deployment is improving but younger than TensorFlow's long-established mobile/JS runtimes.

Net: PyTorch trades a bit of turnkey deployment maturity for best-in-class research ergonomics, flexibility and ecosystem momentum.

PyTorch vs TensorFlow & JAX

AspectPyTorchTensorFlowJAX
StyleDynamic, pythonic (define-by-run)Graph + eager; Keras high-levelFunctional, composable transforms
Sweet spotResearch, prototyping, LLMsProduction & deploymentHigh-performance research
DebuggingEasy β€” normal PythonHarder with compiled graphsFunctional purity has a learning curve
DeploymentGood & improvingExcellent (mobile/edge/web)Research-oriented
EcosystemDominant in research / Hugging FaceBroad, mature productionSmaller, cutting-edge
See also: the TensorFlow guide for the other side of this comparison β€” and note Keras 3 now runs on PyTorch, TensorFlow or JAX.

When to choose PyTorch

If you…PyTorch fit
Do research or fast prototypingβœ… The default choice
Work with LLMs / Hugging Face modelsβœ… PyTorch-first ecosystem
Want intuitive, debuggable, flexible codeβœ… Dynamic graphs shine
Are starting a brand-new projectβœ… Most new work uses it
Need heavy mobile / browser / edge deployment⚠️ TensorFlow often smoother
Have an established TensorFlow/TPU stack⚠️ Weigh the switch
Rule of thumb. Building or fine-tuning models, doing research, or working with LLMs β†’ PyTorch. Prioritizing broad deployment (mobile, edge, browser, TPU) β†’ TensorFlow.

Cheat sheet

QuestionShort answer
What is PyTorch?A pythonic, dynamic deep-learning framework β€” research's favorite.
Its defining feature?Define-by-run dynamic graphs β€” models are debuggable Python.
How are gradients computed?Autograd β€” call loss.backward().
How do you train?An explicit loop: forward β†’ loss β†’ backward β†’ step (or use Lightning).
Biggest caveats?Deployment/mobile less mature than TF; you write the training loop.
Main alternative?TensorFlow (deployment); JAX for high-performance research.

Knowledge check

Six questions on PyTorch. Pick an answer for each, then check your score.

1. What is PyTorch's defining characteristic?

2. How do you get gradients in PyTorch?

3. Why is PyTorch dominant for research and LLMs?

4. Compared with Keras's model.fit(), the PyTorch training loop is…

5. Which is a real PyTorch limitation historically?

6. What does torch.compile() (PyTorch 2.x) provide?

Next steps

πŸ”Ά TensorFlow

The other giant β€” the deployment-strong framework, compared head to head.

πŸͺͺ Hands-on CV lab

Train a real model β€” label, train YOLO weights, and read driver-licence fields, from scratch.

🧬 Embedding models

Models that make vectors β€” most are built and trained in PyTorch.

πŸ€– LLM apps

Build on top of models β€” the application layer above the frameworks.

πŸ““ All tutorials

The hub β€” LLMs, embeddings, retrieval, agents & more.