The two engines behind modern artificial intelligence — explained in plain words, with practical notes, runnable examples, and a map of where each one shines.
PyTorch and TensorFlow are software libraries for building, training, and running machine-learning models — most famously the deep neural networks that power image recognition, language models, recommendation engines, and self-driving systems. They are free and open-source, and together they account for the overwhelming majority of deep-learning work done today.
Here is the key idea: training a neural network requires doing an enormous amount of arithmetic, and — crucially — computing derivatives of that arithmetic so the model can learn from its mistakes. You could write all of that by hand, but it would be painful and slow. These frameworks do three heavy things for you automatically:
They give you a fast container for numbers (a tensor), they run the math on a GPU so it's hundreds of times faster, and they compute the gradients for you through automatic differentiation. You describe the model; the framework handles the calculus and the hardware.
Imagine you're a chef inventing recipes. The tensor is your mixing bowl — it holds the ingredients (numbers) in an organized way: a single value, a row, a grid, or a stack of grids. The GPU is an industrial kitchen with a thousand line cooks who can chop in parallel, instead of one cook doing everything alone. And automatic differentiation is a magical assistant who watches every step you take and can instantly tell you, "if you'd used a pinch less salt, the dish would have scored higher" — which is exactly the feedback a model needs to improve.
PyTorch and TensorFlow are two competing brands of this fully-equipped kitchen. They cook the same dishes; they just lay out the counters differently and hand you slightly different utensils.
Almost all deep-learning training — in either framework — is the same five-step cycle, repeated thousands of times. Each lap nudges the model a little closer to correct. Here is the whole idea in one picture:
The top row is the forward pass (making a guess); the bottom row is the backward pass (learning from the error). PyTorch makes you write this loop by hand; Keras hides it inside .fit(). Either way, the cycle underneath is identical.
PyTorch's signature trait is that the computation graph is dynamic: it's created freshly each time you run your code. This "eager" style means a model is just ordinary Python that happens to operate on tensors — which is why beginners and researchers tend to find it the gentler on-ramp.
import torch # x is something we want to learn; track its gradient x = torch.tensor([2.0], requires_grad=True) y = x**2 + 3*x # define some computation y.backward() # autograd computes dy/dx for us print(x.grad) # tensor([7.]) -> derivative 2x+3 at x=2
requires_grad=True flag is what tells PyTorch to watch a tensor. Call .backward() and every watched tensor gets its gradient filled in automatically. This single idea scales from this two-line example all the way up to billion-parameter language models.
TensorFlow arrived first and originally used a "define-then-run" model: you built a static computation graph, then executed it. That was efficient for deployment but awkward to debug. TensorFlow 2.x changed the default to eager execution — so it now feels much more like PyTorch — while keeping the option to compile to a fast graph with a single decorator.
import tensorflow as tf x = tf.Variable([2.0]) # a learnable variable with tf.GradientTape() as tape: # "record" the math y = x**2 + 3*x print(tape.gradient(y, x)) # tf.Tensor([7.]) -> same answer
requires_grad, TensorFlow asks you to record operations inside a GradientTape block. Different spelling, identical idea: watch the math, then ask for the slope.
The "framework wars" shaped how these tools look today. A quick timeline:
Early frameworks — Theano, Torch (Lua), Caffe — prove the idea but are clunky. Google builds an internal system called DistBelief.
TensorFlow 1.0 is open-sourced by Google. Powerful and production-ready, but its static define-then-run graphs are notoriously hard to debug.
PyTorch launches from Meta with dynamic, Pythonic graphs. Researchers love it almost immediately.
TensorFlow 2.0 makes eager execution the default and folds in Keras as the official high-level API — a direct response to PyTorch's ergonomics.
PyTorch overtakes TensorFlow in research publications. Google quietly seeds a third option, JAX, for its own large-scale research.
PyTorch moves to the neutral PyTorch Foundation, adds torch.compile for graph-level speed, and grows edge tooling (ExecuTorch). TensorFlow rebrands TF Lite as LiteRT and keeps its grip on production and on-device inference.
A practical comparison. None of these are absolutes — both frameworks have spent years copying each other's best ideas, so the gaps are smaller than they used to be.
| Dimension | PyTorch | TensorFlow |
|---|---|---|
| Maker / year | Meta, 2016 | Google, 2015 |
| Default style | Dynamic (define-by-run) | Eager since 2.x, compiles to graph on demand |
| Feels like | Plain Python | Python + Keras conventions |
| Learning curve | Gentle, intuitive | Gentle via Keras; deeper layers get complex |
| Debugging | Easy — standard Python tools | Easy in eager mode; trickier once compiled |
| Research mindshare | Dominant in papers & new models | Strong but declining in research |
| Deployment | Rapidly maturing (TorchServe, ExecuTorch, ONNX) | Very mature (Serving, LiteRT, TF.js) |
| Mobile / embedded | ExecuTorch, PyTorch Mobile | LiteRT (formerly TF Lite) — long the standard |
| Specialty hardware | GPU-centric, growing accelerator support | First-class TPU support |
| High-level API | torch.nn, PyTorch Lightning | Keras (built in) |
A tiny neural network — a single linear layer that learns to map an input to an output — written in each framework. Use the tabs to flip between them and notice how similar the shapes are.
import torch, torch.nn as nn model = nn.Linear(1, 1) # 1 input -> 1 output loss_fn = nn.MSELoss() opt = torch.optim.SGD(model.parameters(), lr=0.01) x = torch.tensor([[1.],[2.],[3.]]) y = torch.tensor([[2.],[4.],[6.]]) # target: y = 2x for epoch in range(200): pred = model(x) loss = loss_fn(pred, y) opt.zero_grad() # reset old gradients loss.backward() # compute new gradients opt.step() # update the weights
import tensorflow as tf model = tf.keras.Sequential([ tf.keras.layers.Dense(1, input_shape=(1,)) ]) model.compile(optimizer=tf.keras.optimizers.SGD(0.01), loss="mse") x = [[1.],[2.],[3.]] y = [[2.],[4.],[6.]] # target: y = 2x model.fit(x, y, epochs=200, verbose=0) # training loop is hidden
.fit(), which is wonderfully concise for standard tasks but gives you less to look at when something custom is needed. This trade-off — visible control vs. tidy abstraction — captures the cultural difference between the two camps.
The toy example above learns a single line. Here's something closer to real life: a small network that learns to recognize handwritten digits (the classic MNIST dataset — 28×28 grayscale images of the numbers 0–9). This is the "hello world" of deep learning, and it shows the full arc: load data, define layers, train.
import torch, torch.nn as nn from torch.utils.data import DataLoader from torchvision import datasets, transforms # 1. Data: 28x28 grayscale digits -> tensors tfm = transforms.ToTensor() train = datasets.MNIST("data", train=True, download=True, transform=tfm) loader = DataLoader(train, batch_size=64, shuffle=True) # 2. Model: flatten -> hidden layer -> 10 class scores model = nn.Sequential( nn.Flatten(), nn.Linear(28*28, 128), nn.ReLU(), nn.Linear(128, 10), ) loss_fn = nn.CrossEntropyLoss() opt = torch.optim.Adam(model.parameters(), lr=1e-3) # 3. Train: one full pass over the data model.train() for images, labels in loader: preds = model(images) loss = loss_fn(preds, labels) opt.zero_grad() loss.backward() opt.step()
import tensorflow as tf # 1. Data: built-in MNIST, pixels scaled to 0-1 (x, y), _ = tf.keras.datasets.mnist.load_data() x = x / 255.0 # 2. Model: flatten -> hidden layer -> 10 class scores model = tf.keras.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(128, activation="relu"), tf.keras.layers.Dense(10), ]) model.compile( optimizer="adam", loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=["accuracy"], ) # 3. Train: one full pass over the data model.fit(x, y, epochs=1, batch_size=64)
A trained model is worthless if you can't store it and reload it later. This is one of the first real-project needs — and the two frameworks have slightly different philosophies.
# PyTorch favors saving just the learned weights ("state dict") torch.save(model.state_dict(), "model.pt") # To load: rebuild the SAME architecture, then pour the weights in model = build_model() # your own function/class model.load_state_dict(torch.load("model.pt")) model.eval() # switch to inference mode
# Keras can save the WHOLE model: architecture + weights + config model.save("model.keras") # Load it back in one line, ready to predict model = tf.keras.models.load_model("model.keras") predictions = model.predict(x)
Neither framework lives alone — each anchors a constellation of companion libraries that do the real day-to-day work.
Increasingly you'll hear about JAX, also from Google. It isn't a drop-in replacement — think of it as "NumPy with superpowers": a familiar array API plus composable function transforms (grad for gradients, jit for compilation, vmap for auto-batching), all compiled through XLA to run fast on GPUs and TPUs. It's favored in cutting-edge research and underpins libraries like Flax and Haiku, as well as a good deal of large-scale model training at Google/DeepMind. For most newcomers PyTorch is still the place to start, but JAX is worth knowing exists.
Rough, real-world associations (these shift over time, and most big organizations use more than one):
| Framework | Commonly associated with | Sweet spot |
|---|---|---|
| PyTorch | Meta, OpenAI, much of academia, the Hugging Face model ecosystem, many autonomy/robotics teams | Research, rapid prototyping, large language models, anything using the latest published architectures |
| TensorFlow | Google products historically, many enterprises with established ML pipelines, on-device and mobile apps via LiteRT | Production serving at scale, mobile/embedded/browser inference, mature MLOps pipelines |
| JAX | Google Research / DeepMind and a growing slice of frontier-model labs | High-performance, large-scale research where compilation and TPUs matter |
A few honest heuristics that hold up well:
You're learning deep learning, doing research, prototyping new ideas, or working with the latest published models. Its readability and the size of its research community make it the path of least resistance today.
Your priority is shipping to constrained or massive-scale targets — mobile apps, microcontrollers, browsers, or large serving fleets — or your team and existing codebase already live in the TensorFlow/Keras world.
Building a model is half the job; running it somewhere real is the other half. This is where the frameworks' personalities matter most.
TensorFlow Serving has long been a standard for high-throughput model serving; PyTorch answers with TorchServe and broad support for the open ONNX format, which lets a model trained in one framework run in many runtimes.
This was historically TensorFlow's stronghold via TF Lite (now LiteRT), widely used for on-device inference. PyTorch has closed in with ExecuTorch. A key technique on both sides is quantization — shrinking a model's numbers from 32-bit floats down to 8-bit integers (INT8) so it runs fast and small on limited hardware.
optimizer.zero_grad() in your loop and gradients pile up across steps, quietly wrecking training. It's the single most common beginner bug..to(device); forgetting leads to a loud error. Keep your data and model on the same device.tf.Session() or placeholder, it's outdated — find a 2.x version.model.train() / model.eval() in PyTorch (Keras handles this inside fit/predict). Skipping it gives mysteriously bad predictions.| Tensor | An n-dimensional array of numbers — the data structure everything flows through. |
| Autograd / autodiff | Automatic computation of derivatives (gradients) so the model can learn. |
| Gradient | The slope telling each parameter which way (and how much) to change to reduce error. |
| GPU / TPU | Specialized chips that do the massively parallel math of deep learning fast. |
| Epoch | One full pass over the training dataset. |
| Batch | A small chunk of examples processed together in one step. |
| Loss function | A number measuring how wrong the model currently is; training minimizes it. |
| Optimizer | The rule (e.g. SGD, Adam) for updating parameters using the gradients. |
| ReLU / activation | A simple non-linear function between layers that lets networks learn curved patterns. |
| Eager vs. graph | Running math line-by-line (eager) vs. compiling it into a fixed graph first (graph). |
| Quantization | Shrinking numbers (e.g. 32-bit float → 8-bit int) for smaller, faster models on limited hardware. |
| ONNX | An open format for sharing trained models between frameworks and runtimes. |
Note: framework features and product names evolve quickly (for example, TF Lite was rebranded LiteRT). When precise version-specific behavior matters, the official docs above are the authoritative source.