How Computer Vision Works

Computer vision teaches machines to extract meaning from images and video. Here is the pipeline behind a modern detector, step by step.

Images are grids of numbers

A colour image is a 3D array: height × width × 3 channels (red, green, blue), each value typically 0 to 255. A model never "sees" a car, only these numbers, so the task is to learn which numeric patterns mean "car".

Convolutions find patterns

A convolutional neural network (CNN) slides small filters across the image. Early layers respond to edges and corners; middle layers combine them into textures and parts such as wheels or eyes; deeper layers respond to whole objects. Pooling or strided layers shrink the image progressively so later layers see a wider context.

Transformers for vision

Vision transformers cut the image into patches and apply the same attention mechanism used in language models, letting every patch relate to every other. Many modern systems blend convolutional and attention layers.

Core tasks

• Classification: what is in this image?
• Object detection: what is where? Output is a set of boxes, each with a class and confidence score.
• Segmentation: which pixels belong to each object?
• Tracking: which detection in this frame is the same object as in the last?
• Pose and OCR: body keypoints, text reading and more.

How object detectors like YOLO work

YOLO ("You Only Look Once") predicts boxes and classes in a single forward pass, which makes it fast enough for live video. The image is resized (commonly to 640×640), passed through a backbone that extracts features, and a head predicts candidate boxes with class probabilities. Low-confidence candidates are discarded; older versions then used non-maximum suppression to remove overlapping duplicates, while newer designs aim to avoid that step.

Training

Models learn from labelled images. A loss function measures how far predicted boxes and classes are from the labels, and gradient descent adjusts millions of weights to reduce it. Augmentation (flips, crops, brightness changes) helps the model generalise. Performance is measured on held-out images with metrics such as precision, recall and mAP.

Running it fast

Models are quantised to lower precision and exported to formats like ONNX so they can run efficiently on GPUs, phones and even in a browser using WebGPU or WebAssembly. Our live demo does exactly that.