The mathematics you will actually use

This last stage exists for one reason: the AI Engineering pathway assumes a reader who is not frightened of three ideas. None of them requires a degree; all of them reward a few evenings of real attention. Here is each one taken far enough to be useful.

Linear algebra — the data structure of ML

A vector is a list of numbers standing in for something — a word, a user, an image patch. Once things are vectors, geometry becomes meaning: the dot product measures how aligned two vectors are, and its normalised form, cosine similarity, is precisely the operation behind “find me documents similar to this query” in the retrieval stage later. Embeddings work because training arranges the space so that related things point the same way.

A matrix is a machine that transforms vectors — stretch, rotate, project — and matrix multiplication applies one transformation to a whole batch of vectors at once. The bookkeeping rule worth internalising: an (m×k) matrix times a (k×n) gives (m×n) — inner dimensions must agree; they are what gets “summed away”. When a model card says a layer projects 4096 dimensions to 4096, it is describing one such matrix; “more parameters” largely means bigger and more numerous matrices. Every one of the exotic words — embedding, attention, projection, LoRA — is this operation wearing a costume.

Calculus — one idea, used a billion times

The derivative answers a single question: if I nudge this input, how much does the output move? The gradient is that answer for many inputs at once — a vector pointing uphill. Training a network is: measure how wrong the output was (a loss), compute the gradient of that loss with respect to every parameter, and step each parameter slightly downhill. That is gradient descent; the chain rule is what lets the gradient flow backwards through a stack of layers (hence backpropagation — it is the chain rule, applied systematically); the learning rate is the step size, and almost every training pathology — divergence, stagnation — is a step-size story. You need the picture and the chain rule; nobody will ask you to integrate anything.

Probability — the model’s native language

A language model’s raw output is not text. For every possible next token it emits a score; softmax exponentiates and normalises those scores into a probability distribution — and temperature is a knob that divides the scores first, sharpening the distribution toward the favourite (low T) or flattening it toward variety (high T). Sampling from the distribution is what produces words. This is also why the same prompt yields different answers, and why “the model said” is a draw from a distribution, not a verdict.

Three more terms carry you a long way: expectation (the long-run average — what an eval score estimates), conditional probability (probability given what you already know — the model is one big conditional distribution over the next token given the context), and the idea that training maximises the probability of the data (log-likelihood — the log is there so that multiplying many probabilities becomes adding many logs).

How much is enough?

Enough to read the AI pathway without flinching: cosine similarity, matmul shapes, gradient + chain rule + learning rate, softmax + temperature, expectation. The linked book covers exactly this subset properly; the 3Blue1Brown series makes the geometry visceral in an evening. Resist the completionist detour into proofs — you can always return for them once something concrete demands it.

That closes the foundations. You can price a line of code, name the fictions your process lives inside, follow a request across the wire, estimate before you build, and read ML’s notation without flinching. The next pathway takes the one assumption still standing — that you are on a single machine — and removes it.