Natural Language Processing with Deep Learning

Personal study notes written in my own words from publicly available lectures and course pages. Not affiliated with or endorsed by Stanford University. No assignment solutions are published here.

What the course is about

Human language arrives as a stream of discrete symbols, and nothing about the symbol “cat” tells a computer that it is closer to “dog” than to “carburetor”. CS224n is about turning those symbols into vectors, building networks that read and write sequences of them, and understanding why the models that came out of this line of work — word2vec, LSTMs, Transformers, BERT, GPT-style language models — behave the way they do. The course sits at the point where linguistics, statistics, and deep learning meet, and it treats all three seriously.

You should be comfortable with Python and NumPy, with matrix calculus at the level of taking gradients through a few layers, and with basic probability (conditional probability, expectations, maximum likelihood). Having seen logistic regression, gradient descent, and a feed-forward network before helps a lot; CS229 or CS230 covers that ground. No linguistics background is assumed, but the course introduces just enough syntax and semantics to make the modelling choices make sense.

The lectures follow a clear arc. The first third builds representations and training machinery: word vectors, backpropagation, and a first structured task (dependency parsing). The middle third is sequence modelling: language models, recurrent networks, attention, and the Transformer. The final third is the pretraining era: how large models are trained, adapted, prompted, decoded, evaluated, and analysed, and what their deployment means for the people on the receiving end.

Lecture notes

01 · Word vectors: word2vec and GloVe

The distributional hypothesis says that a word’s meaning is captured by the company it keeps. One-hot vectors ignore this — every pair is orthogonal — so the goal is a dense vector per word in which similar words sit close together. Word2vec learns these vectors by prediction. In the skip-gram model, a center word $c$ is used to predict each context word $o$ within a window of size $m$, and the objective is the average negative log-likelihood over a corpus of $T$ positions:

Each word gets two vectors, $v$ as a center word and $u$ as a context word. The softmax denominator runs over the whole vocabulary, which is far too slow, so negative sampling replaces it with binary classification: push the true pair together and push $k$ random “negative” words apart.

Negatives are usually drawn from a unigram distribution raised to the $3/4$ power, which lifts rare words. GloVe takes the other route: build the global co-occurrence matrix $X$ first and fit vectors so that dot products match log counts, with a weighting function $f$ that caps very frequent pairs and zeroes out unseen ones.

Vectors are evaluated intrinsically (similarity correlation with human ratings, analogies solved as $\arg\max$ cosine of $x_b - x_a + x_c$) and extrinsically (does swapping them into a named-entity tagger help). Common problems: a tiny window gives syntactic neighbours while a large one gives topical ones, rare words get noisy vectors, and one vector per word type blends all senses of a polysemous word. These vectors become the input layer of every model that follows.

02 · Neural classifiers and backpropagation

Softmax regression on word vectors draws a linear boundary; a hidden layer lets the model learn its own features. The running example is window classification: to tag the center word of a five-word window, concatenate the five vectors, pass them through a $\tanh$ layer, and score the classes. Training minimises cross-entropy, $-\log p_y$, and the gradient of the loss with respect to the scores has the memorable form $p - \text{onehot}(y)$: predicted minus observed.

Backpropagation is the chain rule organised on a computation graph. Each node needs only its local gradient; multiplying the upstream gradient by the local one gives the downstream gradient, and the same intermediate values are reused instead of recomputed. Because word vectors are parameters too, the gradient flows all the way into the embedding rows that appear in the window.

import numpy as np
def forward_backward(x, y, W, b, U):
    h = np.tanh(W @ x + b)                 # hidden layer
    s = U @ h                              # class scores
    p = np.exp(s - s.max()); p /= p.sum()  # softmax
    loss = -np.log(p[y])
    ds = p.copy(); ds[y] -= 1              # dL/ds = p - onehot(y)
    dU = np.outer(ds, h)
    dz = (U.T @ ds) * (1 - h**2)           # through tanh
    return loss, dU, np.outer(dz, x), dz   # dW = outer(dz, x), db = dz

Two things go wrong in practice. Shapes: the gradient of a parameter always has the parameter’s shape, and checking that catches most bugs. Fine-tuning embeddings on a small dataset moves only the words that appear in training, so a test word that was a near neighbour of “good” can be left behind when “good” migrates. Numerical gradient checking with finite differences is cheap insurance. The next task, parsing, applies exactly this classifier to a structured decision.

03 · Dependency parsing

A dependency parse connects words with directed arcs from a head to its dependent, forming a tree rooted at the main verb. Compared with constituency trees, dependencies are compact and map naturally onto “who did what to whom”, which is why they became the default structure for downstream use. Ambiguity is the hard part: prepositional-phrase attachment and coordination scope produce an exponential number of legal trees for one sentence.

Transition-based parsing turns tree building into a sequence of classification decisions. A configuration holds a stack, a buffer of unread words, and the arcs so far; the parser repeatedly chooses SHIFT, LEFT-ARC, or RIGHT-ARC. Each choice needs a classifier over the configuration, and the neural version replaces hand-built feature templates with embeddings of the words, part-of-speech tags, and arc labels at fixed stack and buffer positions, concatenated and fed through one hidden layer. It runs in linear time and is greedy.

Accuracy is reported as unlabelled and labelled attachment score (UAS/LAS), the fraction of words whose head (and label) is correct. The usual failure is error propagation: one wrong early action leaves the parser in a configuration it never saw during training. Beam search, dynamic oracles, and better tagging all help. The arc-standard system also cannot produce crossing arcs, which matters for free-word-order languages. The parser still sees a fixed window of context; recurrent models drop that limitation.

04 · Language models and RNNs

A language model assigns probability to a sequence by the chain rule, $P(w_1, \dots, w_T) = \prod_t P(w_t \mid w_{<t})$. N-gram models truncate the history to $n-1$ words and estimate the conditional by counting, which is simple but suffers from sparsity (unseen n-grams get zero unless smoothed) and storage that grows with the corpus. A fixed-window neural model shares statistics across similar words through embeddings, but the window is still fixed and each position has its own weights.

A recurrent network reads any length with one shared weight matrix:

Training uses teacher forcing — feed the true previous word, predict the next — with cross-entropy at every step, and backpropagation through time sums the gradient contributions from each step. The standard evaluation metric is perplexity, the exponential of the average per-token cross-entropy:

Lower is better, and a perplexity of $k$ means the model is roughly as uncertain as a uniform choice among $k$ words. Sampling one token at a time from the output distribution turns the model into a generator.

Practical issues: full backpropagation through a long document is too expensive, so the sequence is chunked (truncated BPTT); recurrence is inherently sequential, so training does not parallelise across time; and the gradient signal through many steps either vanishes or explodes, which is the subject of the next lecture.

05 · Vanishing gradients, LSTMs, and GRUs

The gradient of the loss at step $t$ with respect to the hidden state at an earlier step $k$ contains a product of Jacobians $\prod_{i=k+1}^{t} \partial h_i / \partial h_{i-1}$. If the recurrent weights are small the product shrinks exponentially with distance and the model cannot learn long-range dependencies; if they are large it explodes. Exploding gradients are handled by clipping the gradient norm. Vanishing gradients need an architectural fix.

The LSTM keeps a separate cell state $c_t$ that is updated additively, with gates deciding what to erase, write, and read:

With the forget gate near one, information and its gradient travel along $c_t$ through elementwise multiplications rather than through a weight matrix at every step. The GRU merges cell and hidden state and uses a reset gate and an update gate, with fewer parameters.

Two structural additions matter: bidirectional RNNs read left-to-right and right-to-left and concatenate, which is fine for encoding but impossible for generation; and stacking two to four layers helps. Initialising the forget-gate bias to a positive value keeps early training from erasing everything. The same additive-shortcut idea appears as residual connections in deep feed-forward networks. An unconditional language model can only continue text; conditioning it on an input turns it into a translator.

06 · Seq2seq, attention, and machine translation

Neural machine translation is a conditional language model: an encoder RNN reads the source sentence, and a decoder RNN generates the target one token at a time, $P(y \mid x) = \prod_t P(y_t \mid y_{<t}, x)$, trained end to end. Decoding at test time is a search problem. Greedy decoding takes the argmax at each step and cannot undo a mistake; beam search keeps the $k$ best partial hypotheses, and because longer hypotheses accumulate more negative log-probability, scores are normalised by length before choosing the final output.

The single encoder vector is a bottleneck. Attention lets every decoder step look back at all encoder states: compute a score between the decoder state $s_t$ and each encoder state $h_i$, normalise with a softmax, and take the weighted sum as a context vector.

Attention improves translation quality, gives a soft alignment that can be inspected, and creates a short gradient path from the loss to any source position. Translation is scored with BLEU, a geometric mean of clipped n-gram precisions (usually 1 to 4) multiplied by a brevity penalty. It correlates with human judgment at the corpus level but punishes valid paraphrases.

What goes wrong: a very large beam favours short, safe outputs; rare words fall outside the vocabulary until subword tokenisation (byte-pair encoding) is used; and the encoder still has to be read sequentially. If attention does the heavy lifting, the recurrence itself becomes the thing to remove.

07 · Transformers

Recurrent encoders have two costs: computation is sequential across time, and the path between two distant tokens is long, so signal degrades. Self-attention fixes both by letting every token attend to every other token in one parallel step. Each input vector is projected into a query, a key, and a value, and the output for a token is a value-weighted average determined by query-key similarity:

Dividing by $\sqrt{d_k}$ keeps the dot products from growing with dimension and saturating the softmax. Self-attention alone is missing three things. It is order-blind, so positional encodings (sinusoidal or learned) are added to the input embeddings. It is linear in the values, so a position-wise feed-forward network follows each attention layer. And a decoder must not see the future, so scores for later positions are set to $-\infty$ before the softmax. Multi-head attention runs several smaller attention operations in parallel with separate projections and concatenates the results, letting heads specialise. Residual connections and layer normalisation — normalising each token’s vector to zero mean and unit variance, then rescaling — make deep stacks trainable. In an encoder-decoder Transformer, cross-attention takes queries from the decoder and keys and values from the encoder.

The price is quadratic time and memory in sequence length, which is why long-context work keeps revisiting the attention pattern. Training details matter more than with RNNs: learning-rate warmup is needed, the pre-layer-norm variant is easier to optimise than the original post-norm layout, and forgetting the causal mask produces a decoder that scores perfectly on training data and cannot generate. This architecture scales, and scaling is what the next lectures exploit.

08 · Pretraining: BERT, GPT, and T5

Pretraining trains a large network on raw text with a self-supervised objective, then adapts it to a task. Word vectors already did this for the input layer; the shift is doing it for the whole model, so that syntax, some world knowledge, and coreference cues are learned before any labelled data appears. The three Transformer shapes come with three objectives.

Masked and causal objectives are both cross-entropy, just over different conditioning sets:

BERT masks roughly 15 percent of tokens and, to soften the mismatch with fine-tuning where no mask token appears, sometimes replaces the chosen token with a random word or leaves it unchanged. Its original next-sentence-prediction task was later shown to add little. Encoders see both directions, which makes them strong readers but awkward generators; decoders generate naturally but only condition on the left, so for classification you read out the last position. Sentence-piece or byte-pair vocabularies keep every input representable. A pretrained model is only a starting point; getting task behaviour out of it is the next question.

09 · Fine-tuning and prompting

Full fine-tuning adds a small task head and updates every parameter with a small learning rate for a few epochs. Because the pretrained weights already encode most of what is needed, a few thousand labelled examples are often enough, and this recipe drove the jump on benchmarks like GLUE. Prompting is the alternative for decoder models: describe the task in natural language, optionally include a few worked examples, and read the continuation. No gradients are computed, so one model serves many tasks.

Chain-of-thought prompting asks the model to write intermediate reasoning before the answer and helps on multi-step problems, but mainly once the model is large. Instruction tuning, covered later, fine-tunes on many tasks phrased as prompts so that the two approaches converge.

Things go wrong in specific ways. Fine-tuning on small datasets can vary widely across random seeds, so report several. Prompt results can swing by many points from a reformatted template. Benchmark text may appear in the pretraining corpus, inflating scores, so contamination checks matter. Whichever way the model is adapted, the output still has to be decoded from a probability distribution, which is a design decision of its own.

10 · Natural language generation and decoding

Generation covers translation, summarisation, dialogue, and open-ended writing, and the same trained model can produce very different text depending on how tokens are chosen. Training with teacher forcing means the model never sees its own mistakes, so at test time errors compound — exposure bias. Decoding methods trade off quality against diversity.

Beam search maximises likelihood, and for open-ended tasks high-likelihood text is not what people write; human text sits below the model’s most probable path. Repetition is a known failure of likelihood-trained models and is patched with n-gram blocking or trained away with unlikelihood objectives.

Evaluation is the weak point. Overlap metrics such as BLEU and ROUGE need a reference and reward lexical matches, which is unfair to a good paraphrase. Model-based metrics like BERTScore compare embeddings. Human evaluation is the standard but is expensive, noisy, and hard to reproduce. Typical mistakes: using beam search for a chatbot, tuning temperature without a held-out check, and reading a BLEU gain on dialogue as a real improvement. Question answering is generation constrained by a passage, which is where the next lecture goes.

11 · Question answering and reading comprehension

Reading comprehension gives the model a passage and a question and asks for the answer, usually as a span of the passage. The SQuAD benchmark made this concrete: predict a start index and an end index, and score by exact match and token-level F1 against human answers. Its second version added unanswerable questions, so the model must also learn when to abstain.

Early neural readers such as BiDAF built separate representations of question and passage, computed attention in both directions between them, and ran a recurrent layer over the fused result. The pretrained-model recipe is simpler: feed the question and passage as one sequence separated by a special token, and train two vectors $w_{\text{start}}$ and $w_{\text{end}}$ against the final hidden states:

with the end distribution defined the same way and the answer chosen as the highest-scoring valid span.

Open-domain QA drops the passage: a retriever (sparse, like TF-IDF or BM25, or a dense bi-encoder) pulls candidates from a corpus and a reader answers from them. What goes wrong: models exploit lexical overlap and annotation artifacts, so adding one distracting sentence to a passage can flip the answer; and high F1 on one dataset rarely transfers to another. Large decoder models can answer many questions with no passage at all, which raises the question of what scale buys.

12 · Large language models: scaling, instruction tuning, and RLHF

Empirically, language-model loss falls smoothly as a power law in parameters, data, and compute, and later work showed that for a fixed compute budget parameters and training tokens should grow together rather than parameters alone. Scale also changes behaviour: a large decoder can perform a task specified only in its prompt, with no weight updates — in-context learning — and some abilities appear abruptly across scale, although how much of that is a measurement artifact is debated.

A raw pretrained model completes text; it does not follow instructions. Instruction tuning fine-tunes it on many tasks written as natural-language requests with answers, and the result generalises to unseen instructions. Reinforcement learning from human feedback goes further: collect human preference comparisons between candidate outputs, train a reward model $r_\phi(x, y)$ on them, and optimise the policy to score well while staying close to the supervised starting point:

The KL term keeps the model from drifting into text the reward model has never scored. Direct preference optimisation reaches a similar objective without an explicit reward model or RL loop.

Failure modes are well documented: over-optimising the reward model produces outputs the reward likes and humans do not; models learn to agree with the user; fluent text is confidently wrong. Fine-tuning a model with tens of billions of parameters is also simply expensive, which motivates cheaper adaptation.

13 · Efficient adaptation: adapters, LoRA, and prefix tuning

Parameter-efficient fine-tuning freezes the pretrained weights and trains a small number of new parameters, so each task costs megabytes rather than a full model copy. Adapters insert a small bottleneck network (down-project, nonlinearity, up-project) after the attention and feed-forward sublayers. Prefix tuning prepends trainable vectors to the keys and values at every layer; prompt tuning does the same at the input only. LoRA (low-rank adaptation) keeps the architecture unchanged and writes the weight update as a product of two thin matrices:

$A$ starts random and $B$ starts at zero, so training begins from the pretrained function exactly. After training, $BA$ can be merged into $W_0$, so inference costs nothing extra.

import numpy as np
d, k, r, alpha = 1024, 1024, 8, 16
W0 = np.random.randn(d, k) * 0.02   # frozen pretrained weight
A = np.random.randn(r, k) * 0.01    # trainable, random init
B = np.zeros((d, r))                # trainable, zero init -> dW = 0 at start
def lora_forward(x):                # x has shape (k,)
    return W0 @ x + (alpha / r) * (B @ (A @ x))

Typical mistakes: choosing a rank too small for a large domain shift, and applying LoRA to only one projection matrix when several help. With adaptation solved, the course turns back to the linguistic phenomena models are supposed to capture.

14 · Coreference and linguistic structure

Coreference resolution finds all mentions in a text that refer to the same entity: names, noun phrases, and pronouns. It is needed for summarisation, question answering, and dialogue, and it is hard because pronouns are ambiguous and resolving them often needs world knowledge — Winograd-style sentences flip the referent by changing one verb. Classical approaches used syntactic search heuristics; modern systems learn it.

The end-to-end neural approach treats every span up to some length as a candidate mention, scores each span for mention-likeness, prunes aggressively, and then scores pairs:

where $s_a$ measures how plausible span $j$ is as an antecedent of span $i$, and a dummy antecedent lets the model say “no earlier mention”. Span representations come from a contextual encoder plus attention over the span’s tokens. Because a mention can only link to one earlier antecedent, clusters emerge by following links.

Evaluation averages several cluster metrics (MUC, B-cubed, CEAF) because each rewards different errors. What goes wrong: mention-detection mistakes propagate, pronoun resolution inherits gender stereotypes from data (the WinoBias tests make this measurable), and models lean on surface cues where reasoning is needed. More broadly, pretrained models pick up much of this structure implicitly, which raises the question of what exactly they have learned.

15 · Interpretability and analysis

Analysis of NLP models works at three levels. Behavioural testing feeds crafted inputs — minimal pairs for subject-verb agreement with intervening nouns, negation, or word-order swaps — and checks predictions, treating the model as a black box. Probing trains a small classifier on frozen representations to predict a linguistic property such as part of speech or dependency depth; success shows the information is easily decodable from that layer. A structural probe goes further and finds a linear transformation under which vector distances match tree distances. Mechanistic analysis looks inside: which heads attend to what, which neurons fire, and how information moves between layers.

Two caveats matter. A probe with enough capacity can decode almost anything from anything, so control tasks with random labels are needed to separate what the representation encodes from what the probe learned. And attention weights are not explanations by default: studies have found that very different attention distributions can produce the same output, that many heads can be pruned without loss, and that gradient-based attribution often disagrees with attention. Attention shows where the model looked, not why it answered.

A recurring finding is that lower layers of a pretrained encoder carry more surface and syntactic information while upper layers carry more task-specific and semantic information, though the split is gradual. The usual mistakes are cherry-picking an attention map and reading “decodable” as “used”. Instead of asking what is stored in the parameters, the next lecture puts knowledge outside the model.

16 · Retrieval-augmented generation

Knowledge stored in parameters is frozen at training time, cannot cite its source, and is reproduced from fuzzy memory, which is one root of hallucination. Retrieval-augmented generation adds an external store: a retriever finds passages relevant to the query, and the generator conditions on them. The retriever can be sparse (BM25 over terms) or dense — a bi-encoder trained contrastively so that a question and its supporting passage have a high dot product, with other passages in the batch serving as negatives. Passages are pre-encoded into an approximate nearest-neighbour index so lookup is fast.

The RAG formulation marginalises over the retrieved documents:

Variants differ in where retrieval enters: REALM retrieves during pretraining, RETRO cross-attends to retrieved chunks inside the decoder, and kNN-LM interpolates the model’s next-token distribution with one built from nearest-neighbour contexts.

In practice most work is in the pipeline: how documents are chunked, which embedding model is used, whether a cross-encoder reranks the candidates, and how many passages fit in the context window. What goes wrong: the retriever returns something plausible but irrelevant and the generator either ignores it or gets distracted by it; the index goes stale; and evaluation checks answer correctness without checking that the answer is faithful to the retrieved text. Systems that retrieve and generate at scale are deployed to real users, which is where the course ends.

17 · Bias, safety, and ethics in NLP

Language data is produced by people, and models trained on it inherit their statistical regularities, including the harmful ones. Word vectors encode gender and racial associations that show up in analogy tests and in downstream decisions; coreference systems resolve pronouns along stereotyped occupation lines; toxicity classifiers flag dialects of English at higher rates. Bias is usually separated into allocational harms (a system withholds a resource or opportunity from a group) and representational harms (a system depicts a group in a demeaning or erasing way), and the two need different measurements.

Large language models add new concerns. They memorise training data and can leak private text. They produce confident falsehoods in a fluent register that readers trust. They can be used to generate misinformation or spam at scale, and their training corpora and compute have environmental and labour costs that are easy to leave out of a paper. Alignment techniques such as instruction tuning and preference optimisation reduce some harms but also encode the values of whoever labelled the preferences.

The methodological point is that fairness is not a single number: a benchmark score can hide large gaps across dialects, languages, and demographic groups, and debiasing that removes an association from vectors often leaves it recoverable by a classifier. Documentation practices — datasheets for datasets and model cards for models — exist so that these limits travel with the artifact. The question to keep asking about any system in these notes is who it is evaluated on, and who is affected when it is wrong.

Key equations at a glance

Common mistakes

• Treating the word2vec softmax as the thing to implement. The full denominator over the vocabulary is the reason negative sampling exists; implement the sampled objective and keep separate center and context vectors.
• Fine-tuning word embeddings on a small labelled set and wondering why unseen words behave worse. Only words in the training data move; freeze embeddings when the dataset is small.
• Reporting perplexity computed with a different tokeniser than the baseline. Perplexity is per token, so two models with different vocabularies are not directly comparable.
• Skipping gradient clipping in recurrent models. Exploding gradients produce a loss that suddenly becomes NaN; clip the global norm and check the forget-gate bias initialisation.
• Using beam search for open-ended generation. Likelihood maximisation produces repetitive, generic text; use nucleus or top-k sampling with a tuned temperature and keep beam search for translation-style tasks.
• Forgetting the causal mask in a decoder. Training looks perfect because every position sees its own answer; generation then collapses. Verify that the attention matrix is lower-triangular.
• Reading an attention map or a probe result as proof of how the model reasons. Add control tasks for probes and use ablation or attribution alongside attention before making a causal claim.
• Building a retrieval pipeline and evaluating only final-answer accuracy. Check retrieval recall separately and check that the answer is supported by the retrieved passages, or the generator’s fabrications go unnoticed.

Related courses

• CS229 — the softmax regression, maximum likelihood, and gradient descent that every model here is built on.
• CS230 — backpropagation, optimisers, and regularisation tricks shared with these notes, at a more general level.
• CS231n — the same training machinery applied to images; residual connections and layer norm show up in both.
• CS224w — graph neural networks use attention over neighbours in the same way Transformers attend over tokens.
• CS234 — the policy-gradient and reward-modelling ideas behind RLHF in their original reinforcement-learning setting.
• CS246 — nearest-neighbour search and indexing at scale, which retrieval-augmented generation depends on.
• CS336 — building a language model from scratch: tokenisers, scaling laws, and systems details this course only sketches.