Course Dashboard | From Tokens to Transformers

Week 01 opens 28 Sep_

posted 22 sep 2026

Three 30-minute tasks each weekday, one assignment at the weekend. Register on Unstop to be eligible for assessments and certification.

Week 01

What is NLP, and under the hood

opens 28 sep · 28 sep – 4 oct

Swipe sideways to see all three tasks.

Week 1	What's in there	Task 1	Task 2	Task 3
Day 1	What is NLP, and what an NLP pipeline looks like from raw text to a working model.	VideoIBM · What is NLP?	Videocodebasics · NLP Pipeline	NotebookRun a spaCy pipeline on a paragraph: tokens, POS tags, entities.
Day 2	Under the hood: tokenizers and embeddings. How text becomes numbers, and why "king − man + woman" lands near "queen".	VideoUnder The Hood · Dataset Preparation & Tokenization	VideoUnder The Hood · What Are Word Embeddings?, then The Illustrated Word2Vec	VideoKarpathy · Let's build the GPT Tokenizer
Day 3	Sequence models: why RNNs and LSTMs fail on long text, and what attention fixed.	VideoStatQuest · RNNs	VideoStatQuest · LSTMs	VideoStatQuest · seq2seq and Attention for Neural Networks
Day 3Alternative	The same ground as one continuous lecture, if you prefer that to four clips.	LectureMIT 6.S191 · RNNs, Transformers & Attention, first half	LectureMIT 6.S191, second half
Day 4	Self-attention and the transformer block: what Q, K and V do, and how to read an attention map.	Video3Blue1Brown · But what is a GPT?	Video3Blue1Brown · Attention in Transformers	InteractiveTransformer Explainer: find one head that attends to the previous token; draw the block from memory.
Day 5	The whole pipeline end to end: tokenise one sentence, embed it, attend over it, read the logits.	VideoStatQuest · Decoder-Only Transformers	InteractiveTransformer Explainer: walk one sentence through every stage.	NotebookOne sentence through GPT-2: token IDs → embeddings → attention weights → logits.

Week 02

How models are trained, and how to use them

Releases 5 Oct · 5 – 11 Oct

Swipe sideways to see all three tasks.

Week 2	What's in there	Task 1	Task 2	Task 3
Day 1	From a Transformer to an LLM.	VideoKarpathy · Intro to Large Language Models · pretraining and scaling	VideoSebastian Raschka · Developing an LLM: Building, Training, Finetuning (27:20–35:21) · Understading the whole LLM Cycle	NotebookRun three prompts through Qwen2.5-0.5B and its Instruct counterpart · compare the outputs
Day 2	how RLHF turns one into the other, and how decoding actually picks the next token.	VideoStatQuest · RLHF, Clearly Explained	VideoGary Explains · Temperature, Top-K, Top-P: the secret controls of your LLM	NotebookSampling acts on logits produced by fixed weights. Plot the distribution before reading generated prose
Day 3	Prompting: what reliably works, and how to get structured output you can build on.	VideoIBM · Prompt Engineering · Guidelines and Iterative lessons	BlogEffective context engineering for AI agents	NotebookGet the model to return valid JSON. Then break it. Then make it robust.
Day 4	Evaluate an extractor with a frozen schema, a small regression set, and separate JSON, schema, field-accuracy, and latency results.	ReadHamel Husain · Your AI Product Needs Evals · read the evaluation types, unit tests, and opening human/model eval sections; define the three extractor checks.	ReadIBM · What Are Vision Language Models and LLaVA · sketch encoder → connector → language model and note failures with tiny text, counting, or missing evidence.	NotebookEvaluation and VLM · score prompts v1/v2 on 10 cases, inspect two failures, define five untuned held-out cases, and make a visual checklist; report optional VLM results separately.
Day 5	When to retrieve versus fine-tune: change the input context, model parameters, or both; cover grounding, citations, abstention, SFT, LoRA, and evaluation leakage.	ReadIBM · What is RAG? and RAG vs Fine-Tuning vs Prompt Engineering · draw question → retrieval → prompt evidence → cited answer; note that RAG can fail and “RAG for facts, fine-tuning for behaviour” is only a heuristic.	ReadSebastian Raschka · Developing an LLM (39:48–43:06) and Chapter 7 instruction fine-tuning notebook sections 7.1–7.2; identify instruction, input, target, and create three examples without training.	NotebookW2D5 Context RAG and Adaptation · compare no context, correct, wrong, and retrieved evidence; test one unanswerable question, inspect three SFT examples and LoRA counts, then choose a method for five scenarios and plan the weekend app.
Weekend	Assignment 2. A 15-question quiz, like last week. Due Sunday 11 October, 11:59 PM IST.	Optional depthKarpathy · Deep Dive into LLMs like ChatGPT (3.5 hr) · HF · How to generate text · Prompt Engineering Guide · Hamel Husain · Building LLM evaluation systems

Week 03

Build a RAG app, then train a LoRA adapter

Releases 12 Oct · 12 – 18 Oct

Swipe sideways to see all three tasks.

Week 3	What's in there	Task 1	Task 2	Task 3
Day 1	Turn documents into a searchable index: chunk size and overlap, source metadata, sentence embeddings, cosine similarity, and saving the index.	VideoLance Martin · RAG From Scratch Part 2: Indexing and Sentence Transformers · Semantic Search (opening and “Symmetric vs. Asymmetric”) · sketch the offline indexing path separately from the online query path.	ReadGreg Kamradt · Five Levels of Text Splitting (Levels 1–2, skim Level 3 headings) and play with ChunkViz · change size and overlap and watch what a boundary cuts off.	NotebookW3D1 Chunking and Embeddings · load the eight fictional club documents, compare two window sizes, embed chunks and queries with MiniLM, try an exact-term query against a paraphrase, then save and reload the index.
Day 2	Build a RAG app with visible sources: top-k, context assembly, token budgets, grounded answers, citations, abstention, stale documents and prompt injection.	VideoLance Martin · Part 3: Retrieval and Part 4: Generation · write down the inputs and outputs of the retriever and the generator; skip the account setup.	ReadRAG From Scratch · Parts 1–4 notebook, Part 4 only · trace retrieved documents into the prompt and into the answer. A valid citation ID does not prove the passage supports the claim.	NotebookW3D2 RAG Application · run `ask_club(question)` and read the answer, sources, scores and timing; compare no evidence, a known passage and retrieved evidence; then test an unsupported question, an archived policy and an instruction hidden in a document.
Day 3	Score retrieval separately from answers: hit rate at k, reciprocal rank, oracle context, optional reranking, and held-out questions that stay out of the index.	ReadJames Briggs · Rerankers and Two-Stage Retrieval (“Recall vs. Context Windows” and “Power of Rerankers”, stop before the hosted code) · explain why a reranker sees query and passage together, and why it cannot recover a passage that was never retrieved.	ReadHamel Husain · Your AI Product Needs Evals (“Write Scoped Tests” and “Looking At Your Traces”) then W3D3 RAG Evaluation sections 1–2 · ten development questions, eight answerable and two unsupported; score hit rate and reciprocal rank.	NotebookW3D3 RAG Evaluation sections 3–4 · compare chunk sizes with k fixed, optionally rerank six candidates to two, compare no-context, oracle and retrieved answers on three cases, annotate three failures, and draft five fresh questions for the weekend.
Day 4	Prepare data and train a LoRA adapter for the event extractor: train/validation/test splits, loss masks, rank and alpha, target modules, gradient accumulation. Needs a CUDA GPU runtime.	ReadSebastian Raschka · LoRA from scratch (stop before DoRA) and Practical Tips for Finetuning LLMs Using LoRA (intro and Q1) · count the parameters two low-rank matrices add; contrast full fine-tuning, LoRA and QLoRA.	ReadRaschka · Chapter 7 notebook, section 7.3 padding masks, then W3D4 LoRA Training sections 1–4 · audit the 204 synthetic examples and their group splits, inspect the −100 loss mask and the real token targets before training.	NotebookW3D4 LoRA Training sections 5–6 (with the PEFT quicktour open) · Qwen2.5-0.5B-Instruct, rank 8, alpha 16, q_proj/v_proj; up to 40 optimizer steps with accumulation 4; save the best checkpoint and download `club_adapter_run1.zip` before the runtime disconnects.
Day 5	Reload the adapter in a fresh runtime and compare it against the original checkpoint: overfitting, field accuracy, regressions, one unlocked test run, and an adapter card.	VideoSebastian Raschka · Developing an LLM (46:04–53:59, evaluation) and LoRA tips, Q5: avoiding overfitting · relate training and validation loss to extraction errors. No improvement is also a legitimate result.	ReadPEFT · save model and inference then W3D5 Adapter Evaluation sections 1–3 · upload Thursday’s ZIP, check base model ID, tokenizer and schema, and compare base vs adapter on the 24 validation cases using adapter disabling.	NotebookW3D5 Adapter Evaluation sections 4–5 and the adapter card · freeze the system, unlock the 20-case test once, probe inputs outside the training distribution, and write up data, settings, limits and remaining failures.
Weekend	Assignment 3. A 15-question quiz, like previous weeks. Due Sunday 18 October, 11:59 PM IST.	Optional depthQLoRA paper (abstract and Figure 1) · RAG From Scratch repository (query transformation, multi-query) · Raschka · implement the LoRA layer · Qwen2.5-0.5B-Instruct card · all-MiniLM-L6-v2 card · Colab GPU availability FAQ

Schedule

28 Sep – 18 Oct

28 September	Week 1 released
4 October	Assignment 1 due
5 October	Week 2 released
11 October	Assignment 2 due
12 October	Week 3 released
18 October	Assignment 3 due