Course Materials

# Module 1

Getting Started: Google Colab and GitHub

Learning objectives: Open, run, and save a notebook in Google Colab; navigate a GitHub repository and open a notebook from it; describe the course workflow for accessing, editing, and retaining copies of code.

Before class

• Sign in to Google ColabOptional

During class

• Sign in to or create a GitHub accountRequired
• Practice opening, running, copying, and saving a course notebook.Required

After class

• Code.org, “How Chatbots and Large Language Models Work” (about 7 min)Required
• Karpathy, “Intro to Large Language Models” (1 hr, listenable as a podcast)Optional

# Module 2

Tokens and Embeddings

Learning objectives: Tokenize short texts and interpret token IDs; use sentence embeddings to compare semantic similarity; distinguish representation models from text-generation models; explain how next-token probabilities produce generated text.

Before class

• 3Blue1Brown, “But what is a GPT? Visual intro to transformers” (27 min)Required
• Read Chapter 1, “An Introduction to Large Language Models,” and Chapter 2, “Tokens and Embeddings.”Optional

During class

• Reference notebook: Introduction to Language ModelsOptional
• Reference notebook: Tokens and Token EmbeddingsOptional

After class

• Hugging Face LLM Course, “Tokenizers”Optional
• Jay Alammar, “The Illustrated Word2vec”Optional
• TensorFlow Embedding Projector: explore nearest neighbors and semantic clusters in a pretrained embedding spaceOptional

# Module 3

Text Classification I: Encoder Models

Learning objectives: Distinguish binary, multiclass, and multilabel classification problems; apply task-specific and embedding-based classifiers; evaluate predictions using suitable metrics and structured error analysis.

Before class

• Kim, Soderman, and Sang, “Sentiment analysis of solar energy in U.S. Cities: a 10-year analysis using transformer-based deep learning”: read sections 1, “Introduction,” and 2, “Materials and approach,” onlyRequired
• Jay Alammar, “The Illustrated BERT”: read “Example: Sentence Classification” through “Model Outputs”Optional
• IBM Technology, “Text Classification: AI Techniques and Real-World Applications” (14 min)Required
• StatQuest, “Machine Learning Fundamentals: The Confusion Matrix” (7 min)Required
• 3Blue1Brown, “Attention in transformers, step-by-step”Optional
• Read Chapter 3, “Looking Inside Large Language Models,” and the Chapter 4 sections on task-specific and embedding-based text classification.Optional

During class

• Reference notebook: Looking Inside LLMsOptional
• Companion notebook: Text ClassificationOptional

After class

• Hugging Face LLM Course, “How Transformers Solve Tasks”: read “Text classification”Optional
• Hugging Face LLM Course, “Working with pipelines”: try the sentiment-classification example with your own sentencesOptional
• Hugging Face Transformers, “Text classification”: fine-tune DistilBERT on IMDb and run inferenceOptional
• Hugging Face SetFit, “Sentence Transformers Finetuning”: see how an embedding model and classification head work togetherOptional
• TensorFlow Playground: run a classification problem and observe how the learned decision boundary changesOptional

# Module 4

Text Classification II: Zero-Shot and Few-Shot Models

Learning objectives: Compare NLI-based zero-shot classification with prompted generative classification; use label descriptions, few-shot examples, and constrained outputs; test how label wording and examples affect classification performance.

Before class

• Read Chapter 4, “Text Classification,” from “What If We Do Not Have Labeled Data?” through “Text Classification with Generative Models,” including the Flan-T5 and ChatGPT classification examples.Optional
• IBM Technology, “What is Zero-Shot Learning?” (9 min)Required
• CodeEmporium, “Few Shot Learning – explained!” (10 min)Required

During class

• Companion notebook: Text ClassificationRequired

After class

• Bhavesh Bhatt, “Zero Shot Classification with HuggingFace Pipeline” (hands-on code walkthrough)Optional
• Hugging Face, “Zero-Shot Text Classification”Optional

# Module 5

Text Classification III: Evaluation and Error Analysis

Learning objectives: Compare classifiers on the same held-out examples using aggregate and per-class results; categorize errors and disagreements with reference to the original text; recommend an approach while distinguishing development decisions from final evaluation.

Before class

• scikit-learn, “classification_report”: macro and weighted averages; first worked example (opens in a new tab)Required
• scikit-learn, “Common pitfalls and recommended practices”: “Data leakage” and “How to avoid data leakage” (opens in a new tab)Required
• Benjy Bodner, “Confusion Matrix, Precision, Recall & F1 — Explained Together!” (6 min) (opens in a new tab)Required
• Chapter 4, “Text Classification”: metrics in “Using a Task-Specific Model”Optional

During class

• Lab 1 workshop: compare classifiers on the same examples and inspect prediction errors.Required

After class

• Lab 1: Text Classification and Error AnalysisOptional

# Module 6

Text Clustering and Topic Modeling

Learning objectives: Create document embeddings for an unlabeled text collection; cluster and visualize semantically related documents; interpret topic representations while identifying instability, labeling choices, and other limitations.

Before class

• StatQuest, “K-means clustering”: watch 0:33–7:08Required
• BERTopic, “The Algorithm”: “Visual Overview” and steps 1–5 of “Detailed Overview”Required
• Chapter 5, “Text Clustering and Topic Modeling”: through the BERTopic discussion before “Adding a Special Lego Block”Optional

During class

• BERTopic, “Visualize Topics”: explore the topic map and topic-similarity heatmapRequired
• Companion notebook: Text Clustering and Topic ModelingOptional

After class

• Pinecone, “Advanced Topic Modeling with BERTopic”: pipeline overviewOptional
• BERTopic FAQ: reproducibility, outliers, and stop wordsOptional

# Module 7

RAG I: Dense Retrieval and Grounded Generation

Learning objectives: Trace how a question moves through chunking, embeddings, retrieval, and answer generation; use a scaffolded dense-retrieval workflow; construct a grounded-answer prompt and check whether cited passages support the answer.

Before class

• Microsoft Learn, “Understand retrieval-augmented generation”: text and diagrams (opens in a new tab)Required
• Microsoft Learn, “Prepare data for retrieval”: text and diagrams (opens in a new tab)Required
• Microsoft Learn, “Retrieve information and generate a response”: text and diagrams (opens in a new tab)Required
• Chapter 8: semantic search overview, opening dense-retrieval example, and “From Search to RAG”Optional

During class

• Companion notebook: Semantic Search and Retrieval-Augmented Generation (opens in a new tab)Optional

After class

• Lab 2: Semantic Search, RAG, and EvaluationOptional

# Module 8

RAG II: Evaluation and Improvement

Learning objectives: Distinguish retrieval failures from generation failures; assess evidence relevance, answer correctness, groundedness, and unanswerable questions; compare one retrieval or prompt change using fixed examples and report its tradeoffs.

Before class

• Microsoft Learn, “Evaluate a RAG solution”: text and diagrams (opens in a new tab)Required
• Evidently AI, “How to evaluate a RAG system: methods and metrics” (7 min): retrieval quality, faithfulness, and stress testing (opens in a new tab)Required
• Chapter 8, “Retrieval Evaluation Metrics”: queries and relevance judgments; calculations optionalOptional

After class

• Sentence Transformers, “Retrieve & Re-Rank”: pipeline overview and optional example scripts (opens in a new tab)Optional

# Module 9

Prompt and Context Engineering

Learning objectives: Distinguish prompt engineering from context engineering; select and organize instructions, examples, and source evidence; revise a prompt and assess whether the answer becomes more informative and supported by its sources.

Before class

• Anthropic, “Effective context engineering for AI agents”: “Context engineering vs. prompt engineering” and “The anatomy of effective context” (opens in a new tab)Required
• Chapter 6, “Prompt Engineering”: “The Basic Ingredients of a Prompt” and “Instruction-Based Prompting”Optional

During class

• Prompt-engineering exercise: revise instructions while keeping the source packet fixed; compare useful details and check source support.Required

After class

• Anthropic, “Effective context engineering for AI agents”: “Context retrieval and agentic search” (opens in a new tab)Optional

# Module 10

Evaluating LLM Answers

Learning objectives: Distinguish faithfulness to sources, completeness, and relevance; compare human evaluation, automatic metrics, and LLM-as-a-judge; apply a rubric to Module 9 answers and check the judge’s decisions against source evidence.

Before class

• Evidently AI, “LLM evaluation methods and metrics” (opens in a new tab)Required
• Evidently AI, “LLM evaluation: a beginner’s guide”: “LLM evaluation methods,” including manual and automated evaluation; stop before “LLM evaluation scenarios” (opens in a new tab)Required
• Hugging Face, “Using LLM-as-a-judge”: introduction and prose in Section 1; skip setup and code (opens in a new tab)Required
• Stanford CRFM, “The Need for Holistic Evaluation”: “Holistic evaluation” and its three subsections (opens in a new tab)Required

During class

• Evaluate the fixed Module 9 answers with an LLM judge, then audit its ratings using the sources and a shared rubric.Required

After class

• Microsoft Learn, “RAG evaluators”: “System evaluation” definitions of groundedness, relevance, and response completeness (opens in a new tab)Optional

# Module 11

AI Agents and Model Adaptation

Learning objectives: Trace an agent’s tool-use loop and stopping condition; distinguish prompt, context, and workflow changes from updates to model weights; explain parameter-efficient fine-tuning; use evaluation evidence to choose a feasible project improvement.

Before class

• IBM Technology, “Agentic Engineering: 5 Terms Every AI Developer Should Know” (11 min 40 sec) (opens in a new tab)Required
• Anthropic, “Building effective agents”: introduction through “When (and when not) to use agents” (opens in a new tab)Required
• Hugging Face: “Fine-tuning” (opening definition before login instructions) (opens in a new tab) and “LoRA” (introduction and diagram before “Usage”) (opens in a new tab)Required
• Bring one project result, one failure example, and one question for the project clinic.Required

During class

• Hugging Face Agents Course, “What are Tools?”: trace tool use through the calculator example; stop before “Auto-formatting Tool sections” (opens in a new tab)Required
• Project clinic: discuss one result, one limitation, and a feasible improvement. Fine-tuning is optional.Required

After class

• Anthropic, “Building effective agents”: choose one workflow pattern—prompt chaining, routing, or evaluator-optimizer (opens in a new tab)Optional
• Chapter 7: introductions to chains and agents through ReAct; skip LangChain. Or Chapter 12: SFT, full fine-tuning, and PEFT through LoRA; stop before QLoRA.Optional

# Module 12

Project Presentations

Learning objectives: Present the project question, data, workflow, evaluation, and results as a coherent analytical story; use clear visuals or examples to explain model behavior; discuss limitations and responsible use and respond thoughtfully to questions and peer feedback.

Before class

• Review the project presentation requirementsRequired
• Check that your GitHub repository and README are ready for questions.Required
• Submit presentation materials through Canvas by the published deadline.Required