Syllabus

Topics

When is AI the right tool, and what is an "AI product"? Reason about the tradeoffs of probabilistic systems, treating today's constraints (non-determinism, latency, cost) as examples of a durable pattern, not a fixed list.

Reading

• Sculley et al., "Hidden Technical Debt in ML Systems" (NeurIPS, 2015)
• Yan, "Patterns for Building LLM-based Systems & Products" (2023)
• Optional: Karpathy, "Software Is Changing (Again)" (YC AI Startup School, 2025)

Topics

Surface real user needs through interviews and jobs-to-be-done; use AI to synthesize research while judging whether its output is genuine signal or sophisticated noise.

Deliverable

Discovery brief for a product / agent idea with explicit AI-fit assessment.

Reading

• Christensen et al., "Know Your Customers' Jobs to Be Done" (HBR, 2016)
• Park et al., "Generative Agent Simulations of 1,000 People" (2024)
• Optional: Hämäläinen et al., "Evaluating LLMs in Generating Synthetic HCI Research Data" (CHI, 2023)

Topics

Build a prototype, then learn to tell whether it's any good: choose evaluation dimensions, score outputs, read benchmarks skeptically (Goodhart, gaming), and weigh human against automated judges. In-class lab: interactive case study, followed by a workshop to design evals for students' own prototypes.

Deliverable

Working prototype of the idea from your discovery brief.

Reading

• Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" (NeurIPS, 2023)
• Miller, "Adding Error Bars to Evals" (Anthropic, 2024)
• Optional: Tan et al., "JudgeBench" (2024)
• Optional: Tyser et al., "AI-Driven Review Systems" (2024)

Topics

Turn evaluation into an improvement loop: pick the right lever (prompt → retrieval → scaffolding → fine-tune → RL) and apply self-improvement methods including adversarial self-improvement and recursive self-improvement, while avoiding optimizing your measure into meaninglessness.

Deliverable

Gate 1: Project proposal with your newly formed team (~3 pages)

Reading

• Ouyang et al., "Training Language Models to Follow Instructions with Human Feedback" (InstructGPT, 2022)
• Yuan et al., "Self-Rewarding Language Models" (2024)
• Optional: Christiano et al., "Deep Reinforcement Learning from Human Preferences" (NeurIPS, 2017)

Topics

Judge behavior across whole interactions, and build simulated users to test at scale, anchoring on measuring the sim-to-real gap.

Reading

• Yao et al., "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains" (2024)
• Seshadri et al., "Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations" (2026)
• Optional: Anthropic, "Demystifying Evals for AI Agents" (2026)

Topics

Where qualitative research complements granular evals (sentiment, the felt experience, diary studies, and AI-generated users), and where they break down.

Reading

• Amershi et al., "Guidelines for Human-AI Interaction" (CHI, 2019)
• Salminen et al., "The Use of LLMs in HCI: A Critical Analysis of Synthetic Users" (Augmented Humans, 2025)
• Optional: Weisz et al., "Design Principles for Generative AI Applications" (CHI, 2024)

Topics

Diagnose real failures as systemic rather than single-cause, and identify the eval, guardrail, or research that would have caught each. E.g., Air Canada was held legally liable when its support bot invented a bereavement-refund policy that didn't exist.

Deliverable

Eval feedback for two other teams' projects.

Reading

• Bainbridge, "Ironies of Automation" (Automatica, 1983)
• OpenAI, "Expanding on What We Missed with Sycophancy" (2025)
• Case sourcing: Popchanovska et al., "When AI Fails, What Works? A Data-Driven Taxonomy of Real-World Risk Mitigation Strategies" (2026)

Topics

Red-team for worst-case and harmful behavior, address agentic safety risks (destructive actions, leaked secrets), and weigh who sets the boundaries on AI use.

Deliverable

Gate 2: Midterm report (~5 pages + working build)

Reading

• Amodei et al., "Concrete Problems in AI Safety" (2016)
• Ganguli et al., "Red Teaming Language Models to Reduce Harms" (Anthropic, 2022)
• Lab tool: Debenedetti et al., "AgentDojo" (NeurIPS, 2024)

Topics

Evaluate beyond text, where latency, tone, turn-taking, and cross-modal grounding create new failure modes, with worked-through real product examples.

Reading

• Clark & Brennan, "Grounding in Communication" (1991)
• Sierra, "τ-voice: Benchmarking Full-Duplex Voice Agents" (2026)
• Optional: Selvakumar et al., "MULTIVOX" (EMNLP, 2025)
• Optional: Lin et al., "Full-Duplex-Bench" (2025)
• Optional: Lee et al., "Holistic Evaluation of Text-to-Image Models (HEIM)" (NeurIPS, 2023)

Topics

Defend an evidence-backed ship-or-don't-ship call with your eval results and a concrete next-iteration plan, before outside judges.

Deliverable

Gate 3: Demo Day (12-min demo; 5–7 page report due 72 hrs after class)

Peer feedback and ratings.