Project gates
The gates are checkpoints in one continuous build. Each should show what changed because of evidence, feedback, or evaluation—not just that the team produced more features.
| Gate | Timing | What it establishes |
|---|---|---|
| 1 · Proposal | Week 4 | A specific user need, why AI is appropriate, initial success criteria, a feasible scope, and a team charter. Format: ~3-page proposal. |
| 2 · Midterm | Week 8 | A working build, evaluation evidence, risks and failure modes, and a prioritized plan for the remaining work. Format: ~5 pages plus the working build. |
| 3 · Final | Week 10 | A live demo, final report, evidence-backed ship-or-don’t-ship recommendation, and next-iteration plan. Format: 12-minute demo; 5–7 page report due 72 hours after class. |
Exact due dates, submission instructions, and grading appear in Canvas, which is authoritative.
Team charter
Submit a short charter with Gate 1. It should record:
- regular meeting and working times;
- the team’s normal response-time expectation and how to flag something urgent;
- how product and technical decisions will be made;
- how code review, branches, deployment access, credentials, and documentation will work;
- initial ownership and a plan for rotating or sharing less visible work;
- what the team will do when someone is blocked or misses a commitment.
The charter is a working agreement, not a contract frozen in Week 4. Revisit it after Gate 2 or whenever the team’s needs change.
Individual contribution and grading
Project grades are individual, not purely team-based, so students on the same project may receive different scores. The teaching team considers multiple forms of evidence:
- confidential peer feedback, including a mid-project pulse and final peer evaluation;
- the team charter and subsequent ownership decisions;
- project artifacts such as research notes, decision records, evaluation work, code, design, and writing;
- conversations with the team and each member’s ability to explain and defend the work;
- repository history where it provides useful context.
Commit counts and lines of code are not contribution scores. Research, user testing, evaluation design, product judgment, coordination, and careful review may be just as important as implementation.
Getting help early
Do not wait for the final peer evaluation to reveal a serious team problem. Start by naming the issue concretely and revisiting the charter. If that does not resolve it—or if raising it directly feels unsafe or inappropriate—contact your assigned mentor in the private team Slack channel or contact the teaching team privately. We can clarify responsibilities, facilitate a reset, adjust scope, or discuss changes to the team.
Tell us promptly if a teammate withdraws, has an extended absence, loses access to essential tools, or is consistently unable to participate. Early communication gives us more ways to help without derailing the project.
Demo Day and public release
Demo Day is public, though the expected audience is the class and invited guests. A team should assume that anyone may see what appears on screen. Do not display secrets, private user information, restricted data, or material the team is not authorized to share. Use synthetic or de-identified demonstration data when appropriate.
An audience member may not record or redistribute a presentation without the team’s permission. Any official recording will be opt-in by team.
Project repositories remain private throughout the course. After final submission, a team may release its repository only with unanimous agreement and after completing the privacy, rights, attribution, and credential checks in Privacy, recording, and public work.
A live technical failure is not itself a failed project. Teams should prepare a short fallback—such as a recorded run, screenshots, or saved outputs—and be ready to explain what failed and what evidence supports the intended behavior.
Responsible development
Projects involving sensitive data, real external communities, consequential decisions, or agents that can act in the world require advance discussion with the teaching team. Read the full responsible product development policy before user testing or deployment.
Starter Projects
Some example project ideas teams may pick from or adapt. Each is scoped to a 7-week build and teaches a distinct evaluation lesson.
Meeting Notetaker with Structured Outputs
A desktop assistant that runs alongside calls without joining as a bot, captures the user's typed shorthand alongside the audio, and produces action items, decisions, and owner-tagged open questions. Teaches LLM-as-judge calibration against humans: students collect real meetings, hand-label "good action item" examples, then build an LLM judge and measure its agreement with humans before trusting it for scaled evaluation.
First-Pass Content Moderation for a Real Online Community
Partner with one Discord server, subreddit, or open Slack and build the first-pass moderator: flag, route to a human, or auto-action. Students specify the policy in plain English, build the classifier, and ship a moderator UI. Teaches asymmetric error costs: false-positive harms and false-negative harms have very different shapes, and the team must build separate dashboards for each and defend their threshold choice.
Fantasy Sports Draft Copilot with Model Tiering
Sync to a fantasy league. During the live draft, give real-time recommendations with rationale (fast, cheap inference). During the season, give weekly start/sit advice with confidence intervals and post-mortems (slower, deeper analysis). Teaches cost optimization by tiering models (Haiku class for draft turn, Opus class for weekly recap) and forces students to defend the routing decision.