Module 3: Optimization, loss, and regularization

Theme

Optimization, loss, and regularization

Essential Question

How do we train deep networks reliably when loss surfaces are non-convex and data are noisy?

Professional Scenario

A modeling team has a prototype that trains inconsistently and needs a defensible training configuration before scaling experiments.

Learning Outcomes

By the end of this module, students will be able to:

• explain the core technical idea in precise neural-network vocabulary

• connect architecture and training choices to the shape of the data and task

• run or interpret the module lab as reproducible evidence

• identify limitations, failure modes, and next steps for a defensible experiment

Module Components

• Book prose for conceptual framing and practitioner patterns.

• Assignment notebook for the graded artifact: optimizer comparison memo using loss curves, accuracy, and regularization tradeoffs.

• Slides and narration for structured teaching.

• Lab notebook for hands-on experimentation.

• Rubric for grading technical correctness, evidence, tradeoffs, risk analysis, and communication.

Use This Module in Order

1. Read the learning chapter.

2. Review the slide deck with the matching narration.

3. In Populi, open the private student-repository link for this course and enter modules/module-3.

4. Clone the repository once or open its Codespace/Colab copy; run lab.ipynb and complete exercise.ipynb there.

5. Self-check with the rubric, commit and push the work, then submit exactly what Populi requests.