Module 8: GPU workflows, scale, and deployment

Module 8: GPU workflows, scale, and deployment#

AINS6003 — Deep Learning & Neural Networks

90-minute lecture deck

Essential question: How do practitioners move from notebook experiments to reproducible GPU training pipelines?

Why This Matters#

A prototype model is ready to leave a notebook, and the team needs a repeatable training job with accountable metrics and resource assumptions

For non-CS graduate students, the goal is not to become a software engineer in one class session. The goal is to learn how to read an AI workflow, ask better questions, and explain what the evidence does or does not support.

90-Minute Teaching Arc#

Time

Segment

Purpose

0-10

Orientation and stakes

Connect the topic to a professional decision.

10-25

Conceptual model

Build intuition before code or formulas.

25-40

Worked example

Translate vocabulary into a small concrete case.

40-55

Evidence and interpretation

Read outputs, metrics, or artifacts carefully.

55-70

Guided student activity

Let students change one variable and observe.

70-82

Risk, limits, and communication

Name what could go wrong and how to explain it.

82-90

Assignment handoff

Clarify deliverable, rubric, and next step.

Learning Outcomes#

  • explain the core technical idea in precise neural-network vocabulary

  • connect architecture and training choices to the shape of the data and task

  • run or interpret the module lab as reproducible evidence

  • identify limitations, failure modes, and next steps for a defensible experiment

By the end, students should be able to explain the idea without hiding behind jargon and should know what evidence would make a recommendation stronger.

Plain-Language Framing#

Ask students to complete this sentence before the technical terms arrive:

This method helps a professional decide whether ______ because it uses ______ as evidence.

Then revisit the sentence at the end of the lecture and improve it with module vocabulary.

Core Vocabulary#

  • device placement and tensor movement: define it in one sentence, then connect it to the scenario.

  • random seeds and deterministic settings: define it in one sentence, then connect it to the scenario.

  • environment capture and dependency management: define it in one sentence, then connect it to the scenario.

  • checkpoints, metrics, and experiment tracking: define it in one sentence, then connect it to the scenario.

  • throughput, cost, and deployment gates: define it in one sentence, then connect it to the scenario.

Instructor note: pause after each term and ask for a student-generated example.

Conceptual Model#

Use a three-part model:

  1. Input: What information is available?

  2. Transformation: What does the AI or analytic method do to the information?

  3. Decision: What human or organizational action could change because of the result?

This keeps the discussion accessible for students new to Python.

Worked Example Setup#

Use the professional scenario as the example case. Ask:

  • Who owns the decision?

  • What evidence would they trust?

  • What would count as a bad recommendation?

  • What would a cautious first experiment look like?

Write the answers on the board before opening the lab.

Method Pattern#

  • Record environment, data version, split, seed, hyperparameters, and hardware before comparing runs.

  • Keep training, evaluation, and inference entry points separate.

  • Use checkpoints and metric logs so long runs are recoverable and auditable.

  • Set deployment gates for performance, robustness, cost, and monitoring readiness.

This is the repeatable professional move students should practice across the program.

Lab Bridge#

Lab notebook: Module 8 Lab: GPU profiling checklist

Use Colab as the recommended first environment. Have students run all cells first, then change exactly one value, threshold, feature, or assumption. The point is observation and interpretation, not typing a lot of code from scratch.

Reading Lab Outputs#

When students see a number, plot, table, or printed result, ask four questions:

  1. What changed?

  2. Is the change large enough to matter?

  3. What assumption produced the result?

  4. What would we need before using this outside the toy setting?

Guided Activity#

Students work in pairs or small groups:

  • Run the lab unchanged.

  • Change one small input or parameter.

  • Capture the before/after result.

  • Write a two-sentence interpretation for a nontechnical stakeholder.

Share two examples with the room.

Common Failure Modes#

  • Notebook state that cannot be reproduced by another practitioner.

  • Silent CPU/GPU tensor mismatch or unintended device transfers.

  • Scaling batch size without checking memory, convergence, or metric comparability.

  • Deploying a model artifact without lineage, monitoring, or rollback.

A strong lecture names these risks before students over-trust the output.

Discussion Checkpoint#

Use these prompts at the 60-minute mark:

  • What did the method make easier to see?

  • What did the method hide or simplify?

  • Who might be harmed by a confident but wrong interpretation?

  • What evidence would make you more comfortable recommending action?

Assignment Handoff#

Module 8 Assignment: Reproducible training workflow checklist

  • Write a reproducibility checklist for a deep learning experiment

  • Record environment, device, random seed, data split, metrics, and checkpoint assumptions

  • Use the starter cell to report CUDA availability and tensor placement

  • Explain how to scale the experiment from exploration to a repeatable job

Students should leave knowing the artifact they are producing, the evidence they must include, and the limitation they must state.

Rubric Translation#

Translate grading into student language:

  • Correct: terms and results are used accurately.

  • Evidence-based: claims point to notebook output, scenario facts, or documented assumptions.

  • Context-aware: the recommendation fits the stakeholder decision.

  • Honest: limitations and risks are named clearly.

Closing Reflection#

Exit prompt:

In one paragraph, explain what this module helps you decide, what evidence the lab produced, and what you would still need before trusting the result in a real organization.

Collect this verbally, in the LMS, or as the opening paragraph of the assignment.

Instructor Timing Notes#

If time runs short, preserve the lab bridge and assignment handoff. Compress vocabulary rather than skipping interpretation. Students new to AI need repeated practice moving from output to meaning.

If time runs long, add a second student share-out focused on limitations and stakeholder communication.

If Students Are New to Python#

  • Explain that a notebook mixes text, code, and output in one page.

  • Run the notebook once before asking students to change anything.

  • Ask for one small change, not open-ended coding.

  • Grade interpretation, evidence, and limitation statements more than syntax fluency.

  • Keep Colab as the first-choice environment unless the activity truly needs the full repository.

  • Give students permission to describe the result first, then refine the vocabulary after discussion.