Module 2 Assignment: Gradient trace and autograd check

Module 2 Assignment: Gradient trace and autograd check#

Theme#

Backpropagation and automatic differentiation

Scenario#

A research engineer needs to explain why a custom loss is not training and whether the issue is math, implementation, or scale.

Exercises#

  1. Draw the graph for a two-layer network and identify retained tensors.

  2. Use the starter cell to compare a manual gradient with an autograd result.

  3. Change one operation and predict how the derivative should change before running it.

  4. Explain one gradient failure mode and a practical diagnostic.

Evidence Requirements#

  • A short explanation of the data, tensors, objective, and evaluation signal used in the starter experiment.

  • At least one meaningful modification to the starter code, with the changed variable named explicitly.

  • A comparison against the unmodified starter result or another defensible baseline.

  • A limitation statement that separates what the toy experiment demonstrates from what a production model would require.

Submission#

Submit a 600-900 word technical memo plus code, plots, tables, or shape traces needed to support your claims. The memo should read like a review artifact for another AI practitioner: concise, reproducible, and honest about uncertainty.

Rubric Focus#

  • Technical correctness and appropriate neural-network vocabulary.

  • Evidence from the starter experiment or a documented extension.

  • Connection between design choices and data/problem structure.

  • Clear treatment of limitations, failure modes, or next experimental gates.

import torch

torch.manual_seed(2)
x = torch.tensor([1.5])
w = torch.tensor([0.8], requires_grad=True)
b = torch.tensor([-0.2], requires_grad=True)
target = torch.tensor([1.0])

y_hat = w * x + b
loss = (y_hat - target).pow(2).mean()
loss.backward()

manual_dw = 2 * (y_hat.detach() - target) * x
print(f"autograd dw: {w.grad.item():.3f}")
print(f"manual dw:   {manual_dw.item():.3f}")

autograd dw: 0.000
manual dw:   0.000

Reflection Prompts#

  • What changed when you modified the starter experiment, and why should that change matter?

  • Which result surprised you, and what diagnostic would you run next?

  • What assumption would you document before handing this model to another practitioner?

  • Which failure mode from the module reading is most relevant to your result?