Module 2 Assignment: Gradient trace and autograd check

Module 2 Assignment: Gradient trace and autograd check#

Theme#

Backpropagation and automatic differentiation

Exercises#

  1. Draw the computational graph for a two-layer network with mean squared error.

  2. Identify which intermediate tensors must be retained for the backward pass.

  3. Use the starter cell to compare a manual gradient with PyTorch autograd.

  4. Explain one condition that can make gradients vanish, explode, or become misleading.

Submission#

Submit a 600-900 word technical memo plus any code, plots, or shape traces needed to support your claims. Use the starter cell as a minimum reproducible experiment, then make at least one meaningful modification.

Rubric#

  • Correct use of module vocabulary and notation

  • Clear connection between design choices and data/problem structure

  • Evidence from the starter experiment or your own extension

  • Concise reflection on limitations, failure modes, or next steps

import torch

torch.manual_seed(2)
x = torch.tensor([1.5])
w = torch.tensor([0.8], requires_grad=True)
b = torch.tensor([-0.2], requires_grad=True)
target = torch.tensor([1.0])

y_hat = w * x + b
loss = (y_hat - target).pow(2).mean()
loss.backward()

manual_dw = 2 * (y_hat.detach() - target) * x
print(f"autograd dw: {w.grad.item():.3f}")
print(f"manual dw:   {manual_dw.item():.3f}")

autograd dw: 0.000
manual dw:   0.000

Reflection prompts#

  • What changed when you modified the starter experiment?

  • Which result surprised you, and what diagnostic would you run next?

  • What assumption would you document before handing this model to another practitioner?