Module 2 Lab: Manual vs autograd gradients

Contents

Module 2 Lab: Manual vs autograd gradients#

Check a hand-derived gradient against PyTorch autograd on the same scalar computation.

Run the setup cell, inspect the printed diagnostics, and then complete the exercises at the end. The lab is intentionally small enough to run in GitHub Codespaces without a GPU.

import torch

torch.manual_seed(12)
x = torch.tensor([[1.0, -2.0]])
W = torch.tensor([[0.3], [-0.7]], requires_grad=True)
b = torch.tensor([0.1], requires_grad=True)
target = torch.tensor([[2.0]])

y_hat = x @ W + b
loss = ((y_hat - target) ** 2).mean()
loss.backward()

manual_dy = 2 * (y_hat.detach() - target)
manual_dW = x.T @ manual_dy
manual_db = manual_dy.squeeze()
print("autograd dW:", W.grad.flatten().tolist())
print("manual dW:  ", manual_dW.flatten().tolist())
print("autograd db:", b.grad.item())
print("manual db:  ", manual_db.item())

autograd dW: [-0.39999985694885254, 0.7999997138977051]
manual dW:   [-0.39999985694885254, 0.7999997138977051]
autograd db: -0.39999985694885254
manual db:   -0.39999985694885254

Lab exercises#

  1. Change one model or data parameter and rerun the lab.

  2. Record whether the metric improved, worsened, or stayed roughly the same.

  3. Add one sentence connecting the result to Backpropagation and automatic differentiation.

  4. Identify one limitation of this toy setup before applying the idea to a real dataset.

# Reflection workspace
observation = ""
next_experiment = ""
print({"observation": observation, "next_experiment": next_experiment})

{'observation': '', 'next_experiment': ''}
Contents