Module 7 Lab: Generative vs discriminative

Contents

Module 7 Lab: Generative vs discriminative#

Contrast a discriminative classifier with a small generative autoencoder objective.

Run the setup cell, inspect the printed diagnostics, and then complete the exercises at the end. The lab is intentionally small enough to run in GitHub Codespaces without a GPU.

import torch
from torch import nn

torch.manual_seed(17)
X = torch.randn(180, 2)
y = ((X[:, 0] ** 2 + X[:, 1]) > 0.6).long()

classifier = nn.Sequential(nn.Linear(2, 12), nn.ReLU(), nn.Linear(12, 2))
autoencoder = nn.Sequential(nn.Linear(2, 4), nn.ReLU(), nn.Linear(4, 1), nn.Linear(1, 4), nn.ReLU(), nn.Linear(4, 2))
cls_opt = torch.optim.Adam(classifier.parameters(), lr=0.04)
ae_opt = torch.optim.Adam(autoencoder.parameters(), lr=0.04)
cls_loss = nn.CrossEntropyLoss()
ae_loss = nn.MSELoss()

for _ in range(120):
    cls_opt.zero_grad()
    loss_c = cls_loss(classifier(X), y)
    loss_c.backward()
    cls_opt.step()
    ae_opt.zero_grad()
    loss_a = ae_loss(autoencoder(X), X)
    loss_a.backward()
    ae_opt.step()

print(f"classifier accuracy: {(classifier(X).argmax(1) == y).float().mean().item():.3f}")
print(f"autoencoder reconstruction MSE: {ae_loss(autoencoder(X), X).item():.3f}")

classifier accuracy: 0.994
autoencoder reconstruction MSE: 0.364

Lab exercises#

  1. Change one model or data parameter and rerun the lab.

  2. Record whether the metric improved, worsened, or stayed roughly the same.

  3. Add one sentence connecting the result to Generative models and applications.

  4. Identify one limitation of this toy setup before applying the idea to a real dataset.

# Reflection workspace
observation = ""
next_experiment = ""
print({"observation": observation, "next_experiment": next_experiment})

{'observation': '', 'next_experiment': ''}
Contents