Module 5 Lab: Sequence prediction

Contents

Open in Colab

Student guidance: If you are new to Python, start here. In Colab, choose Runtime > Run all, then change one small value in the lab and write what changed. Use Codespaces only when your instructor asks you to work with the full course repository.

Module 5 Lab: Sequence prediction#

Train a compact recurrent model to predict a trend from ordered observations.

Run the setup cell, inspect the printed diagnostics, and then complete the exercises at the end. The lab is intentionally small enough to run in GitHub Codespaces without a GPU.

import torch
from torch import nn

torch.manual_seed(15)
steps = torch.linspace(0, 1, 7)
X = torch.stack([steps + 0.05 * torch.randn(7) for _ in range(80)]).unsqueeze(-1)
y = (X[:, -1, 0] > X[:, 0, 0]).float().unsqueeze(1)

model = nn.Sequential()
rnn = nn.LSTM(input_size=1, hidden_size=10, batch_first=True)
head = nn.Linear(10, 1)
opt = torch.optim.Adam(list(rnn.parameters()) + list(head.parameters()), lr=0.04)
loss_fn = nn.BCEWithLogitsLoss()

for _ in range(100):
    opt.zero_grad()
    out, _ = rnn(X)
    logits = head(out[:, -1, :])
    loss = loss_fn(logits, y)
    loss.backward()
    opt.step()

with torch.no_grad():
    pred = (torch.sigmoid(logits) > 0.5).float()
    acc = (pred == y).float().mean().item()
print(f"trend accuracy: {acc:.3f}")
print("last hidden state shape:", tuple(out[:, -1, :].shape))

trend accuracy: 1.000
last hidden state shape: (80, 10)

Lab exercises#

  1. Change one model or data parameter and rerun the lab.

  2. Record whether the metric improved, worsened, or stayed roughly the same.

  3. Add one sentence connecting the result to Sequence models: RNNs and LSTMs.

  4. Identify one limitation of this toy setup before applying the idea to a real dataset.

# Reflection workspace
observation = ""
next_experiment = ""
print({"observation": observation, "next_experiment": next_experiment})

{'observation': '', 'next_experiment': ''}
Contents