Module 5 Lab: Sequence prediction

Contents

Module 5 Lab: Sequence prediction#

Train a compact recurrent model to predict a trend from ordered observations.

Run the setup cell, inspect the printed diagnostics, and then complete the exercises at the end. The lab is intentionally small enough to run in GitHub Codespaces without a GPU.

import torch
from torch import nn

torch.manual_seed(15)
steps = torch.linspace(0, 1, 7)
X = torch.stack([steps + 0.05 * torch.randn(7) for _ in range(80)]).unsqueeze(-1)
y = (X[:, -1, 0] > X[:, 0, 0]).float().unsqueeze(1)

model = nn.Sequential()
rnn = nn.LSTM(input_size=1, hidden_size=10, batch_first=True)
head = nn.Linear(10, 1)
opt = torch.optim.Adam(list(rnn.parameters()) + list(head.parameters()), lr=0.04)
loss_fn = nn.BCEWithLogitsLoss()

for _ in range(100):
    opt.zero_grad()
    out, _ = rnn(X)
    logits = head(out[:, -1, :])
    loss = loss_fn(logits, y)
    loss.backward()
    opt.step()

with torch.no_grad():
    pred = (torch.sigmoid(logits) > 0.5).float()
    acc = (pred == y).float().mean().item()
print(f"trend accuracy: {acc:.3f}")
print("last hidden state shape:", tuple(out[:, -1, :].shape))

trend accuracy: 1.000
last hidden state shape: (80, 10)

Lab exercises#

  1. Change one model or data parameter and rerun the lab.

  2. Record whether the metric improved, worsened, or stayed roughly the same.

  3. Add one sentence connecting the result to Sequence models: RNNs and LSTMs.

  4. Identify one limitation of this toy setup before applying the idea to a real dataset.

# Reflection workspace
observation = ""
next_experiment = ""
print({"observation": observation, "next_experiment": next_experiment})

{'observation': '', 'next_experiment': ''}
Contents