Neural Units

Neural Units#

  • Neuron: weighted sum + activation

  • Layers compose representations

  • Depth enables hierarchical features

Multilayer Perceptrons#

  • Input → hidden → output

  • Nonlinear activations (ReLU, GELU)

  • Universal approximation in principle; data and optimization matter in practice