Landmark architectures

by @stats385 · 10 nodes · curated

AlexNet through ResNet plus Dropout, BatchNorm, visualization, and the rethinking-generalization paper.

LeNet-5 (LeCun et al.) yann.lecun.com · Sep 24 yann.lecun.com · Sep 24

Pioneering digit CNN — historical root of modern convnets; banks used it for checks.

Generative Adversarial Networks (Goodfellow et al.) arxiv.org · Sep 24 arxiv.org · Sep 24

GANs as a landmark generative architecture — course ‘extra’ but culturally essential.

Understanding Deep Learning Requires Rethinking Generalization arxiv.org · Sep 24 arxiv.org · Sep 24

Zhang et al. — nets can memorize random labels; forced a rethink of classical generalization.

Zeiler & Fergus — Visualizing CNNs cs.nyu.edu · Sep 24 cs.nyu.edu · Sep 24

Classic visualization paper — how to see what conv layers learn.

Batch Normalization arxiv.org · Sep 24 arxiv.org · Sep 24

Stabilized deep training by reducing internal covariate shift; changed how nets are built.

Dropout (Srivastava et al.) jmlr.org · Sep 24 jmlr.org · Sep 24

Simple regularization that prevents co-adaptation — still a default trick students should know.

ResNet — Deep Residual Learning arxiv.org · Sep 24 arxiv.org · Sep 24

Residual blocks unlocked 100+ layer nets; still the mental model for modern architectures.

GoogLeNet / Inception arxiv.org · Sep 24 arxiv.org · Sep 24

Going Deeper with Convolutions — multi-scale inception modules and careful compute budgeting.

VGG — Very Deep Convolutional Networks arxiv.org · Sep 24 arxiv.org · Sep 24

Showed depth with small 3×3 filters; became the default feature backbone for years.

AlexNet (Krizhevsky, Sutskever, Hinton) papers.nips.cc · Sep 24 papers.nips.cc · Sep 24

2012 ImageNet breakthrough — launched the modern deep CNN era. Know the architecture sketch.