Machine perception

McCulloch and Pitts did not produce a trainable neural network, but they connected nervous-system behaviour to computation. The artificial neuron became a formal building block: it accepts signals, compares their sum with a threshold and produces an output. That abstraction survived for decades and still underpins how neural networks are described.

The perceptron added the key practical property to the formal neuron: learning from data. Its algorithm corrected weights after an error and gradually built a linear boundary between classes. Its capabilities were limited, but the modern loop of data, error and parameter update begins here.

Backpropagation made multilayer training computationally practical: output error was successively converted into updates for every layer. Versions of the method existed earlier, but the 1986 work established it as a general way to learn internal representations rather than only tune an output layer.

LeNet combined convolutions, shared weights and backpropagation in a practical document-recognition system. It was an important bridge from laboratory idea to industrial use: the network learned local visual features itself and operated on a real stream of cheques.

AlexNet did not invent neural networks, but it combined three ingredients at working scale: a large labelled dataset, a deep convolutional architecture and GPU training. The result made feature extraction part of learning and became the practical starting point of the modern deep-learning boom.

Word2vec showed that useful relationships between words could emerge from predicting context without a hand-built feature dictionary. Vector representations became a transferable internal language for neural networks and helped prepare the move from task-specific classifiers to large pretrained models.

Seq2Seq separated input encoding from output generation and showed that a learned system could translate without hand-written linguistic rules. The bottleneck of a fixed memory vector later helped motivate attention.