The Credit Assignment Problem: From Rosenblatt's Perceptron to Backpropagation to Quantum Gradients

The single hardest problem in machine learning, the one every other problem reduces to, is this:

When a learning system makes a mistake, which of its many internal parameters should be blamed, and by how much?

Minsky named the credit assignment problem in 1961 (Steps Toward Artificial Intelligence, Proc. IRE 49(1)). It’s worth being honest about what he meant: Minsky was writing about reinforcement learning–style temporal credit — apportioning blame across a sequence of actions to a delayed reward — not the structural/depth credit that backpropagation later solved. The generalization of the term to cover “any apportionment of blame across the internal state of a learning machine” is something the field did over the next few decades, not something Minsky proposed. I’ll use the broader meaning throughout this post because it is genuinely useful — every algorithm we use to train neural networks, from Rosenblatt’s perceptron rule in 1958 to the trillion-parameter optimizers of 2026, is in the end a particular answer to a credit-assignment problem in this generalized sense. So is reinforcement learning’s original temporal-credit problem. So are the more exotic biologically-plausible alternatives that periodically threaten to displace backpropagation. And so, increasingly, are the quantum analogues being developed under the heading of variational quantum machine learning.

This post walks the lineage. I want to make clear (a) what the original perceptron rule actually was, (b) why backpropagation took roughly two decades to be recognized as the answer, (c) what the modern engineering stack around backprop is — because the algorithm in isolation does not scale, and the literature on what makes it scale is large and underread — (d) what serious alternatives have been proposed and where they actually win, (e) how reinforcement learning poses the same problem in time rather than depth, and (f) whether quantum computers can do this faster, slower, or at all.

Conferences I’ll lean on for citations: NeurIPS, ICML, ICLR, with detours to Nature, Science, COLT, and a few key arXiv preprints. Per the format of my other survey posts, every inline reference to an arXiv ID is a clickable link.

Table of contents

1. The Perceptron and the Original Credit Assignment Rule

Frank Rosenblatt introduced the perceptron in The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain (Psychological Review, 1958). The model is now banal — a single linear threshold unit

$y = \text{sign}\left(\sum_{i} w_i x_i - \theta\right)$

with a learning rule

$\Delta w_i = \eta (t - y) x_i$

— but the credit assignment story it embedded is the one we still tell. The error $(t - y)$ is observed at the output. The rule attributes that error to each weight $w_i$ proportionally to the input $x_i$ that the weight multiplied. There is no notion of internal structure to credit, because there are no hidden units. This is what makes the perceptron rule both elegant and, ultimately, dead-end as a general learning theory.

Rosenblatt’s Principles of Neurodynamics (Spartan Books, 1962) is the book-length treatment. It already contains multilayer perceptron architectures, with sensory (S) units, association (A) units, and response (R) units. Crucially, the A-units’ connections to the sensory layer were fixed random projections — only the R-unit weights were trained. So Rosenblatt had the multilayer architecture but not a learning algorithm for the internal connections. The credit assignment problem for hidden weights was named in that book without being solved, and that is the stronger version of the historical claim. Every subsequent line of work in this post is, more or less, an answer to it.

2. Minsky and Papert: The Limits Made Rigorous

Marvin Minsky and Seymour Papert’s Perceptrons (MIT Press, 1969) is remembered mostly for one result: a single-layer perceptron cannot compute XOR. The actual book is much more careful. It proves limitation theorems about what order-bounded perceptrons (those whose predicates each look at only $k$ inputs) can and cannot decide — connectedness of an image, in particular, is famously beyond any finite-order perceptron.

The book is often blamed for the AI winter of the 1970s, which is partly unfair: Minsky and Papert explicitly note in their epilogue that the limitations they prove are limitations of bounded-order perceptrons, and they speculate that multilayer machines might do more. What the book did not provide was an algorithm for training those multilayer machines. The credit assignment problem for hidden weights remained open through the 1970s, and funding for neural network research evaporated in the absence of a solution. Whether Perceptrons caused the winter or merely diagnosed it is a debate historians of AI still have; the most decisive treatment is Mikel Olazaran’s A Sociological Study of the Official History of the Perceptrons Controversy, Social Studies of Science 26(3):611–659, 1996.

3. Backpropagation’s Long Discovery, 1970–1986

The standard story is that backpropagation was invented by Rumelhart, Hinton, and Williams in 1986. The actual chronology is much longer.

• 1970. Seppo Linnainmaa, then a master’s student in Helsinki, described reverse-mode automatic differentiation in his thesis The Representation of the Cumulative Rounding Error of an Algorithm as a Taylor Expansion of the Local Rounding Errors (University of Helsinki). The algorithm computes the gradient of an arbitrary differentiable composition $f = f_L \circ \cdots \circ f_1$ at a cost that is a small constant multiple — typically 2–3× — of a forward evaluation in FLOPs. This is the algorithm we now call backpropagation. Linnainmaa did not apply it to neural networks; he was studying numerical round-off. The “small constant” matters: it is what makes backprop the asymptotically optimal answer to credit assignment among gradient-based algorithms, and it is the wall that forward-mode AD and other alternatives keep failing to break.
• 1974. Paul Werbos, in his Harvard PhD thesis Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, derived the same algorithm specifically for training multilayer networks. Werbos’s thesis was largely ignored for a decade — not because it was unread, but because of an audience mismatch: he framed the work as nonlinear sensitivity analysis for social-science forecasting, not pattern recognition. The neural network research community of the mid-1970s would not have known to look for backpropagation in a behavioral-sciences econometrics thesis. Werbos has written about this himself in retrospectives. The lesson worth absorbing is that the right algorithm in the wrong language can sit unread for a decade.
• 1982. Werbos’s Applications of advances in nonlinear sensitivity analysis (in System Modeling and Optimization, Springer LNCIS 38) re-published the result for the controls community.
• 1985. David Parker, then at MIT, independently rediscovered backprop and described it in Learning-Logic: Casting the Cortex of the Human Brain in Silicon, a 1985 technical report from the Sloan School / Center for Computational Research in Economics and Management Science. Yann LeCun, then a graduate student in Paris, described a closely related algorithm in the conference paper Une procédure d’apprentissage pour réseau à seuil asymétrique in the Proceedings of Cognitiva 85 (Paris, June 1985, pp. 599–604). LeCun’s PhD thesis is two years later (Université Pierre et Marie Curie, 1987); the 1985 piece is the conference paper, not a thesis.
• 1986. David Rumelhart, Geoffrey Hinton, and Ronald Williams published Learning representations by back-propagating errors in Nature (323, 533–536). This is the paper that made backpropagation famous. The longer treatment, with the autoencoder demonstration, is in the PDP volumes, Volume 1, Chapter 8.

The Rumelhart-Hinton-Williams paper did not “invent” backpropagation in any narrow technical sense. What it did was:

1. Frame the algorithm as a solution to the credit assignment problem for internal representations. The Nature paper’s hero result is the family-tree task: the network learns to predict relationships (“uncle,” “niece”) in two parallel kinship trees, and the hidden-unit representations spontaneously encode high-level features like “generation” and “branch of family” without ever being told to. Credit is being assigned to features, not just weights.
2. The 6-3-6 autoencoder demonstration — discovering a binary code in three hidden units — is in the longer PDP chapter, not the Nature paper. It is the cleaner toy demonstration of the same point.
3. Frame the whole thing in the language of learning rather than control or sensitivity analysis, which is what made the algorithm legible to cognitive scientists and AI researchers rather than just the engineering community Werbos had been writing for.

Hinton has subsequently said in many talks that the paper’s contribution was less the algorithm than the demonstration of what backpropagation could do with hidden layers. That is a credit-assignment story about credit assignment.

A useful retrospective on the multiple discoveries is Jürgen Schmidhuber’s Deep Learning in Neural Networks: An Overview (Neural Networks, 2015) — he has been the most insistent historian on the point that Linnainmaa (1970) and Werbos (1974) preceded RHW (1986), and the chronology there is meticulous.

4. Why It Took So Long to Scale: The Engineering Stack of Backprop

Backpropagation as an algorithm was known by 1986. Backpropagation as a technology that trains 175-billion-parameter language models required another three decades of engineering. The intermediate work is often presented as a list of architectural tricks; it is more usefully read as a sequence of solutions to credit-assignment pathologies that emerge at scale.

Vanishing and exploding gradients. Sepp Hochreiter’s 1991 diploma thesis Untersuchungen zu dynamischen neuronalen Netzen (TU Munich, advisor Schmidhuber) identified the core obstacle: in a deep recurrent network trained by backprop-through-time, the gradient signal decays (or explodes) exponentially with the number of timesteps. Without a fix, credit cannot propagate back through long temporal dependencies at all. The vanishing-side fix that took: Hochreiter and Schmidhuber’s Long Short-Term Memory (Neural Computation, 1997), which introduces gated linear paths through time so that gradient magnitudes are preserved — but the 1997 LSTM did not yet include a forget gate, and in practice failed on long sequences for reasons related to that. The forget gate is Gers, Schmidhuber, and Cummins’s Learning to Forget: Continual Prediction with LSTM (Neural Computation 12(10), 2000), and the Learning to Forget LSTM is the variant the field actually used for the next 17 years. The exploding side has its own canonical fix — Pascanu, Mikolov, and Bengio’s On the difficulty of training recurrent neural networks (ICML 2013) introduced gradient norm clipping, which is now a default in essentially every modern training loop. The LSTM-plus-clipping pairing is what made sequence-to-sequence translation work in 2014 and what dominated NLP until transformers arrived.

Initialization. Backprop on deep MLPs with naive Gaussian weight initialization almost never works. Xavier Glorot and Yoshua Bengio’s Understanding the difficulty of training deep feedforward neural networks (AISTATS 2010) gave the analysis — preserve variance of activations and gradients across layers — and the prescription that is still used: $W \sim \mathcal{U}[-\sqrt{6/(n_{\text{in}}+n_{\text{out}})}, +\sqrt{6/(n_{\text{in}}+n_{\text{out}})}]$. Kaiming He et al.’s Delving Deep into Rectifiers (ICCV 2015) generalized this for ReLU activations. The reason these matter for credit assignment: at the wrong initialization scale, the Jacobians of the chain rule either explode the error signal or kill it before it reaches the lower layers, and backprop on a dead-gradient network is mathematically running but learning nothing.

Activation functions. The ReLU has a longer history than most modern accounts acknowledge. Kunihiko Fukushima’s Neocognitron (Biological Cybernetics, 1980) used a piecewise-linear half-rectifier as its non-linearity from the start. Nair and Hinton’s Rectified Linear Units Improve Restricted Boltzmann Machines (ICML 2010) reintroduced it to deep networks, and Glorot, Bordes, and Bengio’s Deep Sparse Rectifier Neural Networks (AISTATS 2011) is the paper that actually convinced the community: rectifiers beat sigmoids and tanhs for deep MLPs across a range of benchmarks. The ReLU’s piecewise-linear nature gives gradients that are exactly 0 or 1, which keeps the Jacobian product from collapsing — a credit-assignment-preserving non-linearity, in retrospect.

Optimizers. Stochastic gradient descent is the bare backprop recipe. The optimizers that actually train modern networks form two lineages. The first is the second-moment-adaptive line — AdaGrad (Duchi, Hazan, Singer, JMLR 2011) introduced per-parameter learning rates from accumulated gradient magnitudes; RMSProp (Hinton lecture notes, 2012) replaced the unbounded accumulation with an exponentially-decayed second moment; Adam (Kingma and Ba, ICLR 2015) added bias-corrected first-moment momentum to the RMSProp second moment; AdamW (Loshchilov and Hutter, ICLR 2019) decoupled weight decay from the gradient update, which turned out to matter for large transformers. Each step in that chain fixes a specific failure mode of the previous one.

The post-Adam frontier has split into two camps. Lion (Chen et al., NeurIPS 2023, an evolutionary-search-discovered optimizer whose name stands for “EvoLved sign momentum”) uses only the sign of momentum and is memory-cheaper than Adam at comparable performance. Shampoo (Gupta, Koren, Singer, ICML 2018) introduced full-matrix preconditioning via Kronecker-factored second-order information; the practical version is Distributed Shampoo (Anil, Gupta, Koren, Regan, Singer, 2020), and the variant that bridged it into mainstream use is SOAP (Vyas et al., 2024), which applies Shampoo as a preconditioner inside Adam. Muon — Keller Jordan’s original blog write-up (October 2024) and the Moonshot scaling study (Liu et al., 2025, in the Kimi K2 training run) — uses a Newton–Schulz orthogonalization of momentum and has been rapidly adopted at several frontier labs after Kimi K2 demonstrated it at scale. Whether it is the new default outside that single demonstration is not yet publicly verifiable.

All of these optimizers are credit-assignment-rescalers: they take the raw gradient and reshape it parameter-by-parameter so that the update direction is closer to the natural gradient or some computationally cheap approximation of it.

Normalization. Sergey Ioffe and Christian Szegedy’s Batch Normalization (ICML 2015) keeps activations on a stable scale, which keeps the Jacobian on a stable scale, which keeps credit-flow stable. BatchNorm dominated CNN architectures from 2015 on. It also has a specific failure mode that is highly relevant to the next architectural wave: BatchNorm computes statistics over the batch dimension, which makes it (a) brittle for variable-length sequences, (b) brittle for small batch sizes, and (c) coupled across examples in ways that break some inference regimes. Ba, Kiros, and Hinton’s Layer Normalization (2016 arxiv) normalizes within an example rather than across the batch, which is exactly what variable-length sequence models need. This is the reason transformers use LayerNorm and CNNs use BatchNorm — not a stylistic choice but a direct response to BatchNorm’s batch-coupling failure modes in the sequence regime. RMSNorm (Zhang and Sennrich, NeurIPS 2019) is a further simplification (drop the mean-centering term) that most modern transformers now use.

Residual connections. Kaiming He et al.’s Deep Residual Learning for Image Recognition (CVPR 2016) introduced the identity skip $x \mapsto x + F(x)$, which makes the gradient through a deep network always have a path of magnitude 1. The credit assignment story: residuals turn the Jacobian product through depth into a Jacobian sum, so even when individual layer Jacobians are small or poorly conditioned the gradient has a non-vanishing shortcut to every layer. This is the single biggest depth-credit-assignment fix in the modern stack. Before ResNets, 50-layer networks were unstable; after ResNets, 1000-layer networks train. The architectural turning point of the modern era.

Mixed-precision training. Modern frontier-scale runs train weights in bf16 or fp16 with periodic fp32 master copies. Micikevicius et al.’s Mixed Precision Training (ICLR 2018) is the foundational reference. The credit-assignment story is direct: gradients in fp16 underflow before they reach the lower layers, killing credit-flow. The paper’s loss scaling trick multiplies the loss by a large constant before the backward pass and unscales gradients before the optimizer step — a literal credit-magnitude rescaling to prevent underflow.

The transformer. Vaswani et al.’s Attention Is All You Need (NeurIPS 2017) is, from a credit-assignment perspective, an architectural innovation that democratizes credit-flow: any output token can attend directly to any input token in a single layer, so the gradient path between any two tokens is $O(1)$ in sequence length rather than $O(n)$ as in an RNN. (The path is still $O(L)$ in network depth — there is no free lunch on number of layers — but it is constant in $n$, which is what handles long-range dependencies.) The credit assignment graph is flatter, not shorter, and that is the structural reason transformers handle long-range dependencies better than LSTMs.

It is worth saying this plainly: modern deep learning is not “backprop.” Modern deep learning is backprop plus an enormous stack of architectural and optimization choices that together make credit assignment well-conditioned at billion-parameter scale. Each item above is a fix for a specific credit-assignment pathology:

• Initialization keeps the Jacobian from being broken at step zero.
• Activations keep the Jacobian from collapsing to zero through depth.
• Normalization keeps the Jacobian’s scale from drifting across layers and steps.
• Residual connections turn the Jacobian product through depth into a Jacobian sum, providing a constant-magnitude shortcut to every layer.
• Gradient clipping prevents the exploding side of the same Jacobian-product problem.
• Mixed-precision loss scaling prevents arithmetic underflow from killing credit at low magnitudes.
• Adaptive optimizers reshape what comes out the other end so the update direction approximates the natural gradient rather than the raw Euclidean one.
• Transformer attention flattens the credit-assignment graph so sequence-length dependencies don’t compound through depth.

That’s the engineering stack. Read this way, the “lots of tricks” pile becomes a single coherent program: every layer of the stack is keeping some part of the credit-assignment graph well-conditioned. The practical state of the art is in their combination, not in any one of them.

5. Biologically-Implausible Backprop and Its Alternatives

Backpropagation has a long-running embarrassment: it does not look like anything the brain does. The standard list of biological-implausibility complaints, going back to Francis Crick’s The recent excitement about neural networks (Nature 337:129–132, January 1989):

1. Weight transport. Backprop’s reverse pass uses the same weights as the forward pass, transposed. There is no known biological mechanism for a synapse to “know” the value of the symmetric synapse going the other direction.
2. Symmetric forward/backward circuitry. The error signal in backprop travels along a perfectly mirrored network, which the brain does not appear to have.
3. Update locking. A layer cannot update until the full forward pass and the full backward pass have completed — a strong sequential bottleneck that biological synapses don’t seem to obey.
4. Differentiable non-linearities. Spiking neurons are not differentiable in any obvious sense.

A research program persisting since the 1990s has been to find learning algorithms that work on deep networks but avoid (1)–(4). The most-cited results:

Feedback alignment

Timothy Lillicrap, Daniel Cownden, Douglas Tweed, and Colin Akerman first posted Random synaptic feedback weights support error backpropagation for deep learning to arXiv in November 2014; the polished version appeared in Nature Communications in 2016. The result: you can replace the transposed forward weights in the backward pass with fixed random matrices and still train networks. The empirical observation is that the forward weights adjust themselves to align with the random feedback weights — credit assignment is approximate but works. This was startling. It directly addresses complaint (1).

The mechanism is worth one more sentence, because it’s the credit-assignment-quality-degrades-with-depth story in microcosm: the alignment between forward and feedback weights is only partial, and the alignment angle between the true backprop gradient and the feedback-alignment update degrades the deeper you go. That is precisely why FA trains MNIST cleanly, trains CIFAR-10 reasonably, and falls apart on ImageNet. Credit signal in FA is correct in direction near the output and increasingly noisy near the input.

Arild Nøkland’s Direct Feedback Alignment Provides Learning in Deep Neural Networks (NeurIPS 2016) went further: each layer receives error signal directly from the output via a fixed random matrix, no chain-rule propagation needed. This solves complaint (3) (update locking) as well, and Nøkland showed it trains MLPs on MNIST and CIFAR-10.

Sergey Bartunov et al.’s Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures (NeurIPS 2018) gave the decisive negative result on FA and target propagation as implemented at that time: both underperform backprop on ImageNet by a large margin. The honest 2026 reading, though, is not that the question is closed but that the field has split into two branches in response. The first branch tries harder to match backprop’s gradient using local rules — predictive coding (Whittington and Bogacz 2017; Millidge et al. 2022) and equilibrium propagation (Scellier and Bengio 2017; Laborieux et al. 2021) are the load-bearing entries here. The second branch abandons gradient-matching and uses alternative learning signals — Forward-Forward (Hinton 2022), PEPITA (Dellaferrera and Kreiman, ICML 2022), DRTP (Frenkel, Lefebvre, Bol, Frontiers in Neuroscience 2021), and activation relaxation (Millidge et al., 2020) belong here. Neither branch has yet beaten backprop at frontier scale. The biological-plausibility frontier has not yet beaten the engineering frontier at scale.

Target propagation

Yoshua Bengio’s How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation is from July 2014 and was never formally published — it remained an arXiv preprint, which is itself historically interesting: Bengio floated the framing, the community picked it up via D.-H. Lee, Zhang, Fischer, and Bengio’s Difference Target Propagation (ECML PKDD 2015), and the original framing paper stayed off the peer-reviewed record. The reframing: instead of propagating error gradients, propagate target activations — each layer is trained to produce activations close to a target computed by inverting the layer above. Inversion is approximated by learned auto-encoders. Difference target propagation handles the inversion error.

This addresses the biological-plausibility complaints because the inversion can in principle be learned by local connections, and there is no global weight-transport requirement.

Equilibrium propagation

Benjamin Scellier and Yoshua Bengio’s Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation (Frontiers in Computational Neuroscience, 2017) is one of the most beautiful proposals in this area. It applies to energy-based networks (continuous Hopfield nets). The training procedure has two phases:

• Free phase. Network relaxes to a fixed point given the input.
• Clamped phase. Output units are slightly nudged toward the target; the network relaxes to a new fixed point.

The local synaptic update is proportional to the difference in pre/post-synaptic correlations between the two phases. Scellier and Bengio prove that in the limit of small nudging, this update equals the backpropagation gradient. No weight transport. No separate backward circuitry. Updates are local and Hebbian-like. This is mathematically the cleanest biologically-plausible alternative.

There is a subtle catch in the “small nudging” limit: the gradient estimator from a single $\beta > 0$ run is biased, and that bias grows with depth — which is precisely why early EqProp scaled poorly past toy networks. Laborieux et al.’s Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing Its Gradient Estimator Bias (Frontiers in Neuroscience, 2021) introduces a symmetric estimator using both $+\beta$ and $-\beta$ relaxations, which cancels the leading-order bias and makes EqProp competitive with backprop on CIFAR-10 with deep ConvNets. The connect-the-dots story: the small-$\beta$ limit creates a bias that grows with depth, the symmetric estimator removes the leading-order term, and this is exactly the engineering-of-the-credit-assignment-algorithm story that section 4 told for vanilla backprop. Laydevant, Markovic, and Grollier’s Training an Ising machine with equilibrium propagation (Nature Communications, 2024) is the cleanest recent hardware demonstration: an Ising machine trained by EqProp without ever computing a gradient symbolically. It has not yet been demonstrated at ImageNet scale, but the prospect of an algorithm that natively runs on analog physics is one of the few reasons to think a non-backprop algorithm might overtake backprop — not because it learns better, but because the underlying hardware costs less.

Predictive coding

Predictive coding originates outside machine learning entirely: Rajesh Rao and Dana Ballard’s Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects (Nature Neuroscience 2(1):79–87, January 1999) was a neural-coding theory, not a learning algorithm — an account of why visual-cortex receptive fields look the way they do. The reframing of predictive coding as a backprop-approximating learning algorithm is James Whittington and Rafal Bogacz’s An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity (Neural Computation, 2017), almost twenty years later. Beren Millidge et al.’s Predictive Coding Approximates Backprop along Arbitrary Computation Graphs (Neural Computation, 2022) generalizes the result: predictive coding networks of arbitrary topology compute the same gradients as backprop, with local Hebbian updates and no weight transport, in the limit of converged predictive-coding inference.

Predictive coding is, in the 2026 picture, the entry with the cleanest “this is how the cortex might actually do it” story — the inference dynamics are a known motif of cortical microcircuits, and the learning rule is local and Hebbian. Feedback alignment is the historically important entry; predictive coding is the currently-most-defensible-as-cortical-mechanism entry. They are doing different rhetorical jobs and both belong in the discussion.

The Salvatori-Buckley-Tschantz-Millidge cluster (the Oxford / VERSES predictive-coding group) has a 2022–2024 sequence of scaling results: Salvatori et al.’s Learning on Arbitrary Graph Topologies via Predictive Coding (NeurIPS 2022) is the load-bearing entry for the “approximates backprop on arbitrary computation graphs” thread, and Predictive Coding beyond Gaussian Distributions (NeurIPS 2022) extends it past the Gaussian-noise setting. These show predictive coding can train transformers and CNNs at small-to-mid scale. Frontier-scale scaling remains an open question.

Forward-Forward

Geoffrey Hinton’s The Forward-Forward Algorithm: Some Preliminary Investigations (presented as an invited talk at NeurIPS 2022; the paper remains an arXiv preprint and has not been peer-reviewed) is the most recent proposal in this lineage from backprop’s main popularizer. It replaces the forward-backward structure with two forward passes: one over “positive” (real) data, one over “negative” (corrupted) data. Each layer is trained locally to have high activation on positive data and low activation on negative data — a layer-wise goodness function, no global error signal at all.

Forward-Forward has not displaced backprop. The interesting thing about Hinton’s paper is that it is honest about that — he explicitly frames it as a proposal worth investigating because of biological plausibility and the potential for analog hardware that cannot compute gradients but can compute forward passes very cheaply. Subsequent work, including Tosato et al.’s A Comparative Analysis of the Forward-Forward Algorithm (2024) and several ICLR 2025 follow-ups, has refined the algorithm but not pushed it beyond small-scale benchmarks. There is a nice rhetorical arc in the fact that the popularizer of backpropagation has spent the last four decades openly searching for an alternative.

Surrogate gradients for spiking networks

Complaint (4) in the list above — backprop assumes differentiable non-linearities, spiking neurons aren’t — has a clean production-grade answer that the bio-plausible literature mostly skips over. Emre Neftci, Hesham Mostafa, and Friedemann Zenke’s Surrogate Gradient Learning in Spiking Neural Networks (IEEE Signal Processing Magazine 36(6):51–63, 2019) is the canonical reference: in the forward pass, neurons spike (a non-differentiable threshold function); in the backward pass, you substitute a smooth surrogate function (a fast sigmoid, a piecewise-linear ramp, etc.) for differentiation. This pretend-the-spike-is-differentiable trick is the credit-assignment hack that makes spiking-network training tractable, and it is what every modern neuromorphic-hardware training pipeline (Intel Loihi, IBM TrueNorth descendants, BrainScaleS, SpiNNaker) actually uses in practice. It is biologically motivated in the sense that the surrogate corresponds to a stochastic-spike approximation, but it is, mechanically, just backprop with an arithmetic substitution.

Closing synthesis

The approximators (predictive coding, equilibrium propagation) have stronger theory and weaker scaling. The replacers (forward-forward, target prop variants, PEPITA) have weaker theory and weaker scaling. Backprop remains the only credit-assignment procedure that has been demonstrated to scale to frontier-model regimes. That is a fact about engineering economics, not a fact about brains. If a non-backprop algorithm displaces backprop, it will be because the underlying hardware costs less — Ising machines, analog crossbar arrays, neuromorphic chips, quantum systems doing Hamiltonian relaxation — not because the algorithm computes a better gradient. The same logic carries directly into the quantum discussion in section 8: a quantum credit-assignment algorithm doesn’t have to learn better than backprop, only run cheaper or differently.

6. Synthetic Gradients and Decoupled Training

A different attack on the update locking problem of backprop — closer in spirit to engineering than biology — is to learn an approximation of the gradient.

Max Jaderberg et al.’s Decoupled Neural Interfaces using Synthetic Gradients (ICML 2017) introduces this idea: each layer has a small auxiliary network that predicts the gradient flowing into it from the layer above. This synthetic gradient can be used to update the layer immediately, without waiting for the full backward pass. The synthetic-gradient networks are themselves trained, in a bootstrap loop, on the real gradient when it eventually arrives.

The result: layers can be trained asynchronously, recurrent networks can backprop through (nominally) infinite time, and pipeline parallelism becomes trivial. The technique is used in some industrial pipeline-parallel training setups, although deterministic pipeline-parallelism schedules have absorbed most of the same engineering need — GPipe (Huang et al., NeurIPS 2019), PipeDream (Narayanan et al., SOSP 2019), interleaved 1F1B in Megatron-LM (Narayanan et al., 2021), and most recently Zero Bubble Pipeline Parallelism (Qi et al., ICLR 2024) which eliminates pipeline bubbles by splitting the backward pass into two parts. The Zero-Bubble line is what runs in production now; GPipe and PipeDream are the historical named entrants. Synthetic gradients remain a more theoretically interesting answer to the credit-locking problem, but the practical answer in 2026 is to schedule the bubbles out, not to predict the gradient.

A related line is local greedy training — Belilovsky et al.’s Greedy Layerwise Learning Can Scale to ImageNet (ICML 2019) and Löwe et al.’s Greedy InfoMax (NeurIPS 2019) — which trains each layer with a local objective and avoids the global backward pass entirely. These methods underperform backprop on supervised tasks but are surprisingly competitive on self-supervised representation learning.

A more recent revival of the “compute gradients without backprop” theme is Baydin et al.’s Gradients without Backpropagation (2022) and the follow-up forward-gradient line, which uses forward-mode automatic differentiation with random tangent directions to obtain unbiased gradient estimators in a single forward pass — at the cost of high variance, scaling with the number of parameters. The headline appeal is that the algorithm never needs to store intermediate activations. Empirical follow-ups in 2023–2025 have shown forward gradients can train moderate-sized networks but variance reduction at scale is the unsolved problem. Worth tracking as a possible alternative for memory-constrained training regimes.

7. Temporal Credit Assignment: Reinforcement Learning’s Version of the Problem

Credit assignment in supervised learning is structural: how do we apportion blame across the depth of a network? In reinforcement learning, the problem is temporal: how do we apportion blame across time, when an action taken now produces a reward many steps later? This is the original credit assignment problem in Minsky’s 1961 formulation, and it remains active research.

The same engineering-stack frame from section 4 works here. Each algorithm below is a fix for a specific temporal-credit-assignment pathology.

Value-based methods: the bias-variance trade. Pure Monte Carlo (sample full trajectories, attribute final return to every action taken) is unbiased but has unbounded variance for long horizons. Pure one-step temporal-difference learning bootstraps from the next-step value estimate, dramatically reducing variance but introducing bias from the imperfect value function. Richard Sutton’s Learning to Predict by the Methods of Temporal Differences (Machine Learning 3(1):9–44, 1988) introduced TD($\lambda$), which interpolates between the two via a geometric mixing of $n$-step returns. The associated eligibility trace machinery — retroactively credit states/actions in proportion to how recently they were visited — has a longer lineage: A. Harry Klopf’s 1972 Air Force technical report Brain Function and Adaptive Systems — A Heterostatic Theory, then Barto, Sutton, and Anderson’s Neuronlike adaptive elements that can solve difficult learning control problems (IEEE Trans. SMC 13(5), 1983), with the canonical modern treatment in Sutton and Barto’s Reinforcement Learning: An Introduction, 2nd ed. (MIT Press, 2018).

Policy gradients: the score-function lineage. Ronald Williams’s Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning (Machine Learning 8:229–256, 1992) is the foundational paper of the modern RL canon and is missing from too many summaries. REINFORCE assigns credit to actions via the log-likelihood ratio weighted by the return: $\nabla J(\theta) = \mathbb{E}[\nabla \log \pi_\theta(a|s) \cdot R]$. This is unbiased but high-variance — the score-function estimator pays the same Monte Carlo variance cost as raw return-based credit assignment. Every modern policy-gradient method, including PPO and GRPO, is a variance-reduced version of Williams 1992.

The variance-reduction trick that took: Generalized Advantage Estimation — Schulman, Moritz, Levine, Jordan, and Abbeel’s High-Dimensional Continuous Control Using Generalized Advantage Estimation (ICLR 2016). GAE replaces the raw return with a $\lambda$-weighted sum of TD residuals against a learned value baseline, giving the same bias-variance dial as TD($\lambda$) but in advantage form, compatible with policy gradients. GAE is the workhorse of PPO and almost every modern actor-critic method.

Proximal Policy Optimization — Schulman, Wolski, Dhariwal, Radford, and Klimov’s Proximal Policy Optimization Algorithms (2017 arXiv, never formally published in a venue) — fixes a different credit-assignment pathology: vanilla policy gradient updates that move too far in parameter space per step destabilize the value function and the credit-assignment signal it provides. PPO’s clipped objective constrains the step size in policy-ratio space. It is the most-deployed RL algorithm of the modern era.

RLHF and reward-model credit assignment. When the environment provides no reward, the reward signal itself becomes something a model has to learn — and credit assignment now operates over an approximated reward landscape rather than a ground-truth one. Paul Christiano et al.’s Deep Reinforcement Learning from Human Preferences (NeurIPS 2017) is the foundational paper; Ouyang et al.’s Training Language Models to Follow Instructions with Human Feedback (InstructGPT, NeurIPS 2022) is what dragged the approach into the LLM era. The unhappy consequence: when credit is assigned against a learned reward model, the policy can find regions where the reward model is wrong and exploit them. Gao, Schulman, and Hilton’s Scaling Laws for Reward Model Overoptimization (ICML 2023) characterized this reward over-optimization quantitatively, showing reward-model goodness and policy goodness diverge predictably past a certain KL distance from the SFT policy. Reward hacking is a credit-assignment problem in which the assigner is itself the bug.

GRPO and the reasoning-RL era. Shao et al.’s DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (February 2024) introduced Group Relative Policy Optimization — replace PPO’s learned value baseline with a group-relative baseline computed from multiple completions sampled for the same prompt. The motivation was practical: PPO’s value baseline is expensive to learn well on language tasks, and a group-mean baseline is a cheap, low-variance alternative when the prompt is fixed and only completions vary. GRPO is the algorithm that did the heaviest temporal-credit-assignment lifting in the reasoning-RL wave of 2024–2025 (DeepSeek-R1 being the landmark demonstration), and it is arguably the most consequential post-PPO RL algorithm. A nuance worth flagging: GRPO’s credit assignment is trajectory-level, not per-token — the entire completion shares one advantage. Whether per-token credit assignment is needed for long-chain reasoning, and whether the answer should come from learned value baselines, process reward models, or attention-mediated credit, is an active 2025–2026 research front.

Hindsight and counterfactual credit assignment. The hindsight family uses future information to reconstruct past credit. Andrychowicz et al.’s Hindsight Experience Replay (NeurIPS 2017) relabels failed trajectories with the goal they actually achieved, turning every attempt into a useful learning signal; this is the cleanest sparse-reward hack in the literature. Harutyunyan et al.’s Hindsight Credit Assignment (NeurIPS 2019) uses the future outcome to compute credit for past actions in a counterfactual sense. Mesnard et al.’s Counterfactual Credit Assignment in Model-Free Reinforcement Learning (ICML 2021) constructs unbiased lower-variance gradient estimators via learned future-conditional value baselines, particularly useful for partially observed MDPs. Raposo et al.’s Synthetic Returns for Long-Term Credit Assignment (2021 arXiv) predicts synthetic returns from observation pairs as an explicit credit-assigner trained alongside the value function, demonstrating that hindsight inference can solve credit assignment in sparse-reward settings where vanilla policy gradients can’t. Jürgen Schmidhuber’s group has remained on the credit-assignment beat across this whole period — Arjona-Medina, Gillhofer, Widrich, Unterthiner, Brandstetter, and Hochreiter’s RUDDER: Return Decomposition for Delayed Rewards (NeurIPS 2019) reformulates reward as return contributions and learns to redistribute reward back to the timesteps that caused it. The author list is a nice arc: the same Hochreiter who diagnosed vanishing gradients in 1991 is still solving credit-assignment problems 28 years later.

Transformers as credit-assigners. Decision Transformer (Chen et al., NeurIPS 2021) and Trajectory Transformer (Janner et al., NeurIPS 2021) recast offline RL as sequence modeling — instead of computing a value function to bridge actions and far-future rewards, you let the transformer’s attention layers learn which past actions to attend to when predicting future returns. Attention is a credit-assignment mechanism. The transformer is one of the few architectures that handles both spatial/structural and temporal credit assignment with the same primitive, which is part of why it has displaced bespoke RL machinery for offline RL on benchmark tasks.

For long-horizon real-world tasks (robotics, agentic LLM tool use), temporal credit assignment remains the open problem. The 2025 wave of RL-with-verifiable-rewards training for reasoning LLMs is in the end another credit assignment story, with the additional twist that the credit-assigning algorithm (GRPO) currently spreads credit at the trajectory level and the field has not converged on whether and how to push it down to per-token. The 1986 Rumelhart-Hinton-Williams paper does not yet have a moral equivalent for the temporal version of the problem.

8. Quantum Approaches: Can Quantum Computers Help with Credit Assignment?

The natural question to ask, in 2026, is whether quantum computers can compute gradients faster or in qualitatively different ways. The answer turns out to depend sharply on what you’re trying to train: a classical neural network on a quantum computer, a parameterized quantum circuit on classical data, or a quantum machine learning model on intrinsically quantum data. The credit-assignment story is quite different in each case.

Parameter-shift rule: backprop’s quantum cousin

The first thing to know is that parameterized quantum circuits do have a gradient rule. For a circuit with parameters $\theta$ where each gate is of the form $e^{-i\theta_j G_j / 2}$ with Pauli generator $G_j$, the gradient of an expectation $\langle O \rangle (\theta) = \langle \psi(\theta) | O | \psi(\theta) \rangle$ is given exactly by

$\frac{\partial \langle O \rangle}{\partial \theta_j} = \frac{1}{2}\left[\langle O \rangle(\theta + \pi/2 \cdot e_j) - \langle O \rangle(\theta - \pi/2 \cdot e_j)\right]$

— two circuit evaluations with shifted parameters. This is the parameter-shift rule of Mitarai et al., Quantum Circuit Learning (Phys. Rev. A, 2018), and Schuld et al., Evaluating analytic gradients on quantum hardware (Phys. Rev. A, 2019). Banchi and Crooks’s Measuring Analytic Gradients of General Quantum Evolution with the Stochastic Parameter Shift Rule (Quantum, 2021) generalizes it.

The cost is $O(M)$ circuit evaluations to compute the full gradient of an $M$-parameter circuit. That’s the same as forward-mode automatic differentiation. It is not backprop scaling — backprop computes the full gradient in $O(1)$ forward-pass equivalents.

So the question is: can quantum models achieve backprop scaling?

Abbas et al.: an impossibility result, and shadow tomography as the partial workaround

Amira Abbas, Robbie King, Hsin-Yuan Huang, William Huggins, Ramis Movassagh, Dar Gilboa, and Jarrod McClean’s On quantum backpropagation, information reuse, and cheating measurement collapse (NeurIPS 2023 Spotlight) is the landmark paper here. I wrote a longer post on it last month — the punchline is:

1. Negative result. With single-copy measurement access to the trained quantum state, no algorithm can achieve backprop scaling on a general parameterized quantum circuit. The obstruction is information-theoretic: backprop reuses the forward-pass intermediate state to compute all the partial derivatives, and measurement collapse makes that impossible in the quantum setting. They prove an $\Omega(2^n / \varepsilon^2)$ lower bound for the relevant family.
2. Positive result. Given multiple copies of the state — i.e., quantum memory — shadow tomography (Aaronson, STOC 2018; Huang, Kueng, Preskill, Nature Physics, 2020) lets you predict $M$ expectation values from $O(\log M / \varepsilon^4)$ copies, recovering close-to-backprop scaling.

The result is a clean statement of where the quantum-classical gap lives in machine learning: not in the gate set, not in the expressivity, but in the fact that classical computers can read their RAM without destroying it.

Quantum natural gradient

A different and complementary line is to change the metric, not the computational shortcut. Stokes et al.’s Quantum Natural Gradient (Quantum, 2020) defines gradient descent on the manifold of quantum states using the Fubini–Study metric (the quantum Fisher information). The update direction is the inverse-metric times the gradient — like classical natural gradient (Amari, 1998), but with the quantum information geometry. Empirically, QNG converges in dramatically fewer iterations than vanilla quantum gradient descent, at the cost of estimating the (quantum) Fisher matrix.

Follow-ups in this lineage:

• Koczor and Benjamin, Quantum Natural Gradient generalised to non-unitary circuits (Phys. Rev. Research, 2022).
• van Straaten and Koczor, Measurement Cost of Metric-Aware Variational Quantum Algorithms (PRX Quantum, 2021), which gives explicit measurement-budget analyses.

The credit assignment story here is the same as classical natural gradient: the update is the gradient rescaled by the local geometry, so that updates respect the actual information distance between parameter settings rather than the arbitrary Euclidean distance in parameter space.

HHL and quantum linear algebra: a credit-assignment red herring?

A frequent claim in the popular AI/quantum literature is that the Harrow-Hassidim-Lloyd algorithm (Harrow, Hassidim, Lloyd, Phys. Rev. Lett., 2009) — which solves a linear system $Ax = b$ in time polynomial in $\log N$ rather than $N$ — could be used to invert the Hessian or compute Newton-step updates exponentially faster.

This is mostly wishful thinking, for reasons made painfully explicit in the dequantization literature. Ewin Tang’s A Quantum-Inspired Classical Algorithm for Recommendation Systems (STOC 2019) and the cascade of follow-up dequantization results (Chia et al., STOC 2020; Tang, 2019) showed that many of the apparent exponential speedups from HHL-style quantum linear algebra rely on input assumptions (quantum-accessible state preparation, low-rank structure) that, if granted classically, allow classical algorithms to match the quantum scaling up to polynomial factors. The frontier of meaningful HHL speedups for ML is much narrower than the popular framing suggests.

For credit assignment specifically: HHL is not a backprop replacement. It is a way to apply $A^{-1}$ to a vector. If you want to use it for Newton-step optimization, you still have to compute the Hessian, and the cost of getting the Hessian into quantum-accessible form is typically dominant. The genuine quantum speedup for second-order optimization, if there is one, is a much subtler question.

Variational quantum algorithms: where credit assignment actually bites

The credit-assignment problem most relevant to near-term (“NISQ”) quantum computing is in variational quantum algorithms: training a parameterized quantum circuit to minimize an energy or loss, using a classical optimizer in the outer loop and quantum expectation evaluations in the inner loop.

The pathologies:

• Barren plateaus. McClean et al.’s Barren plateaus in quantum neural network training landscapes (Nature Communications, 2018) and Cerezo et al.’s Cost function dependent barren plateaus (Nature Communications, 2021) show that gradients in deep random parameterized quantum circuits decay exponentially in the number of qubits. This is the quantum version of the vanishing gradient problem and is, in a meaningful sense, the core credit-assignment obstacle in quantum ML. Without an inductive bias that escapes barren plateaus, training a parameterized quantum circuit on more than a small number of qubits is information-theoretically infeasible.
• Noise. Wang et al., Noise-induced barren plateaus in variational quantum algorithms (Nature Communications, 2021) shows that hardware noise itself produces barren plateaus, even in shallow circuits.
• Provable absence implies classical simulability. The most consequential recent paper here is Cerezo et al.’s Does provable absence of barren plateaus imply classical simulability? Or, why we need to rethink variational quantum computing (arXiv 2023, Nature Communications 16:7907, 2025). The argument: every architectural strategy proposed to guarantee absence of barren plateaus — symmetry-respecting ansätze, shallow circuits, tensor-network-structured circuits — happens to also place the model in a regime that is classically simulable with polynomial-time algorithms. If that conjecture holds in full generality, the trainable-VQA regime and the classically-hard regime are disjoint, and variational quantum ML cannot deliver a quantum advantage on classical data. The paper does not prove this in full generality, but it gathers enough evidence to make the burden of proof rest with anyone claiming a NISQ VQA advantage.

The implication: classical backprop’s credit-assignment scaling is bad enough on a quantum substrate that NISQ-era variational quantum ML is unlikely to outperform classical ML on classical data, and the 2024 framework gives a principled reason to expect this is not merely a current-engineering problem. The earlier consensus statement is in Cerezo, Verdon, Huang, Cincio, Coles, Challenges and opportunities in quantum machine learning (Nature Computational Science, 2022); the 2024 paper is the much sharper version.

Where quantum credit assignment might genuinely matter

Where quantum machine learning has the strongest argument is not on classical data, but on intrinsically quantum data — outputs of quantum experiments, ground states of chemistry Hamiltonians, etc. Huang, Kueng, Torlai, Albert, Preskill’s Provably efficient machine learning for quantum many-body problems (Science, 2022) shows that learning properties of ground states from classical-shadow training data is provably efficient — credit assignment over a learning problem that no classical method can reach without exponentially much data.

Aharonov, Cotler, Qi’s Quantum algorithmic measurement (Nature Communications, 2022) frames the data-side question more generally: a quantum learning algorithm can extract information from quantum data exponentially faster than any classical algorithm could from the same data, provided the data is presented coherently. This is the most defensible “quantum advantage in ML” frontier as of 2026.

Sweke et al.’s Stochastic gradient descent for hybrid quantum-classical optimization (Quantum, 2020) is the canonical analysis of SGD’s interaction with quantum-shot noise, showing that the noisy quantum gradient estimator from finite measurement samples plays the role of a stochastic gradient in classical SGD theory, with convergence guarantees that scale appropriately. This is, in a precise sense, classical SGD’s credit assignment theory transplanted into a quantum measurement setting.

9. Where This Leaves Us

A few claims, in decreasing order of confidence:

1. Backpropagation is not going away on classical hardware. Sixty years of research has not produced a learning rule that beats backprop on the canonical scale-up benchmarks. The combination of backprop + AdamW (or Lion / Muon at the post-2024 frontier) + residual connections + LayerNorm or RMSNorm + transformer architecture, glued together with mixed-precision loss scaling and gradient clipping, is the answer to credit assignment in 2026 at trillion-parameter scale. Sections 4 and the closing list of section 4 are the actual content of this claim — every entry in that stack is a fix for a specific credit-assignment pathology, and the stack as a whole is what makes the algorithm work.

2. The biologically-plausible alternatives are converging toward backprop, not away from it — but the meaningful goal is different now. Equilibrium propagation, predictive coding, and target propagation all recover backprop’s gradient in some limit, and at small-to-mid scale they work. None has yet beaten backprop at frontier scale, and Bartunov 2018 set the bar that the field is still trying to clear. The honest reading is that if a non-backprop algorithm displaces backprop, it will be because of hardware economics — Ising machines, analog crossbar arrays, neuromorphic chips — not because of mathematical superiority. Laydevant et al.’s 2024 demonstration of equilibrium propagation training an Ising machine is the most concrete recent data point in this direction. Surrogate gradients are already the production-grade answer for spiking neuromorphic hardware.

3. Synthetic gradients are still niche; pipeline scheduling won. Zero Bubble Pipeline Parallelism and 1F1B scheduling have absorbed most of the engineering need that synthetic gradients were proposed for. The remaining niche — forward-mode-with-random-tangents gradients (Baydin et al. 2022) for memory-constrained training — is interesting but not yet competitive on variance reduction. Don’t expect a revival unless the underlying memory cost of activations becomes the dominant scaling bottleneck again, which is plausible at trillion-token regimes but not yet acute.

4. Temporal credit assignment is the hardest open problem in RL, and the GRPO-driven reasoning-RL wave has made it acute. REINFORCE → GAE → PPO is the variance-reduction lineage. RLHF added a learned-reward-model layer with its own credit-assignment failure modes (reward over-optimization, Gao-Schulman-Hilton 2023). GRPO replaced PPO’s learned value baseline with a group-relative baseline and assigns credit at the trajectory level, not per-token. Whether per-token credit assignment matters for long-chain reasoning — and whether the answer comes from process reward models, attention-mediated credit, or learned per-step value functions — is the most consequential open temporal-credit-assignment question of 2025–2026. None of RUDDER, hindsight CA, counterfactual CA, decision transformers, or GRPO is yet the moral equivalent of the 1986 Rumelhart-Hinton-Williams paper for the temporal problem.

5. Quantum credit assignment has a sharp impossibility result, and the trainable-VQA conjecture is much worse than people realized two years ago. Abbas et al.’s NeurIPS 2023 result is the cleanest statement of why quantum learning is hard given measurement collapse: backprop scaling requires multiple copies of a quantum state, which classical computers don’t need and quantum computers don’t have for free. Shadow tomography offers a partial workaround given quantum memory but does not yield a generic exponential speedup for training classical-data ML models. The harder result is Cerezo et al.’s “provable absence implies classical simulability” paper (arXiv 2023, Nat. Comm. 2025): every architectural strategy proposed to guarantee trainability of variational quantum circuits also appears to place them in a classically-simulable regime. If that conjecture survives, NISQ-era variational quantum ML cannot deliver a quantum advantage on classical data even in principle — the trainable regime and the quantum-hard regime are disjoint.

6. The defensible “quantum ML advantage” is on intrinsically quantum data, not on classical-data benchmarks. Huang et al.’s 2022 Science paper is the cleanest example. For everything else — chemistry, materials, possibly quantum sensing and quantum-experiment readout — the credit-assignment story is genuinely different from the classical one, and quantum methods may have a real advantage. For classical data, the parameter-shift rule gives an $O(M)$ forward-mode gradient, the Abbas–King–Huang impossibility blocks backprop scaling without quantum memory, and the Cerezo “absence-of-barren-plateaus-implies-simulability” conjecture (Nat. Comm. 2025) suggests the trainable architectures that survive are exactly the classically-simulable ones. That is not a complete impossibility result, but it is enough to shift the burden of proof.

The credit assignment problem is, in a real sense, what AI is. Every algorithm we use is a particular wager about how to push blame for a global signal back through internal structure. Rosenblatt’s wager was extremely simple and didn’t scale to hidden units. The 1986 wager — chain rule, reverse-mode AD, applied to a non-linear feedforward network with hidden representations — turned out to be the one that scaled, and it took roughly two decades to recognize that (Linnainmaa 1970 → Werbos 1974 → Parker/LeCun 1985 → RHW 1986) and another three decades to develop the surrounding engineering stack — initialization, activations, normalization, residuals, optimizers, mixed precision, the transformer — that makes it work in practice. None of that stack was obvious in 1986; almost all of it had to be discovered the hard way, at scale.

The next paradigm shift, if there is one, will probably look the same way in retrospect: a wager that no one currently takes seriously becoming, in twenty years, the obvious answer. Equilibrium propagation on Ising-machine and neuromorphic hardware; shadow-tomography-based gradients on a quantum-classical hybrid stack; attention-mediated temporal credit assignment in agentic systems trained with GRPO-descendant algorithms — any of these is a plausible candidate. But the structure of the problem — which parameter, by how much, for what error? — is invariant across them. The history of the field is a sequence of increasingly elaborate answers to that single question.

References (selected)

The following list collects the most load-bearing references from the post, organized by section. Inline arXiv references in the body are all hyperlinked above; this list is for convenience.

Founding work. Rosenblatt (1958, Psych. Review, “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain”); Minsky and Papert (1969, Perceptrons, MIT Press); Minsky (1961, “Steps Toward Artificial Intelligence”, Proc. IRE 49(1)); Olazaran (1996, “A Sociological Study of the Official History of the Perceptrons Controversy”, Social Studies of Science 26(3):611–659).

Backpropagation history. Linnainmaa (1970 MSc thesis, University of Helsinki, “The Representation of the Cumulative Rounding Error of an Algorithm as a Taylor Expansion of the Local Rounding Errors”); Werbos (1974 PhD thesis, Harvard, “Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences”); Parker (1985, “Learning-Logic: Casting the Cortex of the Human Brain in Silicon”, MIT Center for Computational Research in Economics and Management Science technical report); LeCun (1985, “Une procédure d’apprentissage pour réseau à seuil asymétrique”, Proc. Cognitiva 85, Paris, pp. 599–604); Rumelhart, Hinton, Williams (1986, “Learning Representations by Back-propagating Errors”, Nature 323:533–536); Schmidhuber (2015, “Deep Learning in Neural Networks: An Overview”, Neural Networks 61:85–117).

Engineering stack. Hochreiter (1991 thesis); Hochreiter and Schmidhuber (1997, Neural Computation); Gers, Schmidhuber, Cummins (Neural Comp. 2000); Pascanu, Mikolov, Bengio (ICML 2013); Glorot and Bengio (AISTATS 2010); Glorot, Bordes, Bengio (AISTATS 2011); Fukushima (Biol. Cybernetics 1980); He et al. (ICCV 2015); Nair and Hinton (ICML 2010); Kingma and Ba (ICLR 2015); Loshchilov and Hutter (ICLR 2019); Chen et al. (NeurIPS 2023, Lion); Gupta, Koren, Singer (ICML 2018, Shampoo); Anil et al. (Distributed Shampoo, 2020); Vyas et al. (SOAP, 2024); Jordan (Muon, 2024) and Liu et al. (Kimi-K2 / Muon scaling, arXiv:2502.16982); Ioffe and Szegedy (ICML 2015); Ba, Kiros, Hinton (2016); Zhang and Sennrich (NeurIPS 2019, RMSNorm); He et al. (CVPR 2016, ResNet); Micikevicius et al. (ICLR 2018, Mixed Precision); Vaswani et al. (NeurIPS 2017).

Biologically-plausible alternatives. Crick (Nature 1989, 337:129–132); Lillicrap et al. (arXiv 2014, Nature Comm. 2016); Nøkland (NeurIPS 2016); Bartunov et al. (NeurIPS 2018); Frenkel et al. (Front. Neurosci. 2021, DRTP); Dellaferrera and Kreiman (ICML 2022, PEPITA); Millidge et al. (2020, Activation Relaxation); Bengio (arXiv 2014); D.-H. Lee et al. (ECML PKDD 2015); Scellier and Bengio (Front. Comput. Neurosci. 2017); Laborieux et al. (Front. Neurosci. 2021); Laydevant, Markovic, Grollier (Nature Comm. 2024); Rao and Ballard (Nature Neurosci. 1999); Whittington and Bogacz (Neural Computation 2017); Millidge et al. (Neural Comp. 2022); Salvatori et al. (NeurIPS 2022, graph topologies); Salvatori et al. (NeurIPS 2022, beyond Gaussian); Hinton (NeurIPS 2022 invited talk, arXiv preprint); Tosato et al. (2024); Neftci, Mostafa, Zenke (IEEE SP Mag. 2019, surrogate gradients).

Synthetic gradients and decoupled training. Jaderberg et al. (ICML 2017); Huang et al. (NeurIPS 2019, GPipe); Narayanan et al. (SOSP 2019, PipeDream); Narayanan et al. (Megatron-LM 1F1B 2021); Qi et al. (ICLR 2024, Zero Bubble); Baydin et al. (2022, Forward Gradients); Belilovsky et al. (ICML 2019); Löwe et al. (NeurIPS 2019).

RL credit assignment. Klopf (1972, AFCRL); Barto, Sutton, Anderson (IEEE SMC 1983); Sutton (Machine Learning 3(1):9–44, 1988); Williams (Machine Learning 8:229–256, 1992, REINFORCE); Schulman et al. (ICLR 2016, GAE); Schulman et al. (2017, PPO); Sutton and Barto (RL: An Introduction, 2nd ed., 2018); Christiano et al. (NeurIPS 2017, DRLHP); Ouyang et al. (NeurIPS 2022, InstructGPT); Gao, Schulman, Hilton (ICML 2023, reward over-optimization); Shao et al. (2024, DeepSeekMath / GRPO); Andrychowicz et al. (NeurIPS 2017, HER); Arjona-Medina et al. (NeurIPS 2019, RUDDER); Harutyunyan et al. (NeurIPS 2019); Mesnard et al. (ICML 2021); Raposo et al. (2021, Synthetic Returns); Chen et al. (NeurIPS 2021, Decision Transformer); Janner et al. (NeurIPS 2021, Trajectory Transformer); DeepSeek-AI (2025, R1).

Quantum. Mitarai et al. (Phys. Rev. A 2018); Schuld et al. (Phys. Rev. A 2019); Banchi and Crooks (Quantum 2021); Abbas et al. (NeurIPS 2023); Aaronson (STOC 2018); Huang, Kueng, Preskill (Nature Physics 2020); Stokes et al. (Quantum 2020); Koczor and Benjamin (Phys. Rev. Research 2022); Harrow, Hassidim, Lloyd (Phys. Rev. Lett. 2009); Tang (STOC 2019); Chia et al. (STOC 2020); McClean et al. (Nature Comm. 2018); Cerezo et al. (Nature Comm. 2021); Wang et al. (Nature Comm. 2021); Cerezo et al. (Nat. Comp. Sci. 2022); Cerezo et al. (arXiv:2312.09121, 2023; Nature Communications 16:7907, 2025, “Does provable absence of barren plateaus imply classical simulability?”); Huang et al. (Science 2022); Aharonov, Cotler, Qi (Nature Comm. 2022); Sweke et al. (Quantum 2020).