Loading Events

« All Events

  • This event has passed.

GRASP Special Seminar: David Balduzzi, Victoria University of Wellington, New Zealand, “Kickback Cuts Backprop’s Red-tape: Rethinking Credit Assignment”

January 22, 2015 @ 10:30 am - 11:30 am


 The main problem of distributed learning is credit assignment, which was solved in the 80s with the invention of error backpropagation. 30 years later, Backprop, along with a few more recent tricks, is the major workhorse underlying machine learning and remains state-of-the-art for supervised learning. However, weight updates under Backprop depend on recursive computations that require distinct output and error signals — features not shared by biological neurons, that are perhaps unnecessary. In this talk, I revisit Backprop and the credit assignment problem. The main results are: (1) that Backprop decomposes into a collection of local learning algorithms; (2) regret bounds for these sub-algorithms; and (3) a factorization of Backprop’s error signals. Using these results, I derive a new algorithm for nonparametric regression, Kickback, that is significantly simpler than Backprop. Finally, I provide a sufficient condition for Kickback to follow error gradients, and show that Kickback matches Backprop’s performance on real-world regression benchmarks.


- Learn More

David Balduzzi is a Senior Lecturer in Mathematics and Statistics at Victoria University Wellington. He received a PhD in algebraic geometry from the University of Chicago, after which he worked on computational neuroscience at UW-Madison and machine learning at the Max Planck Institute for Intelligent Systems and ETH Zürich.


January 22, 2015
10:30 am - 11:30 am
Event Category: