Many new data scientists have voiced what they feel is the lack of a satisfying way to learn the concepts of back propagation/gradient computation in neural networks when taking undergrad level ML classes. So I thought I’d put together a number of useful learning resources to jump-start an understanding for this important process. The following list, curated from an informal Twitter poll, appears in no particular order. After consuming one or more of these resources, I’m confident you’ll feel empowered in your ability to take the next step in your development. Many research papers and texts will make a lot more sense. Enjoy!
Video: What is backpropagation really doing?
Lecture Slides: Deep Learning – Spring 2021, Lecture 5 – Intro to Optimization
Lecture Slides: Deep Learning Systems: Algorithms and Implementation, Fall 2021, Carnegie Mellon University – Automatic Differentiation
Blog Post: Yet another backpropagation tutorial
Video: How Backpropagation Works
eBook Chapter: How the backpropagation algorithm works
Blog Post: Backprop and systolic arrays
Lecture slides with Python code: Backprop
Video: Backprop Bootcamp: Introduction to Backpropagation
Blog Post: Calculus on Computational Graphs: Backpropagation
Blog Post: A Visual Explanation of Gradient Descent Methods
Video: Neural Nets via MIT Open Courseware
Blog Post: Reverse-mode automatic differentiation from scratch, in Python
Lecture Video: Backprop by Geoff Hinton
Video: Stanford Winter Quarter 2016 class: CS231n: Convolutional Neural Networks for Visual Recognition. Lecture 4 on Backprop
Blog Post: Hacker’s guide to Neural Networks with Javascript code by Andreg Karpathy
Blog Post: Back-propagation, an introduction
Lecture Slides: Introduction to Artificial Intelligence, CS188 UC Berkeley
Video: Backpropagation Intuition by Andrew Ng
Blog Post: Backpropagation 101
Blog Post: A Step by Step Backpropagation Example
Contributed by Daniel D. Gutierrez, Editor-in-Chief and Resident Data Scientist for insideAI News. In addition to being a tech journalist, Daniel also is a consultant in data scientist, author, educator and sits on a number of advisory boards for various start-up companies.
Sign up for the free insideAI News newsletter.
Join us on Twitter: @InsideBigData1 – https://twitter.com/InsideBigData1
