Great visualizations. Really enjoyed having a well-written example where mathematical proofs directly help with understanding a practical application.
I wonder what would happen with this analysis if a momentum term was added to the gradient descent. It seems that it would fix the specific failure modes in the examples, but I wonder if there's a corresponding mathematical way of categorizing what kinds of functions can(not) be quickly optimized with GD + momentum.
We studied it in our peparation for college entrance exams in India. Though the detail the article goes in is exhaustive. But I thought that this maybe common or almost common knowledge.
We used to call it sandwich theorem
Great visualizations. Really enjoyed having a well-written example where mathematical proofs directly help with understanding a practical application.
I wonder what would happen with this analysis if a momentum term was added to the gradient descent. It seems that it would fix the specific failure modes in the examples, but I wonder if there's a corresponding mathematical way of categorizing what kinds of functions can(not) be quickly optimized with GD + momentum.
The animation is very good, making the article easy to understand
We studied it in our peparation for college entrance exams in India. Though the detail the article goes in is exhaustive. But I thought that this maybe common or almost common knowledge. We used to call it sandwich theorem
The sandwich theorem would normally refer to this one: https://en.wikipedia.org/wiki/Squeeze_theorem