From Gradient Descent to Adam. Here are some optimizers you should know. And an easy way to remember them.

SUBSCRIBE to my channel for more good stuff!

REFERENCES

[1] Have fun plotting equations : [ Ссылка ]
[2] Original paper on the Adam optimizer: [ Ссылка ]
[3] Blog on types of optimizers: [ Ссылка ]
[4] Blog on optimizing gradient descent: [ Ссылка ]
[5] Github gist of code for rending animation of a math function: [ Ссылка ]
[6] Another Blog to quench your thirst for knowledge on optimizers cuz the other links weren't good enough: [ Ссылка ]

свернуть