Resolved: How do I change the learning rate?
Good evening. Where can I find the learning rate in the code to change it? Thank you
Hi, you can check the resources for assignment MNIST - Exercises, there is a described solution for adjusting the learning rate.
I have looked for it in the "Tensorflow MNIST complete with comments", but I can't find the learning rate. Are you able to find the line of code with the learning rate?
Once you download these resources > go to MNIST - Exercises.zip > 12_8_mnist-exercises-solutions.rar > folder MNIST - Exercise Solutions > 8. TensorFlow_MNIST_Learning_rate_Part_1_Solution.ipynb
You will see the described solution.
Basically it is just this code:
custom_optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001)
model.compile(optimizer=custom_optimizer, loss='sparse_categorical_crossentropy', metrics=['accuracy'])
Ok thanks, I remember this code. But why there's no learning rate in the "Tensorflow MNIST complete with comments" to change, too ?
I guess, this is because custom optimizer is not good for providing best result. Therefore they left original ADAM optimizer.
It makes sense, in fact ADAM automatically uses a learning rate changing with epochs. Thank you!