Introduction to TensorFlow 2 – Model layout: learning rate, input size not specified

Question

In lesson https://learn.365datascience.com/courses/deep-learning-with-tensorflow-2-0/model-layout-inputs-outputs-targets-weights-bias-optimizer-and-loss we defined a variable input_size but we don't seem to use it, so why define it?
We also don't specify a learning rate, contrary to the numpy implementation; how is this managed in tensorflow?

Answer 1

Hi JeanMichel, Thanks for reaching out and very good questions indeed! TensorFlow 2.0 can now automatically detect that and there is no need to explicitly write it. However, sometimes (especially when dealing with CNNs) we need to preprocess the input layer in some way. Therefore, it is a good practice to specify it. Regarding the learning rate, since we are using Adam, the initial learning rate is mostly irrelevant. The algorithm will find (adapt) the best learning rate for each weight individually! Best,
Iliya

Introduction to TensorFlow 2 – Model layout: learning rate, input size not specified

Submit an answer