Hello again, I hope all your team are enjoying good health. I have a quick question. Can neural networks be used for optimization? I mean after developing my algorithm, can I get the inputs which lead to maximum or minimum output? Thank you in advance. Stay safe.

Hi Hady, That's an excellent question and is especially relevant when you go on to learn about CNNs (Convolutional Neural Networks). Consider one step of training your network, i.e. one input/output pair.The basic forward propagation recipe is: input + weights = output, where "+" stands for all the linear algebra you perform under the hood of multiplying matrices etc.After that you run backpropagation in order to adjust the weights: weights <= weights + correction. Having this in mind, how would we approach the problem of finding an input that produces a certain output, given the trained network, i.e. the weights?In the exact same way!The recipe is the same: input + weights = output.This time around, however, you keep the weights fixed, and you vary the input values, so the update step changes to: input <= input + correction. Using this strategy you can find inputs that correspond to arbitrary outputs, not simply minimal / maximal outputs. To summarise: Training the network: fixed input + varying weights = outputFinding an input that produces a certain output: varying input + fixed weights = output As I mentioned, this is particularly relevant for CNNs, and I would recommend the following article as an example description of why this is the case: https://arxiv.org/pdf/1412.0035.pdf Hope this helped, and please feel free to ask more questions as they arise. Stay safe and keep learning!Iskren

Follow this topic

Hady Ali

Last answered:

27 Mar 2020

Posted on:

26 Mar 2020

Optimization in deep learning

1 answers ( 0 marked as helpful)

Iskren Vankov

Posted on:

27 Mar 2020

Optimization in deep learning

Submit an answer