🛠️ Scheduled Maintenance | We’ll be undergoing scheduled maintenance and upgrades between 00:00 PST Jan 26th until 00:00 PST Jan 28th. There may be brief interruption of services in that period. We apologize for the inconvenience.

The 365 Data Science team is proud to invite you to our own community forum. A very well built system to support your queries, questions and give the chance to show your knowledge and help others in their path of becoming Data Science specialists.
Anybody can ask a question
Anybody can answer
The best answers are voted up and moderated by our team

Optimization in deep learning

Optimization in deep learning


Hello again,
I hope all your team are enjoying good health.
I have a quick question.
Can neural networks be used for optimization?
I mean after developing my algorithm, can I get the inputs which lead to maximum or minimum output?
Thank you in advance.
Stay safe.

1 Answer

365 Team

Hi Hady,
That’s an excellent question and is especially relevant when you go on to learn about CNNs (Convolutional Neural Networks).
Consider one step of training your network, i.e. one input/output pair.
The basic forward propagation recipe is: input + weights = output, where “+” stands for all the linear algebra you perform under the hood of multiplying matrices etc.
After that you run backpropagation in order to adjust the weights: weights <= weights + correction.
Having this in mind, how would we approach the problem of finding an input that produces a certain output, given the trained network, i.e. the weights?
In the exact same way!
The recipe is the same: input + weights = output.
This time around, however, you keep the weights fixed, and you vary the input values, so the update step changes to: input <= input + correction.
Using this strategy you can find inputs that correspond to arbitrary outputs, not simply minimal / maximal outputs.
To summarise:
Training the network: fixed input + varying weights = output
Finding an input that produces a certain output: varying input + fixed weights = output
As I mentioned, this is particularly relevant for CNNs, and I would recommend the following article as an example description of why this is the case: https://arxiv.org/pdf/1412.0035.pdf
Hope this helped, and please feel free to ask more questions as they arise.
Stay safe and keep learning!