How many nodes should exist within a hidden layer of a neural network?
I understand that each node in an input layer corresponds to a variable and maybe even a node for bias; I do not understand the amount of nodes in a hidden layer. From all of the visualizations I have seen, there is always one more node in the hidden layer compared to the input layer. For example, in the image below there are two nodes that make up the input layer. Next, there are three nodes that make up the hidden layer. I understand combinatorics but I am totally missing something here. What dictates the amount of nodes in a hidden layer?
1 answers ( 0 marked as helpful)
Hi John,
Thanks for reaching out.
The number of nodes in a hidden layer can vary. There is no "rule" to determine it.
There is one big consideration though! If you have 5 input nodes and you have 2 nodes in the consequent hidden layer, you are not creating information, you are decreasing the dimensionality of the problem (thus losing information). Therefore, in general, we want to have more hidden nodes in the hidden layers than there are nodes in the input layer.
This is also the reason why in all pictures we represent the hidden layers with more nodes than the input layer.
Best,
Iliya