Resolved: Does a lower batch size always mean a better accuracy?
Good morning. I'm quite sure that lowering the batch size always increases the time for training, but does a lower batch size always mean a better accuracy, too? So does a batch size of 1 always give a better accuracy compared to a batch size of 2, 3, 4 and so on? Thank you
Hi,
No, this is definitely not the case.
Since the model is practically learning from the samples it sees in a batch, there needs to be a good mix of all types of data points in one batch for it to learn properly. With single digit batch sizes, there is no variety in the types of inputs, so the model will struggle to understand what to do.
Depending on the problem and dataset at hand, smaller or larger batches might be more optimal, there is no "one size fits all".
Hope this helps!
Best,
Nikola, 365 Team
