What is an epoch?
Hi Team,
By the term 'epoch', I understand it as a full pass through training data and then a test it on validation data.
So it means.. we are doing complete gradient descent steps to reach global minima in one epoch. If that is correct, then what will happen in next epoch? Are we going to use same training data once again with the previous calculated weights?
Could you please help me with this.
Thanks in advance!
1 answers ( 0 marked as helpful)
Hello again, Darien!
First, an epoch in simplest terms is -> all the code inside the loop. In other simple term is -> a full iteration over the whole training set.
***
Now here's the detailed one:
So there are several things here. We've got:
1) Many epochs
2) Many batches
3) Only 1 training set (and 1 validation set) So what happens in 1 epoch? In 1 epoch, we take all the batches (say we have 50 batches). We train on the first batch and then validate on it. We train on the second batch and then validate on it. We train on the third batch and then validate on it. We continue in this way until the 50th batch. So we have trained the model on 50 batches (50 small datasets, which are subset of the training data). Once we've gone through all of them, there is no training data left to train on. And that's the end of the epoch!!! In the 2nd epoch we do the same. We train 50 more times. So in fact, just in 2 'iterations of the loop' we have trained 100 times, instead of 2 times. That is why we don't use the word 'iteration' but epoch. Some networks may train 1,000,000 times inside a single epoch, and then have 100 epochs!!! Hope this helps! Best, The 365 Team
2) Many batches
3) Only 1 training set (and 1 validation set) So what happens in 1 epoch? In 1 epoch, we take all the batches (say we have 50 batches). We train on the first batch and then validate on it. We train on the second batch and then validate on it. We train on the third batch and then validate on it. We continue in this way until the 50th batch. So we have trained the model on 50 batches (50 small datasets, which are subset of the training data). Once we've gone through all of them, there is no training data left to train on. And that's the end of the epoch!!! In the 2nd epoch we do the same. We train 50 more times. So in fact, just in 2 'iterations of the loop' we have trained 100 times, instead of 2 times. That is why we don't use the word 'iteration' but epoch. Some networks may train 1,000,000 times inside a single epoch, and then have 100 epochs!!! Hope this helps! Best, The 365 Team