In Deep learning course, we were taught to balance the dataset in order to avoid some problems with accuracy. Suppose a case where the targets are skewed for one output but balancing it led to reduction in the size of the dataset. How to approach this issue?
What you are describing is quite typical for some industries. For instance, fraudulent transactions in banking are less than 0.01% of all transactions.
If you are trying to solve such an issue using feedforward Neural Networks – you will not be able to. It is much more advisable to turn to other models.