I was curious about log and exp transformations. As far as I understood, we use them to transform non-linear relationship to linear. But doesn’t it kill the sole purpose of linear regression, if we are transforming the data to make it linear? in other words, if the relationship is not linear, why should we even transform the data and explore it with linear regression, instead of some other relationship or “algorithm”?
Additionally, there are different types of log transformations. Is there a guideline on which one to use, or is it a matter of preference of the user?
It is always better to use simpler models as they are more easily interpreted and troubleshooted. On top of that, if you are using a linear regression, chances are many people can collaborate with you on it, give feedback or take it over from you. Even more, there are a lot of resources online on that topic.
The more complicated the model, the more niche it becomes. This makes it hard to troubleshoot, hard to monitor, hard to maintain and very hard to handover to other colleagues.
That aside, you can always use a neural network or an XGBoosted decision tree and you would have the best result. However, it will take you much more time to build properly.
The 365 Team