Resolved: Can we also use Correlation Coefficient in Feature Selection?
Hi!
In this lecture Sklearn's feature_selection module is used to drop features that don't have strong explanatory power on the Target. Can we also use Correlation Coefficient between each Feature and Target to do the same? If the Feature and Target are related, their correlation coef will be near 1, while if they are not it will be near 0.
Thank you.
Hey Shivaprasad,
Thank you for your question and engagement in the Machine Learning course!
As sklearn's documentation suggests, the f_regression() method (which returns the F-statistics and p-values for each feature) is derived from the r_regression() method (which returns the Pearson correlation coefficient for each feature). A large p-value is automatically penalized with a correlation coefficient close to 0. Therefore, you could use both metrics to identify features with low explanatory power. However, here is a citation from the documentation of the f_regression() method:
"Note however that contrary to f_regression, r_regression values lie in [-1, 1] and can thus be negative. f_regression is therefore recommended as a feature selection criterion to identify potentially predictive feature for a downstream classifier, irrespective of the sign of the association with the target variable."
Moreover, it would be much easier for you to follow the course and answer exam questions if you use the p-value rather than the correlation coefficient.
Considering everything said above, you are of course strongly encouraged to experiment with the code and the various methods that sklearn provides. You are very welcome to examine the values from the r_regression() method and compare them with the respective p-values.
Hope this answer helps! Thank you and keep up the good work!
Kind regards,
365 Hristina