Logistic Regression
I know it may be a more advanced topic, but please provide an explanation or a link to where myself and others may go to understand the linear algebra and operations of the following code:
from sklearn import linear_model
import scipy.stats as stat class LogisticRegression_with_p_values:
def __init__(self,*args,**kwargs):
self.model = linear_model.Logisticregression(*args,**kwargs)
def fit(self,X,y):
self.model.fit(X,y)
denom = (2.0 * (1.0 + np.cosh(self.model.decision_function(X))))
denom = np.tile(denom,(X.shape[1],1)).T
F_ij = np.dot((X / denom).T,X)
Cramer_Rao = np.linalg.inv(F_ij)
sigma_estimates = np.sqrt(np.diagonal(Cramer_Rao))
z_scores = self.model.coef_[0] / sigma_estimates
p_values = [stat.norm.sf(abs(x)) * 2 for x in z_scores]
self.coef_ = self.model.coef_
self.intercept_ = self.model.intercept_
self.p_values = p_values (taken from a video by 365 Data Science about Logistic Regression and how to add p_values to the coefficient (beta estimates) once you fit a model using Sklearn).
import scipy.stats as stat class LogisticRegression_with_p_values:
def __init__(self,*args,**kwargs):
self.model = linear_model.Logisticregression(*args,**kwargs)
def fit(self,X,y):
self.model.fit(X,y)
denom = (2.0 * (1.0 + np.cosh(self.model.decision_function(X))))
denom = np.tile(denom,(X.shape[1],1)).T
F_ij = np.dot((X / denom).T,X)
Cramer_Rao = np.linalg.inv(F_ij)
sigma_estimates = np.sqrt(np.diagonal(Cramer_Rao))
z_scores = self.model.coef_[0] / sigma_estimates
p_values = [stat.norm.sf(abs(x)) * 2 for x in z_scores]
self.coef_ = self.model.coef_
self.intercept_ = self.model.intercept_
self.p_values = p_values (taken from a video by 365 Data Science about Logistic Regression and how to add p_values to the coefficient (beta estimates) once you fit a model using Sklearn).
2 answers ( 0 marked as helpful)
Hi John,
We haven't really been into this, because you don't need explanation, but rather a whole book on econometrics, to be honest.
I suggest using "Econometric Analysis" by William H. Greene. Unfortunately, I don't think any single explanation can help, unless you've been through a thorough journey through econometrics (and the actual formulas, so you can turn them in code).
Best,
The 365 Team
Hi guys, in my case this customized function is not running, it gives me the following error:
---> 20 Cramer_Rao = np.linalg.inv(F_ij) ## Inverse Information Matrix
LinAlgError: Singular matrix
Any idea about how to fix it?
Thanks.
---> 20 Cramer_Rao = np.linalg.inv(F_ij) ## Inverse Information Matrix
LinAlgError: Singular matrix
Any idea about how to fix it?
Thanks.
Submit an answer
