使用 Logit() 和 fit() 在 python 中进行逻辑回归

Question

我正在尝试使用以下代码在 python 中执行逻辑回归 -

from patsy import dmatrices
import numpy as np
import pandas as pd
import statsmodels.api as sm

df=pd.read_csv('C:/Users/Documents/titanic.csv')
df=df.drop(['ticket','cabin','name','parch','sibsp','fare'],axis=1) #remove columns from table
df=df.dropna() #dropping null values

formula = 'survival ~ C(pclass) + C(sex) + age' 
df_train = df.iloc[ 0: 6, : ] 
df_test = df.iloc[ 6: , : ]

#spliting data into dependent and independent variables
y_train,x_train = dmatrices(formula, data=df_train,return_type='dataframe')
y_test,x_test = dmatrices(formula, data=df_test,return_type='dataframe')

#instantiate the model
model = sm.Logit(y_train,x_train)
res=model.fit()
res.summary()

我在这一行遇到错误-

--->res=model.fit()

我的数据集中没有缺失值。但是，我的数据集非常小，只有 10 个条目。我不确定这里出了什么问题，我该如何解决？我是运行 Jupyter notebook 中的程序。整个错误消息如下 -

    ---------------------------------------------------------------------------
PerfectSeparationError                    Traceback (most recent call last)
<ipython-input-37-c6a47ec170d5> in <module>()
     19 y_test,x_test = dmatrices(formula, data=df_test,return_type='dataframe')
     20 model = sm.Logit(y_train,x_train)
---> 21 res=model.fit()
     22 res.summary()

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py in fit(self, start_params, method, maxiter, full_output, disp, callback, **kwargs)
   1374         bnryfit = super(Logit, self).fit(start_params=start_params,
   1375                 method=method, maxiter=maxiter, full_output=full_output,
-> 1376                 disp=disp, callback=callback, **kwargs)
   1377 
   1378         discretefit = LogitResults(self, bnryfit)

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py in fit(self, start_params, method, maxiter, full_output, disp, callback, **kwargs)
    201         mlefit = super(DiscreteModel, self).fit(start_params=start_params,
    202                 method=method, maxiter=maxiter, full_output=full_output,
--> 203                 disp=disp, callback=callback, **kwargs)
    204 
    205         return mlefit # up to subclasses to wrap results

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\base\model.py in fit(self, start_params, method, maxiter, full_output, disp, fargs, callback, retall, skip_hessian, **kwargs)
    423                                                        callback=callback,
    424                                                        retall=retall,
--> 425                                                        full_output=full_output)
    426 
    427         #NOTE: this is for fit_regularized and should be generalized

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\base\optimizer.py in _fit(self, objective, gradient, start_params, fargs, kwargs, hessian, method, maxiter, full_output, disp, callback, retall)
    182                             disp=disp, maxiter=maxiter, callback=callback,
    183                             retall=retall, full_output=full_output,
--> 184                             hess=hessian)
    185 
    186         # this is stupid TODO: just change this to something sane

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\base\optimizer.py in _fit_newton(f, score, start_params, fargs, kwargs, disp, maxiter, callback, retall, full_output, hess, ridge_factor)
    246             history.append(newparams)
    247         if callback is not None:
--> 248             callback(newparams)
    249         iterations += 1
    250     fval = f(newparams, *fargs)  # this is the negative likelihood

C:\Program Files\Anaconda3\lib\site-packages\statsmodels\discrete\discrete_model.py in _check_perfect_pred(self, params, *args)
    184                 np.allclose(fittedvalues - endog, 0)):
    185             msg = "Perfect separation detected, results not available"
--> 186             raise PerfectSeparationError(msg)
    187 
    188     def fit(self, start_params=None, method='newton', maxiter=35,

PerfectSeparationError: Perfect separation detected, results not available

Answer 1

你有完美的分离，这意味着你的数据可以通过超平面完美分离。发生这种情况时，您的参数的最大似然估计是无限的，因此您的错误。

完美分离示例：

Gender   Outcome  
male     1
male     1
male     0
female   0
female   0

在这种情况下，如果我得到一个女性观察结果，我可以 100% 确定结果将为 0。也就是说，我的数据完美地将结果分开。没有不确定性，找到我的系数的数值计算不收敛。

根据您的错误，您正在发生类似的事情。只需 10 个条目，您就可以想象这将如何发生，而不是拥有 1000 个条目或类似的条目。所以获取更多数据:)

使用 Logit() 和 fit() 在 python 中进行逻辑回归

Logistic Regression in python using Logit() and fit()

python

regression

logistic-regression

jupyter-notebook