Introduction to Classification using Logistic Regression with Scikit-Learn
This post, including the source .ipynb notebook file, will be used as a basis for other topics. You can obtain a copy of the source by clicking the Source link at the post of this post.
To keep things simple, we're going to utilise one of the many toy datasets built into Scikit-Learn! (And yes, it is a real dataset.)
We're also not going to explain how Scikit-Learn's LogisticRegression is implemented in this post.
To structure our code, we will define our model in two parts:
- The code we need to fit our model
- The code we need to use our fitted model to generate predictions
When it comes to model building, these are the two main functional components - so, and for reasons which will be explained in other posts, we're going to build a Python class called CustomModel, with a function for each of these components:
from sklearn.linear_model import LogisticRegression
class CustomModel(object):
def fit(self, X, y):
# LogisticRegression implements a number of parameters, you can read about them here:
# http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
#
# With the exception of `random_state`, each of these are the defaults.
model_params = {
'penalty': 'l2',
'dual': False,
'tol': 0.0001,
'C': 1.0,
'fit_intercept': True,
'intercept_scaling': 1,
'class_weight': None,
'random_state': 1234, # Fixed to 1234 for reproducibility
'solver': 'liblinear',
'max_iter': 100,
'multi_class': 'ovr',
'verbose': 0,
'warm_start': False,
'n_jobs': 1
}
self.clf = LogisticRegression(**model_params)
self.clf.fit(X, y)
return self # fun fact: returning self enables method chaining i.e. .fit().predict()
def predict(self, X):
# We only want to output the positive case (the second column returned by `predict_proba`:
return self.clf.predict_proba(X)[:,1]
Now we're ready to use our model!
In the next section we're going to load the sample data discussed above, and divide it into two portions:
- 75% for model fitting
- 25% for predictions
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y,
train_size=0.75,
test_size=0.25,
random_state=1234) # more reproducibility
Now we have everything we need, lets load up our model, fit it with the training data, and generate some predictions:
# load our model
model = CustomModel()
# fit our model
model.fit(X_train, y_train)
# generate some predictions
model.predict(X_test)