Description

In this project, we will work on a classification algorithm that makes predictions when a dependent variable assumes discrete values. Logistic regression is arguably the simplest solution. In the case of binary classification (class 0 or class 1), it uses a sigmoid function to estimate how likely an observation belongs to class 1. It looks as follows:
$S(t) = {{1} \over {1 + e^{-t}}}$where

$t = b_0 + b_1x_1 + b_2x_2+...$

$b_0$ is the bias; $b_1, b_2, ...$ are the coefficients. You can fit logistic models without a bias term. In this case, $t$ would look like this:

$t = b_1x_1 + b_2x_2+...$

To complete this stage, the CustomLogisticRegression class should contain the following attributes and methods:

class CustomLogisticRegression:

    def __init__(self, fit_intercept=True, l_rate=0.01, n_epoch=100):
        self.fit_intercept = ...
        self.l_rate = ...
        self.n_epoch = ...

    def sigmoid(self, t):
        return ...

    def predict_proba(self, row, coef_):
        t = ...
        return self.sigmoid(t)

fit_intercept attribute is set to True when the logistic model is fitted with a bias ($t$ contains $b_0$). Otherwise, it is set to False, and the logistic model is fitted without any bias ($t$ doesn't contain $b_0$). We will discuss the l_rate and n_epoch attributes in later stages.

The sigmoid method based on the formula above takes a value of t and returns a float in the range of $[0, 1]$. In this case, the predict_proba method:

• Takes a row from a dataset and coefficients including a bias (designated as coef_);

• Calculates t;

• Calls the sigmoid method and then returns its value.

The coef_ array is provided in this stage. The values for bias and coefficients are obtained from minimizing the cost function using gradient descent. You will be able to retrieve these values in later stages. Row is the only row in the dataframe. The row and coef_ are represented as $x_1, x_2, ...$ and $b_0, b_1, b_2, ...$ respectively in t. Remember that the bias $b_0$ is present only when fit_intercept is True. You can get t from row and coef_ with numpy.dot.

In this project, we will work with the Wisconsin Breast Cancer Dataset from the sklearn library. Learn carefully how to load and use dataset from the given link. The target variable is denoted by y. The matrix of selected independent variables will be referred to as X.

You may want to standardize the features if they are measured in different units. Suppose $x$ is a feature. Z-standardization can be calculated as follows:

$z_i = {{x_i-\mu} \over \sigma}$$z_i$ is the standard score for the $i^{th}$ sample of the $x$ feature; $x_i$ is the value of the $i^{th}$ sample in $x$; $\mu$ and $\sigma$ are the mean and standard deviation of $x$, respectively. See an example below where the standard score is calculated with $\mu = 2.0, \ \sigma=3.207$:

Objectives

1. Create the CustomLogisticRegression class;

2. Create the __init__ method;

3. Create the sigmoid method;

4. Create the predict_proba method;

5. Load the Breast Cancer Wisconsin dataset. Select worst concave points and worst perimeter as features and target as the target variable;

6. Standardize X;

7. Split the dataset including the target variable into training and test sets. Set train_size=0.8 and random_state=43;

8. Given the coefficients below, calculate the probabilities of the first 10 rows in the test set. You don't need the training set in this stage;

9. Print these probabilities as a Python list.

Take the following values as bias and coefficients: [0.77001597, -2.12842434, -2.39305793]. It means that the model is fitted with an intercept, so t contains the bias term $b_0$ of 0.77001597; the coefficient of worst concave points $b_1$ is -2.12842434, and the coefficient of worst perimeter $b_2$ is -2.39305793.

Examples

Example 1: an example test set; features have been standardized

Download as a file

Output:

[0.38601, 0.99885, 0.03703, 0.12322]

Example 2: an example test set; features have been standardized

Download as a file

Output:

[0.26804, 0.47657, 0.91722, 0.878]

Example 3: an example test set; features have been standardized

Download as a file

Output:

[0.80045, 0.98387, 0.85675, 0.12745]