How to make a neural network on python

Learn how to create a neural network in Python with an example, from setting up the environment to training the model.

Building a Neural Network from Scratch in Python

Building a neural network from scratch in python can be a daunting task, but is an incredibly rewarding experience. In this tutorial, we will create a neural network using only the Python programming language.

First, let's import some necessary libraries. We will be using NumPy for matrix operations, matplotlib for plotting, and scikit-learn for our dataset.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons

# Make a dataset of moons
X, y = make_moons(n_samples=1000, noise=0.1)
plt.scatter(X[:,0], X[:,1], c=y)
plt.show()

Now that we have our dataset, we can begin building our neural network. The first step is to define the model, which is composed of the layers, their activation functions, and the weights. We will create a three-layer neural network, with the input layer, a hidden layer, and an output layer. The activation function for the hidden and output layers will be the sigmoid function. We will also add a bias to each layer.

# Define the model
input_size = 2 # The number of features in the dataset
layer_1_size = 3 # The size of the first layer
layer_2_size = 3 # The size of the second layer
output_size = 2 # The number of classes in the dataset

# Initialize the weights
w1 = np.random.randn(input_size, layer_1_size)
b1 = np.random.randn(1, layer_1_size)
w2 = np.random.randn(layer_1_size, layer_2_size)
b2 = np.random.randn(1, layer_2_size)
w3 = np.random.randn(layer_2_size, output_size)
b3 = np.random.randn(1, output_size)

# Define the activation functions
def sigmoid(x):
    return 1/(1+np.exp(-x))

def sigmoid_derivative(x):
    return x * (1-x)

Now that we have defined the model, we need to define the loss function. We will use the binary cross-entropy loss function. The function takes in the true labels and the predicted labels and returns the loss.

# Define the loss function
def binary_cross_entropy(true_labels, predictions):
    # Clip the predictions to make sure they are between 0 and 1
    predictions = np.clip(predictions, 1e-10, 1-1e-10)
    # Calculate the loss
    loss = -1 * np.mean(true_labels * np.log(predictions) + (1 - true_labels) * np.log(1 - predictions))
    return loss

Now that we have our model and loss function defined, we can begin training our network. We will train it using mini-batch gradient descent with a learning rate of 0.001. We will iterate through the dataset in batches of size 32 and update the weights after each batch.

# Train the model
batch_size = 32
learning_rate = 0.001

# Iterate over the dataset in batches
for i in range(0, len(X), batch_size):
    # Get the batch data
    X_batch = X[i:i+batch_size]
    y_batch = y[i:i+batch_size]
    
    # Forward pass
    layer_1 = sigmoid(np.dot(X_batch, w1) + b1)
    layer_2 = sigmoid(np.dot(layer_1, w2) + b2)
    predictions = sigmoid(np.dot(layer_2, w3) + b3)
    
    # Calculate the loss
    loss = binary_cross_entropy(y_batch, predictions)
    
    # Backward pass
    d_predictions = predictions - y_batch
    d_w3 = np.dot(layer_2.T, d_predictions)
    d_b3 = np.sum(d_predictions)
    d_layer_2 = np.dot(d_predictions, w3.T)
    d_w2 = np.dot(layer_1.T, d_layer_2 * sigmoid_derivative(layer_2))
    d_b2 = np.sum(d_layer_2 * sigmoid_derivative(layer_2))
    d_layer_1 = np.dot(d_layer_2 * sigmoid_derivative(layer_2), w2.T)
    d_w1 = np.dot(X_batch.T, d_layer_1 * sigmoid_derivative(layer_1))
    d_b1 = np.sum(d_layer_1 * sigmoid_derivative(layer_1))
    
    # Update the weights
    w1 -= learning_rate * d_w1
    b1 -= learning_rate * d_b1
    w2 -= learning_rate * d_w2
    b2 -= learning_rate * d_b2
    w3 -= learning_rate * d_w3
    b3 -= learning_rate * d_b3

Once the model is trained, we can evaluate it on the test set. We will calculate the accuracy, which is the percentage of correct predictions.

# Evaluate the model
predictions = sigmoid(np.dot(sigmoid(np.dot(X, w1) + b1), w2) + b2)
accuracy = np.mean(predictions == y)
print("Accuracy: {:.2f}%".format(accuracy * 100))

That's it! You have successfully built a neural network from scratch in Python. With a few lines of code, you can create a powerful model that can be used for a variety of tasks.