Initial Thoughts

Do you often hear terms like deep learning and neural networks but don't know where to start? As a programmer who learned deep learning from scratch, I deeply relate to this. I remember when I first encountered deep learning, I was overwhelmed by the technical terms and complex concepts. However, through continuous exploration and practice, I discovered that deep learning isn't as difficult as imagined when you grasp the right approach.

Today, I want to share how to build your first neural network using Python and TensorFlow. I'll guide you step by step through building a handwritten digit recognition system using plain language. I believe after reading this article, you'll have a fresh perspective on deep learning.

Basic Foundations

Before we start coding, let's understand some key concepts. You can think of a neural network as a structure composed of multiple layers of neurons, similar to the neural network in our brains. Each neuron receives input information, processes it, and passes it to the next layer. Through this layer-by-layer transmission and processing, neural networks can ultimately complete complex recognition tasks.

For a simple example, imagine recognizing a handwritten number. Your brain first identifies features like stroke shapes and positions, then combines this information to determine which number it is. Neural networks work on a similar principle.

Environment Setup

Before we start coding, we need to set up our development environment. It's like preparing all ingredients and cooking utensils before making a dish. We need to install the following tools:

pip install tensorflow numpy matplotlib


import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

Data Acquisition

For deep learning, data is like a teacher, and the model is like a student. Let's first obtain the MNIST dataset, which contains 60,000 training images and 10,000 test images of handwritten digits. Each image is a 28x28 pixel grayscale image.

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()


x_train = x_train / 255.0
x_test = x_test / 255.0


print(f"Training set shape: {x_train.shape}")
print(f"Test set shape: {x_test.shape}")

Model Construction

Now let's build the neural network model. You can imagine this process as building a bridge, where each layer must be carefully designed to ensure the stability of the overall structure. Our model uses the most classic multilayer perceptron structure:

model = tf.keras.Sequential([
    # Input layer: flatten 28x28 image into 784 neurons
    tf.keras.layers.Flatten(input_shape=(28, 28)),

    # First hidden layer: 128 neurons with ReLU activation function
    tf.keras.layers.Dense(128, activation='relu'),

    # Dropout layer: prevent overfitting
    tf.keras.layers.Dropout(0.2),

    # Output layer: 10 neurons corresponding to digits 0-9
    tf.keras.layers.Dense(10, activation='softmax')
])


model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

Model Training

Training a model is like teaching a child to learn reading - it requires repeated practice and correction. We'll have the model repeatedly examine the training data and continuously adjust its internal parameters until it can accurately recognize handwritten digits:

history = model.fit(
    x_train, y_train,
    epochs=10,
    batch_size=32,
    validation_split=0.2,
    verbose=1
)


plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.title('Model Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Model Loss')
plt.xlabel('Epoch')
plt.ylabel('Loss Value')
plt.legend()
plt.show()

Results Demonstration

After training, our model has acquired the ability to recognize handwritten digits. Let's see how it performs:

test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
print(f"Test set accuracy: {test_accuracy:.4f}")


sample_size = 5
sample_indices = np.random.randint(0, len(x_test), sample_size)

plt.figure(figsize=(15, 3))
for i, idx in enumerate(sample_indices):
    plt.subplot(1, sample_size, i+1)
    plt.imshow(x_test[idx], cmap='gray')
    prediction = model.predict(x_test[idx:idx+1])
    predicted_number = np.argmax(prediction)
    plt.title(f'Prediction: {predicted_number}')
    plt.axis('off')
plt.show()

Practical Experience

Through developing this project, I've summarized several important lessons:

1. Importance of data preprocessing: Normalizing pixel values from 0-255 to 0-1 range, though simple, significantly improves model performance. I forgot this step in my first training attempt, which resulted in very slow convergence.

2. Choice of model structure: Initially, I tried more complex network structures with more layers and neurons, but found they actually affected model performance. This taught me that simpler structures are sometimes better.

3. Handling overfitting: By adding a Dropout layer, we effectively prevented the model from overmemoizing training data. It's like learning through understanding rather than rote memorization.

4. Impact of batch size: I experimented with different batch sizes and finally chose 32, which achieved a good balance between training speed and model performance in my experiments.

Advanced Considerations

Through this project, we've mastered the basics of deep learning, but this is just the beginning. Have you thought about:

1. How to handle more complex image recognition tasks? Like facial or object recognition?

2. What other scenarios could this model architecture be used for besides handwritten digit recognition?

3. How to optimize model performance? Should we add more layers? Or adjust the number of neurons?

These questions are worth exploring further. I suggest trying to modify the model structure or use other datasets to expand your deep learning skills.

Learning Outcomes

Through this project, we've not only learned how to build neural networks using TensorFlow but more importantly understood the basic principles and practical methods of deep learning. Remember, learning deep learning is like learning any new skill - it requires patience and practice.

I believe through this article, you now have a clearer understanding of deep learning. Next, I suggest you try:

1. Experimenting with different optimizers like SGD, RMSprop, etc.
2. Adjusting the network structure to see how performance changes
3. Using other datasets like Fashion-MNIST
4. Exploring applications of Convolutional Neural Networks (CNN)

What do you think about this tutorial? Feel free to share your thoughts and questions in the comments. If you encounter any issues during practice, feel free to discuss them.

Let's explore more possibilities in the ocean of deep learning together. Remember, every expert was once a beginner - what's important is maintaining enthusiasm and perseverance for learning.