# 卷积神经网络(CNN)的理解与可视化

**1. 卷积层的基本概念**

The first layer in a CNN is always a **Convolutional Layer**. First thing to make sure you remember is what the input to this conv (I’ll be using that abbreviation a lot) layer is. Like we mentioned before, the input is a 32 x 32 x 3 array of pixel values. Now, the best way to explain a conv layer is to imagine a flashlight that is shining over the top left of the image. Let’s say that the light this flashlight shines covers a 5 x 5 area. And now, let’s imagine this flashlight sliding across all the areas of the input image. In machine learning terms, this flashlight is called a **filter**(or sometimes referred to as a **neuron **or a **kernel**) and the region that it is shining over is called the **receptive field**. Now this filter is also an array of numbers (the numbers are called **weights** or**parameters**). A very important note is that the depth of this filter has to be the same as the depth of the input (this makes sure that the math works out), so the dimensions of this filter is 5 x 5 x 3. Now, let’s take the first position the filter is in for example. It would be the top left corner. As the filter is sliding, or **convolving**, around the input image, it is multiplying the values in the filter with the original pixel values of the image (aka computing **dot products**). These multiplications are all summed up (mathematically speaking, this would be 75 multiplications in total). So now you have a single number. Remember, this number is just representative of when the filter is at the top left of the image. Now, we repeat this process for every location on the input volume. (Next step would be moving the filter to the right by 1 unit, then right again by 1, and so on). Every unique location on the input volume produces a number. After sliding the filter over all the locations, you will find out that what you’re left with is a 28 x 28 x 1 array of numbers, which we call an **activation map** or **feature map**. The reason you get a 28 x 28 array is that there are 784 different locations that a 5 x 5 filter can fit on a 32 x 32 input image. These 784 numbers are mapped to a 28 x 28 array.

Let’s say now we use two 5 x 5 x 3 filters instead of one. Then our output volume would be 28 x 28 x 2. By using more filters, we are able to preserve the spatial dimensions better. Mathematically, this is what’s going on in a convolutional layer.

（强烈推荐原文：神经网络与深度学习-第6章）

**2. 第一层卷积层剖析**

However, let’s talk about what this convolution is actually doing from a high level. Each of these filters can be thought of as **feature identifiers**. When I say features, I’m talking about things like straight edges, simple colors, and curves. Think about the simplest characteristics that all images have in common with each other. Let’s say our first filter is 7 x 7 x 3 and is going to be a curve detector. (In this section, let’s ignore the fact that the filter is 3 units deep and only consider the top depth slice of the filter and the image, for simplicity.)As a curve detector, the filter will have a pixel structure in which there will be higher numerical values along the area that is a shape of a curve (Remember, these filters that we’re talking about as just numbers!).

Now, let’s go back to visualizing this mathematically. When we have this filter at the top left corner of the input volume, it is computing dot products between the filter and pixel values at that region. Now let’s take an example of an image that we want to classify, and let’s put our filter at the top left corner.

Remember, what we have to do is multiply the values in the filter with the original pixel values of the image.

Basically, in the input image, if there is a shape that generally resembles the curve that this filter is representing, then all of the dot products summed together will result in a large value! Now let’s see what happens when we move our filter.

The value is much lower! This is because there wasn’t anything in the image section that responded to the curve detector filter. Remember, the output of this conv layer is an activation map. So, in the simple case of a one filter convolution (and if that filter is a curve detector), the activation map will show the areas in which there at mostly likely to be curves in the picture. In this example, the top left value of our 28 x 28 x 1 activation map will be 6600. This high value means that it is likely that there is some sort of curve in the input volume that caused the filter to activate. The top right value in our activation map will be 0 because there wasn’t anything in the input volume that caused the filter to activate (or more simply said, there wasn’t a curve in that region of the original image). Remember, this is just for one filter. This is just a filter that is going to detect lines that curve outward and to the right. We can have other filters for lines that curve to the left or for straight edges. The more filters, the greater the depth of the activation map, and the more information we have about the input volume.

（原文见博客：A Beginner's Guide To Understanding Convolutional Neural Networks）

**3. 卷积神经网络多层架构**

The first convolutional layer filters the 224×224×3 input image with 96 kernels of size 11×11×3
with a stride of 4 pixels (this is the distance between the receptive field centers of neighboring neurons in a kernel map).

**Figure 2:** An illustration of the architecture of our CNN, explicitly showing the delineation of responsibilities
between the two GPUs. One GPU runs the layer-parts at the top of the figure while the other runs the layer-parts
at the bottom. The GPUs communicate only at certain layers. The network’s input is 150,528-dimensional, and
the number of neurons in the network’s remaining layers is given by 253,440–186,624–64,896–64,896–43,264–
4096–4096–1000.
neurons in a kernel map).

The second convolutional layer takes as input the (response-normalized
and pooled) output of the first convolutional layer and filters it with 256 kernels of size 5 × 5 × 48.
The third, fourth, and fifth convolutional layers are connected to one another without any intervening
pooling or normalization layers. The third convolutional layer has 384 kernels of size 3 × 3 ×
256 connected to the (normalized, pooled) outputs of the second convolutional layer. The fourth
convolutional layer has 384 kernels of size 3 × 3 × 192 , and the fifth convolutional layer has 256
kernels of size 3 × 3 × 192. The fully-connected layers have 4096 neurons each.

（原文链接：ImageNet Classification with Deep Convolutional Neural Networks，by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton (2012).）

**4. 卷积层的可视化**

**1. **A guide to convolution arithmetic for deep learning及代码链接conv_arithmetic

**2.**论文Visualizing and Understanding Convolutional Networks，对卷积层的可视化进行了深入研究，首先看反卷积架构图，总共5层，多看架构图有利于模型的设计与性能整体把控

下面一层一层看每个filter的图，及其对应的输出图像（只看响应最高的9个）

再贴一个三维可视化，Project Page：和一个基于JavaScript纯浏览器可使用的版本ConvNetjs:最高，还要说明epoch的概念：

The process of forward pass, loss function, backward pass, and parameter update is generally called one **epoch**. The program will repeat this process for a fixed number of epochs for each training image.

**5. 基于Theano的卷积网络实现**

**6. CNN最新进展**

1. 科大讯飞在语音识别领域的应用拓展（深度全序列卷积神经网络， DFCNN，Deep Fully CNN)，将语谱图直接作为输入，这个思路牛

2. 卷积神经网络模型理解（非常Nice，原文）

如有不妥之处，欢迎拍砖