图像协方差矩阵_深度学习的预处理：从协方差矩阵到图像白化

图像协方差矩阵

by hadrienj

由hadrienj

The goal of this post is to go from the basics of data preprocessing to modern techniques used in deep learning. My point is that we can use code (such as Python/NumPy) to better understand abstract mathematical notions. Thinking by coding! ?

这篇文章的目标是从数据预处理的基础知识到深度学习中使用的现代技术。 我的观点是，我们可以使用代码(例如Python / NumPy)更好地理解抽象的数学概念。 通过编码思考！ ？

We will start with basic but very useful concepts in data science and machine learning/deep learning, like variance and covariance matrices. We will go further to some preprocessing techniques used to feed images into neural networks. We will try to get more concrete insights using code to actually see what each equation is doing.

我们将从数据科学和机器学习/深度学习中的基本但非常有用的概念入手，例如方差和协方差矩阵。 我们将进一步介绍一些用于将图像馈入神经网络的预处理技术。 我们将尝试使用代码来获得更具体的见解，以实际查看每个方程式在做什么。

Preprocessing refers to all the transformations on the raw data before it is fed to the machine learning or deep learning algorithm. For instance, training a convolutional neural network on raw images will probably lead to bad classification performances (Pal & Sudeep, 2016). The preprocessing is also important to speed up training (for instance, centering and scaling techniques, see Lecun et al., 2012; see 4.3).

预处理是指对原始数据进行的所有转换，然后再将其转换为机器学习或深度学习算法。 例如，在原始图像上训练卷积神经网络可能会导致分类性能不佳( Pal＆Sudeep，2016 )。 预处理对于加快训练速度也很重要(例如，定心和缩放技术，请参阅Lecun等人，2012；请参阅4.3 )。

Here is the syllabus of this tutorial:

这是本教程的课程表：

1. Background: In the first part, we will get some reminders about variance and covariance. We will see how to generate and plot fake data to get a better understanding of these concepts.

1.背景：在第一部分中，我们将提醒您一些方差和协方差。 我们将看到如何生成和绘制虚假数据以更好地理解这些概念。

2. Preprocessing: In the second part we will see the basics of some preprocessing techniques that can be applied to any kind of data — mean normalization, standardization, and whitening.

2.预处理：在第二部分中，我们将看到可应用于任何类型数据的一些预处理技术的基础- 均值归一化 ， 标准化和白化 。

3. Whitening images: In the third part, we will use the tools and concepts gained in 1. and 2. to do a special kind of whitening called Zero Component Analysis (ZCA). It can be used to preprocess images for deep learning. This part will be very practical and fun ☃️!

3.增白图像：在第三部分中，我们将使用在1.和2.中获得的工具和概念来进行一种特殊的增白，称为零分量分析 (ZCA)。 它可以用于预处理图像以进行深度学习。 这部分将非常实用且有趣☃️！

Feel free to fork the notebook associated with this post! For instance, check the shapes of the matrices each time you have a doubt.

随意分叉与此帖子相关的笔记本 ！ 例如，每当您有疑问时，请检查矩阵的形状。

1.背景 (1. Background)

A.方差和协方差 (A. Variance and covariance)

The variance of a variable describes how much the values are spread. The covariance is a measure that tells the amount of dependency between two variables.

变量的方差描述了值的分散程度。 协方差是衡量两个变量之间依存度的量度。

A positive covariance means that the values of the first variable are large when values of the second variables are also large. A negative covariance means the opposite: large values from one variable are associated with small values of the other.

正协方差意味着当第二变量的值也很大时，第一变量的值就很大。 负协方差则相反：一个变量的大值与另一个变量的小值相关。

The covariance value depends on the scale of the variable so it is hard to analyze it. It is possible to use the correlation coefficient that is easier to interpret. The correlation coefficient is just the normalized covariance.

协方差值取决于变量的大小，因此很难对其进行分析。 可以使用更容易解释的相关系数。 相关系数只是归一化协方差。

The covariance matrix is a matrix that summarises the variances and covariances of a set of vectors and it can tell a lot of things about your variables. The diagonal corresponds to the variance of each vector:

协方差矩阵是一个汇总一组向量的方差和协方差的矩阵，它可以告诉您很多有关变量的信息。 对角线对应于每个向量的方差：

Let’s just check with the formula of the variance:

让我们检查方差的公式：

with n the length of the vector, and x̄ the mean of the vector. For instance, the variance of the first column vector of A is:

其中向量的长度为n ，向量的平均值为x̄ 。 例如， A的第一列向量的方差为：

This is the first cell of our covariance matrix. The second element on the diagonal corresponds of the variance of the second column vector from A and so on.

这是我们协方差矩阵的第一个单元格。 对角线上的第二个元素对应于第二列向量相对于A的方差，依此类推。

Note: the vectors extracted from the matrix A correspond to the columns of A.

注意 ：从所述矩阵A对应于A的列中提取的矢量。

The other cells correspond to the covariance between two column vectors from A. For instance, the covariance between the first and the third column is located in the covariance matrix as the column 1 and the row 3 (or the column 3 and the row 1).

其他像元对应于来自A的两个列向量之间的协方差。 例如，第一列和第三列之间的协方差位于协方差矩阵中，作为列1和行3(或列3和行1)。

Let’s check that the covariance between the first and the third column vector of A is equal to -2.67. The formula of the covariance between two variables X and Y is:

我们检查A的第一列向量和第三列向量之间的协方差是否等于-2.67。 两个变量X和Y之间的协方差公式为：

The variables X and Y are the first and the third column vectors in the last example. Let’s split this formula to be sure that it is crystal clear:

变量X和Y是最后一个示例中的第一列向量和第三列向量。 让我们对这个公式进行拆分，以确保它非常清晰：

1. The sum symbol (Σ) means that we will iterate on the elements of the vectors. We will start with the first element (i=1) and calculate the first element of X minus the mean of the vector X.

   和符号( Σ )表示我们将迭代向量的元素。 我们将从第一个元素( i = 1 )开始，然后计算X的第一个元素减去向量X的平均值。

2. Multiply the result with the first element of Y minus the mean of the vector Y.

2.将结果乘以Y的第一个元素减去向量Y的平均值。

3. Reiterate the process for each element of the vectors and calculate the sum of all results.

3.对向量的每个元素重复该过程，并计算所有结果的总和。

4. Divide by the number of elements in the vector.

4.除以向量中元素的数量。

Example 1.

范例1。

Let’s start with the matrix A:

让我们从矩阵A开始：

We will calculate the covariance between the first and the third column vectors:

我们将计算第一列向量和第三列向量之间的协方差：

and

和

x̄=3, ȳ=4, and n=3 so we have:

x̄ = 3 ， ȳ= 4和n = 3，所以我们有：

Ok, great! That’s the value of the covariance matrix.

好的太棒了！ 那就是协方差矩阵的值。

Now the easy way. With NumPy, the covariance matrix can be calculated with the function np.cov.

现在简单的方法 。 使用NumPy，可以使用函数np.cov计算协方差矩阵。

It is worth noting that if you want NumPy to use the columns as vectors, the parameter rowvar=False has to be used. Also, bias=True divides by n and not by n-1.

值得注意的是 ，如果希望NumPy将列用作向量，则必须使用参数rowvar=False 。 此外， bias=True除以n 而不是n-1 。

Let’s create the array first:

让我们首先创建数组：

array([[1, 3, 5],       [5, 4, 1],       [3, 8, 6]])

Now we will calculate the covariance with the NumPy function:

现在，我们将使用NumPy函数计算协方差：

array([[ 2.66666667, 0.66666667, -2.66666667],       [ 0.66666667, 4.66666667, 2.33333333],       [-2.66666667, 2.33333333, 4.66666667]])

Looks good!

看起来挺好的！

Finding the covariance matrix with the dot product

用点积找到协