PyTorch机器学习之反向传播

一、手动实现线性模型

首先，利用numpy库的random生成500个加噪声的二维数据坐标，其分布如下图所示，代码如下：

# 生成随机数
np.random.seed(42)
n_examples = 500
x_data = 2*np.random.randn(n_examples) + 1.5
y_data = 3*x_data + 2.5*np.random.randn(n_examples)+1.5
# 画图展示
plt.plot(x_data,y_data,'b.')
plt.xlabel('x')
plt.ylabel('y')
plt.grid()
plt.show()

主要模块如下：

# 正向传播
def forward(x,w,b):
    return x*w+b
# 计算单点损失
def loss(x,y,w,b):
    y_predict = forward(x,w,b)
    return (y_predict-y)**2
# 计算所有数据均方损失和
def mse(w,b):
    l_sum = 0
    for x_val, y_val in zip(x_data, y_data):
        y_pred_val = forward(x_val,w,b)
        loss_val = loss(x_val, y_val, w, b)
        l_sum += loss_val
        # print('\t', x_val, y_val, y_pred_val, loss_val)
    mse = l_sum/len(x_data)
    print('MSE={0}'.format(mse))
    return mse

最后，画出均方损失随权重和偏置变化的三维曲面图，代码和结果图如下：

##画图
fig = plt.figure()
## 画在一个fig里
ax = Axes3D(fig,auto_add_to_figure=False)
fig.add_axes(ax)
##定义网格化数据
w_list = np.arange(-20,20,0.1)
b_list = np.arange(-20,20,0.1)
##生成网格化数据
xx, yy = np.meshgrid(w_list,b_list,sparse=False, indexing='xy')
##每个点的对应高度
zz = mse(xx,yy)
ax.plot_surface(xx, yy, zz, rstride=1, cstride=1, cmap=plt.get_cmap('rainbow'))
# ax.contourf(xx, yy, zz, zdir='z', offset=0.99987, cmap='summer')
#设置坐标轴
ax.set_xlabel('weight')
ax.set_ylabel('bias')
ax.set_zlabel('loss')
plt.show()

二、PyTorch实现反向传播

如题：
意思是利用这个二次模型来预测数据，减小损失函数(MSE)的值。
代码如下：

import torch
import matplotlib.pyplot as plt
import os
os.environ["KMP_DUPLICATE_LIB_OK"]  =  "TRUE"
# 数据集
x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]
# 权重参数初始值均为1
w = torch.tensor([1.0,1.0,1.0])
w.requires_grad = True    # 需要计算梯度

# 前向传播
def forward(x):
    return w[0]*(x**2)+w[1]*x+w[2]
# 计算损失
def loss(x,y):
    y_pred = forward(x)
    return (y_pred-y) ** 2

# 训练模块
print('predict (before tranining) ',4, forward(4).item())
epoch_list = []
w_list = []
loss_list = []
for epoch in range(1000):
    for x,y in zip(x_data,y_data):
        l = loss(x,y)
        l.backward()		# 后向传播
        print('\tgrad: ',x,y,w.grad.data)
        w.data = w.data - 0.01 * w.grad.data		# 梯度下降
        
        w.grad.data.zero_()	# 梯度清零操作
        
    print('progress: ',epoch,l.item())
    epoch_list.append(epoch)
    w_list.append(w.data)
    loss_list.append(l.item())
print('predict (after tranining) ',4, forward(4).item())

# 绘图
plt.plot(epoch_list,loss_list,'b')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.grid()
plt.show()

运行结果如下：

predict (before tranining)  4 21.0
	grad:  1.0 2.0 tensor([2., 2., 2.])
	grad:  2.0 4.0 tensor([22.8800, 11.4400,  5.7200])
	grad:  3.0 6.0 tensor([77.0472, 25.6824,  8.5608])
progress:  0 18.321826934814453
	grad:  1.0 2.0 tensor([-1.1466, -1.1466, -1.1466])
	grad:  2.0 4.0 tensor([-15.5367,  -7.7683,  -3.8842])
	grad:  3.0 6.0 tensor([-30.4322, -10.1441,  -3.3814])
progress:  1 2.858394145965576
	grad:  1.0 2.0 tensor([0.3451, 0.3451, 0.3451])
	grad:  2.0 4.0 tensor([2.4273, 1.2137, 0.6068])
	grad:  3.0 6.0 tensor([19.4499,  6.4833,  2.1611])
progress:  2 1.1675907373428345
	grad:  1.0 2.0 tensor([-0.3224, -0.3224, -0.3224])
	grad:  2.0 4.0 tensor([-5.8458, -2.9229, -1.4614])
	grad:  3.0 6.0 tensor([-3.8829, -1.2943, -0.4314])
progress:  3 0.04653334245085716
	grad:  1.0 2.0 tensor([0.0137, 0.0137, 0.0137])
	grad:  2.0 4.0 tensor([-1.9141, -0.9570, -0.4785])
	grad:  3.0 6.0 tensor([6.8557, 2.2852, 0.7617])
progress:  4 0.14506366848945618
	grad:  1.0 2.0 tensor([-0.1182, -0.1182, -0.1182])
	grad:  2.0 4.0 tensor([-3.6644, -1.8322, -0.9161])
	grad:  3.0 6.0 tensor([1.7455, 0.5818, 0.1939])
progress:  5 0.009403289295732975
	grad:  1.0 2.0 tensor([-0.0333, -0.0333, -0.0333])
	grad:  2.0 4.0 tensor([-2.7739, -1.3869, -0.6935])
	grad:  3.0 6.0 tensor([4.0140, 1.3380, 0.4460])
progress:  6 0.04972923547029495
	grad:  1.0 2.0 tensor([-0.0501, -0.0501, -0.0501])
	grad:  2.0 4.0 tensor([-3.1150, -1.5575, -0.7788])
	grad:  3.0 6.0 tensor([2.8534, 0.9511, 0.3170])
progress:  7 0.025129113346338272
	grad:  1.0 2.0 tensor([-0.0205, -0.0205, -0.0205])
	grad:  2.0 4.0 tensor([-2.8858, -1.4429, -0.7215])
	grad:  3.0 6.0 tensor([3.2924, 1.0975, 0.3658])
progress:  8 0.03345605731010437
	grad:  1.0 2.0 tensor([-0.0134, -0.0134, -0.0134])
	grad:  2.0 4.0 tensor([-2.9247, -1.4623, -0.7312])
	grad:  3.0 6.0 tensor([2.9909, 0.9970, 0.3323])
progress:  9 0.027609655633568764
	grad:  1.0 2.0 tensor([0.0033, 0.0033, 0.0033])
	grad:  2.0 4.0 tensor([-2.8414, -1.4207, -0.7103])
	grad:  3.0 6.0 tensor([3.0377,