利用TensorFlow进行经典机器学习及神经网络入门

# 利用TensorFlow进行经典机器学习及神经网络入门 ## 1. 回归模型 ### 1.1 岭回归（Ridge Regression） 岭回归是一种用于线性回归的正则化方法，旨在防止模型过拟合。以下是使用TensorFlow实现岭回归来预测波士顿房价的完整代码： ```python import tensorflow as tf import numpy as np # 假设X_train和y_train已经定义 num_outputs = y_train.shape[1] num_inputs = X_train.shape[1] x_tensor = tf.placeholder(dtype=tf.float32, shape=[None, num_inputs], name='x') y_tensor = tf.placeholder(dtype=tf.float32, shape=[None, num_outputs], name='y') w = tf.Variable(tf.zeros([num_inputs, num_outputs]), dtype=tf.float32, name='w') b = tf.Variable(tf.zeros([num_outputs]), dtype=tf.float32, name='b') model = tf.matmul(x_tensor, w) + b ridge_param = tf.Variable(0.8, dtype=tf.float32) ridge_loss = tf.reduce_mean(tf.square(w)) * ridge_param loss = tf.reduce_mean(tf.square(model - y_tensor)) + ridge_loss learning_rate = 0.001 optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss) mse = tf.reduce_mean(tf.square(model - y_tensor)) y_mean = tf.reduce_mean(y_tensor) total_error = tf.reduce_sum(tf.square(y_tensor - y_mean)) unexplained_error = tf.reduce_sum(tf.square(y_tensor - model)) rs = 1 - tf.div(unexplained_error, total_error) num_epochs = 1500 loss_epochs = np.empty(shape=[num_epochs],dtype=np.float32) mse_epochs = np.empty(shape=[num_epochs],dtype=np.float32) rs_epochs = np.empty(shape=[num_epochs],dtype=np.float32) mse_score = 0.0 rs_score = 0.0 with tf.Session() as tfs: tfs.run(tf.global_variables_initializer()) for epoch in range(num_epochs): feed_dict = {x_tensor: X_train, y_tensor: y_train} loss_val, _ = tfs.run([loss, optimizer], feed_dict=feed_dict) loss_epochs[epoch] = loss_val feed_dict = {x_tensor: X_test, y_tensor: y_test} mse_score, rs_score = tfs.run([mse, rs], feed_dict=feed_dict) mse_epochs[epoch] = mse_score rs_epochs[epoch] = rs_score print('For test data : MSE = {0:.8f}, R2 = {1:.8f} '.format( mse_score, rs_score)) ``` 运行上述代码后，得到测试数据的均方误差（MSE）为30.64177132，决定系数（R²）为0.63988018。 ### 1.2 ElasticNet正则化 ElasticNet正则化结合了Lasso和Ridge正则化的方法。完整代码可在`ch-04a_Regression`笔记本中找到。运行该模型后，测试数据的MSE为30.64861488，R²为0.63979971。 ## 2. 分类模型 ### 2.1 逻辑回归基础 逻辑回归是一种常用的分类方法，是一种概率性和线性分类器。输入特征向量属于特定类别的概率可以用以下公式表示： \[P(Y = i|x, w, b) = \phi(z)\] 其中： - \(Y\) 表示输出 - \(i\) 表示其中一个类别 - \(x\) 表示输入 - \(w\) 表示权重 - \(b\) 表示偏置 - \(z\) 表示回归方程 - \(\phi\) 表示平滑函数或模型 ### 2.2 二元分类的逻辑回归 对于二元分类，我们将模型函数 \(\phi(z)\) 定义为Sigmoid函数： \[\phi(z) = \frac{1}{1 + e^{-z}}\] Sigmoid函数的输出值在 \([0, 1]\) 范围内。因此，我们可以使用 \(y = \phi(z)\) 的值来预测类别：如果 \(y > 0.5\)，则类别为1；否则，类别为0。 二元逻辑回归的损失函数可以表示为： \[J(w) = -\sum_{i=1}^{n}[y_i\log(\phi(z_i)) + (1 - y_i)\log(1 - \phi(z_i))]\] 以下是实现二元分类逻辑回归的代码： ```python import tensorflow as tf import numpy as np from sklearn.datasets import make_classification from sklearn.model_selection import train_test_split import matplotlib.pyplot as plt # 生成数据集 X, y = make_classification(n_samples=200, n_features=2, n_informative=2, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=1) if (y.ndim == 1): y = y.reshape(-1, 1) # 绘制数据 plt.scatter(X[:, 0], X[:, 1], marker='o', c=y) plt.show() # 转换为one-hot编码 num_outputs = 2 y = np.eye(num_outputs)[y] # 划分训练集和测试集 X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=.4, random_state=42) # 定义模型和损失函数 num_outputs = y_train.shape[1] num_inputs = X_train.shape[1] learning_rate = 0.001 x = tf.placeholder(dtype=tf.float32, shape=[None, num_inputs], name="x") y = tf.placeholder(dtype=tf.float32, shape=[None, num_outputs], name="y") w = tf.Variable(tf.zeros([num_inputs, num_outputs]), name="w") b = tf.Variable(tf.zeros([num_outputs]), name="b") model = tf.nn.sigmoid(tf.matmul(x, w) + b) loss = tf.reduce_mean(-tf.reduce_sum( (y * tf.log(model)) + ((1 - y) * tf.log(1 - model)), axis=1)) optimizer = tf.train.GradientDescentOptimizer( learning_rate=learning_rate).minimize(loss) # 运行模型 num_epochs = 1 with tf.Session() as tfs: tf.global_variables_initializer().run() for epoch in range(num_epochs): tfs.run(optimizer, feed_dict={x: X_train, y: y_train}) y_pred = tfs.run(tf.argmax(model, 1), feed_dict={x: X_test}) y_orig = tfs.run(tf.argmax(y, 1), feed_dict={y: y_test}) preds_check = tf.equal(y_pred, y_orig) accuracy_op = tf.reduce_mean(tf.cast(preds_check, tf.float32)) accuracy_score = tfs.run(ac ```