{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Tce3stUlHN0L" }, "source": [ "##### Copyright 2020 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2022-12-14T22:31:52.272372Z", "iopub.status.busy": "2022-12-14T22:31:52.271925Z", "iopub.status.idle": "2022-12-14T22:31:52.275889Z", "shell.execute_reply": "2022-12-14T22:31:52.275299Z" }, "id": "tuOe1ymfHZPu" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "qFdPvlXBOdUN" }, "source": [ "# 变量简介" ] }, { "cell_type": "markdown", "metadata": { "id": "MfBg1C5NB3X0" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
在 TensorFlow.org 上查看在 Google Colab 中运行在 GitHub 上查看源代码下载笔记本
" ] }, { "cell_type": "markdown", "metadata": { "id": "AKhB9CMxndDs" }, "source": [ "TensorFlow **变量**是用于表示程序处理的共享持久状态的推荐方法。本指南介绍在 TensorFlow 中如何创建、更新和管理 `tf.Variable` 的实例。\n", "\n", "变量通过 `tf.Variable` 类进行创建和跟踪。`tf.Variable` 表示张量，对它执行运算可以改变其值。利用特定运算可以读取和修改此张量的值。更高级的库（如 `tf.keras`）使用 `tf.Variable` 来存储模型参数。 " ] }, { "cell_type": "markdown", "metadata": { "id": "xZoJJ4vdvTrD" }, "source": [ "## 设置\n", "\n", "本笔记本讨论变量布局。如果要查看变量位于哪一个设备上，请取消注释这一行代码。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:52.279261Z", "iopub.status.busy": "2022-12-14T22:31:52.279047Z", "iopub.status.idle": "2022-12-14T22:31:54.188922Z", "shell.execute_reply": "2022-12-14T22:31:54.188167Z" }, "id": "7tUZJk7lDiGo" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2022-12-14 22:31:53.211168: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 22:31:53.211265: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 22:31:53.211274: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] } ], "source": [ "import tensorflow as tf\n", "\n", "# Uncomment to see where your variables get placed (see below)\n", "# tf.debugging.set_log_device_placement(True)" ] }, { "cell_type": "markdown", "metadata": { "id": "vORGXDarogWm" }, "source": [ "## 创建变量\n", "\n", "要创建变量，请提供一个初始值。`tf.Variable` 与初始值的 `dtype` 相同。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:54.193586Z", "iopub.status.busy": "2022-12-14T22:31:54.192773Z", "iopub.status.idle": "2022-12-14T22:31:57.586549Z", "shell.execute_reply": "2022-12-14T22:31:57.585864Z" }, "id": "dsYXSqleojj7" }, "outputs": [], "source": [ "my_tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]])\n", "my_variable = tf.Variable(my_tensor)\n", "\n", "# Variables can be all kinds of types, just like tensors\n", "bool_variable = tf.Variable([False, False, False, True])\n", "complex_variable = tf.Variable([5 + 4j, 6 + 1j])" ] }, { "cell_type": "markdown", "metadata": { "id": "VQHwJ_Itoujf" }, "source": [ "变量与张量的定义方式和操作行为都十分相似，实际上，它们都是 `tf.Tensor` 支持的一种数据结构。与张量类似，变量也有 `dtype` 和形状，并且可以导出至 NumPy。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.590192Z", "iopub.status.busy": "2022-12-14T22:31:57.589637Z", "iopub.status.idle": "2022-12-14T22:31:57.596328Z", "shell.execute_reply": "2022-12-14T22:31:57.595673Z" }, "id": "GhNfPwCYpvlq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape: (2, 2)\n", "DType: \n", "As NumPy: [[1. 2.]\n", " [3. 4.]]\n" ] } ], "source": [ "print(\"Shape: \", my_variable.shape)\n", "print(\"DType: \", my_variable.dtype)\n", "print(\"As NumPy: \", my_variable.numpy())" ] }, { "cell_type": "markdown", "metadata": { "id": "eZmSBYViqDoU" }, "source": [ "大部分张量运算在变量上也可以按预期运行，不过变量无法重构形状。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.599665Z", "iopub.status.busy": "2022-12-14T22:31:57.599047Z", "iopub.status.idle": "2022-12-14T22:31:57.610191Z", "shell.execute_reply": "2022-12-14T22:31:57.609471Z" }, "id": "TrIaExVNp_LK" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A variable: \n", "\n", "Viewed as a tensor: tf.Tensor(\n", "[[1. 2.]\n", " [3. 4.]], shape=(2, 2), dtype=float32)\n", "\n", "Index of highest value: tf.Tensor([1 1], shape=(2,), dtype=int64)\n", "\n", "Copying and reshaping: tf.Tensor([[1. 2. 3. 4.]], shape=(1, 4), dtype=float32)\n" ] } ], "source": [ "print(\"A variable:\", my_variable)\n", "print(\"\\nViewed as a tensor:\", tf.convert_to_tensor(my_variable))\n", "print(\"\\nIndex of highest value:\", tf.math.argmax(my_variable))\n", "\n", "# This creates a new tensor; it does not reshape the variable.\n", "print(\"\\nCopying and reshaping: \", tf.reshape(my_variable, [1,4]))" ] }, { "cell_type": "markdown", "metadata": { "id": "qbLCcG6Pc29Y" }, "source": [ "如上所述，变量由张量提供支持。您可以使用 `tf.Variable.assign` 重新分配张量。调用 `assign`（通常）不会分配新张量，而会重用现有张量的内存。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.613448Z", "iopub.status.busy": "2022-12-14T22:31:57.612906Z", "iopub.status.idle": "2022-12-14T22:31:57.621093Z", "shell.execute_reply": "2022-12-14T22:31:57.620527Z" }, "id": "yeEpO309QbB2" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ValueError: Cannot assign value to variable ' Variable:0': Shape mismatch.The variable shape (2,), and the assigned value shape (3,) are incompatible.\n" ] } ], "source": [ "a = tf.Variable([2.0, 3.0])\n", "# This will keep the same dtype, float32\n", "a.assign([1, 2]) \n", "# Not allowed as it resizes the variable: \n", "try:\n", " a.assign([1.0, 2.0, 3.0])\n", "except Exception as e:\n", " print(f\"{type(e).__name__}: {e}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "okeywjLdQ1tY" }, "source": [ "如果在运算中像使用张量一样使用变量，那么通常会对支持张量执行运算。\n", "\n", "从现有变量创建新变量会复制支持张量。两个变量不能共享同一内存空间。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.623937Z", "iopub.status.busy": "2022-12-14T22:31:57.623567Z", "iopub.status.idle": "2022-12-14T22:31:57.634598Z", "shell.execute_reply": "2022-12-14T22:31:57.634039Z" }, "id": "2CnfGc6ucbXc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[5. 6.]\n", "[2. 3.]\n", "[7. 9.]\n", "[0. 0.]\n" ] } ], "source": [ "a = tf.Variable([2.0, 3.0])\n", "# Create b based on the value of a\n", "b = tf.Variable(a)\n", "a.assign([5, 6])\n", "\n", "# a and b are different\n", "print(a.numpy())\n", "print(b.numpy())\n", "\n", "# There are other versions of assign\n", "print(a.assign_add([2,3]).numpy()) # [7. 9.]\n", "print(a.assign_sub([7,9]).numpy()) # [0. 0.]" ] }, { "cell_type": "markdown", "metadata": { "id": "ZtzepotYUe7B" }, "source": [ "## 生命周期、命名和监视\n", "\n", "在基于 Python 的 TensorFlow 中，`tf.Variable` 实例与其他 Python 对象的生命周期相同。如果没有对变量的引用，则会自动将其解除分配。\n", "\n", "为了便于跟踪和调试，您还可以为变量命名。两个变量可以使用相同的名称。" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.637538Z", "iopub.status.busy": "2022-12-14T22:31:57.637173Z", "iopub.status.idle": "2022-12-14T22:31:57.644727Z", "shell.execute_reply": "2022-12-14T22:31:57.644161Z" }, "id": "VBFbzKj8RaPf" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[False False]\n", " [False False]], shape=(2, 2), dtype=bool)\n" ] } ], "source": [ "# Create a and b; they will have the same name but will be backed by\n", "# different tensors.\n", "a = tf.Variable(my_tensor, name=\"Mark\")\n", "# A new variable with the same name, but different value\n", "# Note that the scalar add is broadcast\n", "b = tf.Variable(my_tensor + 1, name=\"Mark\")\n", "\n", "# These are elementwise-unequal, despite having the same name\n", "print(a == b)" ] }, { "cell_type": "markdown", "metadata": { "id": "789QikItVA_E" }, "source": [ "保存和加载模型时会保留变量名。默认情况下，模型中的变量会自动获得唯一变量名，所以除非您希望自行命名，否则不必多此一举。\n", "\n", "虽然变量对微分很重要，但某些变量不需要进行微分。在创建时，通过将 `trainable` 设置为 False 可以关闭梯度。例如，训练计步器就是一个不需要梯度的变量。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.647981Z", "iopub.status.busy": "2022-12-14T22:31:57.647348Z", "iopub.status.idle": "2022-12-14T22:31:57.652568Z", "shell.execute_reply": "2022-12-14T22:31:57.651978Z" }, "id": "B5Sj1DqhbZvx" }, "outputs": [], "source": [ "step_counter = tf.Variable(1, trainable=False)" ] }, { "cell_type": "markdown", "metadata": { "id": "DD_xfDLDTDNU" }, "source": [ "## 放置变量和张量\n", "\n", "为了提高性能，TensorFlow 会尝试将张量和变量放在与其 `dtype` 兼容的最快设备上。这意味着如果有 GPU，那么大部分变量都会放置在 GPU 上。\n", "\n", "不过，您可以对此进行重写。在此代码段中，即使存在可用的 GPU，我们也可以在 CPU 上放置一个浮点张量和一个变量。通过打开设备放置日志记录（请参阅[设置](#scrollTo=xZoJJ4vdvTrD)），可以查看变量的放置位置。\n", "\n", "注：虽然可以手动放置变量，但使用[分布策略](distributed_training.ipynb)是一种可优化计算的更便捷且可扩展的方式。\n", "\n", "如果在有 GPU 和没有 GPU 的不同后端上运行此笔记本，则会看到不同的记录。*请注意，必须在会话开始时打开设备布局记录。*" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.655867Z", "iopub.status.busy": "2022-12-14T22:31:57.655337Z", "iopub.status.idle": "2022-12-14T22:31:57.681605Z", "shell.execute_reply": "2022-12-14T22:31:57.680992Z" }, "id": "2SjpD7wVUSBJ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[22. 28.]\n", " [49. 64.]], shape=(2, 2), dtype=float32)\n" ] } ], "source": [ "with tf.device('CPU:0'):\n", "\n", " # Create some tensors\n", " a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n", " b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])\n", " c = tf.matmul(a, b)\n", "\n", "print(c)" ] }, { "cell_type": "markdown", "metadata": { "id": "PXbh-p2BXKcr" }, "source": [ "您可以将变量或张量的位置设置在一个设备上，然后在另一个设备上执行计算。但这样会产生延迟，因为需要在两个设备之间复制数据。\n", "\n", "不过，如果您有多个 GPU 工作进程，但希望变量只有一个副本，则可以这样做。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:31:57.684587Z", "iopub.status.busy": "2022-12-14T22:31:57.684198Z", "iopub.status.idle": "2022-12-14T22:31:57.693430Z", "shell.execute_reply": "2022-12-14T22:31:57.692860Z" }, "id": "dgWHN3QSfNiQ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[ 1. 4. 9.]\n", " [ 4. 10. 18.]], shape=(2, 3), dtype=float32)\n" ] } ], "source": [ "with tf.device('CPU:0'):\n", " a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n", " b = tf.Variable([[1.0, 2.0, 3.0]])\n", "\n", "with tf.device('GPU:0'):\n", " # Element-wise multiply\n", " k = a * b\n", "\n", "print(k)" ] }, { "cell_type": "markdown", "metadata": { "id": "fksvRaqoYfay" }, "source": [ "注：由于 `tf.config.set_soft_device_placement` 默认处于打开状态，所以，即使在没有 GPU 的设备上运行此代码，它也会运行，只不过乘法步骤会在 CPU 上执行。\n", "\n", "有关分布式训练的详细信息，请参阅[指南](distributed_training.ipynb)。" ] }, { "cell_type": "markdown", "metadata": { "id": "SzCkWlF2S4yo" }, "source": [ "## 后续步骤\n", "\n", "要了解变量的一般用法，请参阅有关[自动微分](autodiff.ipynb)的指南。" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "variable.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 0 }