Neural Networks with Google TensorFlow
Download as pptx, pdf61 likes57,180 views
This slides explains how Convolution Neural Networks can be coded using Google TensorFlow. Video available at : https://www.youtube.com/watch?v=EoysuTMmmMc
More Related Content
What's hot (20)
Viewers also liked (7)
Similar to Neural Networks with Google TensorFlow (20)
Recently uploaded (20)
Neural Networks with Google TensorFlow
- 1. Neural Networks with Google TensorFlow Darshan Patel Northeastern University
- 2. • Overview: 1) Computer Vision Tasks 2) Convolution Neural Network (CNNs) Architecture 3) CNNs using Google TensorFlow 4) Google TensorBoard Neural Networks with Google TensorFlow
- 13. • Mathematical Definition: A function derived from two given functions by integration that expresses how the shape of one is modified by the other What is Convolution?
- 14. Neural Networks
- 15. Neural Networks - Forward Pass
- 16. Neural Networks - Back Propagation Source : http://cs231n.github.io
- 17. • ConvNet architectures make the explicit assumption that the inputs are images, which allows us to encode certain properties into the architecture. • This assumption makes the forward function more efficient to implement and vastly reduces the amount of parameters in the network. How CNN/ConvNets is different?
- 18. Cont. How CNN/ConvNets is different? Source : http://cs231n.github.io/convolutional-networks
- 19. LeNet-5 1990 Yann LeCun Director of AI Research at Facebook Handwritten Digits Classification
- 21. AlexNet Architecture - ImageNet 2012
- 26. Layers in ConvNets 1. Convolution Layer 2. ReLU (Activation) Layer 3. Pooling Layer 4. Fully Connected Layer
- 37. Source : http://cs231n.stanford.edu/slides/ Activation Layer
- 43. • A ConvNet architecture is a list of Layers that transform the image volume into an output volume (e.g. holding the class scores) • There are a few distinct types of Layers (e.g. CONV/FC/RELU/POOL are by far the most popular) • Each Layer accepts an input 3D volume and transforms it to an output 3D volume through a differentiable function • Each Layer may or may not have parameters (e.g. CONV/FC do, RELU/POOL don't) ConvNets Summary
- 46. • Second generation Machine Learning system, followed by DistBelief • TensorFlow grew out of a project at Google, called Google Brain, aimed at applying various kinds of neural network machine learning to products and services across the company. • An open source software library for numerical computation using data flow graphs • Used in following projects at Google 1. DeepDream 2. RankBrain 3. Smart Reply And many more.. Google TensorFlow
- 47. Data Flow Graph • Data flow graphs describe mathematical computation with a directed graph of nodes & edges • Nodes in the graph represent mathematical operations. • Edges represent the multidimensional data arrays (tensors) communicated between them. • Edges describe the input/output relationships between nodes. • The flow of tensors through the graph is where TensorFlow gets its name.
- 48. Google TensorFlow Basic Elements • Tensor • Variable • Operation • Session • Placeholder • TensorBoard
- 49. • TensorFlow programs use a tensor data structure to represent all data • Think of a TensorFlow tensor as an n-dimensional array or list In the following example, c, d and e are symbolic Tensor Objects, where as result is a numpy array Tensor
- 50. 1. Constant Value Tensors 2. Sequences 3. Random Tensors Tensor Types
- 52. Sequence Tensors
- 53. Random Tensors
- 54. • In-memory buffers containing tensors • Initial value defines the type and shape of the variable. • They must be explicitly initialized and can be saved to disk during and after training. Variable
- 55. • An Operation is a node in a TensorFlow Graph • Takes zero or more Tensor objects as input, and produces zero or more Tensor objects as output. • Example: c = tf.matmul(a, b) Creates an Operation of type "MatMul" that takes tensors a and b as input, and produces c as output. Operation
- 56. • A class for running TensorFlow operations • InteractiveSession is a TensorFlow Session for use in interactive contexts, such as a shell and Ipython notebook. Session & Interactive Session
- 57. • A value that we'll input when we ask TensorFlow to run a computation. Placeholder
- 58. TensorBoard : Visual Learning
- 59. MNIST Dataset
- 60. MNIST Dataset
- 62. Load MNIST Data
- 63. Load MNIST Data Start a session
- 66. Convolution and Pooling Stride of one Max Pooling over 2x2 blocks Stride of two
- 67. First Convolution Layer including ReLU It will consist of convolution, followed by max pooling Filter/Patch Dimension Number of Input Channels Number of Output Channel Number of Output Channel
- 68. Second Convolution Layer including ReLU It will consist of convolution, followed by max pooling
- 69. Fully Connected Layer • Reshape the tensor from the pooling layer into a batch of vectors • Multiply by a weight matrix, add a bias, and apply a ReLU
- 70. Dropout • To reduce over fitting, we will apply dropout before the readout layer. • Dropout is an extremely effective, simple and recently introduced regularization technique by Srivastava et al. in “Dropout: A Simple Way to Prevent Neural Networks from Overfitting” that complements the other methods (L1, L2, maxnorm). Source : http://cs231n.github.io/neural-networks-2/
- 71. Dropout • We create a placeholder for the probability that a neuron's output is kept during dropout. • This allows us to turn dropout on during training, and turn it off during testing. • While training, dropout is implemented by only keeping a neuron active with some probability pp (a hyperparameter)
- 72. Readout Layer • Finally, we add a softmax layer, just like for the one layer softmax regression.
- 73. Train and Evaluate the Model Initialize All Variables Training Accuracy Testing Optimizer Loss Function
- 74. • TensorBoard operates by reading TensorFlow events files, which contain summary data that you can generate when running TensorFlow. TensorBoard
- 75. • TensorBoard operates by reading TensorFlow events files, which contain summary data that you can generate when running TensorFlow. • First, create the TensorFlow graph that we'd like to collect summary data from, and decide which nodes should be annotated with summary operation. • For example, • For MNIST digits CNNs, we'd like to record how the learning rate varies over time, and how the objective function is changing • We’d like to record distribution of gradients or weights TensorBoard
- 77. TensorBoard
- 78. TensorBoard
Editor's Notes
- #15: Regular Neural Nets don't scale well to full images.