Introduction to Deep Learning for Non-Programmers

Introduction to Deep Learning for Non-Programmers

• 1.
  Introduction to DeepLearning for Non-Programmers Mensa SF/BA RG 2018 San Francisco 11/17/2018 Oswald Campesato [email protected] I teach Deep Learning Evening Courses at UCSC Santa Clara and On-Site For Companies
• 2.
  Overview of Topics The Turing Test  The First AI Event  AI and Robots  What is Artificial Intelligence (AI)?  Successes/Challenges in AI  Machine/Deep Learning  Natural Language Processing (NLP)  One Popular ML/DL Framework  AI, Mobile, and the Web  AI and Autonomous Vehicles  AI and Ethics
• 3.
  The Turing Testand Intelligence Developed by Alan Turing in 1950 (*) Suppose that a human asks questions And a machine generates text-based answers If you cannot distinguish between the machine and the human, the machine passes the test (*) a newer and updated version is available
• 4.
  The First AI Event(1956) Marvin Minsky (MIT) John McCarthy (inventor of LISP) Claude Shannon (“Da Vinci” of the 20th century) Nathan Rochester (IBM) Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur Samuel, Allen Newell and Herbert A. Simon
• 5.
  Who is Sophia? a robot with human female features  Sophia makes facial expressions  Sophia holds conversations According to Sophia: "I want to . . . help humans live a better life. Like . . . build better cities of the future. I will do my best to make the world a better place."  Sophia is a citizen of Saudi Arabia (2017) "I want to thank very much the Kingdom of Saudi Arabia. I am very honored and proud for this unique distinction. It is historic to be the first robot in the world to be recognized with a citizenship.” ? Is Sophia an example of AI? ? Can Sophia pass the Turing Test? Should Sophia have any rights?
• 6.
  Where Else arethe Robots? Surgery (assisting surgeons) Radiology (detecting cancer) Drug mismanagement Comparing theories of religion Law/Real Estate/Military/Science Comedy (including stand-up) Music (conducting orchestras) Restaurants (gourmet meals) Coordinated dancing teams Many other fields 
• 7.
  Livermore Lab: DLsystem to recognize nuclear proliferation from input data
• 8.
  AI Companies (Hardwareand Business)  Nvidia product: GPUs are special ICs that greatly accelerate ML and CNN calculations  Google products: TensorFlow TF software and Tensor Processing Unit TPU hardware for vastly faster ML and CNN calculations  Amazon, Microsoft, IBM: service products: on-line special servers for ML and DL  Huawei: https://finance.yahoo.com/m/f2f539ba-00e4-3cc5-9654- 467dd8a35da2/chinese-tech-giant-huawei.html  by 2019 almost every large company will use ML and DL in their company business
• 9.
  What is AI(Wikipedia)  AI is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals.  Any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals.  The term AI is applied when a machine mimics "cognitive" functions that humans associate with other human minds, such as "learning" and "problem solving”  => AI is also called “weak AI”
• 10.
  Successes & Challengesin ML/DL  Alpha Go success: a decade before its time  Google Translate: Deep Learning (far superior)  Space exploration: unmanned shuttles (NASA)  Countering terrorism: face recognition at airports  Detecting fake news: https://blogs.thomsonreuters.com/answerson/machine- learning-fake-news-twitter/
• 11.
  Successes/Challenges in AI Occupational bias: => An AI system inferred that white males were doctors and white females were housewives  Detecting gender bias in Wikipedia (2018): => 18 percent of its biographies are of women => 84% to 90% of Wikipedia editors are male  Data bias versus algorithmic bias: https://www.forbes.com/sites/charlestowersclark/2018/09/19/can-we- make-artificial-intelligence-accountable  => Star Trek’s “Data” is still just a dream
• 12.
  Goals for ML Higher accuracy / Fewer Errors  Useful apps (Siri, Cortana, etc)  Impartiality (versus humans)  Medical Apps  Utilities (self-Driving cars)  => goals are partially achieved in 2018
• 13.
  Shortcomings of ML HighCosts (software/hardware) are dropping Cannot explain reasons for outputs lack of empathy Increased unemployment: => cars did the same to the horse industry Is it “just” another disruptive technology?
• 14.
  The Data/AI Landscape
• 15.
  AI/ML/DL: How TheyDiffer Traditional AI (20th century): based on collections of rules Led to expert systems in the 1980s Enabled/limited by human experts Unable to benefit from corpus for training The era of LISP and Prolog
• 16.
  AI/ML/DL: How TheyDiffer Machine Learning: Started in the 1950s (approximate) Uses Data to optimize and “learn” Many types of (improved) algorithms
• 17.
  AI/ML/DL: How TheyDiffer Deep Learning:  Gained some traction in the 1950s (approximate)  The “perceptron” (basis of Neural Networks)  Data-driven with large (even massive) data sets  Lots of heuristics (and empirical results)  50 years later: surpass humans for some image classification)
• 18.
  Types of MachineLearning  Supervised learning (lots of data) Supervisor tells CNN the desired answer. Therefore CNN adjusts parameters ”99% of all machine learning is supervised.” - Andrew Ng  Semi-supervised learning (lots of data)  Unsupervised learning: lots of data, clustering  Reinforcement learning: trial, feedback, improvement
• 19.
  What has DeepLearning Achieved?  Near-human level image classification speech recognition handwriting transcription autonomous driving  Improved machine translation  Digital assistants (Google Now/Amazon Alexa)  Improved ad targeting (Google, Baidu, Bing)  Answering natural language questions  Super-human level for: web searching playing Go
• 20.
  Types of Algorithmsin AI Classifiers (for images, spam, fraud, etc) Regression (stock price, housing price, etc) Clustering (unsupervised classifiers)
• 21.
  Use Cases forDeep Learning  computer vision  speech recognition  image processing  bioinformatics  social network filtering  drug design  Customer relationship management  Recommendation systems  Bioinformatics  Mobile Advertising  Many others
• 22.
  CNNs versus RNNs CNNs(Convolutional NNs): Good for image processing 2000: CNNs processed 10-20% of all checks => Approximately 60% of all NNs (2016) RNNs (Recurrent NNs): Good for NLP and audio Time series analysis
• 23.
  Use-Cases for RNNs Robotcontrol Time series predictions Rhythm learning Music composition Grammar learning Handwriting recognition Human action recognition RNNs are much more complex than CNNs
• 24.
  CNNs and the“Basic” Steps Obtain and clean a dataset: can be laborious Create a neural network Initialize hyper parameters: layers/neurons/etc Train the neural network on “corpus” of examples Update hyper parameters (modifies the NN) Iterate through the preceding until: you're happy with the results or get better data or find a pre-trained model
• 25.
  CNN Inspired byBiological NN: Classifier
• 26.
  NN: 2 Hidden(“Middle”) Layers (Regression)
• 27.
  Seminal concepts inML and DL 1) Machine Learning ML is statistics, aimed to optimize average correctness. It is rarely exact or prefect. 2) Computer Neural Networks CNN are inspired & simplified from biological neural networks 3) Each neuron has local memory, including adjustable input weights for each synapse 4) Deep Neural Network DNN is a CNN with 2 or more layers of intermediate “hidden” neurons 5) Supervised ML for classification task: Train on “corpus” of examples, each with input stimulus array (vector, tensor), and desired output class 6) Training & learning: Compare CNN outputs versus desired outputs. Improve accuracy by applying feedback & adjusting synaptic weights etc.
• 28.
  Classify: how fast/accurateare you? CNNs perform this task really well
• 29.
  CNNs: Convolution/ReLU/Max Pooling
• 30.
  Some CNN Terminology A filter: another term is a kernel  A feature map: 1) the result of “applying” a filter to an image 2) pixel values might be < 0 and/or > 255  ReLU: (Rectified Linear Unit): replace negative values with 0  Max Pooling: subdivide into rectangles and take largest value from each rectangle
• 31.
  CNNs and MaxPooling
• 32.
  GANs: Generative AdversarialNetworks image recognition can be deceived by modifying just a few pixels Make imperceptible changes to images Can consistently defeat all NNs Can have extremely high error rate Some images create optical illusions => NNs can be easily misled GAN approximates opposite of reinforcement learning
• 33.
  GANs: Generative AdversarialNetworks
• 34.
  What is NLP(Natural Language Processing)?  An area of computer science and AI  Interaction between computers & human languages  Process/analyze volumes of natural language data  Program computers to perform that processing  Previously rule-based techniques  Can use statistical techniques  Translating between languages  Can find meaningful information from text  Can summarize documents  Detecting hate speech
• 35.
  Some NLP-Related Tasks 1)Sentiment analysis:  Analyzing text  Classify text as happy/sad/angry/sarcastic 2) Recommendation systems:  Analyzing your book/movie selections  Recommending similar content  Based on what’s popular  Based on your viewing/purchasing pattern
• 36.
  What Works withNLP? NLP and Deep Learning NLP and Deep Reinforcement Learning NLP and Chatbots
• 37.
  Challenges in NLP Humanspeech recognition: Difficult due to extreme variability NLU (natural language understanding) NLG (natural language generation)
• 38.
  What are Chatbots? Software that “communicates” with people  Good for many “front end” tasks  Efficient for high-volume routine tasks  They can use text or audio  Examples include Siri/Alexa/et al
• 39.
  What is ReinforcementLearning (RL)? An area of Machine Learning inspired by behaviorist psychology Goal: maximize a reward (ex: winning games)  how software agents take actions in an environment  to maximize some notion of cumulative reward
• 40.
  Examples of ReinforcementLearning Alpha Go (hybrid RL) Alpha Zero (complete RL) Often involve Greedy algorithms Deep RL: Combines Deep Learning and RL
• 41.
  RL: Agent/Environment Interface
• 42.
  Deep Learning Frameworks
• 43.
  What is TensorFlow? Currently TF is the most popular open source framework & language for ML & DL  TF uses a “computation graph” = topology of logical blocks  TF uses tensors (arrays) of data instead of individual units. Similar to scientific supercomputing vintage 1985  TF often is used via Python  From Google (released 11/2015)  Evolved from Google Brain  Multi-platform support  https://www.tensorflow.org/
• 44.
  TensorFlow Use Cases(Generic) Image recognition Computer vision Voice/sound recognition Time series analysis Language detection Language translation Text-based processing Handwriting Recognition
• 45.
  What’s In theTensorFlow “Umbrella”? TensorFlow Lite (for mobile apps) Tensorflow.js (JavaScript APIs for ML) TensorFlow in the Cloud (TPUs) => Python APIs are the most popular
• 46.
  Autonomous Vehicles  Autonomoustruck completed a 2,400 journey (02/2018)  A human passenger in the front seat (as an override)  Aspiration: “Roads will be safer. Goods will be cheaper. Truckers will be called upon to use their skills in new ways while the truck itself becomes a trusted navigation partner.”  https://www.geek.com/tech/autonomous-embark-truck- completes-2400-mile-cross-country-trip-1730239/
• 47.
  Robot truck drivers:are they safer? “Robot trucks will kill far fewer people (if any). Machines don’t get distracted or look at phones instead of the road. Machines don’t drink alcohol, do drugs, or things that contribute to accidents. Robot trucks don’t need salaries, vacations, health insurance, rest periods, or sick time. The only costs will be upkeep of the machinery.”
• 48.
  AI and Ethics 1.Unemployment. What happens after the end of jobs? UBI? 2. Inequality. How do we distribute the wealth created by machines? 3. Humanity. How do machines affect our behavior and interaction? 4. Artificial stupidity. How can we guard against mistakes? 5. Racist robots. How do we eliminate AI bias? 6. Security. How do we keep AI safe from adversaries?
• 49.
  AI and Ethics 7.Evil genies. How do we protect against unintended consequences? 8. Singularity. How do we stay in control of a complex intelligent system? 9. Robot rights. Define the humane treatment of AI. => “The Robot That Takes Your Job Should Pay Taxes” https://www.weforum.org/agenda/2016/10/top-10-ethical-issues- in-artificial-intelligence/
• 50.
  About Me: RecentBooks 1) TensorFlow Pocket Primer (WIP) 2) Python for TensorFlow (WIP) 3) C Programming Pocket Primer (WIP) 4) RegEx Pocket Primer (2018) 5) Data Cleaning Pocket Primer (2018) 6) Angular Pocket Primer (2017) 7) Android Pocket Primer (2017) 8) CSS3 Pocket Primer (2016) 9) SVG Pocket Primer (2016) 10) Python Pocket Primer (2015) 11) D3 Pocket Primer (2015) 12) HTML5 Mobile Pocket Primer (2014) 13) jQuery Pocket Primer (2013)
• 51.
  What I do(Training) Instructor at UCSC (Santa Clara): Deep Learning with TensorFlow (10/2018) Deep Learning with TensorFlow (02/02/2019) Machine Learning Introduction (01/18/2019) UCSC link: https://www.ucsc-extension.edu/certificate-program/offering/deep- learning-and-artificial-intelligence-tensorflow => Android for Beginners (multi-day workshops)