Machine Intelligence at Google Scale: TensorFlow
12 likes3,357 views
- TensorFlow is Google's open source machine learning library for developing and training neural networks and deep learning models. It operates using data flow graphs to represent computation. - TensorFlow can be used across many platforms including data centers, CPUs, GPUs, mobile phones, and IoT devices. It is widely used at Google across many products and research areas involving machine learning. - The TensorFlow library is used along with higher level tools in Google's machine learning platform including TensorFlow Cloud, Machine Learning APIs, and Cloud Machine Learning Platform to make machine learning more accessible and scalable.
1 of 28
Downloaded 235 times
Ad
Recommended
Accelerating query processing with materialized views in Apache Hive
Accelerating query processing with materialized views in Apache HiveDataWorks Summit Over the last few years, the Apache Hive community has been working on advancements to enable a full new range of use cases for the project, moving from its batch processing roots towards a SQL interactive query answering platform. Traditionally, one of the most powerful techniques used to accelerate query processing in data warehouses is the precomputation of relevant summaries or materialized views.
This talk presents our work on introducing materialized views and automatic query rewriting based on those materializations in Apache Hive. In particular, materialized views can be stored natively in Hive or in other systems such as Druid using custom storage handlers, and they can seamlessly exploit new exciting Hive features such as LLAP acceleration. Then the optimizer relies in Apache Calcite to automatically produce full and partial rewritings for a large set of query expressions comprising projections, filters, join, and aggregation operations. We shall describe the current coverage of the rewriting algorithm, how Hive controls important aspects of the life cycle of the materialized views such as the freshness of their data, and outline interesting directions for future improvements. We include an experimental evaluation highlighting the benefits that the usage of materialized views can bring to the execution of Hive workloads.
Speaker
Jesus Camacho Rodriguez, Member of Technical Staff, Hortonworks
MLOps in action
MLOps in actionPieter de Bruin Machine Learning operations brings data science to the world of devops. Data scientists create models on their workstations. MLOps adds automation, validation and monitoring to any environment including machine learning on kubernetes. In this session you hear about latest developments and see it in action.
プログラムを自動生成する技術 ～ Programming by Example ～（NTTデータ テクノロジーカンファレンス 2020 発表資料）
プログラムを自動生成する技術 ～ Programming by Example ～（NTTデータ テクノロジーカンファレンス 2020 発表資料）NTT DATA Technology & Innovation プログラムを自動生成する技術
～ Programming by Example ～
（NTTデータ テクノロジーカンファレンス 2020 発表資料）
2020年10月15日（木）
NTTデータ
システム技術本部 デジタル技術部
竹之内 啓太
講演動画は、YouTubeチャンネル「NTT DATA Tech」にて公開中！
https://www.youtube.com/watch?v=3Ccq2bY6BkE
Real-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotXiang Fu Trino summit 2021:
Overview of Trino Pinot Connector, which bridges the flexibility of Trino's full SQL support to the power of Apache Pinot's realtime analytics, giving you the best of both worlds.
Cloud arch patterns
Cloud arch patternsCorey Huinker This document discusses PostgreSQL database architecture patterns for running PostgreSQL at scale when a relational database as a service like Amazon RDS won't meet needs. It describes challenges faced with MySQL, Redshift and Vertica and how PostgreSQL was better suited through techniques like partitioning by date, TOAST compression, foreign data wrappers, and poor man's parallel processing. Key takeaways are that PostgreSQL supported scaling to petabytes of data, sub-second queries across large date ranges, and custom extensions needed while avoiding limitations and expenses of other database options.
MLflow Model Serving
MLflow Model ServingDatabricks Discuss the different ways model can be served with MLflow. We will cover both the open source MLflow and Databricks managed MLflow ways to serve models. Will cover the basic differences between batch scoring and real-time scoring. Special emphasis on the new upcoming Databricks production-ready model serving.
Secrets in Kubernetes
Secrets in KubernetesJerry Jalava This is a presentation I held at "DevOps and Security" -meetup on 5th of April 2016 at RedHat.
Source is available at: https://github.com/jerryjj/devsec_050416
DevOps for Databricks
DevOps for DatabricksDatabricks Applying DevOps to Databricks can be a daunting task. In this talk this will be broken down into bite size chunks. Common DevOps subject areas will be covered, including CI/CD (Continuous Integration/Continuous Deployment), IAC (Infrastructure as Code) and Build Agents.
We will explore how to apply DevOps to Databricks (in Azure), primarily using Azure DevOps tooling. As a lot of Spark/Databricks users are Python users, will will focus on the Databricks Rest API (using Python) to perform our tasks.
Terraform introduction
Terraform introductionJason Vance This document introduces infrastructure as code (IaC) using Terraform and provides examples of deploying infrastructure on AWS including:
- A single EC2 instance
- A single web server
- A cluster of web servers using an Auto Scaling Group
- Adding a load balancer using an Elastic Load Balancer
It also discusses Terraform concepts and syntax like variables, resources, outputs, and interpolation. The target audience is people who deploy infrastructure on AWS or other clouds.
Monitoring With Prometheus
Monitoring With PrometheusKnoldus Inc. In this session, we will start with the importance of monitoring of services and infrastructure. We will discuss about Prometheus an opensource monitoring tool. We will discuss the architecture of Prometheus. We will also discuss some visualization tools which can be used over Prometheus. Then we will have a quick demo for Prometheus and Grafana.
A Practical Enterprise Feature Store on Delta Lake
A Practical Enterprise Feature Store on Delta LakeDatabricks The feature store is a data architecture concept used to accelerate data science experimentation and harden production ML deployments. Nate Buesgens and Bryan Christian describe a practical approach to building a feature store on Delta Lake at a large financial organization. This implementation has reduced feature engineering “wrangling” time by 75% and has increased the rate of production model delivery by 15x. The approach described focuses on practicality. It is informed by innovative approaches such as Feast, but our primary goal is evolutionary extensions of existing patterns that can be applied to any Delta Lake architecture.
Key Takeaways:
– Understand the key use cases that motivate the feature store from both a data science and engineering perspective.
– Consider edge cases where there may be opportunities for simplification such as “online” predictions.
– Review a typical logical data model for a feature store and how that can be applied to your business domain.
– Consider options for physical storage of the feature store in the Delta Lake.
– Understand common access patterns including metadata-based feature discovery.
Apache Arrow Flight – ビッグデータ用高速データ転送フレームワーク #dbts2021
Apache Arrow Flight – ビッグデータ用高速データ転送フレームワーク #dbts2021Kouhei Sutou ビッグデータは現実的な時間で処理できてこそ活きます。処理時間はデータ処理アルゴリズムの計算量だけで決まり…ません！適切なノードにデータを配置しないとそもそもデータ処理できませんが、大量ノードが協調するビッグデータ処理ではデータ移動コストを無視できません。Apache Arrow Flightを使えばネットワーク帯域限界まで高速にデータ転送できます。
このセッションではApache Arrow Flightの仕組みを説明します。また、分散計算プラットフォームApache Arrow BallistaがApache Arrow Flightをどのように利用しているかも紹介します。
Productionzing ML Model Using MLflow Model Serving
Productionzing ML Model Using MLflow Model ServingDatabricks Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate.
MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.
Big Data Security in Apache Projects by Gidon Gershinsky
Big Data Security in Apache Projects by Gidon GershinskyGidonGershinsky ApacheCon 2022 talk - Big Data Security in Apache Projects.
Encryption in Parquet, Spark, Iceberg, Trino, Flink, Arrow.
Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...
Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...HostedbyConfluent Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Ethan Guo | Current 2022
Back in 2016, Apache Hudi brought transactions, change capture on top of data lakes, what is today referred to as the Lakehouse architecture. In this session, we first introduce Apache Hudi and the key technology gaps it fills in the modern data architecture. Bridging traditional data lakes and warehouses, Hudi helps realize the Lakehouse vision, by bringing transactions, optimized table metadata to data lakes and powerful storage layout optimizations, moving them closer to cloud warehouses of today. Viewed from a data engineering lens, Hudi also plays a key unifying role between the batch and stream processing worlds, by acting as a columnar, server-less ""state store"" for batch jobs, ushering in what we call the incremental processing model, where batch jobs can consume new data, update/delete intermediate results in a Hudi table, instead of re-computing/re-write entire output like old-school big batch jobs.
Rest of talk focusses on a deep dive into the some of the time-tested design choices and tradeoffs in Hudi, that helps power some of the largest transactional data lakes on the planet today. We will start by describing a tour of the storage format design, including data, metadata layouts and of course Hudi's timeline, an event log that is central to implementing ACID transactions and concurrency control. We will delve deeper into the practical concurrency control pitfalls in data lakes, and show how Hudi's hybrid approach combining MVCC with optimistic concurrency control, lowers contention and unlocks minute-level near real-time commits to Hudi tables. We will conclude with code examples that showcase Hudi's rich set of table services that perform vital table management such as cleaning older file versions, compaction of delta logs into base files, dynamic re-clustering for faster query performance, or the more recently introduced indexing service that maintains Hudi's multi-modal indexing capabilities.
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & KubeflowJan Kirenz Machine Learning Operations (MLOps) for Machine Learning Lifecycle Management: Building pipelines with Tensor Flow Extended (TFX) and Kubeflow.
Kubernetes 基盤における非機能試験の deepdive（Kubernetes Novice Tokyo #17 発表資料）
Kubernetes 基盤における非機能試験の deepdive（Kubernetes Novice Tokyo #17 発表資料）NTT DATA Technology & Innovation Kubernetes 基盤における非機能試験の deepdive
〜 性能試験は CI/CD にお任せ！ 〜
（Kubernetes Novice Tokyo #17 発表資料）
2022年3月23日（水）
NTTデータ
ITSP事業本部 C＆P事業部
逆井 啓佑
What is Puppet? | How Puppet Works? | Puppet Tutorial For Beginners | DevOps ...
What is Puppet? | How Puppet Works? | Puppet Tutorial For Beginners | DevOps ...Simplilearn This presentation on DevOps Puppet will help you understand what is Puppet, why we need Puppet, components of Puppet, working with Puppet, companies adopting Puppet along with Puppet manifest. Puppet is basically a configuration management tool. A configuration management tool is a tool which runs in client mode or client-server mode and has some configuration language wherein you specify what your state of the system would be. Puppet tool enables you to automate all your IT infrastructures and delivering you control over each Puppet agent in a network. With Puppet, a simple code can be written which then is deployed onto the servers. All servers are rolled back to their previous working states or set to the new desired states in a matter of seconds. Puppet ensures that all your systems are configured the desired states. Now, let us get started and understand Puppet tool in detail.
Below topics are explained in this Puppet presentation:
1. Why Puppet?
2. What is Puppet?
3. Components of Puppet
4. Working on Puppet
5. Companies adopting Puppet
6. Writing manifest in Puppet
Simplilearn's DevOps Certification Training Course will prepare you for a career in DevOps, the fast-growing field that bridges the gap between software developers and operations. You’ll become an expert in the principles of continuous development and deployment, automation of configuration management, inter-team collaboration and IT service agility, using modern DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios. DevOps jobs are highly paid and in great demand, so start on your path today.
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at: https://www.simplilearn.com/
Delight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-PlatformDatabricks This document introduces Delight, an improved Apache Spark UI created by Data Mechanics. Delight provides high-level visualizations of Spark jobs to help users identify inefficiencies and reduce costs. It collects metrics during and after jobs to show CPU usage, task duration, and efficiency. Delight is open source and works on any Spark platform by installing an agent. Data Mechanics aims to further enhance Delight with real-time metrics, driver memory collection, and automated recommendations.
Kubeflow
KubeflowKarane Vieira Kubeflow is an open-source project that makes deploying machine learning workflows on Kubernetes simple and scalable. It provides components for machine learning tasks like notebooks, model training, serving, and pipelines. Kubeflow started as a Google side project but is now used by many companies like Spotify, Cisco, and Itaú for machine learning operations. It allows running workflows defined in notebooks or pipelines as Kubernetes jobs and serves models for production.
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowDatabricks This document summarizes a talk on building an ML platform with Ray and MLflow. Ray is an open-source framework for distributed computing and machine learning. It provides libraries like Ray Tune for hyperparameter tuning and Ray Serve for model serving. MLflow is a tool for managing the machine learning lifecycle including tracking experiments, managing models, and deploying models. The talk demonstrates how to build an end-to-end ML platform by integrating Ray and MLflow for distributed training, hyperparameter tuning, model tracking, and low-latency serving.
DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDatabricks The document discusses migrating a data warehouse to the Databricks Lakehouse Platform. It outlines why legacy data warehouses are struggling, how the Databricks Platform addresses these issues, and key considerations for modern analytics and data warehousing. The document then provides an overview of the migration methodology, approach, strategies, and key takeaways for moving to a lakehouse on Databricks.
MLOps by Sasha Rosenbaum
MLOps by Sasha RosenbaumSasha Rosenbaum This document discusses MLOps, which is applying DevOps practices and principles to machine learning to enable continuous delivery of ML models. It explains that ML models need continuous improvement through retraining but data scientists currently lack tools for quick iteration, versioning, and deployment. MLOps addresses this by providing ML pipelines, model management, monitoring, and retraining in a reusable workflow similar to how software is developed. Implementing even a basic CI/CD pipeline for ML can help iterate models more quickly than having no pipeline at all. The document encourages building responsible AI through practices like ensuring model performance and addressing bias.
Introductory Overview to Managing AWS with Terraform
Introductory Overview to Managing AWS with TerraformMichael Heyns The document provides an overview of Terraform including:
- Terraform is an open source tool from HashiCorp that allows defining and provisioning infrastructure in a code-based declarative way across multiple cloud platforms and services.
- Key concepts include providers that define cloud resources, configuration files that declare the desired state, and a plan-apply workflow to provision and manage infrastructure resources.
- Common Terraform commands are explained like init, plan, apply, destroy, output and their usage.
PostgreSQLでpg_bigmを使って日本語全文検索 （MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）
PostgreSQLでpg_bigmを使って日本語全文検索 （MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）NTT DATA OSS Professional Services ■PostgreSQLでpg_bigmを使って日本語全文検索
～pg_bigmで全文検索するときに知っておくべき8のこと～
（MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）
NTTデータ 澤田 雅彦
Building an MLOps Stack for Companies at Reasonable Scale
Building an MLOps Stack for Companies at Reasonable ScaleMerelda A practical talk on showing the following:
1. Challenges of Deploying ML today
2. How to do MLOps:
- Principles over Technology
- Convention over Configuration
3. What's a reasonable MLOps Stack
4. Demo on Google Collab to Deployed Endpoint
コンテナとimmutableとわたし。あとセキュリティ。（Kubernetes Novice Tokyo #15 発表資料）
コンテナとimmutableとわたし。あとセキュリティ。（Kubernetes Novice Tokyo #15 発表資料）NTT DATA Technology & Innovation コンテナとimmutableとわたし。あとセキュリティ。
（Kubernetes Novice Tokyo #15 発表資料）
2021年12月7日（火）
NTTデータ
システム技術本部生産技術部 クラウド技術センタ
望月 敬太
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...Sam Putnam [Deep Learning] 1) The document discusses TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM as presented by Sam Putnam on 6/8/2017.
2) It provides information on exporting models for TensorFlow Serving, deploying TensorFlow to Android, tools for mobile deep learning like Inception and MobileNets, and using Deeplearning4j on the JVM with integration with Spark.
3) The document shares links to resources on these topics and thanks sponsors while inviting people to join future discussions.
Introducing TensorFlow: The game changer in building "intelligent" applications
Introducing TensorFlow: The game changer in building "intelligent" applicationsRokesh Jankie This is the slidedeck used for the presentation of the Amsterdam Pipeline of Data Science, held in December 2016. TensorFlow in the open source library from Google to implement deep learning, neural networks. This is an introduction to Tensorflow.
Note: Videos are not included (which were shown during the presentation)
More Related Content
What's hot (20)
Terraform introduction
Terraform introductionJason Vance This document introduces infrastructure as code (IaC) using Terraform and provides examples of deploying infrastructure on AWS including:
- A single EC2 instance
- A single web server
- A cluster of web servers using an Auto Scaling Group
- Adding a load balancer using an Elastic Load Balancer
It also discusses Terraform concepts and syntax like variables, resources, outputs, and interpolation. The target audience is people who deploy infrastructure on AWS or other clouds.
Monitoring With Prometheus
Monitoring With PrometheusKnoldus Inc. In this session, we will start with the importance of monitoring of services and infrastructure. We will discuss about Prometheus an opensource monitoring tool. We will discuss the architecture of Prometheus. We will also discuss some visualization tools which can be used over Prometheus. Then we will have a quick demo for Prometheus and Grafana.
A Practical Enterprise Feature Store on Delta Lake
A Practical Enterprise Feature Store on Delta LakeDatabricks The feature store is a data architecture concept used to accelerate data science experimentation and harden production ML deployments. Nate Buesgens and Bryan Christian describe a practical approach to building a feature store on Delta Lake at a large financial organization. This implementation has reduced feature engineering “wrangling” time by 75% and has increased the rate of production model delivery by 15x. The approach described focuses on practicality. It is informed by innovative approaches such as Feast, but our primary goal is evolutionary extensions of existing patterns that can be applied to any Delta Lake architecture.
Key Takeaways:
– Understand the key use cases that motivate the feature store from both a data science and engineering perspective.
– Consider edge cases where there may be opportunities for simplification such as “online” predictions.
– Review a typical logical data model for a feature store and how that can be applied to your business domain.
– Consider options for physical storage of the feature store in the Delta Lake.
– Understand common access patterns including metadata-based feature discovery.
Apache Arrow Flight – ビッグデータ用高速データ転送フレームワーク #dbts2021
Apache Arrow Flight – ビッグデータ用高速データ転送フレームワーク #dbts2021Kouhei Sutou ビッグデータは現実的な時間で処理できてこそ活きます。処理時間はデータ処理アルゴリズムの計算量だけで決まり…ません！適切なノードにデータを配置しないとそもそもデータ処理できませんが、大量ノードが協調するビッグデータ処理ではデータ移動コストを無視できません。Apache Arrow Flightを使えばネットワーク帯域限界まで高速にデータ転送できます。
このセッションではApache Arrow Flightの仕組みを説明します。また、分散計算プラットフォームApache Arrow BallistaがApache Arrow Flightをどのように利用しているかも紹介します。
Productionzing ML Model Using MLflow Model Serving
Productionzing ML Model Using MLflow Model ServingDatabricks Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate.
MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.
Big Data Security in Apache Projects by Gidon Gershinsky
Big Data Security in Apache Projects by Gidon GershinskyGidonGershinsky ApacheCon 2022 talk - Big Data Security in Apache Projects.
Encryption in Parquet, Spark, Iceberg, Trino, Flink, Arrow.
Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...
Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...HostedbyConfluent Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Ethan Guo | Current 2022
Back in 2016, Apache Hudi brought transactions, change capture on top of data lakes, what is today referred to as the Lakehouse architecture. In this session, we first introduce Apache Hudi and the key technology gaps it fills in the modern data architecture. Bridging traditional data lakes and warehouses, Hudi helps realize the Lakehouse vision, by bringing transactions, optimized table metadata to data lakes and powerful storage layout optimizations, moving them closer to cloud warehouses of today. Viewed from a data engineering lens, Hudi also plays a key unifying role between the batch and stream processing worlds, by acting as a columnar, server-less ""state store"" for batch jobs, ushering in what we call the incremental processing model, where batch jobs can consume new data, update/delete intermediate results in a Hudi table, instead of re-computing/re-write entire output like old-school big batch jobs.
Rest of talk focusses on a deep dive into the some of the time-tested design choices and tradeoffs in Hudi, that helps power some of the largest transactional data lakes on the planet today. We will start by describing a tour of the storage format design, including data, metadata layouts and of course Hudi's timeline, an event log that is central to implementing ACID transactions and concurrency control. We will delve deeper into the practical concurrency control pitfalls in data lakes, and show how Hudi's hybrid approach combining MVCC with optimistic concurrency control, lowers contention and unlocks minute-level near real-time commits to Hudi tables. We will conclude with code examples that showcase Hudi's rich set of table services that perform vital table management such as cleaning older file versions, compaction of delta logs into base files, dynamic re-clustering for faster query performance, or the more recently introduced indexing service that maintains Hudi's multi-modal indexing capabilities.
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
MLOps - Build pipelines with Tensor Flow Extended & KubeflowJan Kirenz Machine Learning Operations (MLOps) for Machine Learning Lifecycle Management: Building pipelines with Tensor Flow Extended (TFX) and Kubeflow.
Kubernetes 基盤における非機能試験の deepdive（Kubernetes Novice Tokyo #17 発表資料）
Kubernetes 基盤における非機能試験の deepdive（Kubernetes Novice Tokyo #17 発表資料）NTT DATA Technology & Innovation Kubernetes 基盤における非機能試験の deepdive
〜 性能試験は CI/CD にお任せ！ 〜
（Kubernetes Novice Tokyo #17 発表資料）
2022年3月23日（水）
NTTデータ
ITSP事業本部 C＆P事業部
逆井 啓佑
What is Puppet? | How Puppet Works? | Puppet Tutorial For Beginners | DevOps ...
What is Puppet? | How Puppet Works? | Puppet Tutorial For Beginners | DevOps ...Simplilearn This presentation on DevOps Puppet will help you understand what is Puppet, why we need Puppet, components of Puppet, working with Puppet, companies adopting Puppet along with Puppet manifest. Puppet is basically a configuration management tool. A configuration management tool is a tool which runs in client mode or client-server mode and has some configuration language wherein you specify what your state of the system would be. Puppet tool enables you to automate all your IT infrastructures and delivering you control over each Puppet agent in a network. With Puppet, a simple code can be written which then is deployed onto the servers. All servers are rolled back to their previous working states or set to the new desired states in a matter of seconds. Puppet ensures that all your systems are configured the desired states. Now, let us get started and understand Puppet tool in detail.
Below topics are explained in this Puppet presentation:
1. Why Puppet?
2. What is Puppet?
3. Components of Puppet
4. Working on Puppet
5. Companies adopting Puppet
6. Writing manifest in Puppet
Simplilearn's DevOps Certification Training Course will prepare you for a career in DevOps, the fast-growing field that bridges the gap between software developers and operations. You’ll become an expert in the principles of continuous development and deployment, automation of configuration management, inter-team collaboration and IT service agility, using modern DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios. DevOps jobs are highly paid and in great demand, so start on your path today.
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at: https://www.simplilearn.com/
Delight: An Improved Apache Spark UI, Free, and Cross-Platform
Delight: An Improved Apache Spark UI, Free, and Cross-PlatformDatabricks This document introduces Delight, an improved Apache Spark UI created by Data Mechanics. Delight provides high-level visualizations of Spark jobs to help users identify inefficiencies and reduce costs. It collects metrics during and after jobs to show CPU usage, task duration, and efficiency. Delight is open source and works on any Spark platform by installing an agent. Data Mechanics aims to further enhance Delight with real-time metrics, driver memory collection, and automated recommendations.
Kubeflow
KubeflowKarane Vieira Kubeflow is an open-source project that makes deploying machine learning workflows on Kubernetes simple and scalable. It provides components for machine learning tasks like notebooks, model training, serving, and pipelines. Kubeflow started as a Google side project but is now used by many companies like Spotify, Cisco, and Itaú for machine learning operations. It allows running workflows defined in notebooks or pipelines as Kubernetes jobs and serves models for production.
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowDatabricks This document summarizes a talk on building an ML platform with Ray and MLflow. Ray is an open-source framework for distributed computing and machine learning. It provides libraries like Ray Tune for hyperparameter tuning and Ray Serve for model serving. MLflow is a tool for managing the machine learning lifecycle including tracking experiments, managing models, and deploying models. The talk demonstrates how to build an end-to-end ML platform by integrating Ray and MLflow for distributed training, hyperparameter tuning, model tracking, and low-latency serving.
DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDatabricks The document discusses migrating a data warehouse to the Databricks Lakehouse Platform. It outlines why legacy data warehouses are struggling, how the Databricks Platform addresses these issues, and key considerations for modern analytics and data warehousing. The document then provides an overview of the migration methodology, approach, strategies, and key takeaways for moving to a lakehouse on Databricks.
MLOps by Sasha Rosenbaum
MLOps by Sasha RosenbaumSasha Rosenbaum This document discusses MLOps, which is applying DevOps practices and principles to machine learning to enable continuous delivery of ML models. It explains that ML models need continuous improvement through retraining but data scientists currently lack tools for quick iteration, versioning, and deployment. MLOps addresses this by providing ML pipelines, model management, monitoring, and retraining in a reusable workflow similar to how software is developed. Implementing even a basic CI/CD pipeline for ML can help iterate models more quickly than having no pipeline at all. The document encourages building responsible AI through practices like ensuring model performance and addressing bias.
Introductory Overview to Managing AWS with Terraform
Introductory Overview to Managing AWS with TerraformMichael Heyns The document provides an overview of Terraform including:
- Terraform is an open source tool from HashiCorp that allows defining and provisioning infrastructure in a code-based declarative way across multiple cloud platforms and services.
- Key concepts include providers that define cloud resources, configuration files that declare the desired state, and a plan-apply workflow to provision and manage infrastructure resources.
- Common Terraform commands are explained like init, plan, apply, destroy, output and their usage.
PostgreSQLでpg_bigmを使って日本語全文検索 （MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）
PostgreSQLでpg_bigmを使って日本語全文検索 （MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）NTT DATA OSS Professional Services ■PostgreSQLでpg_bigmを使って日本語全文検索
～pg_bigmで全文検索するときに知っておくべき8のこと～
（MySQLとPostgreSQLの日本語全文検索勉強会 発表資料）
NTTデータ 澤田 雅彦
Building an MLOps Stack for Companies at Reasonable Scale
Building an MLOps Stack for Companies at Reasonable ScaleMerelda A practical talk on showing the following:
1. Challenges of Deploying ML today
2. How to do MLOps:
- Principles over Technology
- Convention over Configuration
3. What's a reasonable MLOps Stack
4. Demo on Google Collab to Deployed Endpoint
コンテナとimmutableとわたし。あとセキュリティ。（Kubernetes Novice Tokyo #15 発表資料）
コンテナとimmutableとわたし。あとセキュリティ。（Kubernetes Novice Tokyo #15 発表資料）NTT DATA Technology & Innovation コンテナとimmutableとわたし。あとセキュリティ。
（Kubernetes Novice Tokyo #15 発表資料）
2021年12月7日（火）
NTTデータ
システム技術本部生産技術部 クラウド技術センタ
望月 敬太
Viewers also liked (7)
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...Sam Putnam [Deep Learning] 1) The document discusses TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM as presented by Sam Putnam on 6/8/2017.
2) It provides information on exporting models for TensorFlow Serving, deploying TensorFlow to Android, tools for mobile deep learning like Inception and MobileNets, and using Deeplearning4j on the JVM with integration with Spark.
3) The document shares links to resources on these topics and thanks sponsors while inviting people to join future discussions.
Introducing TensorFlow: The game changer in building "intelligent" applications
Introducing TensorFlow: The game changer in building "intelligent" applicationsRokesh Jankie This is the slidedeck used for the presentation of the Amsterdam Pipeline of Data Science, held in December 2016. TensorFlow in the open source library from Google to implement deep learning, neural networks. This is an introduction to Tensorflow.
Note: Videos are not included (which were shown during the presentation)
On-device machine learning: TensorFlow on Android
On-device machine learning: TensorFlow on AndroidYufeng Guo This document discusses building machine learning models for mobile apps using TensorFlow. It describes the process of gathering training data, training a model using Cloud ML Engine, optimizing the model for mobile, and integrating it into an Android app. Key steps involve converting video training data to images, retraining an InceptionV3 model, optimizing the model size with graph transformations, and loading the model into an Android app. TensorFlow allows developing machine learning models that can run efficiently on mobile devices.
Neural Networks with Google TensorFlow
Neural Networks with Google TensorFlowDarshan Patel This slides explains how Convolution Neural Networks can be coded using Google TensorFlow.
Video available at : https://www.youtube.com/watch?v=EoysuTMmmMc
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...Andrew Gardner Note: these are the slides from a presentation at Lexis Nexis in Alpharetta, GA, on 2014-01-08 as part of the DataScienceATL Meetup. A video of this talk from Dec 2013 is available on vimeo at http://bit.ly/1aJ6xlt
Note: Slideshare mis-converted the images in slides 16-17. Expect a fix in the next couple of days.
---
Deep learning is a hot area of machine learning named one of the "Breakthrough Technologies of 2013" by MIT Technology Review. The basic ideas extend neural network research from past decades and incorporate new discoveries in statistical machine learning and neuroscience. The results are new learning architectures and algorithms that promise disruptive advances in automatic feature engineering, pattern discovery, data modeling and artificial intelligence. Empirical results from real world applications and benchmarking routinely demonstrate state-of-the-art performance across diverse problems including: speech recognition, object detection, image understanding and machine translation. The technology is employed commercially today, notably in many popular Google products such as Street View, Google+ Image Search and Android Voice Recognition.
In this talk, we will present an overview of deep learning for data scientists: what it is, how it works, what it can do, and why it is important. We will review several real world applications and discuss some of the key hurdles to mainstream adoption. We will conclude by discussing our experiences implementing and running deep learning experiments on our own hardware data science appliance.
Large Scale Deep Learning with TensorFlow 
Large Scale Deep Learning with TensorFlow Jen Aman Large-scale deep learning with TensorFlow allows storing and performing computation on large datasets to develop computer systems that can understand data. Deep learning models like neural networks are loosely based on what is known about the brain and become more powerful with more data, larger models, and more computation. At Google, deep learning is being applied across many products and areas, from speech recognition to image understanding to machine translation. TensorFlow provides an open-source software library for machine learning that has been widely adopted both internally at Google and externally.
Google TensorFlow Tutorial
Google TensorFlow Tutorial台灣資料科學年會 TensorFlow Tutorial given by Dr. Chung-Cheng Chiu at Google Brain on Dec. 29, 2015
http://datasci.tw/event/google_deep_learning
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...
TensorFlow Serving, Deep Learning on Mobile, and Deeplearning4j on the JVM - ...Sam Putnam [Deep Learning]
Ad
Similar to Machine Intelligence at Google Scale: TensorFlow (20)
Tensorflow
Tensorflowmarwa Ayad Mohamed Purpose: Machine Learning & deep Learning research
Open Source (By Google)
Numerical Computation using data flow graphs
BY :Marwa Ayad Mohamed
2017 arab wic marwa ayad machine learning
2017 arab wic marwa ayad machine learningmarwa Ayad Mohamed This document provides information about Marwa Ayad Mohamed and her presentation on machine learning with Google tools. It discusses artificial intelligence, machine learning, deep learning, and how these concepts are used for applications like image recognition, object recognition, smart email reply, voice recognition, self-driving cars, and more. It then describes TensorFlow, a popular machine learning library developed by Google, how the programming model works, and provides steps for installing TensorFlow and running demos like image recognition on Windows systems. Contact information is also included at the end.
ML in Android
ML in AndroidJose Antonio Corbacho Machine learning is a subset of artificial intelligence that allows systems to learn from data without being explicitly programmed. The document discusses machine learning concepts like supervised vs unsupervised learning and neural networks. It then provides a step-by-step example of using a convolutional neural network model to classify handwritten digits on Android devices using TensorFlow. Key steps include exploring data, choosing an architecture, training the model, exporting it, and using it for inference in an Android app.
Computation graphs - Tensorflow & CNTK
Computation graphs - Tensorflow & CNTKA H M Forhadul Islam This document provides an overview of computational graphs and several deep learning frameworks. It describes how computational graphs represent functions as connected nodes that propagate data through edges. Frameworks like TensorFlow, CNTK, and PyTorch use computational graphs for forward and backward propagation in neural networks. TensorFlow is highlighted as an open-source framework developed by Google that supports data and model parallelism across CPU, GPU, and custom hardware. CNTK is also introduced as a Microsoft framework that describes neural networks as directed graphs.
Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)wesley chun 1.5-hr tech talk introducing Machine Learning and the GCP ML APIs and other Google Cloud developer tools.
Google Big Data Expo
Google Big Data ExpoBigDataExpo The document discusses Google Cloud AI services including Cloud ML Engine for machine learning model training and prediction. It provides examples of using Cloud ML Engine to train models locally and in the cloud, perform distributed training, and hyperparameter tuning. It also covers deploying trained models and making predictions against them.
Aprendizaje automático aplicado utilizando TensorFlow
Aprendizaje automático aplicado utilizando TensorFlowNicolas Bortolotti The document discusses TensorFlow, an open source machine learning framework developed by Google. It provides an overview of TensorFlow, describing it as a dataflow graph used to express machine learning algorithms like neural networks. Examples are given of how TensorFlow is used across Google products and by other companies for applications like computer vision, speech recognition, and machine translation. Resources for learning TensorFlow are also listed, including videos, books, and code labs.
Getting Started with TensorFlow on Google Cloud
Getting Started with TensorFlow on Google CloudMariam Aslam Getting Started with TensorFlow on Google Cloud
Xavier Amatriain, VP of Engineering, Quora at MLconf SF - 11/13/15
Xavier Amatriain, VP of Engineering, Quora at MLconf SF - 11/13/15MLconf 10 More Lessons Learned from Building Real-Life ML Systems: A year ago I presented a collection of 10 lessons in MLConf. These goal of the presentation was to highlight some of the practical issues that ML practitioners encounter in the field, many of which are not included in traditional textbooks and courses. The original 10 lessons included some related to issues such as feature complexity, sampling, regularization, distributing/parallelizing algorithms, or how to think about offline vs. online computation.
Since that presentation and associated material was published, I have been asked to complement it with more/newer material. In this talk I will present 10 new lessons that not only build upon the original ones, but also relate to my recent experiences at Quora. I will talk about the importance of metrics, training data, and debuggability of ML systems. I will also describe how to combine supervised and non-supervised approaches or the role of ensembles in practical ML systems.
10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systemsXavier Amatriain 1. Machine learning applications at Quora include answer ranking, feed ranking, topic recommendations, user recommendations, and more. A variety of models are used including logistic regression, gradient boosted decision trees, neural networks, and matrix factorization.
2. Implicit signals like watching and clicking tend to be more useful than explicit signals like ratings. However, both implicit and explicit signals combined can better represent long-term goals.
3. The outputs of machine learning models will often become inputs to other models, so models need to be designed with this in mind to avoid issues like feedback loops.
10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConfXavier Amatriain 1. Machine learning applications at Quora include answer ranking, feed ranking, topic recommendations, user recommendations, and more. A variety of models are used including logistic regression, gradient boosted decision trees, neural networks, and matrix factorization.
2. Implicit signals like watching and clicking tend to be more useful than explicit signals like ratings. However, both implicit and explicit signals combined can better represent long-term goals.
3. It is important to focus on feature engineering to create features that are reusable, transformable, interpretable, and reliable. The outputs of models may become inputs to other models, so care must be taken to avoid feedback loops and ensure proper data dependencies.
Google Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningDataWorks Summit/Hadoop Summit Google Cloud Platform empowers TensorFlow and machine learning by providing scalable computing resources and APIs. It allows developers to build neural networks with TensorFlow, and easily integrate pre-trained machine learning models into applications using Cloud Vision and Speech APIs. Cloud Machine Learning offers a managed service for distributed TensorFlow training and prediction at scale in the cloud.
Kaz Sato, Evangelist, Google at MLconf ATL 2016
Kaz Sato, Evangelist, Google at MLconf ATL 2016MLconf Machine Intelligence at Google Scale: Tensor Flow and Cloud Machine Learning: The biggest challenge of Deep Learning technology is the scalability. As long as using single GPU server, you have to wait for hours or days to get the result of your work. This doesn’t scale for production service, so you need a Distributed Training on the cloud eventually. Google has been building infrastructure for training the large scale neural network on the cloud for years, and now started to share the technology with external developers. In this session, we will introduce new pre-trained ML services such as Cloud Vision API and Speech API that works without any training. Also, we will look how TensorFlow and Cloud Machine Learning will accelerate custom model training for 10x – 40x with Google’s distributed training infrastructure.
TensorFlow - La IA detrás de Google
TensorFlow - La IA detrás de GoogleIsrael Blancas The document discusses machine learning and TensorFlow. It provides three ways to get started with machine learning of varying complexity: using cloud APIs, retraining existing models, or developing new models. It then discusses Google Cloud machine learning APIs for vision, natural language, speech and translation. It provides examples of using the vision API and mobile vision API. It introduces TensorFlow as an open source machine learning library for research and production with Python and C++ frontends.
Deep learning with TensorFlow
Deep learning with TensorFlowNdjido Ardo BAR This document provides an agenda for a presentation on deep learning with TensorFlow. It includes:
1. An introduction to machine learning and deep networks, including definitions of machine learning, neural networks, and deep learning.
2. An overview of TensorFlow, including its architecture, evolution, language features, computational graph, TensorBoard, and use in Google Cloud ML.
3. Details of TensorFlow hands-on examples, including linear models, shallow and deep neural networks for MNIST digit classification, and convolutional neural networks for MNIST.
"Large-Scale Deep Learning for Building Intelligent Computer Systems," a Keyn...
"Large-Scale Deep Learning for Building Intelligent Computer Systems," a Keyn...Edge AI and Vision Alliance For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit-google-keynote
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Jeff Dean, Senior Fellow at Google, presents the "Large-Scale Deep Learning for Building Intelligent Computer Systems" keynote at the May 2016 Embedded Vision Summit.
Over the past few years, Google has built two generations of large-scale computer systems for training neural networks, and then applied these systems to a wide variety of research problems that have traditionally been very difficult for computers. Google has released its second generation system, TensorFlow, as an open source project, and is now collaborating with a growing community on improving and extending its functionality. Using TensorFlow, Google's research group has made significant improvements in the state-of-the-art in many areas, and dozens of different groups at Google use it to train state-of-the-art models for speech recognition, image recognition, various visual detection tasks, language modeling, language translation, and many other tasks.
In this talk, Jeff highlights some of ways that Google trains large models quickly on large datasets, and discusses different approaches for deploying machine learning models in environments ranging from large datacenters to mobile devices. He will then discuss ways in which Google has applied this work to a variety of problems in Google's products, usually in close collaboration with other teams. This talk describes joint work with many people at Google.
Introduction to Machine Learning with TensorFlow
Introduction to Machine Learning with TensorFlowPaolo Tomeo This document introduces TensorFlow, an open source machine learning library for deep learning. It discusses how TensorFlow uses data flow graphs to optimize objective functions and allows computation across CPU and GPU devices. It provides an example of classifying the Iris dataset using TensorFlow's high-level tf.contrib.learn API. It concludes with pointers to additional TensorFlow tutorials and guides.
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning SystemsXavier Amatriain Talk about practical machine learning lessons learned over the years. Given at the Hardcore Data Science track in Strata San Jose 2016
Machine Learning Overview
Machine Learning OverviewMykhailo Koval Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it.
We will review some modern machine learning applications, understand variety of machine learning problem definitions, go through particular approaches of solving machine learning tasks.
This year 2015 Amazon and Microsoft introduced services to perform machine learning tasks in cloud. Microsoft Azure Machine Learning offers a streamlined experience for all data scientist skill levels, from setting up with only a web browser, to using drag and drop gestures and simple data flow graphs to set up experiments.
We will briefly review Azure ML Studio features and run machine learning experiment.
"Large-Scale Deep Learning for Building Intelligent Computer Systems," a Keyn...
"Large-Scale Deep Learning for Building Intelligent Computer Systems," a Keyn...Edge AI and Vision Alliance
Ad
More from DataWorks Summit/Hadoop Summit (20)
Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in ProductionDataWorks Summit/Hadoop Summit This document discusses running Apache Spark and Apache Zeppelin in production. It begins by introducing the author and their background. It then covers security best practices for Spark deployments, including authentication using Kerberos, authorization using Ranger/Sentry, encryption, and audit logging. Different Spark deployment modes like Spark on YARN are explained. The document also discusses optimizing Spark performance by tuning executor size and multi-tenancy. Finally, it covers security features for Apache Zeppelin like authentication, authorization, and credential management.
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache ZeppelinDataWorks Summit/Hadoop Summit This document discusses Spark security and provides an overview of authentication, authorization, encryption, and auditing in Spark. It describes how Spark leverages Kerberos for authentication and uses services like Ranger and Sentry for authorization. It also outlines how communication channels in Spark are encrypted and some common issues to watch out for related to Spark security.
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerDataWorks Summit/Hadoop Summit The document discusses the Virtual Data Connector project which aims to leverage Apache Atlas and Apache Ranger to provide unified metadata and access governance across data sources. Key points include:
- The project aims to address challenges of understanding, governing, and controlling access to distributed data through a centralized metadata catalog and policies.
- Apache Atlas provides a scalable metadata repository while Apache Ranger enables centralized access governance. The project will integrate these using a virtualization layer.
- Enhancements to Atlas and Ranger are proposed to better support the project's goals around a unified open metadata platform and metadata-driven governance.
- An initial minimum viable product will be built this year with the goal of an open, collaborative ecosystem around shared
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformDataWorks Summit/Hadoop Summit This document discusses using a data science platform to enable digital diagnostics in healthcare. It provides an overview of healthcare data sources and Yale/YNHH's data science platform. It then describes the data science journey process using a clinical laboratory use case as an example. The goal is to use big data and machine learning to improve diagnostic reproducibility, throughput, turnaround time, and accuracy for laboratory testing by developing a machine learning algorithm and real-time data processing pipeline.
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and ZeppelinDataWorks Summit/Hadoop Summit This document discusses using Apache Spark and MLlib for text mining on big data. It outlines common text mining applications, describes how Spark and MLlib enable scalable machine learning on large datasets, and provides examples of text mining workflows and pipelines that can be built with Spark MLlib algorithms and components like tokenization, feature extraction, and modeling. It also discusses customizing ML pipelines and the Zeppelin notebook platform for collaborative data science work.
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDataWorks Summit/Hadoop Summit This document compares the performance of Hive and Spark when running the BigBench benchmark. It outlines the structure and use cases of the BigBench benchmark, which aims to cover common Big Data analytical properties. It then describes sequential performance tests of Hive+Tez and Spark on queries from the benchmark using a HDInsight PaaS cluster, finding variations in performance between the systems. Concurrency tests are also run by executing multiple query streams in parallel to analyze throughput.
Hadoop Crash Course
Hadoop Crash CourseDataWorks Summit/Hadoop Summit The document discusses modern data applications and architectures. It introduces Apache Hadoop, an open-source software framework for distributed storage and processing of large datasets across clusters of commodity hardware. Hadoop provides massive scalability and easy data access for applications. The document outlines the key components of Hadoop, including its distributed storage, processing framework, and ecosystem of tools for data access, management, analytics and more. It argues that Hadoop enables organizations to innovate with all types and sources of data at lower costs.
Data Science Crash Course
Data Science Crash CourseDataWorks Summit/Hadoop Summit This document provides an overview of data science and machine learning. It discusses what data science and machine learning are, including extracting insights from data and computers learning without being explicitly programmed. It also covers Apache Spark, which is an open source framework for large-scale data processing. Finally, it discusses common machine learning algorithms like regression, classification, clustering, and dimensionality reduction.
Apache Spark Crash Course
Apache Spark Crash CourseDataWorks Summit/Hadoop Summit This document provides an overview of Apache Spark, including its capabilities and components. Spark is an open-source cluster computing framework that allows distributed processing of large datasets across clusters of machines. It supports various data processing workloads including streaming, SQL, machine learning and graph analytics. The document discusses Spark's APIs like DataFrames and its libraries like Spark SQL, Spark Streaming, MLlib and GraphX. It also provides examples of using Spark for tasks like linear regression modeling.
Dataflow with Apache NiFi
Dataflow with Apache NiFiDataWorks Summit/Hadoop Summit This document provides an overview of Apache NiFi and dataflow. It begins with an introduction to the challenges of moving data effectively within and between systems. It then discusses Apache NiFi's key features for addressing these challenges, including guaranteed delivery, data buffering, prioritized queuing, and data provenance. The document outlines NiFi's architecture and components like repositories and extension points. It also previews a live demo and invites attendees to further discuss Apache NiFi at a Birds of a Feather session.
Schema Registry - Set you Data Free
Schema Registry - Set you Data FreeDataWorks Summit/Hadoop Summit Many Organizations are currently processing various types of data and in different formats. Most often this data will be in free form, As the consumers of this data growing it’s imperative that this free-flowing data needs to adhere to a schema. It will help data consumers to have an expectation of about the type of data they are getting and also they will be able to avoid immediate impact if the upstream source changes its format. Having a uniform schema representation also gives the Data Pipeline a really easy way to integrate and support various systems that use different data formats.
SchemaRegistry is a central repository for storing, evolving schemas. It provides an API & tooling to help developers and users to register a schema and consume that schema without having any impact if the schema changed. Users can tag different schemas and versions, register for notifications of schema changes with versions etc.
In this talk, we will go through the need for a schema registry and schema evolution and showcase the integration with Apache NiFi, Apache Kafka, Apache Storm.
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...DataWorks Summit/Hadoop Summit There is increasing need for large-scale recommendation systems. Typical solutions rely on periodically retrained batch algorithms, but for massive amounts of data, training a new model could take hours. This is a problem when the model needs to be more up-to-date. For example, when recommending TV programs while they are being transmitted the model should take into consideration users who watch a program at that time.
The promise of online recommendation systems is fast adaptation to changes, but methods of online machine learning from streams is commonly believed to be more restricted and hence less accurate than batch trained models. Combining batch and online learning could lead to a quickly adapting recommendation system with increased accuracy. However, designing a scalable data system for uniting batch and online recommendation algorithms is a challenging task. In this talk we present our experiences in creating such a recommendation engine with Apache Flink and Apache Spark.
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...DataWorks Summit/Hadoop Summit DeepLearning is not just a hype - it outperforms state-of-the-art ML algorithms. One by one. In this talk we will show how DeepLearning can be used for detecting anomalies on IoT sensor data streams at high speed using DeepLearning4J on top of different BigData engines like ApacheSpark and ApacheFlink. Key in this talk is the absence of any large training corpus since we are using unsupervised machine learning - a domain current DL research threats step-motherly. As we can see in this demo LSTM networks can learn very complex system behavior - in this case data coming from a physical model simulating bearing vibration data. Once draw back of DeepLearning is that normally a very large labaled training data set is required. This is particularly interesting since we can show how unsupervised machine learning can be used in conjunction with DeepLearning - no labeled data set is necessary. We are able to detect anomalies and predict braking bearings with 10 fold confidence. All examples and all code will be made publicly available and open sources. Only open source components are used.
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLDataWorks Summit/Hadoop Summit QE automation for large systems is a great step forward in increasing system reliability. In the big-data world, multiple components have to come together to provide end-users with business outcomes. This means, that QE Automations scenarios need to be detailed around actual use cases, cross-cutting components. The system tests potentially generate large amounts of data on a recurring basis, verifying which is a tedious job. Given the multiple levels of indirection, the false positives of actual defects are higher, and are generally wasteful.
At Hortonworks, we’ve designed and implemented Automated Log Analysis System - Mool, using Statistical Data Science and ML. Currently the work in progress has a batch data pipeline with a following ensemble ML pipeline which feeds into the recommendation engine. The system identifies the root cause of test failures, by correlating the failing test cases, with current and historical error records, to identify root cause of errors across multiple components. The system works in unsupervised mode with no perfect model/stable builds/source-code version to refer to. In addition the system provides limited recommendations to file/open past tickets and compares run-profiles with past runs.
How Hadoop Makes the Natixis Pack More Efficient 
How Hadoop Makes the Natixis Pack More Efficient DataWorks Summit/Hadoop Summit Improving business performance is never easy! The Natixis Pack is like Rugby. Working together is key to scrum success. Our data journey would undoubtedly have been so much more difficult if we had not made the move together.
This session is the story of how ‘The Natixis Pack’ has driven change in its current IT architecture so that legacy systems can leverage some of the many components in Hortonworks Data Platform in order to improve the performance of business applications. During this session, you will hear:
• How and why the business and IT requirements originated
• How we leverage the platform to fulfill security and production requirements
• How we organize a community to:
o Guard all the players, no one gets left on the ground!
o Us the platform appropriately (Not every problem is eligible for Big Data and standard databases are not dead)
• What are the most usable, the most interesting and the most promising technologies in the Apache Hadoop community
We will finish the story of a successful rugby team with insight into the special skills needed from each player to win the match!
DETAILS
This session is part business, part technical. We will talk about infrastructure, security and project management as well as the industrial usage of Hive, HBase, Kafka, and Spark within an industrial Corporate and Investment Bank environment, framed by regulatory constraints.
HBase in Practice 
HBase in Practice DataWorks Summit/Hadoop Summit HBase is a distributed, column-oriented database that stores data in tables divided into rows and columns. It is optimized for random, real-time read/write access to big data. The document discusses HBase's key concepts like tables, regions, and column families. It also covers performance tuning aspects like cluster configuration, compaction strategies, and intelligent key design to spread load evenly. Different use cases are suitable for HBase depending on access patterns, such as time series data, messages, or serving random lookups and short scans from large datasets. Proper data modeling and tuning are necessary to maximize HBase's performance.
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)DataWorks Summit/Hadoop Summit There has been an explosion of data digitising our physical world – from cameras, environmental sensors and embedded devices, right down to the phones in our pockets. Which means that, now, companies have new ways to transform their businesses – both operationally, and through their products and services – by leveraging this data and applying fresh analytical techniques to make sense of it. But are they ready? The answer is “no” in most cases.
In this session, we’ll be discussing the challenges facing companies trying to embrace the Analytics of Things, and how Teradata has helped customers work through and turn those challenges to their advantage.
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS HadoopDataWorks Summit/Hadoop Summit In this talk, we will present a new distribution of Hadoop, Hops, that can scale the Hadoop Filesystem (HDFS) by 16X, from 70K ops/s to 1.2 million ops/s on Spotiy's industrial Hadoop workload. Hops is an open-source distribution of Apache Hadoop that supports distributed metadata for HSFS (HopsFS) and the ResourceManager in Apache YARN. HopsFS is the first production-grade distributed hierarchical filesystem to store its metadata normalized in an in-memory, shared nothing database. For YARN, we will discuss optimizations that enable 2X throughput increases for the Capacity scheduler, enabling scalability to clusters with >20K nodes. We will discuss the journey of how we reached this milestone, discussing some of the challenges involved in efficiently and safely mapping hierarchical filesystem metadata state and operations onto a shared-nothing, in-memory database. We will also discuss the key database features needed for extreme scaling, such as multi-partition transactions, partition-pruned index scans, distribution-aware transactions, and the streaming changelog API. Hops (www.hops.io) is Apache-licensed open-source and supports a pluggable database backend for distributed metadata, although it currently only support MySQL Cluster as a backend. Hops opens up the potential for new directions for Hadoop when metadata is available for tinkering in a mature relational database.
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...DataWorks Summit/Hadoop Summit In high-risk manufacturing industries, regulatory bodies stipulate continuous monitoring and documentation of critical product attributes and process parameters. On the other hand, sensor data coming from production processes can be used to gain deeper insights into optimization potentials. By establishing a central production data lake based on Hadoop and using Talend Data Fabric as a basis for a unified architecture, the German pharmaceutical company HERMES Arzneimittel was able to cater to compliance requirements as well as unlock new business opportunities, enabling use cases like predictive maintenance, predictive quality assurance or open world analytics. Learn how the Talend Data Fabric enabled HERMES Arzneimittel to become data-driven and transform Big Data projects from challenging, hard to maintain hand-coding jobs to repeatable, future-proof integration designs.
Talend Data Fabric combines Talend products into a common set of powerful, easy-to-use tools for any integration style: real-time or batch, big data or master data management, on-premises or in the cloud.
Backup and Disaster Recovery in Hadoop 
Backup and Disaster Recovery in Hadoop DataWorks Summit/Hadoop Summit While you could be tempted assuming data is already safe in a single Hadoop cluster, in practice you have to plan for more. Questions like: "What happens if the entire datacenter fails?, or "How do I recover into a consistent state of data, so that applications can continue to run?" are not a all trivial to answer for Hadoop. Did you know that HDFS snapshots are handling open files not as immutable? Or that HBase snapshots are executed asynchronously across servers and therefore cannot guarantee atomicity for cross region updates (which includes tables)? There is no unified and coherent data backup strategy, nor is there tooling available for many of the included components to build such a strategy. The Hadoop distributions largely avoid this topic as most customers are still in the "single use-case" or PoC phase, where data governance as far as backup and disaster recovery (BDR) is concerned are not (yet) important. This talk first is introducing you to the overarching issue and difficulties of backup and data safety, looking at each of the many components in Hadoop, including HDFS, HBase, YARN, Oozie, the management components and so on, to finally show you a viable approach using built-in tools. You will also learn not to take this topic lightheartedly and what is needed to implement and guarantee a continuous operation of Hadoop cluster based solutions.
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...DataWorks Summit/Hadoop Summit
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...DataWorks Summit/Hadoop Summit
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)DataWorks Summit/Hadoop Summit
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...DataWorks Summit/Hadoop Summit
Recently uploaded (20)
ECS25 - The adventures of a Microsoft 365 Platform Owner - Website.pptx
ECS25 - The adventures of a Microsoft 365 Platform Owner - Website.pptxJasper Oosterveld My slides for ECS 2025.
6th Power Grid Model Meetup - 21 May 2025
6th Power Grid Model Meetup - 21 May 2025DanBrown980551 6th Power Grid Model Meetup
Join the Power Grid Model community for an exciting day of sharing experiences, learning from each other, planning, and collaborating.
This hybrid in-person/online event will include a full day agenda, with the opportunity to socialize afterwards for in-person attendees.
If you have a hackathon proposal, tell us when you register!
About Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
End-to-end Assurance for SD-WAN & SASE with ThousandEyes
End-to-end Assurance for SD-WAN & SASE with ThousandEyesThousandEyes End-to-end Assurance for SD-WAN & SASE with ThousandEyes
AI Emotional Actors: “When Machines Learn to Feel and Perform"
AI Emotional Actors: “When Machines Learn to Feel and Perform"AkashKumar809858 Welcome to the era of AI Emotional Actors.
The entertainment landscape is undergoing a seismic transformation. What started as motion capture and CGI enhancements has evolved into a full-blown revolution: synthetic beings not only perform but express, emote, and adapt in real time.
For reading further follow this link -
https://akash97.gumroad.com/l/meioex
European Accessibility Act & Integrated Accessibility Testing
European Accessibility Act & Integrated Accessibility TestingJulia Undeutsch Emma Dawson will guide you through two important topics in this session.
Firstly, she will prepare you for the European Accessibility Act (EAA), which comes into effect on 28 June 2025, and show you how development teams can prepare for it.
In the second part of the webinar, Emma Dawson will explore with you various integrated testing methods and tools that will help you improve accessibility during the development cycle, such as Linters, Storybook, Playwright, just to name a few.
Focus: European Accessibility Act, Integrated Testing tools and methods (e.g. Linters, Storybook, Playwright)
Target audience: Everyone, Developers, Testers
New Ways to Reduce Database Costs with ScyllaDB
New Ways to Reduce Database Costs with ScyllaDBScyllaDB How ScyllaDB’s latest capabilities can reduce your infrastructure costs
ScyllaDB has been obsessed with price-performance from day 1. Our core database is architected with low-level engineering optimizations that squeeze every ounce of power from the underlying infrastructure. And we just completed a multi-year effort to introduce a set of new capabilities for additional savings.
Join this webinar to learn about these new capabilities: the underlying challenges we wanted to address, the workloads that will benefit most from each, and how to get started. We’ll cover ways to:
- Avoid overprovisioning with “just-in-time” scaling
- Safely operate at up to ~90% storage utilization
- Cut network costs with new compression strategies and file-based streaming
We’ll also highlight a “hidden gem” capability that lets you safely balance multiple workloads in a single cluster. To conclude, we will share the efficiency-focused capabilities on our short-term and long-term roadmaps.
Co-Constructing Explanations for AI Systems using Provenance
Co-Constructing Explanations for AI Systems using ProvenancePaul Groth Explanation is not a one off - it's a process where people and systems work together to gain understanding. This idea of co-constructing explanations or explanation by exploration is powerful way to frame the problem of explanation. In this talk, I discuss our first experiments with this approach for explaining complex AI systems by using provenance. Importantly, I discuss the difficulty of evaluation and discuss some of our first approaches to evaluating these systems at scale. Finally, I touch on the importance of explanation to the comprehensive evaluation of AI systems.
Jira Administration Training – Day 1 : Introduction
Jira Administration Training – Day 1 : IntroductionRavi Teja This presentation covers the basics of Jira for beginners. Learn how Jira works, its key features, project types, issue types, and user roles. Perfect for anyone new to Jira or preparing for Jira Admin roles.
Securiport - A Border Security Company
Securiport - A Border Security CompanySecuriport Securiport is a border security systems provider with a progressive team approach to its task. The company acknowledges the importance of specialized skills in creating the latest in innovative security tech. The company has offices throughout the world to serve clients, and its employees speak more than twenty languages at the Washington D.C. headquarters alone.
Create Your First AI Agent with UiPath Agent Builder
Create Your First AI Agent with UiPath Agent BuilderDianaGray10 Join us for an exciting virtual event where you'll learn how to create your first AI Agent using UiPath Agent Builder. This session will cover everything you need to know about what an agent is and how easy it is to create one using the powerful AI-driven UiPath platform. You'll also discover the steps to successfully publish your AI agent. This is a wonderful opportunity for beginners and enthusiasts to gain hands-on insights and kickstart their journey in AI-powered automation.
Cyber Security Legal Framework in Nepal.pptx
Cyber Security Legal Framework in Nepal.pptxGhimire B.R. The presentation is about the review of existing legal framework on Cyber Security in Nepal. The strength and weakness highlights of the major acts and policies so far. Further it highlights the needs of data protection act .
UiPath Community Zurich: Release Management and Build Pipelines
UiPath Community Zurich: Release Management and Build PipelinesUiPathCommunity Ensuring robust, reliable, and repeatable delivery processes is more critical than ever - it's a success factor for your automations and for automation programmes as a whole. In this session, we’ll dive into modern best practices for release management and explore how tools like the UiPathCLI can streamline your CI/CD pipelines. Whether you’re just starting with automation or scaling enterprise-grade deployments, our event promises to deliver helpful insights to you. This topic is relevant for both on-premise and cloud users - as well as for automation developers and software testers alike.
📕 Agenda:
- Best Practices for Release Management
- What it is and why it matters
- UiPath Build Pipelines Deep Dive
- Exploring CI/CD workflows, the UiPathCLI and showcasing scenarios for both on-premise and cloud
- Discussion, Q&A
👨🏫 Speakers
Roman Tobler, CEO@ Routinuum
Johans Brink, CTO@ MvR Digital Workforce
We look forward to bringing best practices and showcasing build pipelines to you - and to having interesting discussions on this important topic!
If you have any questions or inputs prior to the event, don't hesitate to reach out to us.
This event streamed live on May 27, 16:00 pm CET.
Check out all our upcoming UiPath Community sessions at:
👉 https://community.uipath.com/events/
Join UiPath Community Zurich chapter:
👉 https://community.uipath.com/zurich/
TrustArc Webinar: Mastering Privacy Contracting
TrustArc Webinar: Mastering Privacy ContractingTrustArc As data privacy regulations become more pervasive across the globe and organizations increasingly handle and transfer (including across borders) meaningful volumes of personal and confidential information, the need for robust contracts to be in place is more important than ever.
This webinar will provide a deep dive into privacy contracting, covering essential terms and concepts, negotiation strategies, and key practices for managing data privacy risks.
Whether you're in legal, privacy, security, compliance, GRC, procurement, or otherwise, this session will include actionable insights and practical strategies to help you enhance your agreements, reduce risk, and enable your business to move fast while protecting itself.
This webinar will review key aspects and considerations in privacy contracting, including:
- Data processing addenda, cross-border transfer terms including EU Model Clauses/Standard Contractual Clauses, etc.
- Certain legally-required provisions (as well as how to ensure compliance with those provisions)
- Negotiation tactics and common issues
- Recent lessons from recent regulatory actions and disputes
Dr Jimmy Schwarzkopf presentation on the SUMMIT 2025 A
Dr Jimmy Schwarzkopf presentation on the SUMMIT 2025 ADr. Jimmy Schwarzkopf Introduction and Background:
Study Overview and Methodology: The study analyzes the IT market in Israel, covering over 160 markets and 760 companies/products/services. It includes vendor rankings, IT budgets, and trends from 2025-2029. Vendors participate in detailed briefings and surveys.
Vendor Listings: The presentation lists numerous vendors across various pages, detailing their names and services. These vendors are ranked based on their participation and market presence.
Market Insights and Trends: Key insights include IT market forecasts, economic factors affecting IT budgets, and the impact of AI on enterprise IT. The study highlights the importance of AI integration and the concept of creative destruction.
Agentic AI and Future Predictions: Agentic AI is expected to transform human-agent collaboration, with AI systems understanding context and orchestrating complex processes. Future predictions include AI's role in shopping and enterprise IT.
Offshore IT Support: Balancing In-House and Offshore Help Desk Technicians
Offshore IT Support: Balancing In-House and Offshore Help Desk Techniciansjohn823664 In today's always-on digital environment, businesses must deliver seamless IT support across time zones, devices, and departments. This SlideShare explores how companies can strategically combine in-house expertise with offshore talent to build a high-performing, cost-efficient help desk operation.
From the benefits and challenges of offshore support to practical models for integrating global teams, this presentation offers insights, real-world examples, and key metrics for success. Whether you're scaling a startup or optimizing enterprise support, discover how to balance cost, quality, and responsiveness with a hybrid IT support strategy.
Perfect for IT managers, operations leads, and business owners considering global help desk solutions.
Introducing the OSA 3200 SP and OSA 3250 ePRC
Introducing the OSA 3200 SP and OSA 3250 ePRCAdtran Adtran's latest Oscilloquartz solutions make optical pumping cesium timing more accessible than ever. Discover how the new OSA 3200 SP and OSA 3250 ePRC deliver superior stability, simplified deployment and lower total cost of ownership. Built on a shared platform and engineered for scalable, future-ready networks, these models are ideal for telecom, defense, metrology and more.
ELNL2025 - Unlocking the Power of Sensitivity Labels - A Comprehensive Guide....
ELNL2025 - Unlocking the Power of Sensitivity Labels - A Comprehensive Guide....Jasper Oosterveld Sensitivity labels, powered by Microsoft Purview Information Protection, serve as the foundation for classifying and protecting your sensitive data within Microsoft 365. Their importance extends beyond classification and play a crucial role in enforcing governance policies across your Microsoft 365 environment. Join me, a Data Security Consultant and Microsoft MVP, as I share practical tips and tricks to get the full potential of sensitivity labels. I discuss sensitive information types, automatic labeling, and seamless integration with Data Loss Prevention, Teams Premium, and Microsoft 365 Copilot.
Protecting Your Sensitive Data with Microsoft Purview - IRMS 2025
Protecting Your Sensitive Data with Microsoft Purview - IRMS 2025Nikki Chapple Session | Protecting Your Sensitive Data with Microsoft Purview: Practical Information Protection and DLP Strategies
Presenter | Nikki Chapple (MVP| Principal Cloud Architect CloudWay) & Ryan John Murphy (Microsoft)
Event | IRMS Conference 2025
Format | Birmingham UK
Date | 18-20 May 2025
In this closing keynote session from the IRMS Conference 2025, Nikki Chapple and Ryan John Murphy deliver a compelling and practical guide to data protection, compliance, and information governance using Microsoft Purview. As organizations generate over 2 billion pieces of content daily in Microsoft 365, the need for robust data classification, sensitivity labeling, and Data Loss Prevention (DLP) has never been more urgent.
This session addresses the growing challenge of managing unstructured data, with 73% of sensitive content remaining undiscovered and unclassified. Using a mountaineering metaphor, the speakers introduce the “Secure by Default” blueprint—a four-phase maturity model designed to help organizations scale their data security journey with confidence, clarity, and control.
🔐 Key Topics and Microsoft 365 Security Features Covered:
Microsoft Purview Information Protection and DLP
Sensitivity labels, auto-labeling, and adaptive protection
Data discovery, classification, and content labeling
DLP for both labeled and unlabeled content
SharePoint Advanced Management for workspace governance
Microsoft 365 compliance center best practices
Real-world case study: reducing 42 sensitivity labels to 4 parent labels
Empowering users through training, change management, and adoption strategies
🧭 The Secure by Default Path – Microsoft Purview Maturity Model:
Foundational – Apply default sensitivity labels at content creation; train users to manage exceptions; implement DLP for labeled content.
Managed – Focus on crown jewel data; use client-side auto-labeling; apply DLP to unlabeled content; enable adaptive protection.
Optimized – Auto-label historical content; simulate and test policies; use advanced classifiers to identify sensitive data at scale.
Strategic – Conduct operational reviews; identify new labeling scenarios; implement workspace governance using SharePoint Advanced Management.
🎒 Top Takeaways for Information Management Professionals:
Start secure. Stay protected. Expand with purpose.
Simplify your sensitivity label taxonomy for better adoption.
Train your users—they are your first line of defense.
Don’t wait for perfection—start small and iterate fast.
Align your data protection strategy with business goals and regulatory requirements.
💡 Who Should Watch This Presentation?
This session is ideal for compliance officers, IT administrators, records managers, data protection officers (DPOs), security architects, and Microsoft 365 governance leads. Whether you're in the public sector, financial services, healthcare, or education.
🔗 Read the blog: https://nikkichapple.com/irms-conference-2025/
Machine Intelligence at Google Scale: TensorFlow
- 1. Machine Intelligence at Google Scale: TensorFlow
- 2. Data is Everything. How well you use your data can determine the degree of your success.
- 3. What is Machine Learning? data algorithm insight
- 6. Human: Someone is using a small grill to melt his sandwich. Neural Model: A person is cooking some food on a grill.
- 7. One example (of 100s): “Neural Art” in TensorFlow An implementation of "A neural algorithm of Artistic style" in TensorFlow, for ● Introductory, hackable demos for TensorFlow, and ● Demonstrating the use of importing various Caffe cnn models (VGG and illustration2vec) in TF. github.com/woodrush/neural-art-tf
- 8. Growing Use of Deep Learning at Google Number of directories containing model description files UniqueProjectDirectories Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 2000 1500 1000 500 0 Across many products/areas ● Apps ● Maps ● Photos ● Gmail ● Speech ● Android ● YouTube ● Translation ● Robotics Research ● Image Understanding ● Natural Language Understanding ● Drug Discovery 2012 2013 2014 2015 Q4
- 9. The Machine Learning Spectrum TensorFlow Cloud Machine Learning Machine Learning APIs Data Scientist
- 10. TensorFlow
- 11. ● Google's open source library for machine intelligence ● Tensor: N-dimensional array ● Flow: data flow computation framework (like MapReduce) ● tensorflow.org launched in Nov 2015 ○ Most popular Machine Learning library ○ 20,000+ stars / 7000+ forks ○ 165 contributors / > 3000 commits What is TensorFlow?
- 12. Portable ● Training on: ○ Data Center ○ CPUs, GPUs and etc ● Running on: ○ Mobile phones ○ IoT devices
- 13. Computation as a Dataflow Graph Graph of Nodes, also called operations (ops) MatMul Add Softmax biases weights inputs targets Xent
- 14. Dataflow Graph (forward) Edges are N-dimensional arrays: Tensors MatMul Add Softmax biases weights inputs targets Xent
- 15. Dataflow Graph (backward graph and updates) Backward graph and update are added automatically to graph Add Mul biases ... learning rate −=... 'Biases' is a variable −= updates biasesSome ops compute gradients
- 16. Dataflow Graph ● In practice often very complex with 100s or 1000s of nodes and edges ● “Inference” means to execute just the forward path of the graph
- 17. Example: Logistic Regression 1. N examples of data ○ Input xn ○ Target tn 2. Model with parameters (weights W, biases b) ○ yn = Softmax(W * xn + b) 3. CrossEntropy loss to minimize ○ loss = -Sum(tn * ln(yn )) 4. Minimization by Gradient Descent ○ requires gradients of loss with respect to parameters W, b ● TensorFlow does automatic differentiation ● Gradient calculation done by TF once model & loss defined
- 20. Portable ● Training on: ○ Data Center ○ CPUs, GPUs and etc ● Running on: ○ Mobile phones ○ IoT devices
- 22. The Machine Learning Spectrum TensorFlow Cloud Machine Learning Machine Learning APIs Application Developer Data Scientist
- 23. Cloud Machine Learning Platform Pre-Processing Training ML Authoring ML Analysis Hosted Model Monitoring & Logging Training Prediction ● Intelligent Text Processing ● Image Feature Processing ● Managed Infrastructure ● Scalable Algorithms ● HW Acceleration Evaluation ● Detailed Model Metrics ● In-process model viz ● No-ops experience Batch Real-Time input output ● Hosted Model experience ● AutoScaling ● Anytime access Data Console Model & Job Mgmt Datalab
- 24. Can I Hug That?
- 25. Can I Hug That? Images Labels Trained Classifier Question Hug / Not Hug
- 26. Demo
- 28. Confidential & ProprietaryGoogle Cloud Platform 28 Easy to use API: pass in an image, we give you insight: Google Cloud Vision API Label Detection OCR Explicit Content Detection Facial Detection Landmark Detection Logo Detection