Introduction to Huawei Cloud

HiLens Service
 Foreword
⚫ Huawei HiLens is a multimodal AI development platform featuring a device-cloud synergy. It
provides an easy-to-use framework, easy-to-setup development environments, a cloud-based
management console, and an AI skill market (a skill can be seen as an AI application ready
to run on a camera or another end user device, and it consists of a model and logic code).
With Huawei HiLens, you can easily develop and deploy visual and auditory AI applications
online, and manage a multitude of connected compute devices.
⚫ This chapter introduces you to the features, advantages, and applications of Huawei HiLens.

1
 Objectives
⚫ Upon completion of this course, you will be familiar with:
 The architecture and key features of HiLens.
 The use cases of HiLens.

2
 Contents
1. Challenges to Device-Edge-Cloud Synergy

2. Huawei HiLens Service

3. Huawei HiLens Use Cases

3
Challenges Faced by Developers of End-to-End AI
Applications/Solutions
• Difficult to set up the
development environment
Smart City
• High level of expertise required
• Time-consuming development
License plate Vehicle flow and debugging
recognition statistics

• Expensive hardware for end-to-

Smart end verification
Campus • High adaptation costs when a
large number of devices are used
FaceID Intrusion Hidden costs
• Takes long to develop an AI
detection
application from scratch

Smart Store • Complex deployment in a live

environment
Foot traffic People flow
• Professionals needed
heatmap

Examples
4
Advantages of AI with Device-Edge-Cloud Synergy
A brief comparison
Lower operating costs
A conventional solution consumes large amounts of
Conventional solution HiLens bandwidth, storage, and computing resources.
HiLens can run on customer premises, which is less costly.

Cloud Skill Market

Better privacy protection
Analytics Storage Cloud APIs Conventional solution: all video data on the cloud
Edge computing with HiLens: no need to transmit data
Video streams out of the campus
On-
premises
Video
VS deployment
frames Faster development
of AI skills
Easy access to many AI applications on the cloud
Storage

Devices
Local AI skills Rich application ecosystem
Fast 3rd-party skill development, easy access

5
Device-Edge-Cloud Synergy
⚫ What's edge computing
 With edge computing, data is processed and stored at the edge of the network to support real-time
services and intelligent applications.

⚫ What's intelligent edge

 Intelligent edge is about deploying AI to edge nodes for intelligent, local data processing.

⚫ What's device-edge-cloud synergy

 Device-edge-cloud synergy requires a unified technology ecosystem across both cloud and on-
premises scenarios, a unified AI architecture, and streamlined AI data flows across the cloud, edge,
and end devices, supporting more intelligent edge and end devices.

6
 Contents
1. Challenges to Device-Edge-Cloud Synergy

2. Huawei HiLens Service

3. Huawei HiLens Use Cases

7
 Huawei Cloud HiLens: Agile Development and Deployment of AI
Applications with Device-Edge-Cloud Synergy
Develop once, deploy on various devices, with agile debugging and testing
Huawei Cloud
Develop
on cloud Multimodal AI applications (processes/containers) Video/image/speech/other AI applications Release
ModelBox, AI application development Huawei Cloud AI
Cloud management platform AI development plug-ins/tools Gallery/Marketplace
framework (open source) Buy
Huawei Cloud ModelArts Data Algorithms Models

Device-edge-cloud synergy

Edge/Devices HiLens Agent Inside

Deploy Developer hardware Smart devices/lite edge nodes Smart cameras Consumer devices

remotely
HiHope AI 3rd-party
Atlas500 IVS 1800 Bozz EC02 3rd-party edge Smart mobility smart
HiLens Kit development board Atlas800
(RK3568/3588) (RK3399) inference SDC devices cameras
Linux/LiteOS Android + 3rd-party Linux/LiteOS
Ascend 310 + Linux X86+GPU + HiSilicon 35xx intelligent chips /Harmony OS+
Ascend 310 + Linux/Ubuntu
(Adaptation needed) HiSilicon 35xx chips

Intelligent solutions

Industrial Transport Campus Finance City ...

8
 Device-Cloud Synergy: Flexible Control on the Cloud and
Collaboration Across the Cloud and End Devices
Online deployment of AI applications, and real-time running on end devices; further verification
and flexible control on the cloud, supporting lightweight application needs.

9
Quick Application Development and Deployment with Pre-
built AI Skills from the HiLens Skill Market
Skill templates (based on
Skill Market HiLens Framework)

Select a skill development

template

Pre-built model and

inference code

One-click
installation

Real-time recognition on
end devices

HiLens Kit

10
HiLens Framework + AI Application Runtime Framework
ModelBox (1)
HiLens Framework, the AI application development framework, is fully integrated with ModelBox.

HiLens Framework is an AI inference framework ModelBox provides a unified AI application development and runtime
that provides audio and video processing, with a framework featuring device-edge-cloud synergy, helping developers
device-cloud synergy. It helps developers easily quickly develop audio, video, NLP, and more AI applications and
deploy AI applications to Ascend devices. deploy these applications on the cloud, edge, and end devices.

HiLens Framework ModelBox Architecture

Basic capabilities O&M Security
Log management
Graph orchestration API
Monitoring and statistics
Functional unit APIs
Audio and Media Model Resource License management
video input processing inference management Graph orchestration engine
Model encryption and decryption
Output Deployment management
Log system EI APIs
management Basic functional unit library
Component
Media Libraries D Libraries Functional unit Resource Collaborative Distributed Task configuration management
framework management inference framework
Service plugin management

NPU: Ascend 310 Development and debugging

Adaptation Layer
Remote debugging
Inference framework
Hardware adaptation System adaptation
adaptation Performance analysis

11
HiLens Framework + AI Application Runtime Framework
ModelBox (2)
Unified application development, deployment, and running across the cloud, edge, and end devices

Device-edge-cloud joint
AI development and deployment
applications • Joint development and debugging of AI
Public Education Finance Manufacturing Oil & Gas models/applications across the cloud, edge,
and end devices
• Real-time model performance management
ModelArts: model training integration ModelArts: inference (edge/IoT deployment)

A variety of deployment modes

ModelBox, a unified AI application framework with cloud-edge-device synergy
• Online services: high throughput, low latency,
Hardware chips OS Inference frameworks and auto scaling
CPU GPU Euler HarmonyOS MindSpore TensorFlow • Batch inference: efficient resource utilization
and distributed compute
Ascend Hi35XX Centos Ubuntu PyTorch TensorRT
• Edge inference: software-hardware integrated
... ... ... solution
• On-device inference: model compression and
conversion, lightweight deployment
management, support for a wide range of end
Cloud Edge Devices devices

ModelBox open source home page:

Underlying software and All-scenario https://modelbox-ai.com/

Fast inference
hardware differences hidden development modes
12
Efficient Development and Deployment with ModelBox
Plugins/Tools
ModelBox SDKs lower the barriers to entry for developers, who can access more resources in the open-
source community. The ModelBox VSCode plugin improves developer experience.

13
ModelBox: Joint Development with Device-Edge-Cloud Synergy;
Train on Cloud, Deploy Anywhere; and Real-Time Model Update
⚫ Unified algorithm/model packaging formats with ModelBox, same APIs across the cloud, edge, and devices
⚫ A seamless AI marketplace with HiLens for seamless deployment and a robust AI ecosystem

Inference
AI Gallery/KooGallery
data

3rd-party system
management
Training
data
Model AI application Inference result

Inference
Data Algorithm development AI application deployment resources
AI application development
preprocessing and training and management
(Based on ModelBox)

Integration
Cloud/edge Cloud
service
deployment Northbound
APIs
Huawei Cloud ModelArts/on-
premises/other cloud platforms Edge

Lightweight edge/device
HiLens Lightweight edge/device
deployment deployment

14
Huawei HiLens Advantages
1. Device-cloud synergy 4. Cross-platform design
for inference
Huawei-developed Ascend and
Devices analyze the collected data HiSilicon chips, and other popular
locally, reducing the costs of chips are supported, covering the
transmitting data to the cloud. needs of mainstream monitoring
scenarios.

HiLens
2. Unified skill development 5. An abundance pre-
framework built AI skills
The skill market provides a rich
An easy-to-use skill development
selection of pre-built AI skills for all
framework simplifies development.
kinds of applications. Users can quickly
deploy these skills onto devices without
any development work.

3. ModelBox 6. Developer community

Unified application development, After developing new skills, developers
deployment, and running across the can share them as templates or release
cloud, edge, and end devices. them to the skill market, where other
users can buy them.

15
 Contents
1. Challenges to Device-Edge-Cloud Synergy

2. Huawei HiLens Service

3. Huawei HiLens Use Cases

16
HiLens: Multimodal AI Development Application Platform
Featuring Device-Cloud Synergy

HiLens AI-enabled devices

A powerful platform with a Drone Home camera Home bot
(babysitting and early
multimodal AI kernel education)

Unified APIs for inference and

computing
Autonomous Vehicle-mounted
vehicle devices
Huawei Cloud Device-cloud synergy
HiLens and low latency

Lower costs and HiLens deep Atlas edge station AI appliance

learning camera
One-stop platform for AI application development, higher efficiency Multimodal
distribution, deployment, and management, skill
development kit
development, a skill market, and device management,
and data management
Dynamic scheduling of inference resources
across the cloud and end devices Ascend 310
An AI SoC with ultra-high
Dynamic model tuning HiSilicon Hi35XX series
computing and energy efficiency
17
Home + AI, Making Homes Safer and More Comfortable
Human figure detection Human figure detection & Face tagging & attribute
Object detection
& crying recognition facial recognition recognition from video

Intelligent family member

Intelligent care for elderly Stranger monitoring Home object & food
analysis and retrieval
and children and alarms recognition
Children's time album
18
Shopping Mall + AI, Empowering Intelligent Monitoring and
Automated Analysis
Customer foot Statistics on old and
Crowd heat map
traffic new customers

Counting people entering or Real-time display of crowd density Based on facial recognition
leaving a specific area
19
Campus + AI, Higher Security and Smarter Management
License plate Abnormal sound Access control based
Intrusion detection
recognition detection on facial recognition

Real-time campus Alarms for abnormal Intelligent facial

Vehicle access control
monitoring sounds (e.g. broken glass) recognition
20
Danale Smart Cameras for Home Surveillance
Fast local inference on
home cameras, much
Devices: Human figure and facial
less data to transmit to
detection, frontal face recognition, baby
the cloud
cry recognition, and gesture recognition

Huawei
Cloud Further confirmation,
Algorithm SDK fast training, and fast
Lightweight integration iteration on the cloud
models
Huawei HiLens
Skill Market
Models with high
accuracy and Cloud-based
performance algorithm APIs Secondary
calling Device-cloud synergy,
lower latency and costs
Commercial
and higher accuracy
cameras
Cloud: facial recognition, facial
attribute recognition, and object
recognition Face image library
input and alarm
configuration on the
Danale platform

Device-cloud synergy: Devices call cloud-based APIs for further confirmation, with results returned in seconds.
21
Huawei HiLens + NTT Data: Smart Auto 4S Store
⚫ NTT Data provides an overall smart automobile 4S store solution that
integrates Huawei services. 3
4
⚫ Huawei provides a HiLens-based solution to support in-store customer
journey and experience analysis and optimization.
2
Multiple video
channels
Multi-skill AI 1
analysis

One-stop development Hardware differences Multi-channel and

on cloud and hidden, one-click skill multi-skill running, at
deployment on devices deployment, and fast low costs
iteration
Application scenario
• Entrance: customer profile (gender, age group, number of visits
• No personalized precision marketing solutions to the store, and license plate)
• Reception area: VIP identification and customer flow statistics
• Limited bandwidth, incapable of centralized analysis of all video data
• Dashboard area: recognition of customer interest per car model
• Different algorithm requirements for different stores and emotion
• Car experiencing: stay duration and path
22
Zhi Yi Ting + Huawei HiLens: Powering a Hassle-Free
Parking Management System with Device-Cloud Synergy
Intelligent parking service

Parking Network Comprehensive Data

management server management statistics
client system and analysis

HiLens-based license
VM & storage
plate recognition

Huawei Cloud Device-cloud synergy, low-

Automatic locating and Camera
position, wide-angle, all-
navigation to idle parking Automatic day/night mode
Private weather license plate
spaces switchover
network recognition

3G/4G/NB-IOT
75% 50% 30%
Recognition Manual Complaint
accuracy review rate rate

License plate capture &

quality filtering

23
ModelArts & HiLens Device-Cloud Synergy for Unmanned
Vehicle Developer Challenge

Traffic light Driving route

recognition recognition

Dynamic obstacle Automated valet

detection parking
Challenges:
Deploying a model on a Training on Platform
Data processing Built-in framework Real-time
conventional industrial cloud development
ModelArts, Huawei's HiLens Framework update
computer is complex. ModelArts-trained HiLens Kit cameras,
AI development provides access, data Skills are
Autonomous vehicles models are USB cameras,
platform, integrates processing, model updated in real
run at a high speed, developed into network cameras,
dataset upload, management, and time and
requiring real-time skills through and Wi-Fi interfaces
storage, auto output interfaces. delivered
inference. HiLens and are provided to
labeling, and Developers can focus through the
Algorithms must be able delivered to facilitate
processing. on algorithm tuning. platform.
to be updated instantly. HiLens Kit devices. development.

24
 Section Summary
⚫ This chapter describes the main functions, advantages, and use
cases of Huawei HiLens.

25
 Q&A
1. (Multiple-answer question) Which of the following are
advantages of Huawei HiLens?
A.Collaborative inference with device-cloud synergy
B. Unified skill development platform
C. Cross-platform design
D.A rich selection of skills in the skill market

26
 Q&A
2. (Multiple-answer question) Which of the following are use cases
of Huawei HiLens?
A.Intelligent home monitoring
B. Intelligent campus monitoring
C. Intelligent shopping mall and store monitoring
D.Smart in-vehicle devices

27
 Acronyms and Abbreviations
⚫ VR: virtual reality
⚫ AR: augmented reality
⚫ OBS: object storage service
⚫ SDK: software development kit
⚫ OCR: optical character recognition

28
 Recommendations
⚫ Huawei Cloud: https://www.huaweicloud.com/en-us/
⚫ HiLens documentation: https://support.huaweicloud.com/en-us/productdesc-hilens/hilens_01_0001.html
⚫ Huawei HiLens official website: https://www.huaweicloud.com/en-us/product/hilens.html
⚫ Huawei HiLens developer community: https://bbs.huaweicloud.com/forum/forum-771-1.html
⚫ HiLens Kit introduction: https://www.huaweicloud.com/product/hilens/hilenskit.html
⚫ Huawei HiLens developer courses: https://education.huaweicloud.com/programs/7515dce4-3f29-4124-
a42d-5b1fc11389ce/about
⚫ ModelArts: https://www.huaweicloud.com/product/modelarts.html

29
 Thank You.
Copyright © 2024 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including,
without limitation, statements regarding the future financial and operating results,
future product portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially from those
expressed or implied in the predictive statements. Therefore, such information is
provided for reference purpose only and constitutes neither an offer nor an
acceptance. Huawei may change the information at any time without notice.

Huawei Cloud | Grow with Intelligence