ESP32-C3 Wireless Adventure

A Comprehensive Guide to IoT

Espressif Systems

June 12, 2023

 Contents
I Preparation 1

1 Introduction to IoT 2
1.1 Architecture of IoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 IoT Application in Smart Homes . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Introduction and Practice of IoT Projects 7

2.1 Introduction to Typical IoT Projects . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Basic Modules for Common IoT Devices . . . . . . . . . . . . . . . . . 7
2.1.2 Basic Modules of Client Applications . . . . . . . . . . . . . . . . . . . 8
2.1.3 Introduction to Common IoT Cloud Platforms . . . . . . . . . . . . . . 9
2.2 Practice: Smart Light Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Project Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.2 Project Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Hardware Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.4 Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Introduction to ESP RainMaker 16

3.1 What is ESP RainMaker? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 The Implementation of ESP RainMaker . . . . . . . . . . . . . . . . . . . . . . 18
3.2.1 Claiming Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.2 RainMaker Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.3 Cloud Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2.4 RainMaker Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Practice: Key Points for Developing with ESP RainMaker . . . . . . . . . . . . 22
3.4 Features of ESP RainMaker . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 User Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.2 End User Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.3 Admin Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4 Setting Up Development Environment 27

4.1 ESP-IDF Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1.1 ESP-IDF Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2
 4.1.2 ESP-IDF Git Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.1.3 Choosing a Suitable Version . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.4 Overview of ESP-IDF SDK Directory . . . . . . . . . . . . . . . . . . . . 30
4.2 Setting Up ESP-IDF Development Environment . . . . . . . . . . . . . . . . . 34
4.2.1 Setting up ESP-IDF Development Environment on Linux . . . . . . . . 34
4.2.2 Setting up ESP-IDF Development Environment on Windows . . . . . . 36
4.2.3 Setting up ESP-IDF Development Environment on Mac . . . . . . . . . 41
4.2.4 Installing VS Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.5 Introduction to Third-Party Development Environments . . . . . . . . 42
4.3 ESP-IDF Compilation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.1 Basic Concepts of Compilation System . . . . . . . . . . . . . . . . . . 43
4.3.2 Project File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.3 Default Build Rules of the Compilation System . . . . . . . . . . . . . 46
4.3.4 Introduction to the Compilation Script . . . . . . . . . . . . . . . . . . 47
4.3.5 Introduction to Common Commands . . . . . . . . . . . . . . . . . . . 48
4.4 Practice: Compiling Example Program “Blink” . . . . . . . . . . . . . . . . . . 49
4.4.1 Example Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4.2 Compiling the Blink Program . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.3 Flashing the Blink Program . . . . . . . . . . . . . . . . . . . . . . . . 55
4.4.4 Serial Port Log Analysis of the Blink Program . . . . . . . . . . . . . . 56
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

II Hardware and Driver Development 60

5 Hardware Design of Smart Light Products based on ESP32-C3 61

5.1 Features and Composition of Smart Light Products . . . . . . . . . . . . . . . 61
5.2 Hardware Design of ESP32-C3 Core System . . . . . . . . . . . . . . . . . . . 64
5.2.1 Power Supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.2.2 Power-on Sequence and System Reset . . . . . . . . . . . . . . . . . . 68
5.2.3 SPI Flash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.2.4 Clock Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.2.5 RF and Antenna . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2.6 Strapping Pins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2.7 GPIO and PWM Controller . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Practice: Building a Smart Light System with ESP32-C3 . . . . . . . . . . . . . 74
5.3.1 Selecting Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.3.2 Configuring GPIOs of PWM Signals . . . . . . . . . . . . . . . . . . . . 76
5.3.3 Downloading Firmware and Debugging Interface . . . . . . . . . . . . 76
 5.3.4 Guidelines for RF Design . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3.5 Guidelines for Power Supply Design . . . . . . . . . . . . . . . . . . . 80
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6 Driver Development 81
6.1 Driver Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2 ESP32-C3 Peripheral Applications . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.3 LED Driver Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.3.1 Color Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.3.2 LED Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.3 LED Dimming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.4 Introduction to PWM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.4 LED Dimming Driver Development . . . . . . . . . . . . . . . . . . . . . . . . 90
6.4.1 Non-Volatile Storage (NVS) . . . . . . . . . . . . . . . . . . . . . . . . 91
6.4.2 LED PWM Controller (LEDC) . . . . . . . . . . . . . . . . . . . . . . . 92
6.4.3 LED PWM Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.5 Practice: Adding Drivers to Smart Light Project . . . . . . . . . . . . . . . . . 97
6.5.1 Button Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.5.2 LED Dimming Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

III Wireless Communication and Control 103

7 Wi-Fi Configuration and Connection 104

7.1 Basics of Wi-Fi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
7.1.1 Introduction to Wi-Fi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
7.1.2 Evolution of IEEE 802.11 . . . . . . . . . . . . . . . . . . . . . . . . . 104
7.1.3 Wi-Fi Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.1.4 Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
7.2 Basics of Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.2.1 Introduction to Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.2.2 Bluetooth Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.2.3 Bluetooth Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.3 Wi-Fi Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.3.1 Wi-Fi Network Configuration Guide . . . . . . . . . . . . . . . . . . . . 124
7.3.2 SoftAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.3.3 SmartConfig . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.3.4 Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.3.5 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
 7.4 Wi-Fi Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.4.1 Wi-Fi Components in ESP-IDF . . . . . . . . . . . . . . . . . . . . . . . 132
7.4.2 Exercise: Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . . . . 134
7.4.3 Exercise: Smart Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . 138
7.5 Practice: Wi-Fi Configuration in Smart Light Project . . . . . . . . . . . . . . . 149
7.5.1 Wi-Fi Connection in Smart Light Project . . . . . . . . . . . . . . . . . 149
7.5.2 Smart Wi-Fi Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 150
7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

8 Local Control 152

8.1 Introduction to Local Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
8.1.1 Application of Local Control . . . . . . . . . . . . . . . . . . . . . . . . 154
8.1.2 Advantages of Local Control . . . . . . . . . . . . . . . . . . . . . . . . 154
8.1.3 Discovering Controlled Devices through Smartphones . . . . . . . . . . 154
8.1.4 Data Communication Between Smartphones and Devices . . . . . . . . 155
8.2 Common Local Discovery Methods . . . . . . . . . . . . . . . . . . . . . . . . 155
8.2.1 Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
8.2.2 Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.2.3 Comparison Between Broadcast and Multicast . . . . . . . . . . . . . . 169
8.2.4 Multicast Application Protocol mDNS for Local Discovery . . . . . . . . 169
8.3 Common Communication Protocols for Local Data . . . . . . . . . . . . . . . 172
8.3.1 Transmission Control Protocol (TCP) . . . . . . . . . . . . . . . . . . . 172
8.3.2 HyperText Transfer Protocol (HTTP) . . . . . . . . . . . . . . . . . . . 178
8.3.3 User Datagram Protocol (UDP) . . . . . . . . . . . . . . . . . . . . . . 182
8.3.4 Constrained Application Protocol (CoAP) . . . . . . . . . . . . . . . . 185
8.3.5 Bluetooth Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
8.3.6 Summary of Data Communication Protocols . . . . . . . . . . . . . . . 196
8.4 Guarantee of Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
8.4.1 Introduction to Transport Layer Security (TLS) . . . . . . . . . . . . . 200
8.4.2 Introduction to Datagram Transport Layer Security (DTLS) . . . . . . . 206
8.5 Practice: Local Control in Smart Light Project . . . . . . . . . . . . . . . . . . 210
8.5.1 Creating a Wi-Fi-based Local Control Server . . . . . . . . . . . . . . . 210
8.5.2 Verifying Local Control Functionality using Scripts . . . . . . . . . . . 214
8.5.3 Creating a Bluetooth-based Local Control Server . . . . . . . . . . . . 215
8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

9 Cloud Control 218

9.1 Introduction to Remote Control . . . . . . . . . . . . . . . . . . . . . . . . . . 218
9.2 Cloud Data Communication Protocols . . . . . . . . . . . . . . . . . . . . . . 219
 9.2.1 MQTT Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
9.2.2 MQTT Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
9.2.3 MQTT Message Format . . . . . . . . . . . . . . . . . . . . . . . . . . 221
9.2.4 Protocol Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
9.2.5 Setting Up MQTT Broker on Linux and Windows . . . . . . . . . . . . 226
9.2.6 Setting Up MQTT Client Based on ESP-IDF . . . . . . . . . . . . . . . . 228
9.3 Ensuring MQTT Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
9.3.1 Meaning and Function of Certificates . . . . . . . . . . . . . . . . . . . 230
9.3.2 Generating Certificates Locally . . . . . . . . . . . . . . . . . . . . . . 232
9.3.3 Configuring MQTT Broker . . . . . . . . . . . . . . . . . . . . . . . . . 234
9.3.4 Configuring MQTT Client . . . . . . . . . . . . . . . . . . . . . . . . . 234
9.4 Practice: Remote Control through ESP RainMaker . . . . . . . . . . . . . . . . 236
9.4.1 ESP RainMaker Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
9.4.2 Node and Cloud Backend Communication Protocol . . . . . . . . . . . 237
9.4.3 Communication between Client and Cloud Backend . . . . . . . . . . . 242
9.4.4 User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
9.4.5 Basic Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
9.4.6 Smart Light Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
9.4.7 RainMaker App and Third-Party Integrations . . . . . . . . . . . . . . . 255
9.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

10 Smartphone App Development 262

10.1 Introduction to Smartphone App Development . . . . . . . . . . . . . . . . . . 262
10.1.1 Overview of Smartphone App Development . . . . . . . . . . . . . . . 263
10.1.2 Structure of the Android Project . . . . . . . . . . . . . . . . . . . . . . 263
10.1.3 Structure of the iOS Project . . . . . . . . . . . . . . . . . . . . . . . . 264
10.1.4 Lifecycle of an Android Activity . . . . . . . . . . . . . . . . . . . . . . 265
10.1.5 Lifecycle of iOS ViewController . . . . . . . . . . . . . . . . . . . . . . 266
10.2 Creating a New Smartphone App Project . . . . . . . . . . . . . . . . . . . . . 268
10.2.1 Preparing for Android Development . . . . . . . . . . . . . . . . . . . 268
10.2.2 Creating a New Android Project . . . . . . . . . . . . . . . . . . . . . . 268
10.2.3 Adding Dependencies for MyRainmaker . . . . . . . . . . . . . . . . . 269
10.2.4 Permission Request in Android . . . . . . . . . . . . . . . . . . . . . . 270
10.2.5 Preparing for iOS Development . . . . . . . . . . . . . . . . . . . . . . 270
10.2.6 Creating a New iOS Project . . . . . . . . . . . . . . . . . . . . . . . . 271
10.2.7 Adding Dependencies for MyRainmaker . . . . . . . . . . . . . . . . . 272
10.2.8 Permission Request in iOS . . . . . . . . . . . . . . . . . . . . . . . . . 273
10.3 Analysis of the App’s Functional Requirements . . . . . . . . . . . . . . . . . . 274
10.3.1 Analysis of the Project’s Functional Requirements . . . . . . . . . . . . 275
 10.3.2 Analysis of User Management Requirements . . . . . . . . . . . . . . . 275
10.3.3 Analysis of Device Provisioning and Binding Requirements . . . . . . . 276
10.3.4 Analysis of Remote-Control Requirements . . . . . . . . . . . . . . . . 276
10.3.5 Analysis of Scheduling Requirements . . . . . . . . . . . . . . . . . . . 277
10.3.6 Analysis of User Centre Requirements . . . . . . . . . . . . . . . . . . 278
10.4 Development of User Management . . . . . . . . . . . . . . . . . . . . . . . . 278
10.4.1 Introduction to RainMaker APIs . . . . . . . . . . . . . . . . . . . . . . 278
10.4.2 Initiating Communication via Smartphone . . . . . . . . . . . . . . . . 279
10.4.3 Account Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
10.4.4 Account Login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
10.5 Development of Device Provisioning . . . . . . . . . . . . . . . . . . . . . . . 285
10.5.1 Scanning Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
10.5.2 Connecting Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
10.5.3 Generating Secret Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.5.4 Getting Node ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.5.5 Provisioning Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
10.6 Development of Device Control . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.6.1 Binding Devices to Cloud Accounts . . . . . . . . . . . . . . . . . . . . 296
10.6.2 Getting a List of Devices . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.6.3 Getting Device Status . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
10.6.4 Changing Device Status . . . . . . . . . . . . . . . . . . . . . . . . . . 303
10.7 Development of Scheduling and User Centre . . . . . . . . . . . . . . . . . . . 306
10.7.1 Implementing Scheduling Function . . . . . . . . . . . . . . . . . . . . 306
10.7.2 Implementing User Centre . . . . . . . . . . . . . . . . . . . . . . . . . 308
10.7.3 More Cloud APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
10.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

11 Firmware Upgrade and Version Management 313

11.1 Firmware Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
11.1.1 Overview of Partition Tables . . . . . . . . . . . . . . . . . . . . . . . . 314
11.1.2 Firmware Boot Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
11.1.3 Overview of the OTA Mechanism . . . . . . . . . . . . . . . . . . . . . 318
11.2 Firmware Version Management . . . . . . . . . . . . . . . . . . . . . . . . . . 321
11.2.1 Firmware Marking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
11.2.2 Rollback and Anti-Rollback . . . . . . . . . . . . . . . . . . . . . . . . 323
11.3 Practice: Over-the-air (OTA) Example . . . . . . . . . . . . . . . . . . . . . . . 324
11.3.1 Upgrade Firmware Through a Local Host . . . . . . . . . . . . . . . . . 324
11.3.2 Upgrade Firmware Through ESP RainMaker . . . . . . . . . . . . . . . 327
11.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
IV Optimisation and Mass Production 335

12 Power Management and Low-Power Optimisation 336

12.1 ESP32-C3 Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
12.1.1 Dynamic Frequency Scaling . . . . . . . . . . . . . . . . . . . . . . . . 337
12.1.2 Power Management Configuration . . . . . . . . . . . . . . . . . . . . 339
12.2 ESP32-C3 Low-Power Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
12.2.1 Modem-sleep mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
12.2.2 Light-sleep Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
12.2.3 Deep-sleep mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
12.2.4 Current Consumption in Different Power Modes . . . . . . . . . . . . . 349
12.3 Power Management and Low-Power Debugging . . . . . . . . . . . . . . . . . 350
12.3.1 Log Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
12.3.2 GPIO Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
12.4 Practice: Power Management in Smart Light Project . . . . . . . . . . . . . . . 354
12.4.1 Configuring Power Management Feature . . . . . . . . . . . . . . . . . 355
12.4.2 Use Power Management Locks . . . . . . . . . . . . . . . . . . . . . . 356
12.4.3 Verifying Power Consumption . . . . . . . . . . . . . . . . . . . . . . . 357
12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

13 Enhanced Device Security Features 359

13.1 Overview of IoT Device Data Security . . . . . . . . . . . . . . . . . . . . . . . 359
13.1.1 Why Securing IoT Device Data? . . . . . . . . . . . . . . . . . . . . . . 360
13.1.2 Basic Requirements for IoT Device Data Security . . . . . . . . . . . . 361
13.2 Data Integrity Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.2.1 Introduction to Integrity Verification Method . . . . . . . . . . . . . . 362
13.2.2 Integrity Verification of Firmware Data . . . . . . . . . . . . . . . . . . 363
13.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
13.3 Data Confidentiality Protection . . . . . . . . . . . . . . . . . . . . . . . . . . 364
13.3.1 Introduction to Data Encryption . . . . . . . . . . . . . . . . . . . . . . 364
13.3.2 Introduction to Flash Encryption Scheme . . . . . . . . . . . . . . . . . 366
13.3.3 Flash Encryption Key Storage . . . . . . . . . . . . . . . . . . . . . . . 369
13.3.4 Working Mode of Flash Encryption . . . . . . . . . . . . . . . . . . . . 370
13.3.5 Flash Encryption Process . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.3.6 Introduction to NVS Encryption . . . . . . . . . . . . . . . . . . . . . . 373
13.3.7 Examples of Flash Encryption and NVS Encryption . . . . . . . . . . . 374
13.4 Data Legitimacy Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
13.4.1 Introduction to Digital Signature . . . . . . . . . . . . . . . . . . . . . 376
13.4.2 Overview of Secure Boot Scheme . . . . . . . . . . . . . . . . . . . . . 378
 13.4.3 Introduction to Software Secure Boot . . . . . . . . . . . . . . . . . . . 378
13.4.4 Introduction to Hardware Secure Boot . . . . . . . . . . . . . . . . . . 380
13.4.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
13.5 Practice: Security Features In Mass Production . . . . . . . . . . . . . . . . . . 386
13.5.1 Flash Encryption and Secure Boot . . . . . . . . . . . . . . . . . . . . . 386
13.5.2 Enabling Flash Encryption and Secure Boot with Batch Flash Tools . . 387
13.5.3 Enabling Flash Encryption and Secure Boot in Smart Light Project . . . 388
13.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388

14 Firmware Burning and Testing for Mass Production 389

14.1 Firmware Burning in Mass Production . . . . . . . . . . . . . . . . . . . . . . 389
14.1.1 Defining Data Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . 389
14.1.2 Firmware Burning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
14.2 Mass Production Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
14.3 Practice: Mass Production Data in Smart Light Project . . . . . . . . . . . . . 394
14.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

15 ESP Insights: Remote Monitoring Platform 395

15.1 Introduction to ESP Insights . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
15.2 Getting Started with ESP Insights . . . . . . . . . . . . . . . . . . . . . . . . . 399
15.2.1 Getting Started with ESP Insights in the esp-insights Project . . . . . . 399
15.2.2 Running Example in the esp-insights Project . . . . . . . . . . . . . . . 401
15.2.3 Reporting Coredump Information . . . . . . . . . . . . . . . . . . . . . 401
15.2.4 Customising Logs of Interest . . . . . . . . . . . . . . . . . . . . . . . . 402
15.2.5 Reporting Reboot Reason . . . . . . . . . . . . . . . . . . . . . . . . . 403
15.2.6 Reporting Custom Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 403
15.3 Practice: Using ESP Insights in Smart Light Project . . . . . . . . . . . . . . . 406
15.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
 Introduction
ESP32-C3 is a single-core Wi-Fi and Bluetooth 5 (LE) microcontroller SoC, based on the
open-source RISC-V architecture. It strikes the right balance of power, I/O capabilities,
and security, thus offering the optimal cost-effective solution for connected devices. To
show various applications of the ESP32-C3 family, this book by Espressif will take you on
an interesting journey through AIoT, starting from the basics of IoT project development
and environment setup to practical examples. The first four chapters talk about IoT, ESP
RainMaker and ESP-IDF. Chapter 5 and 6 brief on hardware design and driver development.
As you progress, you’ll discover how to configure your project through Wi-Fi networks and
mobile Apps. Finally, you’ll learn to optimize your project and put it into mass production.
If you are an engineer in related fields, a software architect, a teacher, a student, or anyone
who has an interest in IoT, this book is for you.
You may download the code example used in this book from Espressif’s site on GitHub. For
latest information on IoT development, please follow our official account.
 Preface
An Informatising World
Riding the wave of Internet, Internet of Things (IoT) made its grand debut to become a
new type of infrastructure in digital economy. To bring the technology closer to the public,
Espressif Systems works for the vision that developers from all walks of life can use IoT to
solve some of the most pressing problems of our times. A world of “Intelligent Network of
All Things” is what we are expecting from the future.
Designing our own chips makes a critical component of that vision. It is to be a marathon, re-
quiring constant breakthroughs against technological boundaries. From the “Game Changer”
ESP8266 to the ESP32 series integrating Wi-Fi and Bluetoothr (LE) connectivity, followed
by ESP32-S3 equipped by AI acceleration, Espressif never stops researching and develop-
ing products for AIoT solutions. With our open-source software, such as the IoT Develop-
ment Framework ESP-IDF, Mesh Development Framework ESP-MDF, and Device Connectiv-
ity Platform ESP RainMaker, we have created an independent framework for building AIoT
applications.
As of July 2022, the cumulative shipments of Espressif’s IoT chipsets have exceeded 800
million, leading in the Wi-Fi MCU market and powering up a huge number of connected de-
vices worldwide. Pursuit for excellence makes every Espressif product a big hit for its high
level of integration and cost efficiency. The release of ESP32-C3 marks a significant mile-
stone of Espressif’s self-developed technology. It is a single-core, 32-bit, RISC-V-based MCU
with 400KB of SRAM, which can run at 160MHz. It has integrated 2.4 GHz Wi-Fi and Blue-
tooth 5 (LE) with a long-range support. It strikes a fine balance of power, I/O capabilities,
and security, thus offering the optimal cost-effective solution for connected devices. Based
on such powerful ESP32-C3, this book is intended to help readers understand IoT-related
knowledge with detailed illustration and practical examples.

Why we wrote this book?

Espressif Systems is more than a semiconductor company. It is also an IoT platform company,
which always strives for breakthroughs and innovations in the field of technology. At the
same time, Espressif has open-sourced and shared its self-developed operating system and
software framework with the community, forming a unique ecosystem. Engineers, makers,
and technology enthusiasts actively develop new software applications based on Espressif’s
products, freely communicate, and share their experience. You can see developers’ fascinat-
ing ideas on various platforms all the time, such as YouTube and GitHub. The popularity
of Espressif’s products has stimulated an increasing number of authors who have produced
over 100 books based on Espressif chipsets, in more than ten languages, including English,
Chinese, German, French, and Japanese.
It is the support and trust of community partners that encourages Espressif’s continuous
innovation. “We strive to make our chips, operating systems, frameworks, solutions, Cloud,
business practices, tools, documentation, writings, ideas, etc., ever more relevant to the an-
swers people need in contemporary life’s most pressing problems. This is Espressif’s highest
ambition and moral compass.” said Mr. Teo Swee Ann, Founder and CEO of Espressif.
Espressif values reading and ideas. As the continuous upgrading of IoT technology poses
higher requirements on engineers, how can we help more people to quickly master IoT chips,
operating systems, software frameworks, application schemes and cloud service products?
As the saying goes, it is better to teach a man how to fish than to give him fish. In a brain-
storming session, it occurred to us that we could write a book to systematically sort out the
key knowledge of IoT development. We hit it off, quickly gathered a group of senior engi-
neers, and combined the experience of the technical team in embedded programming, IoT
hardware and software development, all contributing to the publishing of this book. In the
process of writing, we tried our best to be objective and fair, stripped of the cocoon, and use
concise expressions to tell the complexity and charm of the Internet of Things. We carefully
summarised the common questions, referred to the feedback and suggestions of the commu-
nity, in order to clearly answer the questions encountered in the development process, and
provide practical IoT development guidelines for relevant technicians and decision-makers.

Book Structure
This book takes an engineer-centred perspective and expounds the necessary knowledge for
IoT project development step by step. It is composed of four parts, as follows:
• Preparation (Chapter 1–4): This part introduces the architecture of IoT, typical IoT
project framework, the ESP RainMakerr cloud platform, and the development envi-
ronment ESP-IDF, so as to lay a solid foundation for IoT project development.
• Hardware and Driver Development (Chapter 5–6): Based on the ESP32-C3 chipset,
this part elaborates on the minimum hardware system and driver development, and
implements the control of dimming, colour grading, and wireless communication.
• Wireless Communication and Control (Chapter 7–11): This part explains the in-
telligent Wi-Fi configuration scheme based on ESP32-C3 chip, local & cloud control
protocols, and local & remote control of devices. It also provides schemes for develop-
ing smartphone apps, firmware upgrade, and version management.
• Optimisation and Mass Production (Chapter 12-15): This part is intended for ad-
vanced IoT applications, focusing on optimisation of products in power management,
low-power optimisation, and enhanced security. It also introduces firmware burning
and testing in mass production, and how to diagnose the running status and logs of
device firmware through the remote monitoring platform ESP Insights.
About the Source Code
Readers can run the example programmes in this book, either by entering the code manually
or by using the source code that accompanies the book. We emphasise the combination of
theory and practice, and thus set a Practice section based on the Smart Light project in
almost every chapter. All the codes are open-sourced. Readers are welcome to download
the source code and discuss it in the sections related to this book on GitHub and our official
forum [Link]. The open-sourced code of this book is subject to the terms of Apache
License 2.0.

Author’s Note
This book is officially produced by Espressif Systems and is written by the company’s senior
engineers. It is suitable for managers and R&D personnel in IoT-related industries, teachers
and students of related majors, and enthusiasts in the field of Internet of Things. We hope
that this book can serve as a work manual, a reference, and a bedside book, to be like a
good tutor and friend.
While compiling this book, we referred to some relevant research results of experts, schol-
ars, and technicians at home and abroad, and we did our best to cite them according to
academic norms. However, it is unavoidable that there should be some omissions, so here
we would like to express our deep respect and gratitude to all the relevant authors. In addi-
tion, we have quoted information from the Internet, so we would like to thank the original
authors and publishers and apologise that we cannot indicate the source of every piece of
information.
In order to produce a book of high quality, we have organised rounds of internal discussions,
and learned from the suggestions and feedback of trial readers and publisher editors. Here,
we would like to thank you again for your help which all contributed to this successful work.
Last, but the most importantly, thanks to everyone at Espressif who has worked so hard for
the birth and popularization of our products.
The development of IoT projects involves a wide range of knowledge. Limited to the length
of the book, as well as the level and experience of the author, omissions are unavoidable.
Therefore, we kindly request that experts and readers criticise and correct our mistakes. If
you have any suggestions for this book, please contact us at book@[Link]. We
look forward to your feedback.
 How to use this book?
The code of the projects in this book has been open sourced. You can download it from our
GitHub repository and share your thoughts and questions on our official forum.
GitHub: [Link]
Forum: [Link]
Throughout the book, there will be parts highlighted as shown below.

Source code
In this book, we emphasise the combination of theory and practice, and thus set a Practice
section about the Smart Light project in almost every chapter. Corresponding steps and
source page will be marked between two lines beginning with the tag Source code.

NOTE/TIPS
This is where you may find some critical information and reminding for successfully de-
bugging your programme. They will be marked between two thick lines beginning with
the tag NOTE or TIPS.

Most of the commands in this book are executed under Linux, prompted by the character
“$”. If the command requires superuser privileges to execute, the prompt will be replaced
by “#”. The command prompt on Mac systems is “%”, as used in Section 4.2.3 Installing
ESP-IDF on Mac.
The body text in this book will be printed in Charter, while the code examples, components,
functions, variables, code file names, code directories, and strings will be in Courier New.
Commands or texts that need to be input by the user, and commands that can be entered by
pressing the “Enter” key will be printed in Courier New bold. Logs and code blocks will
be presented in light blue boxes .

Example:
Second, use esp-idf/components/nvs flash/nvs partition generator/nvs
partition [Link] to generate the NVS partition binary file on the development host with
the following command:
$ python $IDF PATH/components/nvs flash/nvs partition generator/nvs partition
[Link] --input mass [Link] --output mass [Link] --size NVS PARTITION SIZE
 Chapter
Introduction to IoT
1

At the end of the 20th century, with the rise of computer networks and communication tech-
nologies, Internet rapidly integrated into people’s lives. As Internet technology continues
to mature, the idea of Internet of Things (IoT) was born. Literally, IoT means an Internet
where things are connected. While the original Internet breaks the limits of space and time
and narrows the distance between “person and person”, IoT makes “things” an important
participant, bringing “people” and “things” closer together. In the foreseeable future, IoT is
set to become the driving force of the information industry.
So, what is the Internet of Things?
It is hard to accurately define the Internet of Things, as its meaning and scope are constantly
evolving. In 1995, Bill Gates first brought up the idea of IoT in his book The Road Ahead.
Simply put, IoT enables objects to exchange information with each other through Internet.
Its ultimate goal is to establish an “Internet of Everything”. This is an early interpretation of
IoT, as well as a fantasy of future technology. Thirty years later, with the rapid development
of economy and technology, the fantasy is coming into reality. From smart devices, smart
homes, smart cities, Internet of Vehicles and wearable devices, to the “metaverse” supported
by IoT technologies, new concepts are constantly emerging. In this chapter, we will begin
with an explanation of the architecture of Internet of Things, and then introduce the most
common IoT application, the smart home, in order to help you get a clear understanding of
IoT.

1.1 Architecture of IoT

Internet of Things involves multiple technologies which have different application needs
and forms in different industries. In order to sort out the structure, the key technologies
and application characteristics of IoT, it is necessary to establish a unified architecture and
a standard technical system. In this book, the architecture of IoT is simply divided into four
layers: perception & control layer, network layer, platform layer, and application layer.
Perception & Control Layer
As the most basic element of the IoT architecture, perception & control layer is the core
to realise the comprehensive sensing of IoT. Its main function is to collect, identify and
control information. It consists of a variety of devices with the ability of perception,

2
 identification, control and execution, and is responsible for retrieving and analysing data
such as material properties, behavioural trends, and device status. In this way, IoT gets
to recognise the real physical world. Besides, the layer is also able to control the status of
the device.
The most common devices of this layer are various sensors, which play an important
role in information collection and identification. Sensors are like human sensory organs,
such as photosensitive sensors equalling to vision, acoustic sensors to hearing, gas sensors
to smelling, and pressure- and temperature-sensitive sensors to touching. With all these
“sensory organs”, objects become “alive” and capable of intelligent perception, recognition
and manipulation of the physical world.
Network Layer
The main function of the network layer is to transmit information, including data obtained
from the perception & control layer to specified target, as well as commands issued from
the application layer back to the perception & control layer. It serves as an important
communication bridge connecting different layers of an IoT system. To set up a basic
model of Internet of Things, it involves two steps to integrate objects into a network:
access to Internet and transmission through Internet.
Access to Internet
Internet enables interconnection between person and person, but fails to include things
into the big family. Before the advent of IoT, most things were not “network-able”.
Thanks to the continuous development of technology, IoT manages to connect things to
the Internet, thus realizing interconnection between “people and things”, and “things
and things”. There are two common ways to implement Internet connection: wired
network access and wireless network access.
Wired network access methods include Ethernet, serial communication (e.g., RS-232,
RS-485) and USB, while wireless network access depends on wireless communication,
which can be further divided into short-range wireless communication and long-range
wireless communication.
Short-range wireless communication includes ZigBee, Bluetoothr , Wi-Fi, Near-Field
Communication (NFC), and Radio Frequency Identification (RFID). Long-range wire-
less communication includes Enhanced Machine Type Communication (eMTC), LoRa,
Narrow Band Internet of Things (NB-IoT), 2G, 3G, 4G, 5G, etc.
Transmission through Internet
Different methods of Internet access lead to corresponding physical transmission link
of data. The next thing is to decide which communication protocol to use to transmit
the data. Compared with Internet terminals, most IoT terminals currently have fewer

Chapter 1. Introduction to IoT 3

 available resources, such as processing performance, storage capacity, network rate,
etc., so it is necessary to choose a communication protocol that occupies fewer resources
in IoT applications. There are two communication protocols that are widely used today:
Message Queuing Telemetry Transport (MQTT) and Constrained Application Protocol
(CoAP).
Platform Layer
The platform layer mainly refers to IoT cloud platforms. When all IoT terminals are net-
worked, their data need to be aggregated on an IoT cloud platform to be calculated and
stored. The platform layer mainly supports IoT applications in facilitating access and
management of massive devices. It connects IoT terminals to the cloud platform, collects
terminal data, and issues commands to terminals, so as to implement remote control. As
an intermediate service to assign equipment to industry applications, the platform layer
plays a connecting role in the entire IoT architecture, carrying abstract business logic
and standardized core data model, which can not only realize rapid access of devices,
but also provide powerful modular capabilities to meet various needs in industry appli-
cation scenarios. The platform layer mainly includes functional modules such as device
access, device management, security management, message communication, monitoring
operation and maintenance, and data applications.
• Device access, realising the connection and communication between terminals and IoT
cloud platforms.
• Device management, including functions such as device creation, device maintenance,
data conversion, data synchronization, and device distribution.
• Security management, ensuring the security of IoT data transmission from the per-
spectives of security authentication and communication security.
• Message communication, including three transmission directions, that is, the terminal
sends data to the IoT cloud platform, the IoT cloud platform sends data to the server
side or other IoT cloud platforms, and the server side remotely controls IoT devices.
• Monitoring O&M, involving monitoring and diagnosis, firmware upgrade, online de-
bugging, log services, etc.
• Data applications, involving the storage, analysis and application of data.
Application Layer
The application layer uses the data from the platform layer to manage the application,
filtering and processing them with tools such as databases and analysis software. The
resulting data can be used for real-world IoT applications such as smart healthcare, smart
agriculture, smart homes, and smart cities.
Of course, the architecture of IoT can be subdivided into more layers, but no matter how
many layers it consists of, the underlying principle remains essentially the same. Learning

4 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

about the architecture of IoT helps deepen our understanding of IoT technologies and build
fully functional IoT projects.

1.2 IoT Application in Smart Homes

IoT has penetrated into all walks of life, and the most closely related IoT application to us
is the smart home. Many traditional appliances are now equipped with one or more IoT
devices, and many newly built houses are designed with IoT technologies from the start.
Figure 1.1 shows some common smart home devices.

Figure 1.1. Common smart home devices

The development of smart home can be simply divided into smart product stage, scene
interconnection stage and intelligent stage, as shown in Figure 1.2.

Figure 1.2. Development stage of smart home

Chapter 1. Introduction to IoT 5

The first stage is about smart products. Different from traditional homes, in smart homes,
IoT devices receive signals with sensors, and are networked through wireless communication
technologies such as Wi-Fi, Bluetooth LE, and ZigBee. Users can control smart products in a
variety of ways, such as smartphone apps, voice assistants, smart speaker control, etc.
The second stage focuses on scene interconnection. In this stage, developers are no longer
considering controlling single smart product, but interconnecting two or more smart prod-
ucts, automating to a certain extent, and finally forming a custom scene mode. For example,
when the user presses any scene mode button, the lights, curtains, and air conditioners will
be automatically adapted to the presets. Of course, there is the prerequisite that the linkage
logic are readily set up, including trigger conditions and execution actions. Imagine that the
air conditioning heating mode is triggered when the indoor temperature drops below 10°C;
that at 7 o’clock in the morning, music is played to wake up the user, smart curtains are
opened, and the rice cooker or bread toaster starts through a smart socket; as the user gets
up and finishes washing, breakfast is already served, so that there will be no delay in going
to work. How convenient has our life become!
The third stage goes to intelligence stage. As more smart home devices are accessed, so
will the types of data generated. With the help of cloud computing, big data and artificial
intelligence, it is like a “smarter brain” has been planted into smart homes, which no longer
require frequent commands from the user. They collect data from previous interactions and
learn the user’s behaviour patterns and preferences, so as to automate activities, including
providing recommendations for decision-making.
Currently, most smart homes are at the scene interconnection stage. As the penetration
rate and intelligence of smart products increase, barriers between communication protocols
are being removed. In the future, smart homes are bound to become really “smart”, just
like the AI system Jarvis in Iron Man, which can not only help the user control various
devices, handle daily affairs, but also have super computing power and thinking ability. In
the intelligent stage, human beings will receive better services both in quantity and quality.

6 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Chapter Introduction and Practice of
2 IoT Projects

In Chapter 1, we introduced the architecture of IoT, and the roles and interrelationships of
the perception & control layer, network layer, platform layer, and application layer, as well
as the development of smart home. However, just like when we learn to paint, knowing
the theoretical knowledge is far from enough. We have to “get our hands dirty” to put IoT
projects into practice in order to truly master the technology. In addition, when a project
moves to the mass production stage, it is necessary to consider more factors such as net-
work connection, configuration, IoT cloud platform interaction, firmware management and
updates, mass production management, and security configuration.
So, what do we need to pay attention to when developing a complete IoT project?
In Chapter 1, we mentioned that smart home is one of the most common IoT application
scenarios, and smart lights are one of the most basic and practical appliances, which can be
used in homes, hotels, gyms, hospitals, etc. Therefore, in this book, we will take the con-
struction of a smart light project as the starting point, explain its components and features,
and provide guidance on project development. We hope that you can draw inferences from
this case to create more IoT applications.

2.1 Introduction to Typical IoT Projects

In terms of development, basic functional modules of IoT projects can be classified into
software and hardware development of IoT devices, client application development, and
IoT cloud platform development. It is important to clarify the basic functional modules,
which will be further described in this section.

2.1.1 Basic Modules for Common IoT Devices

Software and hardware development of IoT devices include the following basic modules:
Data collection
As the bottom layer of the IoT architecture, the IoT devices of the perception & control
layer connect sensors and devices through their chips and peripherals to achieve data
collection and operation control.

7
Account binding and initial configuration
For most IoT devices, account binding and initial configuration are completed in one oper-
ational process, for example, connecting devices with users by configuring Wi-Fi network.
Interaction with IoT cloud platforms
To monitor and control IoT devices, it is also necessary to connect them to IoT cloud
platforms, in order to give commands and report status through interaction between each
other.
Device control
When connected with IoT cloud platforms, devices can communicate with the cloud and
be registered, bound, or controlled. Users can query product status and carry out other
operations on the smartphone app through IoT cloud platforms or local communication
protocols.
Firmware upgrade
IoT devices can also achieve firmware upgrade based on manufacturers’ needs. By re-
ceiving commands sent by the cloud, firmware upgrade and version management will be
realized. With this firmware upgrade feature, you can continuously enhance the functions
of IoT devices, fix defects, and improve user experience.

2.1.2 Basic Modules of Client Applications

Client applications (e.g., smartphone apps) mainly include the following basic modules:
Account system and authorisation
It supports account and device authorisation.
Device control
Smartphone apps are usually equipped with controlling functions. Users can easily con-
nect to IoT devices, and manage them anytime, anywhere through smartphone apps. In
a real-world smart home, devices are mostly controlled through smartphone apps, which
not only enables intelligent management of devices, but also saves the cost of manpower.
Therefore, device control is a must for client applications, such as device function attribute
control, scene control, scheduling, remote control, device linkage, etc. Smart home users
can also customise scenes according to personal needs, controlling lighting, home appli-
ances, entrance, etc., to make home life more comfortable and convenient. They can time
air conditioning, turn off it remotely, set the hallway light on automatically once the door
is unlocked, or switch to the “theater” mode with one single button.
Notification
Client applications update real-time status of IoT devices, and send alerts when devices
go abnormal.

8 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

After-sales customer service
Smartphone apps can provide after-sales services for products, to solve problems related
to IoT device failures and technical operations in a timely manner.
Featured functions
To meet the needs of different users, other functions may be added, such as Shake, NFC,
GPS, etc. GPS can help set the accuracy of scene operations according to location and
distance, while the Shake function allows users to set the commands to be executed for
specific device or scene by shaking.

2.1.3 Introduction to Common IoT Cloud Platforms

IoT cloud platform is an all-in-one platform which integrates functions such as device man-
agement, data security communication, and notification management. According to their
target group and accessibility, IoT cloud platforms can be divided into public IoT cloud
platforms (hereinafter referred to as “public cloud”) and private IoT cloud platforms (here-
inafter referred to as “private cloud”).
Public cloud usually indicates shared IoT cloud platforms for enterprises or individuals,
operated and maintained by platform providers, and shared through the Internet. It can be
free or low-cost, and provides services throughout the open public network, such as Alibaba
Cloud, Tencent Cloud, Baidu Cloud, AWS IoT, Google IoT, etc. As a supporting platform,
public cloud can integrate upstream service providers and downstream end users to create
a new value chain and ecosystem.
Private cloud is built for enterprise use only, thus guaranteeing the best control over data,
security, and service quality. Its services and infrastructure are maintained separately by
enterprises, and the supporting hardware and software are also dedicated to specific users.
Enterprises can customise cloud services to meet the needs of their business. At present,
some smart home manufacturers have already got private IoT cloud platforms and devel-
oped smart home applications based on them.
Public cloud and private cloud have their own advantages, which will be explained later.
To achieve communication connectivity, it is necessary to complete at least embedded devel-
opment on the device side, alongwith business servers, IoT cloud platforms, and smartphone
apps. Facing such a huge project, public cloud normally provides software development kits
for device-side and smartphone apps to speed up the process. Both public and private cloud
provide services including device access, device management, device shadow, and operation
and maintenance.
Device access
IoT cloud platforms need to provide not only interfaces for device access using protocols

Chapter 2. Introduction and Practice of IoT Projects 9

 such as MQTT, CoAP, HTTPS, and WebSocket, but also the function of device security
authentication to block forged and illegal devices, effectively reducing the risk of being
compromised. Such authentication usually supports different mechanisms, so when de-
vices are mass-produced, it is necessary to pre-assign the device certificate according to
the selected authentication mechanism and burn it into the devices.
Device management
The device management function provided by IoT cloud platforms can not only help man-
ufacturers monitor the activation status and online status of their devices in real time,
but also allows options such as adding / removing devices, retrieving, adding / deleting
groups, firmware upgrade, and version management.
Device shadow
IoT cloud platforms can create a persistent virtual version (device shadow) for each de-
vice, and the status of the device shadow can be synchronised and obtained by smartphone
app or other devices through Internet transmission protocols. Device shadow stores the
latest reported status and expected status of each device, and even if the device is offline,
it can still obtain the status by calling APIs. Device shadow provides always-on APIs,
which makes it easier to build smartphone apps that interact with devices.
Operation and maintenance
The O&M function includes three aspects:
• Demonstrating statistical information about IoT devices and notifications.
• Log management allows information retrieval about device behavior, up / down mes-
sage flow, and message content.
• Device debugging supports command delivery, configuration update, and checking the
interaction between IoT cloud platforms and device messages.

2.2 Practice: Smart Light Project

After the theoretical introduction in each chapter, you will find a practice section related to
the Smart Light project to help you get hands-on experience. The project is based on Espres-
sif’s ESP32-C3 chip and ESP RainMaker IoT Cloud Platform, and covers wireless module
hardware in smart light products, embedded software for smart devices based on ESP32-
C3, smartphone apps, and ESP RainMaker interaction.

Source code
For better learning and developing experience, the project in this book has been open-
sourced. You can download the source code from our GitHub repository at [Link]
com/espressif/book-esp32c3-iot-projects.

10 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2.2.1 Project Structure
The Smart Light project consists of three parts:
i. Smart light devices based on ESP32-C3, responsible for interacting with IoT cloud
platforms, and controlling the switch, brightness and color temperature of the LED
lamp beads.
ii. Smartphone apps (including tablet apps running on Android and iOS), responsible
for network configuration of smart light products, as well as querying and controlling
their status.
iii. An IoT cloud platform based on ESP RainMaker. For simplification, we consider
the IoT cloud platform and business server as a whole in this book. Details about ESP
RainMaker will be provided in Chapter 3.
The correspondence between the Smart Light project structure and the architecture of IoT
is shown in Figure 2.1.

Figure 2.1. Structure of smart light project

2.2.2 Project Functions

Divided according to the structure, functions of each part are as follows.
Smart light devices
• Network configuration and connection.
• LED PWM control, such as switch, brightness, color temperature, etc.
• Automation or scene control, e.g., time switch.
• Encryption and secure boot of the Flash.
• Firmware upgrade and version management.

Chapter 2. Introduction and Practice of IoT Projects 11

Smartphone apps
• Network configuration and device binding.
• Smart light product control, such as switch, brightness, color temperature, etc.
• Automation or scene settings, e.g., time switch.
• Local/remote control.
• User registration, login, etc.
ESP RainMaker IoT cloud platform
• Enabling IoT device access.
• Providing device operation APIs accessible to smartphone apps.
• Firmware upgrade and version management.

2.2.3 Hardware Preparation

If interested in putting the project into practice, you will also need the following hardware:
smart lights, smartphones, Wi-Fi routers, and a computer that meets the installation require-
ments of the development environment.
Smart lights
Smart lights are a new type of bulbs, whose shape is the same as the general incandescent
bulb. A smart light is composed of capacitor step-down regulated power supply, wireless
module (with built-in ESP32-C3), LED controller and RGB LED matrix. When connected
to power, the 15 V DC voltage output after capacitor step-down, diode rectification, and
regulation provides energy to the LED controller and LED matrix. The LED controller can
automatically send high and low levels at certain intervals, switching the RGB LED matrix
between closed (lights on) and open (lights off), so that it can emit cyan, yellow, green,
purple, blue, red, and white light. The wireless module is responsible for connecting to
the Wi-Fi router, receiving and reporting the status of smart lights, and sending commands
to control the LED.

Figure 2.2. A simulated smart light

In the early development stage, you can simulate a smart light using the ESP32-C3-
DevKitM-1 board connected with RGB LED lamp beads (see Figure 2.2). But you should

12 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 note that this is not the only way to assemble a smart light. The hardware design of the
project in this book only contains a wireless module (with built-in ESP32-C3), but not a
complete smart light hardware design.
In addition, Espressif also produces a ESP32-C3-based audio development board – ESP32-
C3-Lyra – for controlling lights with audio. The board has interfaces for microphones and
speakers and can control LED strips. It can be used for developing ultra-low-cost, high-
performance audio broadcasters and rhythm light strips. Figure 2.3 shows a ESP32-C3-
Lyra board linked with a strip of 40 LED lights.

Figure 2.3. ESP32-C3-Lyra linked with a strip of 40 LED lights

Smartphones (Android/iOS)
The Smart Light project involves the development of a smartphone app for setting up and
controlling smart light products.
Wi-Fi routers
Wi-Fi routers convert wired network signals and mobile network signals into wireless net-
work signals, for computers, smartphones, tablets, and other wireless devices to connect
to the network. For example, broadband in the home only needs to be connected to a
Wi-Fi router to achieve wireless networking of Wi-Fi devices. The mainstream protocol
standard supported by Wi-Fi routers is IEEE 802.11n, with an average TxRate of 300
Mbps, or 600 Mbps at maximum. They are backward compatible with IEEE 802.11b and
IEEE 802.11g. The ESP32-C3 chip by Espressif supports IEEE 802.11b/g/n, so you can
choose a single-band (2.4 GHz) or dual-band (2.4 GHz and 5 GHz) Wi-Fi router.
A computer (Linux/macOS/Windows)
Development environment will be introduced in Chapter 4.

Chapter 2. Introduction and Practice of IoT Projects 13

2.2.4 Development Process

Figure 2.4. Steps of developing the Smart Light project

Hardware design
Hardware design of IoT devices is essential to an IoT project. A complete smart light
project is intended to produce a lamp working under mains supply. Different manufac-
turers produce lamps of different styles and driver types, but their wireless modules are
usually of the same function. To simplify the development process of the Smart Ligh
project, this book only covers the hardware design and software development of wireless
modules.
IoT cloud platform configuration
To use IoT cloud platforms, you need to configure projects on the backend, such as creat-
ing products, creating devices, setting device properties, etc.
Embedded software development for IoT devices
Implement expected functions with ESP-IDF, Espressif’s device-side SDK, including con-
necting to IoT cloud platforms, developing LED drivers, and upgrading firmware.
Smartphone app development
Develop smartphone apps for Android and iOS systems to realise user registration and
login, device control and other functions.
IoT device optimisation
Once the basic development of IoT device functions is completed, you may turn to opti-
misation tasks, such as power optimisation.
Mass production testing
Carry out mass production tests according to related standards, such as equipment func-
tion test, aging test, RF test, etc.
Despite the steps listed above, a Smart Light project is not necessarily subject to such pro-
cedure as different tasks can also be carried out at the same time. For example, embedded
software and smartphone apps can be developed in parallel. Some steps may also need to
be repeated, such as IoT device optimisation and mass production testing.

14 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2.3 Summary
In this chapter, we first expounded on the basic components and functional modules of
an IoT project, then introduced the Smart Light case for practice, refering to its structure,
functions, hardware preparation, and development process. Readers can draw inferences
from the practice and become confident to carry out IoT projects with minimum mistakes in
the future.

Chapter 2. Introduction and Practice of IoT Projects 15

 Chapter
Introduction to ESP RainMaker
3

The Internet of Things (IoT) offers endless possibilities to change the way people live, yet
the development of IoT engineering is full of challenges. With public clouds, terminal man-
ufacturers can implement product functionality through the following solutions:
Based on solution providers’ cloud platforms
In this way, terminal manufacturers only need to design the product hardware, then
connect the hardware to the cloud using provided communication module, and config-
ure the product functions following the guidelines. This is an efficient approach since
it eliminates the need for server-side and application-side development and operations
and maintenance (O&M). It allows terminal manufacturers to focus on hardware design
without having to consider cloud implementation. However, such solutions (e.g., device
firmware and App) are generally not open source, so the product functions will be lim-
ited by provider’s cloud platform which cannot be customized. Meanwhile, the user and
device data also belong to the cloud platform.
Based on cloud products
In this solution, after completing the hardware design, terminal manufacturers not only
need to implement cloud functions using one or more cloud products provided by the
public cloud, but also need to link the hardware with the cloud. For example, to connect
to Amazon Web Services (AWS), terminal manufacturers need to use AWS products such
as Amazon API Gateway, AWS IoT Core, and AWS Lambda to enable device access, re-
mote control, data storage, user management, and other basic functions. It not only asks
terminal manufacturers to flexibly use and configure cloud products with in-depth un-
derstanding and rich experience, but also requires them to consider the construction and
maintenance cost for initial and later stages This poses great challenges to the company’s
energy and resources.
Compared with public clouds, private clouds are usually built for specific projects and prod-
ucts. Private cloud developers are given highest level of freedom in protocol design and busi-
ness logic implementation. Terminal manufacturers can make products and design schemes
at will, and easily integrate and empower user data. Combining the high security, scalability
and reliability of public cloud with the advantages of private cloud, Espressif launched ESP

16
RainMaker, a deeply integrated private cloud solution based on Amazon cloud. Users can
deploy ESP RainMaker and build private cloud simply with an AWS account.

3.1 What is ESP RainMaker?

ESP RainMaker is a complete AIoT platform built with multiple mature AWS products. It
provides various services required for mass production such as device cloud access, device
upgrade, backend management, third-party login, voice integration, and user management.
By using the Serverless Application Repository (SAR) provided by AWS, terminal manufac-
turers can quickly deploy ESP RainMaker to their AWS accounts, which is time-efficient and
easy to operate. Managed and maintained by Espressif, the SAR used by ESP RainMaker
helps developers reduce cloud maintenance costs and accelerate the development of AIoT
products, thus building secure, stable, and customizable AIoT solutions. Figure 3.1 shows
the architecture of ESP RainMaker.

Figure 3.1. Architecture of ESP RainMaker

The ESP RainMaker public server by Espressif is free for all ESP enthusiasts, makers, and
educators for solution evaluation. Developers can log in with Apple, Google, or GitHub
accounts, and quickly build their own IoT application prototypes. The public server inte-
grates Alexa and Google Home, and provides voice control services, which are supported by
Alexa Skill and Google Actions. Its semantic recognition function is also powered by third
parties. RainMaker IoT devices only respond to specific actions. For an exhaustive list of
supported voice commands, please check the third-party platforms. In addition, Espressif
offers a public RainMaker App for users to control the products through smartphones.

Chapter 3. Introduction to ESP RainMaker 17

3.2 The Implementation of ESP RainMaker
As shown in Figure 3.2, ESP RainMaker consists of four parts:
• Claiming Service, enabling RainMaker devices to dynamically obtain certificates.
• RainMaker Cloud (also known as cloud backend), providing services such as message
filtering, user management, data storage, and third-party integrations.
• RainMaker Agent, enabling RainMaker devices to connect to RainMaker Cloud.
• RainMaker Client (RainMaker App or CLI scripts), for provisioning, user creation,
device association and control, etc.

Figure 3.2. Structure of ESP RainMaker

ESP RainMaker provides a complete set of tools for product development and mass produc-
tion, including:
RainMaker SDK
RainMaker SDK is based on ESP-IDF and provides the source code of the device-side
agent and related C APIs for firmware development. Developers only need to write the
application logic and leave the rest to the RainMaker framework. For more information
about C APIs, please visit [Link]
RainMaker App
The public version of RainMaker App allows developers to complete device provisioning,
and control and query the status of devices (e.g., smart lighting products). It is available
on both iOS and Android app stores. For more details, please refer to Chapter 10.
REST APIs
REST APIs help users build their own applications similar to the RainMaker App. For more
information, please visit [Link]

18 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Python APIs
A Python-based CLI, which comes with the RainMaker SDK, is provided to implement all
functions similar to smartphone features. For more information about Python APIs, please
visit [Link]
Admin CLI
Admin CLI, with higher level of access, is provided for ESP RainMaker private deployment
to generate device certificates in bulk.

3.2.1 Claiming Service

All communication between RainMaker devices and the cloud backend is carried out through
MQTT+TLS. In the context of ESP RainMaker, “Claiming” is the process in which devices
obtain certificates from the Claiming Service to connect to the cloud backend. Note that
Claiming Service is only applicable to the public RainMaker service, while for private de-
ployment, the device certificates need to be generated in bulk through Admin CLI. ESP
RainMaker supports three types of Claiming Service:
Self Claiming
The device itself fetches the certificates through a secret key pre-programmed in eFuse
after connecting to the Internet.
Host Driven Claiming
The certificates are obtained from the development host with the RainMaker account.
Assisted Claiming
The certificates are obtained via smartphone applications during provisioning.

3.2.2 RainMaker Agent

Figure 3.3. Structure of RainMaker SDK

The primary function of the RainMaker Agent is to provide connectivity and assist the appli-
cation layer to process uplink/downlink cloud data. It is built through the RainMaker SDK

Chapter 3. Introduction to ESP RainMaker 19

and developed based on the proven ESP-IDF framework, using ESP-IDF components such as
RTOS, NVS, and MQTT. Figure 3.3 shows the structure of the RainMaker SDK.
The RainMaker SDK includes two major features.
Connection
i. Cooperating with Claiming Service to obtain device certificates.
ii. Connecting to the cloud backend using the secure MQTT protocol to provide remote
connectivity and implement remote control, message reporting, user management,
device management, etc. It uses the MQTT component in ESP-IDF by default and
provides an abstraction layer to interface with other protocol stacks.
iii. Providing wifi provisioning component for Wi-Fi connection and provisioning,
esp https ota component for OTA upgrades, and esp local ctrl component for
local device discovery and connection. All these objectives can be achieved through
simple configuration.
Data processing
i. Storing the device certificates issued by Claiming Service and the data needed when
running RainMaker, by default using the interface provided by the nvs flash com-
ponent, and providing APIs for developers for direct use.
ii. Using the callback mechanism to process uplink/downlink cloud data and automati-
cally unblocking the data to the application layer for easy processing by developers.
For example, the RainMaker SDK provides rich interfaces for establishing TSL (Thing
Specification Language) data, which are required to define TSL models to describe IoT
devices and implement functions such as timing, countdown, and voice control. For
basic interactive features such as timing, RainMaker SDK provides a development-free
solution which can be simply enabled when needed. Then, the RainMaker Agent will
directly process the data, send it to the cloud through the associated MQTT topic, and
feed back the data changes in the cloud backend through callback mechanism.

3.2.3 Cloud Backend

The cloud backend is built on AWS Serverless Computing and achieved through AWS Cog-
nito (identity management system), Amazon API Gateway, AWS Lambda (serverless com-
puting service), Amazon DynamoDB (NoSQL database), AWS IoT Core (IoT access core that
provides MQTT access and rule filtering), Amazon Simple Email Service (SES simple mail
service), Amazon CloudFront (fast delivery network), Amazon Simple Queue Service (SQS
message queuing), and Amazon S3 (bucket storage service). It is aimed to optimize scal-
ability and security. With ESP RainMaker, developers can manage devices without having
to write code in the cloud. Messages reported by devices are transparently transmitted to

20 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

application clients or other third-party services.
Table 3.1 shows the AWS cloud products and functions used in the cloud backend, with
more products and features under development.

Table 3.1. AWS cloud products and functions used by the cloud backend

AWS Cloud Product

Function
Used by RainMaker

AWS Cognito Managing user credentials and supporting third-party logins

AWS Lambda Implementing the core business logic of the cloud backend

Amazon Timestream Storing time series data

Amazon DynamoDB Storing customers’ private information

AWS IoT Core Supporting MQTT communication

Amazon SES Providing email sending services

Amazon CloudFront Accelerating the management of backend website access

Amazon SQS Forwarding messages from AWS IoT Core

3.2.4 RainMaker Client

RainMaker clients, such as App and CLI, communicate with the cloud backend through REST
APIs. Detailed information and instructions about REST APIs can be found in the Swagger
documentation provided by Espressif. RainMaker’s mobile application client is available for
both iOS and Android systems. It allows device provisioning, control, and sharing, as well
as creating and enabling countdown tasks and connecting to third-party platforms. It can
automatically load UI and icons according to the configuration reported by the devices and
fully display the device TSL.
For example, if a smart light is built on the RainMaker SDK-provided examples, the icon
and UI of the bulb light will be loaded automatically when the provisioning is completed.
Users can change the color and brightness of the light through the interface and achieve
third-party control by linking Alexa Smart Home Skill or Google Smart Home Actions to
their ESP RainMaker accounts. Figure 3.4 shows the icon and UI examples of the bulb light
respectively on Alexa, Google Home, and ESP RainMaker App.

Chapter 3. Introduction to ESP RainMaker 21

 (a) Example - Alexa (b) Example - Google Home

(c) Example - ESP RainMaker

Figure 3.4. Examples of icon and UI of the bulb light

on Alexa, Google Home, and ESP RainMaker App

3.3 Practice: Key Points for Developing with ESP RainMaker

Once the device driver layer has been completed, developers may start to create TSL models
and process downlink data using the APIs provided by RainMaker SDK, and enable the ESP
RainMaker basic services based on the product definition and requirements.

22 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Section 9.4 of this book will explain the implementation of the LED smart light in RainMaker.
During debugging, developers can use the CLI tools in the RainMaker SDK to communicate
with the smart light (or call REST APIs from Swagger).
Chapter 10 will elaborate the usage of REST APIs in developing smartphone applications.
The OTA upgrades of LED smart lights will be covered in Chapter 11. If developers have
enabled the ESP Insights remote monitoring, the ESP RainMaker management backend will
display the ESP Insights data. Details will be presented in Chapter 15.
ESP RainMaker supports private deployment, which differs from the public RainMaker
server in the following ways:
Claiming Service
To generate certificates in private deployments, it is required to use the RainMaker Admin
CLI instead of Claiming. With public server, developers must be given admin rights to
implement firmware upgrade, but it is undesirable in commercial deployments. Therefore,
neither separate authentication service can be provided for self-claiming, nor admin rights
for host driven or assisted claiming.
Phone apps
In private deployments, applications need to be configured and compiled separately to
ensure that the account systems are not interoperable.
3rd party logins and voice integration
Developers have to configure separately via Google and Apple Developer accounts to en-
able 3rd party logins, as well as the Alexa Skill and Google Voice Assistant integration.

TIPS
For details about cloud deployment, please visit [Link]
com. In terms of firmware, migration from public server to private server only requires
replacing device certificates, which greatly improves migration efficiency and reduces the
cost of migration and secondary debugging.

3.4 Features of ESP RainMaker

ESP RainMaker features are mainly targeted at three aspect - user management, end users,
and admins. All features are supported in both public and private servers unless otherwise
stated.

3.4.1 User Management

The user management features allow end users to register, log in, change passwords, retrieve
passwords, etc.

Chapter 3. Introduction to ESP RainMaker 23

Register and log in
The registration and login methods supported by RainMaker include:
• Email id + Password
• Phone number + Password
• Google account
• Apple account
• GitHub account (public server only)
• Amazon account (private server only)

NOTE
Sign up using Google/Amazon shares the user’s email address with RainMaker. Sign up
using Apple shares a dummy address that Apple assigns for the user specifically for the
RainMaker service. A RainMaker account will be automatically created for users signing
in with a Google, Apple, or Amazon account for the first time.

Change password
Valid only for Email id/Phone number based logins. All other active sessions will be logged
out after password is changed. As per AWS Cognito behaviour, the logged-out sessions
can stay active upto 1 hour.
Retrieve password
Valid only for Email id/Phone number based logins.

3.4.2 End User Features

Features open to end users include local and remote control and monitoring, scheduling,
device grouping, device sharing, push notifications, and third-party integrations.
Remote control and monitoring
• Query configuration, parameter values, and connection status for one or all devices.
• Set parameters for single or multiple devices.
Local control and monitoring
Mobile phone and the device need to be connected to the same network for local control.
Scheduling
• Users pre-set certain actions at a specific time.
• No Internet connection required for the device while executing the schedule.
• One time or repeat (by specifying days) for single or multiple devices.
Device grouping
Supports multi-level abstract grouping Group metadata can be used to create a Home -
Room structure.

24 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Device sharing
One or more devices can be shared with one or more users.
Push notifications
End users will receive push notifications for events such as
• New device(s) added/removed
• Device connected to cloud
• Device disconnected from cloud
• Device sharing requests created/accepted/declined
• Alert messages reported by devices
Third party integrations
Alexa and Google Voice Assistant are supported to control RainMaker devices, including
lights, switches, sockets, fans, and temperature sensors.

3.4.3 Admin Features

Admin features allow administrators to implement device registration, device grouping, and
OTA upgrades, and to view statistics and ESP Insights data.
Device registration
Generate device certificates and register with Admin CLI (private server only).
Device grouping
Create abstract or structured groups based on device information (private server only).
Over-the-Air (OTA) upgrades
Upload firmware based on version and model, to one or more devices or a group Monitor,
cancel, or archive OTA jobs.
View statistics
Viewable statistics include:
• Device registrations (certificates registered by the admin)
• Device activations (device connected for the first time)
• User accounts
• User-device association
View ESP Insights data
Viewable ESP Insights data include:
• Errors, warnings, and custom logs
• Crash reports and analysis
• Reboot reasons
• Metrics like memory usage, RSSI, etc.
• Custom metrics and variables

Chapter 3. Introduction to ESP RainMaker 25

3.5 Summary
In this chapter, we introduced some key differences between the public RainMaker deploy-
ment and the private deployment. The private ESP RainMaker solution launched by Espres-
sif is highly reliable and extensible. All ESP32 series chips have been connected and adapted
to AWS, which greatly reduces the cost. Developers can focus on prototype verification with-
out having to learn about AWS cloud products. We also explained the implementation and
features of ESP RainMaker, and some key points for development using the platform.

Scan to download ESP RainMaker for Android Scan to download ESP RainMaker for iOS

26 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Chapter Setting Up
4 Development Environment

This chapter focuses on ESP-IDF, the official software development framework for ESP32-C3.
We’ll explain how to set up the environment on various operating systems, and introduce the
project structure and build system of ESP-IDF, as well as the usage of related development
tools. Then we’ll present the compiling and running process of an example project, while
offering a detailed explanation of the output log at each stage.

4.1 ESP-IDF Overview

ESP-IDF (Espressif IoT Development Framework) is a one-stop IoT development framework
provided by Espressif Technology. It uses C/C++ as the main development language and
supports cross-compilation under mainstream operating systems such as Linux, Mac, and
Windows. The example programs included in this book are developed using ESP-IDF, which
offers the following features:
• SoC system-level drivers. ESP-IDF includes drivers for ESP32, ESP32-S2, ESP32-C3,
and other chips. These drivers encompass peripheral low level (LL) library, hardware
abstraction layer (HAL) library, RTOS support and upper-layer driver software, etc.
• Essential components. ESP-IDF incorporates fundamental components required for IoT
development. This includes multiple network protocol stacks such as HTTP and MQTT,
a power management framework with dynamic frequency modulation, and features like
Flash Encryption and Secure Boot, etc.
• Development and production tools. ESP-IDF provides commonly used tools for build-
ing, flash, and debugging during development and mass production (see Figure 4.1),
such as the building system based on CMake, the cross-compilation tool chain based on
GCC, and the JTAG debugging tool based on OpenOCD, etc.
It is worth noting that the ESP-IDF code primarily adheres to the the Apache 2.0 open-source
license. Users can develop personal or commercial software without restrictions while com-
plying with the terms of the open-source license. Additionally, users are granted permanent
patent licenses free of charge, without the obligation to open-source any modifications made
to the source code.

27
 Figure 4.1. Building, flashing, and debug-
ging tools for development and mass production

4.1.1 ESP-IDF Versions

The ESP-IDF code is hosted on GitHub as an open-source project. Currently, there are three
major versions available: v3, v4, and v5. Each major version usually contains various sub-
versions, such as v4.2, v4.3, and so on. Espressif Systems ensures a 30-month support for
bug fixes and security patches for each released sub-version. Therefore, revisions of sub-
versions are also released regularly, such as v4.3.1, v4.2.2, etc. Table 4.1 shows the support
status of different ESP-IDF versions for Espressif chips, indicating whether they are in a
preview stage (offering support for preview versions, which may lack certain features or
documentation) or are officially supported.

Table 4.1. Support status of different ESP-IDF versions for Espressif chips

Series v4.1 v4.2 v4.3 v4.4 v5.0

ESP32 supported supported supported supported supported

ESP32-S2 supported supported supported supported

ESP32-C3 supported supported supported

ESP32-S3 supported supported

ESP32-C2 supported

ESP32-H2 preview preview

28 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The iteration of major versions often involves adjustments to the framework structure and
updates to the compilation system. For example, the major change from v3.* to v4.* was the
gradual migration of the build system from Make to CMake. On the other hand, iteration of
minor versions typically entails the addition of new features or support for new chips.
It is important to distinguish and understand the relationship between stable versions and
GitHub branches. Versions labeled as v*.* or v*.*.* represent stable versions that have
passed complete internal testing by Espressif. Once fixed, the code, tool chain, and release
documents for the same version remain unchanged. However, GitHub branches (e.g., the
release/v4.3 branch) undergo frequent code commits, often on a daily basis. Therefore,
two code snippets under the same branch may differ, necessitating developers to promptly
update their code accordingly.

4.1.2 ESP-IDF Git Workflow

Espressif follows a specific Git workflow for ESP-IDF, outlined as follows:
• New changes are made on the master branch, which serves as the main development
branch. The ESP-IDF version on the master branch always carries a -dev tag to indicate
that it is currently under development, such as v4.3-dev. Changes on the master
branch will first be reviewed and tested in Espressif’s internal repository, and then pushed
to GitHub after automated testing is complete.
• Once a new version has completed feature development on the master branch and met
the criteria for entering beta testing, it transitions to a new branch, such as release/
v4.3. In addition, this new branch is tagged as a pre-release version, like v4.3-beta1.
Developers can refer to the GitHub platform to access the complete list of branches and
tags for ESP-IDF. It’s important to note that the beta version (pre-release version) may
still have a significant number of known issues. As the beta version undergoes continuous
testing, bug fixes are added to both this version and the master branch simultaneously.
Meanwhile, the master branch may have already begun developing new features for the
next version. When testing is nearly complete, a release candidate (rc) label is added
to the branch, indicating that it is a potential candidate for the official release, such as
v4.3-rc1. At this stage, the branch remains a pre-release version.
• If no major bugs are discovered or reported, the pre-release version eventually receives
a major version label (e.g., v5.0) or a minor version label (e.g., v4.3) and becomes an
official release version, which is documented in the release notes page. Subsequently,
any bugs identified in this version are fixed on the release branch. After manual testing
is completed, the branch is assigned a bug-fix version label (e.g., v4.3.2), which is also
reflected on the release notes page.

Chapter 4. Setting Up Development Environment 29

4.1.3 Choosing a Suitable Version
Since ESP-IDF officially began supporting ESP32-C3 from version v4.3, and v4.4 has not
yet been officially released at the time of writing this book, the version used in this book
is v4.3.2, which is a revised version of v4.3. However, it is important to note that by the
time you read this book, v4.4 or newer versions may already be available. When selecting a
version, we recommend the following:
• For entry-level developers, it is advisable to choose the stable v4.3 version or its revised
version, which aligns with the example version used in this book.
• For mass production purposes, it is recommended to use the latest stable version to to
benefit from the most up-to-date technical support.
• If you intend to experiment with new chips or explore new product features, please
use the master branch. The latest version contains all the latest features, but keep in
mind that there may be known or unknown bugs present.
• If the stable version being used does not include the desired new features and you wish to
minimise the risks associated with the master branch, consider using the correspond-
ing release branch, such as the release/v4.4 branch. Espressif’s GitHub repository
will first create the release/v4.4 branch and subsequently release the stable v4.4 ver-
sion based on a specific historical snapshot of this branch, after completing all feature
development and testing.

4.1.4 Overview of ESP-IDF SDK Directory

The ESP-IDF SDK consists of two main directories: esp-idf and .espressif. The for-
mer contains ESP-IDF repository’s source code files and compilation scripts, while the lat-
ter mainly stores compilation tool chains and other software. Familiarity with these two
directories will help developers make better use of available resources and speed up the
development process. The directory structure of ESP-IDF is described below:
(1) ESP-IDF repository code directory (⇠/esp/esp-idf), as shown in Figure 4.2.
a. Component directory components
This core directory integrates numerous essential software components of ESP-IDF. No
project code can be compiled without relying on the components within this directory.
It includes driver support for various Espressif chips. From the LL library and HAL li-
brary interfaces for peripherals to the upper-level Driver and Virtual File System (VFS)
layer support, developers can choose the appropriate components at different levels for
their development needs. ESP-IDF also supports multiple standard network protocol
stacks such as TCP/IP, HTTP, MQTT, WebSocket, etc. Developers can utilise familiar
interfaces like Socket to build network applications. Components provide comprehen-

30 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 4.2. ESP-IDF repository code directory

sive functionality and can be easily integrated into applications, allowing developers to
focus solely on the business logic. Some common components include:
• driver: This component contains peripheral driver programs for various Espressif
chip series, such as GPIO, I2C, SPI, UART, LEDC (PWM), etc. The peripheral driver
programs in this component offer chip-independent abstract interfaces. Each periph-
eral has a common header file (such as gpio.h), eliminating the need to deal with
different chip-specific support questions.
• esp_wifi: Wi-Fi, as a special peripheral, is treated as a separate component. It
includes multiple APIs such as initialisation of various Wi-Fi driver modes, parameter
configuration, and event processing. Certain functions of this component are pro-
vided in the form of static link libraries. ESP-IDF also provides comprehensive driver
documentation for ease of use.

Chapter 4. Setting Up Development Environment 31

 • freertos: This component contains the complete FreeRTOS code. Apart from pro-
viding comprehensive support for this operating system, Espressif has also extended
its support to dual-core chips. For dual-core chips like ESP32 and ESP32-S3, users
can create tasks on specific cores.
b. Document directory docs
This directory contains ESP-IDF related development documents, including the Get
Started Guide, API Reference Manual, Development Guide, etc.

NOTE
After being compiled by automated tools, the contents of this directory are de-
ployed at [Link] Please ensure to switch the
document target to ESP32-C3 and select the specified ESP-IDF version.
c. Script tool tools
This directory contains commonly used compilation front-end tools such as [Link],
and the monitor terminal tool idf_monitor.py, etc. The sub-directory cmake also
contains core script files of the compilation system, serving as the foundation for im-
plementing ESP-IDF compilation rules. When adding the environment variables, the
contents within the tools directory are added to the system environment variable,
allowing [Link] to be executed directly under the project path.
d. Example program directory examples
This directory comprises a vast collection of ESP-IDF example programs that demon-
strate the usage of component APIs. The examples are organised into various sub-
directories based on their categories:
• get-started: This sub-directory includes entry-level examples like “hello world”
and “blink” to help users grasp the basics.
• bluetooth: You can find Bluetooth related examples here, including Bluetooth LE
Mesh, Bluetooth LE HID, BluFi, and more.
• wifi: This sub-directory focuses on Wi-Fi examples, including basic programs like
Wi-Fi SoftAP, Wi-Fi Station, espnow, as well as proprietary communication protocol
examples from Espressif. It also includes multiple application layer examples based
on Wi-Fi, such as Iperf, Sniffer, and Smart Config.
• peripherals: This extensive sub-directory is further divided into numerous sub-
folders based on peripheral names. It mainly contains peripheral driver examples for
Espressif chips, with each example featuring several sub-examples. For instance, the
gpio sub-directory includes two examples: GPIO and GPIO matrix keyboard. It’s
important to note that not all examples in this directory are applicable to ESP32-C3.

32 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 For example, the examples in usb/host are only applicable to peripherals with USB
Host hardware (such as ESP32-S3), and ESP32-C3 does not have this peripheral. The
compilation system typically provides prompts when setting the target. The README
file of each example lists the supported chips.
• protocols: This sub-directory contains examples for various communication pro-
tocols, including MQTT, HTTP, HTTP Server, PPPoS, Modbus, mDNS, SNTP, covering
a wide range of communication protocol examples required for IoT development.
• provisioning: Here, you’ll find provisioning examples for different methods, such
as Wi-Fi provisioning and Bluetooth LE provisioning.
• system: This sub-directory includes system debugging examples (e.g., stack tracing,
runtime tracing, task monitoring), power management examples (e.g., various sleep
modes, co-processors), and examples related to common system components like
console terminal, event loop, and system timer.
• storage: Within this sub-directory, you’ll discover examples of all file systems and
storage mechanisms supported by ESP-IDF (such as reading and writing of Flash, SD
card and other storage media), as well as examples of non-volatile storage (NVS),
FatFS, SPIFFS and other file system operations.
• security: This sub-directory contains examples related to flash encryption.
(2) ESP-IDF compilation tool chain directory (⇠/.espressif), as shown in Figure 4.3.

Figure 4.3. ESP-IDF compilation tool chain directory

Chapter 4. Setting Up Development Environment 33

 a. Software distribution directory dist
The ESP-IDF tool chain and other software are distributed in the form of compressed
packages. During the installation process, the installation tool first downloads the com-
pressed package to the dist directory, and then extracts it to the specified directory.
Once the installation is complete, the contents in this directory can be safely removed.
b. Python virtual environment directory python env
Different versions of ESP-IDF rely on specific versions of Python packages. Installing
these packages directly on the same host can lead to conflicts between package versions.
To address this, ESP-IDF utilises Python virtual environments to isolate different pack-
age versions. With this mechanism, developers can install multiple versions of ESP-IDF
on the same host and easily switch between them by importing different environment
variables.
c. ESP-IDF compilation tool chain directory tools
This directory mainly contains cross-compilation tools required to compile ESP-IDF
projects, such as CMake tools, Ninja build tools, and the gcc tool chain that generates
the final executable program. Additionally, this directory houses the standard library
of the C/C++ language along with the corresponding header files. If a program refer-
ences a system header file like #include <stdio.h>, the compilation tool chain will
locate the stdio.h file within this directory.

4.2 Setting Up ESP-IDF Development Environment

The ESP-IDF development environment supports mainstream operating systems such as Win-
dows, Linux, and macOS. This section will introduce how to set up the development envi-
ronment on each system. It is recommended to develop ESP32-C3 on Linux system, which
will be introduced in detail here. Many instructions are applicable across platforms due to
the similarity of the development tools. Therefore, it is advised to carefully read the content
of this section.

NOTE
You can refer to the online documents available at [Link]
which provide the commands mentioned in this section.

4.2.1 Setting up ESP-IDF Development Environment on Linux

The GNU development and debugging tools required for the ESP-IDF development environ-
ment are native to the Linux system. Additionally, the command-line terminal in Linux is
powerful and user-friendly, making it an ideal choice for ESP32-C3 development. You can

34 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

select your preferred Linux distribution, but we recommend using Ubuntu or other Debian-
based systems. This section provides guidance on setting up the ESP-IDF development envi-
ronment on Ubuntu 20.04.

1. Install required packages

Open a new terminal and execute the following command to install all necessary packages.
The command will automatically skip packages that are already installed.
$ sudo apt-get install git wget flex bison gperf python3 python3-pip python3-
setuptools cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0

TIPS
You need to use the administrator account and password for the command above. By
default, no information will be displayed when entering the password. Simply press the
“Enter” key to continue the procedure.

Git is a key code management tool in ESP-IDF. After successfully setting up the development
environment, you can use the git log command to view all code changes made since the
creation of ESP-IDF. In addition, Git is also used in ESP-IDF to confirm version information,
which is necessary for installing the correct tool chain corresponding to specific versions.
Along with Git, other important system tools include Python. ESP-IDF incorporates numer-
ous automation scripts written in Python. Tools such as CMake, Ninja-build, and Ccache
are widely used in C/C++ projects and serve as the default code compilation and building
tools in ESP-IDF. libusb-1.0-0 and dfu-util are the main drivers used for USB serial
communication and firmware burning.
Once the software packages are installed, you can use the apt show <package_name>
command to obtain detailed descriptions of each package. For example, use apt show git
to print the description information for the Git tool.

Q: What to do if the Python version is not supported?

A: ESP-IDF v4.3 requires a Python version that is not lower than v3.6. For older versions of
Ubuntu, please manually download and install a higher version of Python and set Python3
as the default Python environment. You can find detailed instructions by searching for the
keyword update-alternatives python.

2. Download ESP-IDF repository code

Open a terminal and create a folder named esp in your home directory using the mkdir
command. You can choose a different name for the folder if you prefer. Use the cd command
to enter the folder.

Chapter 4. Setting Up Development Environment 35

$ mkdir -p ⇠/esp
$ cd ⇠/esp

Use the git clone command to download the ESP-IDF repository code, as shown below:
$ git clone -b v4.3.2 --recursive [Link]

In the command above, the parameter -b v4.3.2 specifies the version to download (in
this case, version 4.3.2). The parameter --recursive ensures that all sub-repositories of
ESP-IDF are downloaded recursively. Information about sub-repositories can be found in the
.gitmodules file.

3. Install the ESP-IDF development tool chain

Espressif provides an automated script [Link] to download and install the tool chain.
This script checks the current ESP-IDF version and operating system environment, and then
downloads and installs appropriate version of Python tool packages and compilation tool
chains. The default installation path for the tool chain is ⇠/.espressif. All you need to
do is to navigate to the esp-idf directory and run [Link].
$ cd ⇠/esp/esp-idf
$ ./[Link]

If you install the the tool chain successfully, the terminal will display:
All done!

At this point, you have successfully set up the ESP-IDF development environment.

4.2.2 Setting up ESP-IDF Development Environment on Windows

1. Download ESP-IDF tools installer

TIPS
It is recommended to set up the ESP-IDF development environment on Windows 10 or
above. You can download the installer from [Link] The
installer is also an open-source software, and its source code can be viewed at https:
//[Link]/espressif/idf-installer.

• Online ESP-IDF tools installer

This installer is relatively small, around 4 MB in size, and other packages and code will
be downloaded during the installation process. The advantage of the online installer
is that not only can software packages and code be downloaded on demand during the
installation process, but also allows the installation of all available releases of ESP-IDF
and the latest branch of GitHub code (such as the master branch). The disadvantage
is that it requires a network connection during the installation process, which may
cause installation failure due to network problems.

36 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 • Offline ESP-IDF tools installer
This installer is larger, about 1 GB in size, and contains all the software packages and
code required for environment set up. The main advantage of the offline installer is
that it can be used on computers without Internet access, and generally has a higher
installation success rate. It should be noted that the offline installer can only install
stable releases of ESP-IDF identified by v*.* or v*.*.*.
2. Run the ESP-IDF tools installer
After downloading a suitable version of the installer (take ESP-IDF Tools Offline 4.3.2 for
example here), double-click the exe file to launch the ESP-IDF installation interface. The fol-
lowing demonstrates how to install ESP-IDF stable version v4.3.2 using the offline installer.
(1) In the “Select installation language” interface shown in Figure 4.4, select the language
to be used from the drop-down list.

Figure 4.4. “Select installation language” interface

(2) After selecting the language, click “OK” to pop up the “License agreement” interface
(see Figure 4.5). After carefully reading the installation license agreement, select “I
accept the agreement” and click “Next”.

Figure 4.5. “License agreement” interface

Chapter 4. Setting Up Development Environment 37

 (3) Review the system configuration in the “Pre-installation system check” interface (see
Figure 4.6). Check the Windows version and the installed antivirus software informa-
tion. Click “Next” if all the configuration items are normal. Otherwise, you can click
“Full log” for solutions based on key items.

Figure 4.6. “System check before installation” interface

TIPS
You can submit logs to [Link] for help.
(4) Select the ESP-IDF installation directory. Here, select D:/.espressif, as shown in
Figure 4.7, and click “Next”. Please note that .espressif here is a hidden directory.
After the installation is completed, you can view the specific contents of this directory
by opening the file manager and displaying hidden items.

Figure 4.7. Select the ESP-IDF installation directory

38 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(5) Check the components that need to be installed, as shown in Figure 4.8. It is recom-
mended to use the default option, that is, complete installation, and then click “Next”.

Figure 4.8. Select the components to install

(6) Confirm the components to be installed and click “Install” to start the automated in-
stallation process, as shown in Figure 4.9. The installation process may last tens of
minutes and the progress bar of the installation process is shown in Figure 4.10. Please
wait patiently.

Figure 4.9. Preparing for installation

(7) After the installation is complete, it is recommended to check “Register the ESP-IDF
Tools executables as Windows Defender exclusions...” to prevent antivirus software
from deleting files. Adding exclusion items can also skip frequent scans by antivirus

Chapter 4. Setting Up Development Environment 39

 Figure 4.10. Installation progress bar

software, greatly improving the code compilation efficiency of the Windows system.
Click “Finish” to complete the installation of the development environment, as shown
in Figure 4.11. You can choose to check “Run ESP-IDF PowerShell environment” or
“Run ESP-IDF command prompt”. Run the compilation window directly after installa-
tion to ensure that the development environment functions normally.

Figure 4.11. Installation completed

(8) Open the installed development environment in the program list (either ESP-IDF 4.3
CMD or ESP-IDF 4.3 PowerShell terminal, as shown in Figure 4.12), and the ESP-IDF
environment variable will be automatically added when running in the terminal. After
that, you can use the [Link] command for operations. The opened ESP-IDF 4.3 CMD
is shown in Figure 4.13.

40 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 4.12. Development environment installed

Figure 4.13. ESP-IDF 4.3 CMD

4.2.3 Setting up ESP-IDF Development Environment on Mac

The process of installing the ESP-IDF development environment on a Mac system is the
same as that on a Linux system. The commands for downloading the repository code and
installing the tool chain are exactly the same. Only the commands for installing dependency
packages are slightly different.
1. Install dependency packages
Open a terminal, and install pip, the Python package management tool, by running the
following command:
% sudo easy install pip

Install Homebrew, a package management tool for macOS, by running the following com-
mand:
% /bin/bash -c "$(curl -fsSL [Link]
HEAD/[Link])"

Install the required dependency packages by running the following command:

% brew python3 install cmake ninja ccache dfu-util

2. Download ESP-IDF repository code

Follow the instructions provided in section 4.2.1 to download the ESP-IDF repository code.
The steps are the same as for downloading on a Linux system.

Chapter 4. Setting Up Development Environment 41

3. Install the ESP-IDF development tool chain
Follow the instructions provided in section 4.2.1 to install the ESP-IDF development tool
chain. The steps are the same as for installation on a Linux system.

4.2.4 Installing VS Code

By default, the ESP-IDF SDK does not include a code editing tool (though the latest ESP-IDF
installer for Windows offers the option to install ESP-IDF Eclipse). You can use any text
editing tool of your choice to edit the code and then compile it using terminal commands.
One popular code editing tool is VS Code (Visual Studio Code), which is a free and feature-
rich code editor with a user-friendly interface. It offers various plugins that provide func-
tionalities such as code navigation, syntax highlighting, Git version control, and terminal
integration. Additionally, Espressif has developed a dedicated plugin called Espressif IDF for
VS Code, which simplifies project configuration and debugging.
You can use the code command in the terminal to quickly open the current folder in VS
Code. Alternatively, you can use the shortcut Ctrl+⇠ to open the system’s default terminal
console within VS Code.

TIPS
It is recommended to use VS Code for ESP32-C3 code development. Download and install
the latest version of VS Code at [Link]

4.2.5 Introduction to Third-Party Development Environments

In addition to the official ESP-IDF development environment, which primarily uses the C
language, ESP32-C3 also supports other mainstream programming languages and a wide
range of third-party development environments. Some notable options include:
Arduino:
an open-source platform for both hardware and software, supporting various microcon-
trollers, including ESP32-C3.
It uses the C++ language and offers a simplified and standardised API, commonly re-
ferred to as the Arduino language. Arduino is widely used in prototype development and
educational contexts. It provides an extensible software package and an IDE that allows
for easy compilation and flashing.
MicroPython:
a Python 3 language interpreter designed to run on embedded microcontroller platforms.
With a simple script language, it can directly access ESP32-C3’s peripheral resources (such
as UART, SPI, and I2C) and communication functions (such as Wi-Fi and Bluetooth LE).

42 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 This simplifies hardware interaction. MicroPython, combined with Python’s extensive
mathematical operation library, enables the implementation of complex algorithms on
ESP32-C3, facilitating the development of AI-related applications. As a script language,
there is no need for repeated compilation; modifications can be made and scripts can be
executed directly.
NodeMCU:
an LUA language interpreter developed for ESP series chips.
It supports almost all peripheral functions of ESP chips and is lighter than MicroPython.
Similar to MicroPython, NodeMCU uses a script language, eliminating the need for re-
peated compilation.
Furthermore, ESP32-C3 also supports the NuttX and Zephyr operating systems. NuttX is a
real-time operating system that provides POSIX-compatible interfaces, enhancing applica-
tion portability. Zephyr is a small real-time operating system specifically designed for IoT
applications. It includes numerous software libraries required in IoT development, gradually
evolving into a comprehensive software ecosystem.
This book does not provide detailed installation instructions for the aforementioned develop-
ment environments. You can install a development environment based on your requirements
by following the respective documentation and instructions.

4.3 ESP-IDF Compilation System

4.3.1 Basic Concepts of Compilation System
An ESP-IDF project is a collection of a main program with an entry function and multiple
independent functional components. For example, a project that controls LED switches
mainly consists of an entry program main and a driver component that controls GPIO. If
you want to realise the LED remote control, you also need to add Wi-Fi, TCP/IP protocol
stack, etc.
The compilation system can compile, link, and generate executable files (.bin) for the code
through a set of building rules. The compilation system of ESP-IDF v4.0 and above versions
is based on CMake by default, and the compilation script [Link] can be used to
control the compilation behavior of the code. In addition to supporting the basic syntax of
CMake, the ESP-IDF compilation system also defines a set of default compilation rules and
CMake functions, and you can write the compilation script with simple statements.

4.3.2 Project File Structure

A project is a folder that contains an entry program main, user-defined components, and
files required to build executable applications, such as compilation scripts, configuration

Chapter 4. Setting Up Development Environment 43

files, partition tables, etc. Projects can be copied and passed on, and the same executable file
can be compiled and generated in machines with the same version of ESP-IDF development
environment. A typical ESP-IDF project file structure is shown in Figure 4.14.

Figure 4.14. Typical ESP-IDF project file structure

Since ESP-IDF supports multiple IoT chips from Espressif, including ESP32, ESP32-S series,
ESP32-C series, ESP32-H series, etc., a target needs to be determined before compiling the
code. The target is both the hardware device that runs the application program and the
build target of the compilation system.
Depending on your needs, you can specify one or more targets for your project. For example,
through command [Link] set-target esp32c3, you can set the compilation target to
ESP32-C3, during which the default parameters and compilation tool chain path for ESP32-
C3 will be loaded. After compilation, an executable program can be generated for ESP32-
C3. You can also run the command set-target again to set a different target, and the
compilation system will automatically clean up and reconfigure.
Components
Components in ESP-IDF are modular and independent code units managed within the
compilation system. They are organised as folders, with the folder name representing
the component name by default. Each component has its own compilation script that

44 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 specifies its compilation parameters and dependencies. During the compilation process,
components are compiled into separate static libraries (.a files) and eventually combined
with other components to form the application program.
ESP-IDF provides essential functions, such as the operating system, peripheral drivers,
and network protocol stack, in the form of components. These components are stored
in the components directory located within the ESP-IDF root directory. Developers do
not need to copy these components to the components directory of myProject. In-
stead, they only need to specify the dependency relationships of these components in the
project’s [Link] file using the REQUIRES or PRIV_REQUIRES directives. The
compilation system will automatically locate and compile the required components.
Therefore, the components directory under myProject is not necessary. It is only used
to include some custom components of the project, which can be third-party libraries or
user-defined code. Additionally, components can be sourced from any directory other
than ESP-IDF or the current project, such as from an open-source project saved in another
directory. In this case, you only need to add the path of the component by setting the
EXTRA_COMPONENT_DIRS variable in the [Link] under the root directory.
This directory will override any ESP-IDF component with the same name, ensuring the
correct component is used.
Entry program main
The main directory within the project follows the same file structure as other components
(e.g., component1). However, it holds a special significance as it is a mandatory compo-
nent that must exist in every project. The main directory contains the project’s source code
and the user program’s entry point, typically named app_main. By default, the execu-
tion of the user program starts from this entry point. The main component also differs in
that it automatically depends on all components within the search path. Therefore, there
is no need to explicitly indicate dependencies using the REQUIRES or PRIV_REQUIRES
directives in the [Link] file.
Configuration file
The root directory of the project contains a configuration file called sdkconfig, which
contains the configuration parameters for all the components within the project. The
sdkconfig file is automatically generated by the compilation system and can be modi-
fied and regenerated by the command [Link] menuconfig. The menuconfig options
mainly originate from the [Link] of the project and the Kconfig of the
components. Component developers generally add configuration items in Kconfig to
make the component flexible and configurable.
Build directory
By default, the build directory within the project stores intermediate files and the fi-

Chapter 4. Setting Up Development Environment 45

 nal executable programs generated by the [Link] build command. In general, it is
not necessary to directly access the contents of the build directory. ESP-IDF provides
predefined commands to interact with the directory, such as using the [Link] flash
command to automatically locate the compiled binary file and flash it to the specified
flash address, or using the [Link] fullclean command to clean the entire build
directory.
Partition table ([Link])
Each project requires a partition table to divide the space of flash and specify the size and
starting address of the executable program and user data space. Command [Link]
flash or OTA upgrade program will flash the firmware to the corresponding address
according to this table. ESP-IDF provides several default partition tables in components/
partition_table, such as partitions_singleapp.csv and partitions_two_
[Link], which can be selected in menuconfig.
If the default partition table of the system cannot meet the requirements of the project,
a custom [Link] can be added to the project directory and be selected in
menuconfig.

4.3.3 Default Build Rules of the Compilation System

Rules for overriding components with the same name
During the component search process, the compilation system follows a specific order.
It first searches for internal components of ESP-IDF, then looks for components of the
user project, and finally searches for components in EXTRA_COMPONENT_DIRS. In cases
where multiple directories contain components with the same name, the component
found in the last directory will override any previous components with the same name.
This rule allows for the customisation of ESP-IDF components within the user project,
while keeping the original ESP-IDF code intact.
Rules for including common components by default
As mentioned in section 4.3.2, components need to explicitly specify their dependencies
on other components in the [Link]. However, common components such as
freertos are automatically included in the build system by default, even if their depen-
dency relationships are not explicitly defined in the compilation script. ESP-IDF common
components include freertos, Newlib, heap, log, soc, esp_rom, esp_common,
xtensa/riscv, and cxx. Using these common components avoids repetitive work when
writing [Link] and make it more concise.
Rules for overriding configuration items
Developers can add default configuration parameters by adding a default configuration
file named [Link] to the project. For example, adding CONFIG_LOG_

46 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 DEFAULT_LEVEL_NONE = y can configure the UART interface to not print log data
by default. Furthermore, if specific parameters need to be set for a particular target, a
configuration file named [Link].TARGET_NAME can be added, where
TARGET_NAME can be esp32s2, esp32c3, and so on. These configuration files are im-
ported into the sdkconfig during compilation, with the general default configuration
file [Link] being imported first, followed by the target-specific configu-
ration file, such as [Link].esp32c3. In cases where there are configu-
ration items with the same name, the latter configuration file will override the former.

4.3.4 Introduction to the Compilation Script

When developing a project using ESP-IDF, developers not only need to write source code but
also need to write [Link] for the project and components. [Link] is
a text file, also known as a compilation script, which defines a series of compilation objects,
compilation configuration items, and commands to guide the compilation process of the
source code. The compilation system of ESP-IDF v4.3.2 is based on CMake. In addition
to supporting native CMake functions and commands, it also defines a series of custom
functions, making it much easier to write compilation scripts.
The compilation scripts in ESP-IDF mainly include the project compilation script and the
component compilation scripts. The [Link] in the root directory of the project
is called the project compilation script, which guides the compilation process of the entire
project. A basic project compilation script typically includes the following three lines:
1. cmake_minimum_required(VERSION 3.5)
2. include($ENV{IDF_PATH}/tools/cmake/[Link])
3. project(myProject)

Among them, the cmake_minimum_required (VERSION 3.5) must be placed on the

first line, which is used to indicate the minimum CMake version number required by the
project. Newer versions of CMake are generally backward compatible with older versions,
so adjust the version number accordingly when using newer CMake commands to ensure
compatibility.
include($ENV {IDF_PATH}/tools/cmake/[Link]) imports pre-defined
configuration items and commands of ESP-IDF compilation system, including the default
build rules of the compilation system described in Section 4.3.3. project(myProject)
creates the project itself and specifies its name. This name will be used as the final output
binary file name, i.e., [Link] and [Link].
A project can have multiple components, including the main component. The top-level di-
rectory of each component contains a [Link] file, which is called the component
compilation script. Component compilation scripts are mainly used to specify component
dependencies, configuration parameters, source code files, and included header files for

Chapter 4. Setting Up Development Environment 47

compilation. With ESP-IDF’s custom function idf_component_register, the minimum
required code for a component compilation script is as follows:
1. idf_component_register(SRCS "src1.c"
2. INCLUDE_DIRS "include"
3. REQUIRES component1)

The SRCS parameter provides a list of source files in the component, separated by spaces if
there are multiple files. The INCLUDE_DIRS parameter provides a list of public header file
directories for the component, which will be added to the include search path for other
components that depend on the current component. The REQUIRES parameter identifies the
public component dependencies for the current component. It is necessary for components
to explicitly state which components they depend on, such as component2 depending on
component1. However, for the main component, which depends on all components by
default, the REQUIRES parameter can be omitted.
In addition, native CMake commands can also be used in the compilation script. For exam-
ple, use the command set to set variables, such as set(VARIABLE "VALUE").

4.3.5 Introduction to Common Commands

ESP-IDF uses CMake (project configuration tool), Ninja (project building tool) and esptool
(flash tool) in the process of code compilation. Each tool plays a different role in the com-
pilation, building, and flash process, and also supports different operating commands. To
facilitate user operation, ESP-IDF adds a unified front-end [Link] that allows the above
commands to be called quickly.
Before using [Link], make sure that:
• The environment variable IDF_PATH of ESP-IDF has been added to the current terminal.
• The command execution directory is the root directory of the project, which includes the
project compilation script [Link].
The common commands of [Link] are as follows:
• [Link] --help: displaying a list of commands and their usage instructions.
• [Link] set-target <target>: setting the compilation [Link] fullcleanrget, such
as replacing <target> with esp32c3.
• [Link] menuconfig: launching menuconfig, a terminal graphical configuration
tool, which can select or modify configuration options, and the configuration results
are saved in the sdkconfig file.
• [Link] build: initiating code compilation. The intermediate files and the final exe-
cutable program generated by the compilation will be saved in the build directory of
the project by default. The compilation process is incremental, which means that if only
one source file is modified, only the modified file will be compiled next time.

48 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

• [Link] clean: cleaning the intermediate files generated by the project compilation.
The entire project will be forced to compile in the next compilation. Note that the CMake
configuration and the configuration modifications made by menuconfig will not be
deleted during cleanup.
• [Link] fullclean: deleting the entire build directory, including all CMake config-
uration output files. When building the project again, CMake will configure the project
from scratch. Please note that this command will recursively delete all files in the build
directory, so use it with caution, and the project configuration file will not be deleted.
• [Link] flash: flashing the executable program binary file generated by build to
the target ESP32-C3. The options -p <port_name> and -b <baud_rate> are used
to set the device name of the serial port and the baud rate for flashing, respectively. If
these two options are not specified, the serial port will be automatically detected and the
default baud rate will be used.
• [Link] monitor: displaying the serial port output of the target ESP32-C3. The option
-p can be used to specify the device name of the host-side serial port. During serial port
printing, press the key combination Ctrl+] to exit the monitor.
The above commands can also be combined as needed. For example, the command [Link]
build flash monitor will perform code compilation, flash, and open the serial port
monitor in sequence.
You can visit [Link] to know more about ESP-IDF com-
pilation system.

4.4 Practice: Compiling Example Program “Blink”

4.4.1 Example Analysis
This section will take the program Blink as an example to analyse the file structure and
coding rules of a real project in detail. The Blink program implements the LED blinking
effect, and the project is located in the directory examples/get-started/blink, which
contains a source file, configuration files, and several compilation scripts.
The smart light project introduced in this book is based on this example program. Functions
will be gradually added in later chapters to finally complete it.

Source code
In order to demonstrate the entire development process, the Blink program has been
copied to esp32c3-iot-projects/device firmware/1 blink.

The directory structure of the blink project files is shown in Figure 4.15.
The blink project contains only one main directory, which is a special component that

Chapter 4. Setting Up Development Environment 49

 Figure 4.15. File directory structure of the blink project

must be included as described in section 4.3.2. The main directory is mainly used to store
the implementation of the app_main() function, which is the entry point to the user pro-
[Link] blink project does not include the components directory, because this example
only needs to use the components that come with ESP-IDF and does not require additional
components. The [Link] included in the blink project is used to guide the
compilation process, while [Link] is used to add configuration items for
this example program in menuconfig. Other unnecessary files will not affect the compila-
tion of the code, so they will not be discussed here. A detailed introduction to the blink
project files is as follows.
1. /*blink.c includes the following header files*/
2. #include <stdio.h> //Standard C library header file
3. #include "freertos/freeRTOS.h" //FreeRTOS main header file
4. #include "freertos/task.h" //FreeRTOS Task header file
5. #include "sdkconfig.h" //Configuration header file generated by kconfig
6. #include "driver/gpio.h" //GPIO driver header file

The source file blink.c contains a series of header files corresponding to function declara-
tions. ESP-IDF generally follows the order of including standard library header files, FreeR-
TOS header files, driver header files, other component header files, and project header files.
The order in which header files are included may affect the final compilation result, so try to
follow the default rules. It should be noted that sdkconfig.h is automatically generated
by kconfig and can only be configured through the command [Link] menuconfig.
Direct modification of this header file will be overwritten.
1. /*You can select the GPIO corresponding to the LED in [Link] menuconfig,
and the modification result of menuconfig is that the value of CONFIG_BLINK
_GPIO will be changed. You can also directly modify the macro definition
here, and change CONFIG_BLINK_GPIO to a fixed value.*/
2. #define BLINK_GPIO CONFIG_BLINK_GPIO
3. void app_main(void)
4. {
5. /*Configure IO as the GPIO default function, enable pull-up mode, and
6. disable input and output modes*/
7. gpio_reset_pin(BLINK_GPIO);

50 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

8. /*Set GPIO to output mode*/
9. gpio_set_direction(BLINK_GPIO, GPIO_MODE_OUTPUT);
10. while(1) {
11. /*Print log*/
12. printf("Turning off the LED\n");
13. /*Turn off the LED (output low level)*/
14. gpio_set_level(BLINK_GPIO, 0);
15. /*Delay (1000 ms)*/
16. vTaskDelay(1000 / portTICK_PERIOD_MS);
17. printf("Turning on the LED\n");
18. /*Turn on the LED (output high level)*/
19. gpio_set_level(BLINK_GPIO, 1);
20. vTaskDelay(1000 / portTICK_PERIOD_MS);
21. }
22. }

The app_main() function in the Blink example program serves as the entry point for user
programs. It is a simple function with no parameters and no return value. This function is
called after the system has completed initialisation, which includes tasks such as initialising
the log serial port, configuring single/dual core, and configuring the watchdog.
The app_main() function runs in the context of a task named main. The stack size and
priority of this task can be adjusted in menuconfig → Componentconfig → Common
ESP-related.
For simple tasks like blinking an LED, all the necessary code can be implemented directly
in the app_main() function. This typically involves initialising the GPIO corresponding to
the LED and using a while(1) loop to toggle the LED on and off. Alternatively, you can
use FreeRTOS API to create a new task that handles the LED blinking. Once the new task is
successfully created, you can exit the app_main() function.
The content of main/[Link] file, which guides the compilation process for the
main component, is as follows:
1. idf_component_register(SRCS "blink.c" INCLUDE_DIRS "." )

Among them, main/[Link] only calls one compilation system function, that
is idf_component_register. Similar to the [Link] for most other compo-
nents, blink.c is added to SRCS, and the source files added to SRCS will be compiled.
At the same time, “.”, which represents the path where [Link] is located,
should be added to INCLUDE_DIRS as the search directories for header files. The content
of [Link] is as follows:
1. #Specify v3.5 as the oldest CMake version supported by the current project
2. #Versions lower than v3.5 must be upgraded before compilation continues
3. cmake_minimum_required(VERSION 3.5)
4. #Include the default CMake configuration of the ESP-IDF compilation system

Chapter 4. Setting Up Development Environment 51

5. include($ENV{IDF_PATH}/tools/cmake/[Link])
6. #Create a project named "blink"
7. project(myProject)

Among them, the [Link] in the root directory mainly includes $ENV{IDF_
PATH}/tools/cmake/[Link], which is the main CMake configuration file pro-
vided by ESP-IDF. It is used to configure the default rules of the ESP-IDF compilation system
and define common functions such as idf_component_register; project(blink)
creates a project called blink, and the final firmware will be named [Link].

4.4.2 Compiling the Blink Program

This section takes the Blink program as an example to demonstrate the compilation process
of a simple ESP-IDF program. It is important to note that this section uses the high/low level
of GPIO to drive the LED. However, the WS2812 indicator light requires a special communi-
cation protocol. You can refer to the example program in esp-idf/examples/periphe-
rals/rmt/led strip for more information.

1. Open a new terminal and import the ESP-IDF environment variables

For Linux and Mac systems, use cd ⇠/esp/esp-idf to navigate to the ESP-IDF folder.
Then, import the ESP-IDF environment variables using the command . ./[Link].
This process also performs a complete integrity check of the development environment.

TIPS
Please note that the dot before the space should not be omitted in . ./[Link].
The dot is equivalent to the source directive, which refers to executing the script and
changing the environment variables in the current shell.

Figure 4.16. Automatic addition of environment variables in Windows system

52 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

For Windows systems, you can directly find and open ESP-IDF 4.3 CMD or ESP-IDF 4.3
PowerShell in the program list. After the terminal is opened, the environment variables will
be automatically added, as shown in Figure 4.16.

2. Navigate to the root directory of the blink project

Before compiling the project, navigate to the root directory of the project. To do this, use
the command cd examples/get-started/blink.

3. Set the compilation target to ESP32-C3

Use the command [Link] set-target esp32c3 to set the compilation target to ESP32-
C3, as shown in Figure 4.17. If this step is skipped, the compilation target defaults to ESP32.

Figure 4.17. Set the compilation target to ESP32-C3

4. Configure GPIOs

Use the command [Link] menuconfig to enter the configuration interface. Navigate
using the up/down keys and press Enter key to enter the Example Configuration. Enter
a number to change the GPIO to the specified pin, as shown in Figure 4.18. Save the
configuration by following the prompts.

5. Build the code

Use the command [Link] build to build the code. The code building process is shown
in Figure 4.19. Relevant prompts and flash commands will be printed once the build is
complete, as shown in Figure 4.20.

Chapter 4. Setting Up Development Environment 53

 Figure 4.18. Configure GPIO using menuconfig

Figure 4.19. Code compilation process

Figure 4.20. Prompt after code compilation is complete

54 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

4.4.3 Flashing the Blink Program
For Linux systems, connect the ESP32-C3 to the computer via USB-UART chip (such as
CP2102), and use the command ls /dev/ttyUSB* to view the serial port number. If the
current serial port number printed is /dev/ttyUSB0, use the command [Link] -p /dev
/ttyUSB0 flash to flash the program onto the ESP32-C3.
For Mac systems, connect the ESP32-C3 to the computer via USB-UART chip (such as
CP2102), and use the command ls /dev/cu.* to view the serial port number. If the cur-
rent serial port number printed is /dev/cu.SLAB_USBtoUART, use the command [Link]
-p /dev/cu.SLAB_USBtoUART flash to flash the program onto the ESP32-C3.
For Windows systems, connect the ESP32-C3 to the computer via USB-UART chip (such as
CP2102), and view the serial port number through the device manager. If the current serial
port number is COM5, use the command [Link] -p COM5 flash to flash the program
onto the ESP32-C3.
After the flashing process is completed, you will see a prompt as shown in Figure 4.21 in the
console. When the following log appears, the code will start executing, and the LED on the
development board will start flashing.
Hard resetting via RTS pin...
Done

Figure 4.21. Prompt in the console after flashing is completed

Chapter 4. Setting Up Development Environment 55

4.4.4 Serial Port Log Analysis of the Blink Program
Once the firmware compilation and download are completed, navigate to the project folder,
and run the command [Link] monitor. This will open a monitor with coloured font. The
monitor will output the serial port log of the target ESP32-C3. The content is divided into
three parts by default: first-level bootloader information, second-level bootloader infor-
mation, and user program output. During the output of log, you can press the Ctrl+]
key combination to exit the log output.
ESP-ROM:esp32c3-api1-20210207
Build:Feb 7 2021
rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd6100,len:0x1798
load:0x403ce000,len:0x8dc
load:0x403d0000,len:0x2984
entry 0x403ce000

First-level bootloader information

By default, the first-level bootloader information is output from UART and cannot be turned
off through configuration in ESP-IDF version 4.3.2. This information includes the ROM code
version information fixed internally in the chip. Different chips in the same series may
have different ROM code versions due to ROM repairs and feature expansions. It also in-
cludes the reason for the chip restart, such as rst:0x1 indicating power-on restart of the
chip, rst:0x3 indicating software-triggered restart, rst:0x4 indicating software excep-
tion restart, etc. You can use this information to assess the status of the chip. Additionally,
it provides details about the chip’s boot mode, such as boot:0xc indicating SPI Flash Boot
mode (normal operation mode, in which the code in flash is loaded and executed), and
boot:0x4 indicating Flash Download mode, in which the content of flash can be erased
and programmed.

Second-level bootloader information

The output of second-level bootloader information can be disabled by setting menuconfig

(Top) → Bootloader config → Bootloader log verbosity to No output.
This information mainly includes the ESP-IDF version, flash operating mode and speed,
system partition and stack allocation, as well as the application name and version.
I (30) boot: ESP-IDF v4.3.2-1-g887e7c0c73-dirty 2nd stage bootloader
I (30) boot: compile time [Link]
I (30) boot: chip revision: 3
I (34) boot.esp32c3: SPI Speed : 80MHz

56 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

I (38) boot.esp32c3: SPI Mode : DIO
I (43) boot.esp32c3: SPI Flash Size : 2MB
I (48) boot: Enabling RNG early entropy source...
I (53) boot: Partition Table:
I (57) boot: ## Label Usage Type ST Offset Length
I (64) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (72) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (79) boot: 2 factory factory app 00 00 00010000 00100000
I (86) boot: End of partition table
I (91) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=06058h (24664) map
I (103) esp_image: segment 1: paddr=00016080 vaddr=3fc89c00 size=01a88h (6792) load
I (109) esp_image: segment 2: paddr=00017b10 vaddr=40380000 size=08508h (34056) load
I (122) esp_image: segment 3: paddr=00020020 vaddr=42000020 size=15c54h (89172) map
I (138) esp_image: segment 4: paddr=00035c7c vaddr=40388508 size=0157ch (5500) load
I (139) esp_image: segment 5: paddr=00037200 vaddr=50000000 size=00010h (16) load
I (147) boot: Loaded app from partition at offset 0x10000
I (150) boot: Disabling RNG early entropy source...
I (166) cpu_start: Pro cpu up.
I (179) cpu_start: Pro cpu start user code
I (179) cpu_start: cpu freq: 160000000
I (179) cpu_start: Application information:
I (182) cpu_start: Project name: blink
I (186) cpu_start: App version: v4.3.2-1-g887e7c0c73-dirty
I (193) cpu_start: Compile time: Jan 26 2022 [Link]
I (199) cpu_start: ELF file SHA256: dadcae8e7bb964ab...
I (205) cpu_start: ESP-IDF: v4.3.2-1-g887e7c0c73-dirty
I (212) heap_init: Initializing. RAM available for dynamic allocation:
I (219) heap_init: At 3FC8C4D0 len 00033B30 (206 KiB): DRAM
I (225) heap_init: At 3FCC0000 len 0001F060 (124 KiB): STACK/DRAM
I (232) heap_init: At 50000010 len 00001FF0 (7 KiB): RTCRAM
I (238) spi_flash: detected chip: generic
I (243) spi_flash: flash io: dio
W (247) spi_flash: Detected size(4096k) larger than the size in the binary image
header(2048k). Using the size in the binary image header.
I (260) sleep: Configure to isolate all GPIO pins in sleep state
I (267) sleep: Enable automatic switching of GPIO sleep configuration
I (274) cpu_start: Starting scheduler.

User program output

The user program output includes all information that is printed using the printf() func-
tion, which is the standard output function in the C language, or the ESP_LOG() func-
tion, which is a custom output function provided by ESP-IDF. It is recommended to use
ESP_LOG() because it allows you to specify the log level for better organisation and filter-
ing of logs.
You can configure which logs above a certain level are output through menuconfig(Top)

Chapter 4. Setting Up Development Environment 57

→ Component config → Log output. This allows you to control the verbosity of the
logs and customise the level of detail that is displayed during runtime.
I (278) gpio: GPIO[5]| InputEn: 0| OutfgputEn: 0| OpenDrain: 0| Pullup: 1|
Pulldown: 0| Intr:0
Turning off the LED
Turning on the LED
Turning off the LED
Turning on the LED

In addition to log output, [Link] monitor can also parse system exceptions and trace
software errors. For example, when the application crashes, the following register dump
and traceback information will be generated:
Guru Meditation Error of type StoreProhibited occurred on core 0. Exception was
unhandled.
Register dump:
PC : 0x400f360d PS : 0x00060330 A0 : 0x800dbf56 A1 : 0x3ffb7e00
A2 : 0x3ffb136c A3 : 0x00000005 A4 : 0x00000000 A5 : 0x00000000
A6 : 0x00000000 A7 : 0x00000080 A8 : 0x00000000 A9 : 0x3ffb7dd0
A10 : 0x00000003 A11 : 0x00060f23 A12 : 0x00060f20 A13 : 0x3ffba6d0
A14 : 0x00000047 A15 : 0x0000000f SAR : 0x00000019 EXCCAUSE : 0x0000001d
EXCVADDR: 0x00000000 LBEG : 0x4000c46c LEND : 0x4000c477 LCOUNT : 0x00000000

Backtrace: 0x400f360d:0x3ffb7e00 0x400dbf56:0x3ffb7e20 0x400dbf5e:0x3ffb7e40

0x400dbf82:0x3ffb7e60 0x400d071d:0x3ffb7e90

Based on the register address, the IDF monitor will query the compiled ELF file and trace
the code call process when the application crashes, outputting the function call information
to the monitor:
Guru Meditation Error of type StoreProhibited occurred on core 0. Exception was
unhandled.
Register dump:
PC : 0x400f360d PS : 0x00060330 A0 : 0x800dbf56 A1 : 0x3ffb7e00
0x400f360d: do_something_to_crash at /home/gus/esp/32/idf/examples/get-started/
hello_world/main/./hello_world_main.c:57
(inlined by) inner_dont_crash at /home/gus/esp/32/idf/examples/get-started/hello
_world/main/./hello_world_main.c:52
A2 : 0x3ffb136c A3 : 0x00000005 A4 : 0x00000000 A5 : 0x00000000
A6 : 0x00000000 A7 : 0x00000080 A8 : 0x00000000 A9 : 0x3ffb7dd0
A10 : 0x00000003 A11 : 0x00060f23 A12 : 0x00060f20 A13 : 0x3ffba6d0
A14 : 0x00000047 A15 : 0x0000000f SAR : 0x00000019 EXCCAUSE : 0x0000001d
EXCVADDR: 0x00000000 LBEG : 0x4000c46c LEND : 0x4000c477 LCOUNT : 0x00000000

Backtrace: 0x400f360d:0x3ffb7e00 0x400dbf56:0x3ffb7e20 0x400dbf5e:0x3ffb7e40

0x400dbf82:0x3ffb7e60 0x400d071d:0x3ffb7e90
0x400f360d: do_something_to_crash at /home/gus/esp/32/idf/examples/get-started/
hello_world/main/./hello_world_main.c:57

58 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(inlined by) inner_dont_crash at /home/gus/esp/32/idf/examples/get-started/hello
_world/main/./hello_world_main.c:52
0x400dbf56: still_dont_crash at /home/gus/esp/32/idf/examples/get-started/hello
_world/main/./hello_world_main.c:47
0x400dbf5e: dont_crash at /home/gus/esp/32/idf/examples/get-started/hello_world/
main/./hello_world_main.c:42
0x400dbf82: app_main at /home/gus/esp/32/idf/examples/get-started/hello_world/
main/./hello_world_main.c:33
0x400d071d: main_task at /home/gus/esp/32/idf/components/esp32/./cpu_start.c:254

The trace information of the monitor shows that the application crashes in the function
do_something_to_crash(), which is called by the function app_main() → dont_
crash() → still_dont_crash() → inner_dont_crash() → do_something
_to_crash(). Based on this, the input/output parameters of each link can be checked to
determine the cause of the crash.

4.5 Summary
In this chapter, we have covered the setup of the official software development environment,
ESP-IDF, for ESP32-C3. We have introduced the code resources and file structure of ESP-IDF
and provided a demonstration of the ESP-IDF project structure, compilation system, and
related development tools using a simple example.
By following the instructions in this chapter, you can start developing with ESP-IDF for
simple projects. However, for more specific and advanced compilation requirements, it is
recommended to refer to both the ESP-IDF official documentation and the CMake official
documentation.

Chapter 4. Setting Up Development Environment 59

 Hardware Design of Smart
Chapter
Light Products based on
5
ESP32-C3

In this chapter, we will first introduce the main components of smart light products and their
application scenarios, and take LED smart lights as an example to demonstrate their major
hardware blocks. Then, we will use ESP32-C3 chips and modules to design a smart light
product capable of dimming, colour changing, and wireless communication. The design
provided in this chapter can also be extended and applied to various LED products such as
light strips, ceiling lights, spotlights, etc.

5.1 Features and Composition of Smart Light Products

Smart light products generally use LEDs as light sources. LEDs are solid-state light sources
and semiconductor light devices, characterised by low power consumption and long lifes-
pan, easy to control, and pollution-free. Compared with traditional lighting products, they
have higher efficiency of light energy conversion. At the same time, smart light products
have wireless connectivity functionality, supporting connection to wireless routers or smart
gateways through Wi-Fi, Bluetooth LE, or ZigBee, and then connection to the Internet or
cloud servers. You can not only use smartphones, tablets, smart speakers that support voice
control, and smart control panels to adjust their brightness and colour, as well as setting
timers for turning on/off the lights. You can also group multiple lights together and control
their brightness and colour in batch. You can pre-set lighting scenes for different occasions,
such as “theatre mode” for dimming the ambient lighting, “reading mode” for a soft and
eye-friendly brightness, “music mode” for colour changing and light blinking following the
beat of the music, and “sleep mode” for turning off all the lights except the night lamp. The
structure of a smart light system is shown in Figure 5.1.
From the description above, we can see that the main features of smart light products is
to be controlled through wireless connection. Now we will take the colour-changing smart
LED light as an example to explain the main components of smart light products and how
to control them.
Figure 5.2 shows the structure of a smart LED bulb, including an E27 standard lamp holder,

61
 Figure 5.1. Structure of smart light system

a plastic-wrapped aluminium lamp body, a power supply & an LED driver board, a Wi-Fi
module, LED beads & an aluminium substrate, and a highly transparent lampshade. Com-
pared with traditional LED bulbs, a smart LED bulb has an additional Wi-Fi module. So how
does this Wi-Fi module help control the light wirelessly? The following sections will further
elaborate on the functional implementation.

Figure 5.2. Structure of smart LED bulb

Figure 5.3 shows the functional block diagram of a smart LED bulb, which mainly includes
a 220 V AC-DC power supply module, a constant-current LED driver, a 3.3 V output auxil-

62 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

iary power supply, a PWM control and wireless communication module, and LED beads of
various colours.

Figure 5.3. Block diagram of functional units for smart LED bulb

Before detailing each functional unit, let’s first take a glance at how they manage to change
the brightness and colour of the lighting as a whole. The key lies in the LED lamp beads.
LED lamp beads can be dimmed in two ways: analogue dimming and digital dimming. Ana-
logue dimming changes LED light output by simply adjusting the DC current in the circuit;
while digital dimming, also known as PWM dimming, is achieved by varying the conduction
time of forward current through turning on/off LEDs using PWM signals of different pulse
widths. Section 6.3.3 will describe PWM dimming in detail. Here we will briefly introduce
PWM dimming using PWM signals.
When using a controllable constant-current source to drive LED beads, to adjust colour
temperature, you can change the duty cycles of PWM signals on two channels to adjust the
current of warm-white (WW) and cool-white (CW) LED beads; to adjust light colours, you
can change the duty cycles of PWM signals on three channels to adjust the brightness of
corresponding colours so that the smart LED light emits the mixed colour of different lamp
beads.
Knowing the basics of light dimming and colour change, now let’s dig into the functional
units one by one.
220 V AC-DC power supply
The input power of smart LED lights is usually high-voltage AC, and the standard house-
hold AC in China is 220 V. The 220 V AC-DC power module first converts the AC to DC
through a rectifier bridge, and then reduces it to 18 ⇠ 40 V for the constant-current LED

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 63

 drivers. Since the operating voltage of the PWM control and wireless communication
module is 3.3 V, there is another auxiliary power supply to reduce the DC power to 3.3 V.
Constant-current LED driver
To ensure consistency in the emission of multiple LED beads, you can use a series circuit
and drive the LEDs with a constant current source. The brightness of the LEDs can be
adjusted by controlling the constant current source using PWM signals. Constant-current
LED driver 1 is used to drive the LEDs in cool white (CW) and warm white (WW), and the
output power is relatively higher; constant-current LED driver 2 is used to drive red (R) /
green (G) / blue (B) LEDs, mainly for changing colours, and the output power is lower.
LED beads
In smart LED lights, there are usually warm-white, cool-white, red, green, and blue LED
beads, among which more warm-white and cool-white beads are used for lighting, and
less red, green or blue beads for colour adjustment.
PWM control and wireless communication
In smart light products, to realise PWM control and wireless communication functions,
a highly-integrated system-on-a-chip (SoC) is usually used. SoC supports multiple PWM
signal outputs, as well as one or more mainstream wireless communication protocols such
as Wi-Fi, Bluetooth LE, or ZigBee. It can run embedded RTOS, and supports software
application development. With chips of Wi-Fi connectivity, you can connect your product
to the Internet and cloud servers through a Wi-Fi router; with chips of Bluetooth LE or
ZigBee functions, you need to configure a gateway device to connect to an Ethernet or
Wi-Fi router first and then get it connected to the Internet and cloud servers.
The introduction above explains the main components of smart LED lights, as well as the
realisation of dimming and colour change functions. It can be concluded that the biggest
difference between smart light products and ordinary light products lies in the use of PWM
control and wireless communication. The following sections of this chapter will focus on
how to design the minimal hardware system based on the ESP32-C3 chip to realise PWM
dimming, colour change, and wireless communication. The design is also applicable to other
types of smart light products such as spotlights, ceiling lights, lamps, light strips, etc.

5.2 Hardware Design of ESP32-C3 Core System

Through Section 5.1, we can see that the PWM control and wireless communication module
is the core unit of smart light products, which distinguishes them from traditional light
products. Then how should we design this core system to implement the functions of smart
light products? In this section, we’ll use the ESP32-C3 chip to demonstrate its hardware
design.

64 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

ESP32-C3 is a highly-integrated SoC equipped with a 32-bit RISC-V processor, supporting
2.4 GHz Wi-Fi and Bluetooth LE connectivity. The functional block diagram of ESP32-C3 is
shown in Figure 5.4.

Figure 5.4. Block diagram of ESP32-C3 functions

ESP32-C3 has the following features:

• A 32-bit RISC-V single-core processor with a four-stage pipeline which operates at up
to 160 MHz.
• A complete Wi-Fi subsystem which complies with IEEE 802.11b/g/n protocol and
supports Station mode, SoftAP mode, SoftAP + Station mode, and promiscuous mode.
• A Bluetooth LE subsystem which supports Bluetooth 5 and Bluetooth mesh.
• Storage capacities ensured by 400 KB SRAM and 384 KB ROM on the chip, and SPI,
Dual SPI, Quad SPI, and QPI interfaces that allow connection to external flash.
• Reliable security mechanisms ensured by cryptographic hardware accelerators that
support AES-128/256, Hash, RSA, HMAC, digital signature and secure boot, external
memory encryption and decryption, random number generator, and permission control
on accessing internal memory, external memory, and peripherals.
• A rich set of peripheral interfaces which are ideal for various scenarios and complex
applications; 22 programmable GPIOs that can be configured flexibly to support LED
PWM, UART, I2C, SPI, I2S, ADC, TWAI, RMT, and USB Serial/JTAG applications.

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 65

The ESP32-C3 series of chips has several variants, including the version with in-package SPI
flash. ESP8685 is a small package version of ESP32-C3, as shown in Table 5.1.

Table 5.1. ESP32-C3 series

MPN Flash (MB) Temp (°C) Size (mm)

ESP32-C3 — –40 ⇠ 105 QFN32 (5×5)
ESP32-C3-FN4 4 –40 ⇠ 85 QFN32 (5×5)
ESP32-C3-FH4 4 –40 ⇠ 105 QFN32 (5×5)
ESP32-C3-FH4AZ 4 –40 ⇠ 105 QFN32 (5×5)
ESP8685H2 2 –40 ⇠ 105 QFN32 (4×4)
ESP8685H4 4 –40 ⇠ 105 QFN32 (4×4)

NOTE
• For ESP32-C3FH4AZ, ESP8685H2, and ESP8685H4, pins for flash connection are not
bonded.
• Nomenclature of ESP32-C3 series: F stands for in-package flash, H/N indicates the flash
temperature, and AZ is other identification code.

The core circuit for ESP32-C3 requires about 20 resistors, capacitors, and inductors in total,
as well as one crystal and one SPI flash. The high integration of ESP32-C3 makes it suitable
for small-sized applications such as smart light products. Figure 5.5 and Figure 5.6 show
the block diagram and the schematic of ESP32-C3 core circuit.

Figure 5.5. Block diagram of ESP32-C3 core circuit

The following explains in detail the schematics and PCB layout of ESP32-C3.

66 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 5.6. Schematic of ESP32-C3 core circuit

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 67

5.2.1 Power Supply
Pin 11 and pin 17 are the power supply pins for RTC IO and CPU IO respectively, in a
voltage range of 3.0 V ⇠ 3.6 V. We recommend adding a 0.1 µF capacitor close to each
power supply pin. When working as an output power supply pin, VDD SPI (pin 18) mainly
powers external SPI flash. We recommend adding a 1 µF filter capacitor between VDD SPI
and ground. When VDD SPI works as the power supply pin for in-package flash or external
3.3 V flash, the voltage of VDD3P3 CPU should be maintained at 3.0 V or above, to ensure
the flash’s operation.
Pin 2, pin 3, pin 31, and pin 32 are the analogue power supply pins, working at 3.0 V ⇠
3.6 V. Please note that when ESP32-C3 works in transmission (TX) mode, the instantaneous
current will be higher and may cause power rail collapse. Therefore, it is highly recom-
mended to add a 10 µF capacitor to the power trace, which can work in conjunction with
the 0.1 µF capacitor. In addition, an LC filter circuit needs to be added near pin 2 and pin
3 to suppress high-frequency harmonics. The inductor’s rated current is preferably 500 mA
or above. Refer to the core circuit schematic and place the appropriate decoupling capacitor
near each analogue power pin.
For a single power supply, the recommended voltage is 3.3 V, and the recommended output
current is 500 mA or above. We also suggest adding an ESD protection diode at the power
entrance.

5.2.2 Power-on Sequence and System Reset

ESP32-C3 uses a 3.3 V system power supply. The chip should be activated after the power
rails have stabilised. This is achieved by delaying the activation of pin 7 CHIP EN after
the 3.3 V rails have been brought up. Figure 5.7 shows the power-up and reset timing of
ESP32-C3. Details about the parameters are listed in Table 5.2.

Figure 5.7. ESP32-C3 power-up and reset timing

To ensure that the power supply to ESP32-C3 is stable during power-up, it is advised to add
an RC delay circuit at the CHIP EN pin. The recommended setting for the RC delay circuit

68 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Table 5.2. Parameter description of ESP32-C3 power-up and reset timing

Parameter Description Min.

Time between bringing up the VDDA, VDD3P3, VDD3P3 RTC, and

t0 50 µs
VDD3P3 CPU rails, and activating CHIP EN

t1 Duration of CHIP EN signal level < VIL nRST to reset the chip 50 µs

is usually R = 10k⌦ and C = 1µF , while specific parameters should be adjusted based on
the power-up timing of the power supply and the power-up and reset sequence timing of
the chip.
CHIP EN can also be used as the reset pin of ESP32-C3. When CHIP EN is at low level, the
reset voltage (VIL nRST ) should be (–0.3 ⇠ 0.25) × VDD (where VDD is the I/O voltage for
a particular power domain of pins). To avoid reboots caused by external interference, route
the CHIP EN trace as short as possible, and add a pull-up resistor as well as a capacitor to
ground. Note that CHIP EN pin must not be left floating.

5.2.3 SPI Flash

ESP32-C3 supports external flash of up to 16 MB, which is mainly used for storing program
firmware, system parameters, user parameters, user data, etc. The SPI flash is powered
by VDD SPI. We recommend reserving a serial resistor (initially of 0 ⌦) on the SPI line,
to lower the driving current, adjust timing, reduce crosstalk and external interference, etc.
ESP32-C3FH4/FN4 has an in-package 4 MB SPI flash.

5.2.4 Clock Source

Currently, the ESP32-C3 firmware supports 40 MHz crystal. The specific capacitance of C1
and C2 depends on further testing of, and adjustment to, the overall performance of the
whole circuit. Please add a component (i.e., R1 in Figure 5.6) in series on the XTAL P clock
trace to minimise the impact of crystal harmonics on RF performance. The value of this
component (initially of 24 nH) depends on further RF testing. Note that the accuracy of
the selected crystal needs to be ±10 ppm. In actual use, as the temperature of smart light
products rises, the frequency deviation of the crystal will also increase. Therefore, please
ensure that the frequency deviation of the crystal does not exceed 25 ppm, so as not to affect
Wi-Fi communication.
Although ESP32-C3 has integrated an RC oscillator as the RTC clock source, it also sup-
ports an external 32.768 kHz crystal to act as the RTC clock source. Figure 5.8 shows the
schematic of the external 32.768 kHz crystal.

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 69

 Figure 5.8. Schematic of ESP32-C3’s external crystal (RTC)

NOTE
• Requirements for the 32.768 kHz crystal:
- Equivalent series resistance (ESR)  70 k⌦.
- Load capacitance at both ends should be configured according to the crystal’s specifi-
cation.
• The parallel resistor R10 is used for biasing the crystal circuit (5M ⌦ < R10  10M ⌦).
In general, you do not need to populate R10.
• If the RTC source is not required, then pin 4 (XTAL 32K P) and pin 5 (XTAL 32K N) can
be used as normal GPIOs.

5.2.5 RF and Antenna

In your circuit design, please add a ⇡-matching network between the RF port (LNA IN) and
the antenna, for antenna matching purpose. A CLC network is preferred, as shown in Figure
5.9. The parameters of C8, L2, and C9 in the matching network are subject to the actual
antenna and PCB layout.

Figure 5.9. CLC circuit for ESP32-C3 RF matching

The antenna can be selected based on product design and the overall cost. You can choose
PCB onboard antenna, or an external antenna such as rod antenna, FPC antenna, ceramic
antenna, 3D metal antenna, etc. Commonly-used antenna types are shown in Figure 5.10.
Their installation methods and characteristics are provided in Table 5.3.

70 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 5.10. Commonly-used antenna types

Table 5.3. Installation methods and characteristics of commonly-used antenna types

Antenna Type Installation Methods Characteristics

PCB Low cost, medium gain, usually integrated on
PCB onboard
onboard antenna modules
External connection
High cost, high gain, less susceptible to inter-
Rod antenna through I-PEX
ference, good omni-directional performance
connector
Medium cost, medium gain, can be adhered
FPC antenna Adhesive installation to the package, suitable for products with re-
stricted structure
Medium cost, low gain, small size, suitable for
Ceramic antenna PCB mounting
small-sized modules
High cost, high gain, less susceptible to inter-
3D metal antenna PCB mounting
ference, good omni-directional performance

The RF performance can be optimised through antenna matching. After matching, you
can use CMW500, WT-200, IQ View, IQ Xel or other comprehensive RF testers to test RF
performance of the ESP32-C3 core board. RF test includes conducted test and radiatied test.
Conducted test
In conducted tests, use a 50 ⌦ RF cable to connect the RF output port of the ESP32-
C3 core board to the tester’s RF port, and run the RF test software on the PC. Through
the software, you can communicate with the ESP32-C3 core board and the tester, thus
controlling the test. The conducted test set-up is shown in Figure 5.11.
Radiated test
When performing a radiated test, place the tester’s antenna and ESP32-C3 board’s antenna
close to each other in the shield box. It is recommended that the distance between the
two antennas be about 10 cm. Control the test through PC software. The radiated test
set-up is shown in Figure 5.12.

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 71

 Figure 5.11. Conducted test set-up for ESP32-C3 core board

Figure 5.12. Radiated test set-up for ESP32-C3 core board

For Wi-Fi RF performance test, the primary test parameters are target transmit power, EVM,
receiver sensitivity, and frequency error, as shown in Table 5.4.

Table 5.4. Key parameters for Wi-Fi RF test

Target TX Receiver Sen- Frequency

Working Mode and Rate EVM (dB)
Power (dBm) sitivity (dBm) Error (ppm)

IEEE 802.11b, 1 Mbit/s 21.0±2.0 < 24.5 < 98 ±25

IEEE 802.11g, 54 Mbit/s 19.0±2.0 < 27.5 < 76.2 ±20

IEEE 802.11n, MCS7 HT20 18.5±2.0 < 29 < 74.4 ±20

IEEE 802.11n, MCS7 HT40 18.5±2.0 < 28 < 71.2 ±20

Figure 5.13 shows the spectral mask requirements in different working modes.

72 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 5.13. Spectral mask requirements in different working modes

5.2.6 Strapping Pins

ESP32-C3 has three strapping pins: GPIO2, GPIO8, and GPIO9. During the chip’s system
reset, the strapping pins sample their voltage levels and store them into the latch until the
chip is powered down or shut down. Depending on the stored voltage levels, the chip will
enter different boot modes after system reset. The correspondence between the voltage
levels and the boot modes is shown in Table 5.5. After reset, the strapping pins function as
normal pins.

Table 5.5. Voltage level of strapping pins and corresponding boot mode

Strapping Pins Default SPI Boot Download Boot

GPIO2 N/A 1 1
GPIO8 N/A Irrelevant 1
GPIO9 Weak internal pull-up 1 0

5.2.7 GPIO and PWM Controller

ESP32-C3 has 22 GPIO pins which can be assigned various functions by configuring corre-
sponding registers. All GPIOs can be configured with internal pull-up, pull-down, or set to
high impedance. GPIO MUX and GPIO Matrix are used to collectively control the GPIO pin
signals of the chip. By utilising GPIO MUX and GPIO Matrix (as shown in Figure 5.14), it
is possible to configure the peripheral input signals from any GPIO pin, and the peripheral

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 73

output signals can also be connected to any GPIO pin.

Figure 5.14. IO MUX and GPIO matrix

The PWM controller can generate independent PWM signals on six channels, which can be
configured to any GPIO pins through the GPIO Matrix.

5.3 Practice: Building a Smart Light System with ESP32-C3

Section 5.2 introduced how to design the minimum hardware system (core circuit) and com-
munication system for smart light products based on ESP32-C3. This minimum hardware
system includes the main peripheral components and the antenna part which needs to be
matched with the network analyser and RF tester according to the selected antenna type
and the design of RF circuit. Antenna matching may be difficult for users who are new
to RF. So, is there a ready-made minimum hardware system which has been tuned for RF
performance, for users to get started quickly to develop a smart light product?
Yes, there ARE hardware modules based on the ESP32-C3 chip ready for operation. Apart
from the chip, these modules also integrate a crystal oscillator, flash, antenna, RF circuit and
main peripheral components. In addition, the modules have passed certification of SRRC,
CE, FCC, and KCC, and can be directly applied to smart light products. In the following
sections, we will choose one of the ESP32-C3 modules for smart light products design.

5.3.1 Selecting Modules

As shown in Table 5.6, in terms of the type of antenna, ESP32-C3 modules can be divided
into PCB antenna modules and IPEX external antenna modules; in terms of size and pins,
they can be divided into WROOM series and MINI series. Each module has two temper-
ature range versions: –40 ⇠ 85 °C version and –40 ⇠ 105 °C version, suitable for smart
lights of different temperature requirements. For lighting products such as LED bulbs which
characterise high internal temperature, it is recommended to use the –40 ⇠ 105 °C module.
For other lighting products which do not have high internal temperature, the –40 ⇠ 85 °C
module is suitable.

74 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Table 5.6. ESP32-C3 modules

Module Antenna Temp (°C) Size (mm)

ESP32-C3-WROOM-02

–40 ⇠ 85 °C/
PCB antenna 18×20×3.2
–40 ⇠ 105 °C

ESP32-C3-WROOM-02U

–40 ⇠ 85 °C/
IPEX external antenna 18×14.3×3.2
–40 ⇠ 105 °C

ESP32-C3-MINI-1

–40 ⇠ 85 °C/
PCB antenna 13.2×16.6×2.4
–40 ⇠ 105 °C

ESP32-C3-MINI-1U

–40 ⇠ 85 °C/
IPEX external antenna 13.2×12.5×2.4
–40 ⇠ 105 °C

NOTE
You can also select one module from ESP8685-WROOM-01 to ESP8685-WROOM-07 series
for a smaller package. For more information, please visit [Link].

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 75

5.3.2 Configuring GPIOs of PWM Signals
The PWM controller of ESP32-C3 can generate independent PWM signals on six channels,
which can be assigned to any GPIOs through the GPIO matrix. In our design, five channels
of PWM signals are used to control R (red), G (green), B (blue), CW (cool white), and WW
(warm white) signals. In real application, we can use one channel to control the duty cycles
of WW and CW LEDs to adjust the colour temperature, and another channel to control the
total current to adjust the brightness of WW and CW LEDs. The GPIO configuration of each
PWM signal is shown in Table 5.7.

Table 5.7. GPIO configuration for PWM signals

Function GPIO Configuration

R (Red) GPIO3

G (Green) GPIO4

B (Blue) GPIO5

CW (Cool white) GPIO7

WW (Warm white) GPIO10

When selecting GPIOs, make sure that they are not at high level after chip start-up, otherwise
the LED bulb may flicker when powered on. If no suitable GPIO is available, add a 10 k⌦
pull-down resistor to the GPIO to prevent flickering. Any GPIOs on ESP32-C3 can be used
for PWM function, as long as they are configured during initialisation after chip power-up.
Figure 5.15 shows the minimum control system based on the ESP32-C3-WROOM-02 mod-
ule, which is connected to five LEDs of red, green, blue, cool white, and warm white.

5.3.3 Downloading Firmware and Debugging Interface

1. Connect ESP32-C3 to a PC.

The ESP32-C3 chip integrates a USB Serial/JTAG controller which makes external USB-to-
UART bridge or JTAG adapter unnecessary. The USB on ESP32-C3 uses GPIO19 as D+ and
GPIO18 as D–, and can be directly connected to the USB interface on the PC, so as to realise
firmware download, log printing, and JTAG debugging. Figure 5.16 shows that an ESP32-C3
board is connected to a PC through the built-in USB Serial/JTAG controller. You may visit
[Link] for more applications of the USB Serial/JTAG controller.
For some ESP32-C3 development boards, a USB-to-UART bridge has been connected to the
UART0 interface of the chip. Developers only need to connect the USB interface of the PC to

76 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 5.15. Minimum control system based on ESP32-C3-WROOM-02

Figure 5.16. ESP32-C3 and PC connected through USB Serial/JTAG controller

the development board through the bridge, to realise firmware download and log printing,
as shown in Figure 5.17.

Figure 5.17. USB-to-UART bridge connecting ESP32-C3 development board and PC

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 77

As for a finished board, to save its space and cost, we often use a programmer with USB-
to-UART bridge to connect to the UART0 interface on the ESP32-C3 chip, to implement
firmware download and log printing. Figure 5.18 shows that a programmer with USB-to-
UART bridge is used to connect the development board and the PC.

Figure 5.18. Programmer with USB-to-UART bridge connecting ESP32-C3 and PC

2. Download firmware.

The firmware and system parameters of ESP32-C3 are stored in the SPI flash. To flash
firmware into the chip, first put the chip in download boot mode. According to Table 5.5,
GPIO2 and GPIO8 should be at high level, and GPIO9 should be at low level. Reset the chip
to enter download boot mode. Connect ESP32-C3 to the PC using any of the three methods
above to start firmware download.

3. Debug interface.

There are two ways to debug interface: log printing over serial port and JTAG debugging.
Log printing over serial port
ESP32-C3 ROM code and IDF SDK output log messages through UART0 by default. Con-
nect ESP32-C3 and the PC with any of the three methods above to enable logging in the
PC’s terminal.
JTAG debugging
You can directly use the USB JTAG controller integrated in ESP32-C3 for debugging. To
do this, you need to connect the JTAG pins – MTMS/GPIO4, MTDI/GPIO5, MTCK/GPIO6,
and MTDO/GPIO7 – to an external JTAG adapter to implement debugging.

5.3.4 Guidelines for RF Design

When designing a smart light product using a module with PCB onboard antenna, pay atten-
tion to its placement on the base board to minimize the impact of the board on its antenna
performance. The module should be placed as close to the edge of the base board as possi-
ble. It’s best to place the PCB antenna area outside the base board and keep its feed point
closest to the board.

78 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The antenna feed point of ESP32-C3-WROOM-02 is on the right, while that of ESP32-C3-
MINI-1 is on the left. The placement of these two modules is shown in Figure 5.19 and
5.20.

Figure 5.19. ESP32-C3 module on base board - antenna feed point on the right

Figure 5.20. ESP32-C3 module on base board - antenna feed point on the left

NOTE
For feed points on the right (as in Figure 5.19), position 3 and 4 are preferred. For feed
points on the left (as in Figure 5.20), position 1 and 5 are preferred.

If the positions recommended are unavailable, please make sure that the module is not
covered by any metal shell. The PCB antenna area and the area extended by 15 mm should
be kept clear, namely no copper traces, wiring, or component placement. The clearance
area should be as large as possible, as shown in Figure 5.21. In addition, if there is base

Chapter 5. Hardware Design of Smart Light Products based on ESP32-C3 79

board under the antenna area, it is recommended to cut it off to minimize its impact. When
designing an end product, pay attention to the impact of enclosure on the antenna.

Figure 5.21. Clearance area on the base board

5.3.5 Guidelines for Power Supply Design

When powering up the ESP32-C3 module through a single pin, the power supply should be
of 3.3 V with 500 mA or larger current output. Power ripples can significantly affect the RF
TX performance. Generally, the peak value of the ripple should be less than 80 mV when
transmitting IEEE 802.11n MCS7 packets, and less than 120 mV when transmitting at 11
Mbit/s.

5.4 Summary
After reading this chapter, you should have acquired knowledge of the following subjects
and be able to build your own hardware system for a smart light product:
• Components of a smart light system, implementation of smart light functions, and
functional modules of smart LED lights.
• Principles and methods of LED dimming and color changing.
• Implementing PWM control and wireless communication based on ESP32-C3.
• Selecting antenna for wireless communication, and the main parameters and testing
methods of Wi-Fi RF performance.
• Features of ESP32-C3 and its core circuit design.
• Selecting ESP32-C3 module to simplify application design for smart light products.
• Guidelines for designing smart light products based on ESP32-C3.

80 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Chapter
Driver Development
6

In the last chapter, we introduced the functions and hardware components of an IoT product
(the Smart Light). In this chapter, we will move on to its driver development. Among the
four layers of the IoT architecture, the perception & control layer is intended to control
objects, for example to switch on/off lights, or to open/close a curtain. To control different
objects, corresponding hardware drivers are required, such as LED drivers and motor control
drivers. The perception & control layer can be combined with cloud computing, data mining,
fuzzy recognition, and other AI technologies on the upper layer, to analyse and process
massive data and information, smartly control objects, and realise real-time control, precise
management, and scientific decision-making.

6.1 Driver Development Process

To develop a sensor driver, it generally takes three steps: know about the sensor, develop a
sensor driver, and test the driver.
1. Know about the sensor.
By reading the sensor’s datasheet or other means, learn about the characteristics of the
sensor including its type, communication interface (e.g., I2C, SPI), measurement cycle,
working mode, power mode, etc.
2. Develop a sensor driver.
The main purpose of developing a sensor driver is to control the behaviors of the sensor
through the SoC’s peripheral interfaces.
3. Test the driver.
Once the development is done, write test cases to examine whether the driver can read
data and control the peripheral interface successfully.
It takes similar steps to develop a controller driver: know about the controller, develop a
driver, and test the driver.
1. Know about the controller.
By reading the controller’s datasheet, learn about the controller’s working principles, so
as to select a suitable peripheral interface.

81
2. Develop a driver.
Based on the peripheral interface selected before, develop corresponding driver APIs for
other embedded software modules.
3. Test the driver.
Write test cases to examine whether each driver API can be called and operate as expected.

6.2 ESP32-C3 Peripheral Applications

The ESP32-C3 chip has rich peripheral interfaces, as shown in Figure 6.1. In this section, we
will introduce the application scenarios of ESP32-C3 peripheral interfaces in terms of the
perception & control layer.

Figure 6.1. Peripheral applications of ESP32-C3

Human Machine Interface (HMI)

HMI products are digital devices composed of an input unit (e.g., touch screen and but-
tons) to receive commands and a display to show information, thus realising human ma-
chine interaction. According to their application scenarios, LCD displays, monochrome
displays, and OLED displays can be connected through the SPI and I2C interfaces on
ESP32-C3. GPIOs and ADC are used to read physical button inputs from users. Further-
more, capacitive touch pins of ESP32, ESP32-S2, and ESP32-S3 can be used for touch
buttons, matrix buttons, linear sliders, 2D touch panels, and proximity sensing. These
button and display related functions apply to smart door locks and other devices with
screens. The I2S interface can be used to connect external audio codecs for devices with
voice interaction features. The I2C interface can be used to drive digital tube displays or
LED dot matrix displays, which are common for embedded applications. Compared with

82 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 LCD displays, these displays use fewer GPIOs and less internal memory, and are easier to
be implemented. They are more suitable for scenarios with simpler requirements such as
timing, counting, and status display.
Sensors
Simply speaking, sensors refer to devices and components that can convert various phys-
ical, chemical, and biological quantities in nature into measurable electrical signals. In
this case, different types of sensors are needed. Sensors are the nerve endings of IoT
and the core components for human beings to fully perceive nature. It is indispensable
to deploy various sensors at a large scale for IoT development. We may use temperature
and humidity sensors, inertial sensors, light sensors, air pressure sensors, gesture sensors,
etc., depending on application scenarios. They need to be connected through different
peripheral interfaces to function and collect data. As for ESP32-C3, I2C, SPI, and ADC are
the common peripheral interfaces to drive sensors.
For your reference, drivers compatible with different types of sensors are provided in the
espressif/esp-iot-solution repository on our GitHub.
Controllers
Controlling objects is an important function of the perception & control layer. Control
systems can be divided into two categories: the open-loop system and the closed-loop
system. An open-loop control system, with no feedback mechanism, uses actuators to
directly control objects. Its output signals have no influence or effect on other control
actions within the system. But in closed-loop control systems, output is usually measured
by sensors and fed back for comparison with the set point. The deviation between the
actual output and the expected point is then used to automatically generate the next com-
mand. In smart home applications, common controlled objects include lighting, motors,
and switches, which are mostly controlled by SoCs’ digital and analog signals. The LED
PWM, GPIO, and ADC peripheral interfaces of ESP32-C3 can be used to tranmit the above
signals.

6.3 LED Driver Basics

This section will introduce the basic knowledge of LED drivers, including color spaces in
lighting, LED driver types, LED dimming methods, and PWM.

6.3.1 Color Spaces

Cyan, magenta, and yellow (CMY) are the three primary colors for painting. They mix with
each other and generate a set of colors which constitue the CMY color space. We define the
amount of magenta as the x axis, yellow as the y axis, and cyan as the z axis, thus creating
a 3D space where each color has a unique position.

Chapter 6. Driver Development 83

CMY is not the only color space. Computer monitors generally use the RGB (red, green,
blue) color space, in which the amount of red, green, and blue are assigend as x, y, and z
axis. Another color space is HSV, which describes colors in terms of hue (x axis), saturation
(or chroma, y axis) and value (or brightness, z axis). The lighting industry commonly uses
the HSL color space, which generates colors by changing hue, saturation, and lightness.

1. RGB color space

The RGB color space is the one we are most familiar with. As shown in Figure 6.2, this
color space is represented by mixing the three primary colors to reproduce almost any color.
It is the basic, hardware-oriented color space commonly used in image processing, and
is relatively easy to understand. It uses a linear combination of three primary colors to
represent a secondary color. The three primary components are highly correlated, so it is
not visually intuitive when transitioning colors continuously. To adjust the color of an LED,
you need to change the amount of all three primary colors.

Figure 6.2. RGB color space

Images acquired in natural environments are easily affected by natural light, occlusion, and
shadows. That is, they are sensitive to brightness. The amount of three primary colors in
the RGB color space are closely related to brightness. So long as the brightness changes, the
amount of all three colors will change accordingly. There is, however, no intuitive way to re-
flect this change. The human eye is not equally sensitive to the three colors. In monochrome
vision, the human eye is least sensitive to red and most sensitive to blue. Due to this varia-
tion in sensitivity, the RGB color space is considered to have poor uniformity. The way the
human eye perceives color similarities deviates greatly from the Euclidean distance in the
RGB color space. Therefore, it is difficult for human beings to represent a color accurately
by the amount of three primary colors.

84 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2. HSV color space

The HSV color space is widely used in computers, as shown in Figure 6.3. Compared with
the RGB color space, HSV is closer to the human perception of colors. It can intuitively
represent the hue, saturation, and brightness value of colors for comparison.

Figure 6.3. HSV color space

It is easier to track an object of a particular color in the HSV space than in the RGB space,
and thus the HSV color space is often used to segment objects of a specified color. HSV space
defines colors in terms of hue, saturation, and value (brightness).
Usually, the HSV color space is mapped to a cylinder. The cross section of the cylinder can
be regarded as a polar coordinate system, in which the polar angle is interpreted as hue, the
polar axis length interpreted as saturation, and the height of the cylinder axis as value. Hue
is measured in angle and ranges from 0 to 360°, indicating the position of the spectral color.
Figure 6.4 illustrates hue in the HSV color space.

Figure 6.4. HSV color space – hue

In Figure 6.4, all the colors on the wheel are spectrum colors. Calculated counterclockwise
from red, 0 represents red, 120° represents green, and 240° represents blue.
In the RGB color space, one color is determined by three values. For example, yellow is
represented by (255,255,0). In the HSV color space, yellow is represented by only one
value, i.e., Hue=60.

Chapter 6. Driver Development 85

Figure 6.5 is the semi horizontal cross-section of the cylinder (Hue=60) and illustrates
saturation and value in the HSV color space.

Figure 6.5. HSV color space – saturation and value

In Figure 6.5, the horizontal axis represents saturation, which indicates the deviation from
the spectrum colors. It ranges from 0% to 100%, where 0 represents pure white. The higher
the saturation, the darker the color, the closer to the spectrum color, and vice versa.
The vertical axis represents value, which indicates the brightness of the color in the HSV
color space. Value ranges from 0% to 100%, where 0 represents plain black. The higher the
value, the brighter the color.

3. HSL color space

The HSL color space is similar to the HSV color space. It also has three components: hue,
saturation, and lightness. The difference lies in the last component. Lightness in HSL rep-
resents luminance. A lightness of 100 means white, whereas a lightness of 0 means black.
Value in HSV represents brightness. A value of 100 equals spectrum color, whereas a value
of 0 equals black. Figure 6.6 shows the HSL color space.

Figure 6.6. HSL color space

Figure 6.7 shows hue in the HSL color space, which represents the range of colors the human
eye can perceive. They are distributed on a flat color wheel; each represented by a hue of 0

86 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

to 360°. The significance of hue is that we can change the color by rotating the color wheel
without changing saturation or lightness.

Figure 6.7. HSL color space – hue

Figure 6.8 shows saturation in the HSL color space, ranging from 0% to 100%. It describes
the changes of color purity under the same hue and lightness. The larger the saturation, the
brighter and less gray of the color.

Figure 6.8. HSL color space – saturation

Figure 6.9 shows lightness in the HSL color space, which represents the luminance of a color.
It ranges from 0% to 100%. The smaller the value, the darker the color, and the closer to
black, and vice versa.

Figure 6.9. HSL color space – lightness

The three color spaces introduced above merely describe colors from different dimensions,
and thus can be mutually converted. In practice, the LED lights uses RGB color space as the
brightness of red, green, and blue beads are adjusted to generate various colors. However,
the user interface and control commands usually use the HSV or HSL color space. There-
fore, the LED driver needs to convert values from the HSV or HSL dimension to the RGB
dimension, so as to get the expected LED color.

Chapter 6. Driver Development 87

6.3.2 LED Driver
Compared with traditional light sources, LED is more energy-efficient and eco-friendly with
longer lifespan. It is a low-voltage, high-current semiconductor component. Its luminous
intensity is positively associated with the forward current. When selecting an LED driver,
we need to consider the working environment. If the driver is sensitive to the ambient tem-
perature, we should use components that generate less heat, or dissipate heat. LED driver
is a core component of smart lights and will directly affect the lifespan and use experience
of smart lights. At present, there are mainly two types of LED drivers.
Constant-voltage drivers
Constant-voltage drivers provide stable terminal voltage for the LED, and the current
changes with the load. When driven by a constant-voltage driver, each LED bead needs a
suitable resistor to emit light of the same brightness.
Constant-current drivers
Constant-current drivers stablize the current flowing through the LED, and the voltage
across the LED changes with the load. When driven by a constant-current driver, the LED
can be dimmed by controlling the current flowing through it.

6.3.3 LED Dimming

Dimming is a basic feature of smart LED lights including changing the color, brightness, and
on/off status. Users can adjust LED lights through a smartphone app, a remote controller,
etc. There are three LED dimming methods.
TRIAC dimming
When using TRIAC dimming, the waveform of the input voltage changes with the con-
duction angle of the TRIAC, thereby changing the effective value of the input voltage and
eventually dimming the LED light. TRIAC dimming is suitable for traditional lamps such
as incandescent lamps and fluorescent lamps.
PWM dimming
Basically, PWM switches on/off LED lights and dims lights by sending PWM signals and
changing their frequency and duty cycle.
I2C dimming
The constant-current LED linear controller ICs with I2C interfaces are suitable for driving
low-power LED lights. Such ICs receive control signals through I2C input interfaces and
adjust the current of multiple independent output interfaces to dim the LED.
Among the three LED dimming methods above, PWM dimming performs the best. It guar-
antees no color shifts and stability at low brightness and is therefore widely used.
Figure 6.10 is the block diagram of PWM dimming, which mainly includes on/off signal

88 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

sampling circuit, the main control circuit and the PWM controller. The on/off signal sam-
pling circuit generates a clock signal after detecting the on/off signal in the circuit. The
main control circuit receives the clock signal and generates three pulse signals, which are
respectively output to three PWM controllers. The PWM controllers output different current
signals based on the pulse signals to adjust the brightness of corresponding LED bead. The
main control circuit usually includes a microcontroller unit, whose input is connected to the
on/off signal sampling circuit, and three outputs are respectively connected to three PWM
controllers. The outputs of PWM controllers are connected to the red, green, and blue LED
beads respectively, thereby controlling their brightness to get the expected color. All LED
beads are packaged in one lampshade.

Figure 6.10. Block diagram of PWM dimming

6.3.4 Introduction to PWM

Pulse width modulation (PWM) is a technique that converts analogue signals into pulse
signals (a means of controlling analogue output with digital signals). It can be used to
control the brightness of LEDs, the speed of DC motors, etc.
It has three main parameters: frequency, period, and duty cycle. PWM frequency is the
number of times the PWM signal goes from high level to low level and back to high level
within one second. It is measured in Hz. PWM period is the reciprocal of PWM frequency.
PWM duty cycle refers to the ratio of the high-level time to one PWM period, ranging from
0% to 100%. Figure 6.11 shows the PWM duty cycle.
For example, if the PWM period is 10 ms and the pulse width time is 8 ms, then the PWM
duty cycle is 8/10=80%.

Chapter 6. Driver Development 89

 Figure 6.11. PWM duty cycle

When using PWM to control an LED, if the light is turned on for 1 second and then off for
1 second repeatedly (i.e., period = 2s, duty cycle = 50%), the LED will appear to blink. If
this cycle is shortened to 200 ms, with the LED being on for 100 ms and then off for 100
ms, the LED will appear to blink at a higher frequency. Due to the persistence of vision,
as the cycle continues decreasing, there will be a critical threshold where the human eye
cannot perceive the blinking of the LED. At this point, the persistence of vision blends the on
and off images, resulting in a stable average brightness. This average brightness is directly
related to the PWM duty cycle, as shown in Figure 6.12. Therefore, we can dim LED lights
by adjusting the PWM duty cycle.

Figure 6.12. Relationship between PWM duty cycle and average brightness

6.4 LED Dimming Driver Development

After understanding the basics of LED drivers, we can start developing a dimming driver
based on the ESP32-C3 chip. This mainly includes the development of functional APIs for
controlling the switch, brightness, color, and color temperature. In daily life, it is usually
expected to maintain the color, brightness, and color temperature of a light consistent with
its previous status when turning it on. This requires preserving the light’s status when it is
turned off. To achieve this, we can use the non-volatile storage (NVS) feature provided by
ESP-IDF.
So before writing the driver code, it is also necessary to learn about the LED PWM controller
of ESP32-C3, its programming procedures, and non-volatile storage.

90 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

6.4.1 Non-Volatile Storage (NVS)
The non-volatile storage in ESP-IDF uses a portion of the main flash memory through esp_
partition.h APIs to store key-value pairs. Since NVS is permanent, even if the device is
restarted or powered off, the stored data will not be lost. NVS has been specially designed to
prevent data corruption caused by power failure, and to distribute the written data through-
out NVS in case of flash wear and tear. The dedicated partition in flash used by NVS stores
data of various types, such as integers, NULL-terminated strings, and binary data.
NVS is suitable for storing small data, rather than large data such as strings or binary large
objects (BLOBs) which should be handled by the FAT file system based on wear leveling. In
IoT projects, NVS can store not only the unique mass production data for products, but also
any user data related to the application.
Following are several key concepts of NVS: key-value pairs, namespaces, security, tamper
resistance, and robustness.
Key-value pairs
NVS operates on key-value pairs, as in “key:value”. Keys are ASCII strings of up to 15
characters, while values can be any of the following types:
• Integers: uint8_t, int8_t, uint16_t, int16_t, uint32_t, int32_t,
uint64_t, and int64_t.
• Strings ending with “0”.
• Variable-length binary data.
Namespaces
To mitigate potential conflicts in key names between different components, NVS assigns
a namespace to each key-value pair, which follows the same naming rule as keys, i.e.,
the maximum length is 15 characters. These names are specified in the nvs_open()
or nvs_open_from_part() call. This call returns an opaque handle, which is used in
subsequent calls to nvs_get_*(), nvs_set_*(), and nvs_commit() functions. In
this way, a handle is associated with each namespace, and key names will not collide with
the same names in other namespaces. Please note that the namespaces with the same
name in different NVS partitions are considered as separate namespaces.
Security, tamper resistance, and robustness
After NVS encryption, data will be stored in encrypted form. If NVS encryption is not
enabled, any user with physical access to the flash can modify, erase, or add key-value
pairs. If NVS encryption is enabled, key-value pairs cannot be modified or added without
knowing the corresponding NVS encryption key. However, there is no tamper-resistance
against the erase operation.
When the flash runs into an inconsistent state, NVS will try recovering. Powering off a

Chapter 6. Driver Development 91

 device at any time and then powering it back on will not cause data loss. However, if the
device is powered off while writing a new key-value pair, that specific pair may be lost.

6.4.2 LED PWM Controller (LEDC)

The LED PWM controller of ESP32-C3 can generate six independent digital waveforms, with
the following features:
• Six independent PWM generators (i.e., six channels)
• Four independent timers that support division by fractions
• Automatic duty cycle fading (i.e., gradual increase/decrease of a PWM’s duty cycle
without interference from ESP32-C3) with interrupt generation on fade completion
• Adjustable phase of PWM signal output
• PWM signal in Light-sleep mode (see details of low-power modes in Chapter 12)
• Maximum PWM resolution: 14 bits
The four timers are identical regarding their features and operation. The following sections
refer to the timers collectively as Timerx (where x ranges from 0 to 3). Likewise, the six PWM
generators are also identical in features and operation, and thus are collectively referred to
as PWMn (where n ranges from 0 to 5). Figure 6.13 shows the LED PWM timer.

Figure 6.13. LED PWM timer

The four timers can be independently configured (i.e., configurable clock divider, and counter
overflow value) and each internally maintains a timebase counter (i.e., a counter that counts
on cycles of a reference clock). Each PWM generator selects one of the four timers, uses the
timer’s counter value as a reference to generate PWM signals, and outputs the signals to the
timer.
Figure 6.14 shows the main functional blocks of the timer and the PWM generator.

92 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 6.14. Functional blocks of LED PWM timer and generator

To generate PWM signals, a PWM generator (PWMn) needs to select one of the four timers
(Timerx) and use its counter value as a reference to generate signals. Each PWM gener-
ator has a comparator and two multiplexers. It compares the timer’s 14-bit counter value
(timerx cnt) to two trigger values of the comparator hpointn and lpointn. When timerx cnt
equals hpointn or lpointn, high- or low-level PWM signal will be generated respectively.
Figure 6.15 shows how hpointn and lpointn are used to generate PWM signals with a fixed
duty cycle.

Figure 6.15. Generating PWM signals with a fixed duty cycle using hpointn and lpointn

PWM generators can fade the duty cycle of a PWM output signal. When duty cycle fading
is enabled, the value of lpointn will be incremented/decremented every time the counter
overflows a certain number of times. Figure 6.16 demonstrates the process of duty cycle
fading.

Chapter 6. Driver Development 93

 Figure 6.16. Duty cycle fading

6.4.3 LED PWM Programming

Having learned about the LEDC of ESP32-C3, now we need to configure the controller using
LED PWM APIs provided by ESP-IDF. The configuration includes three steps, as shown in
Figure 6.17.
1. Configure the timer, specifying the frequency and duty resolution of PWM signals.
2. Configure the channel, mapping the timer to the GPIOs that output PWM signals.
3. Output PWM signals to drive the LED. The brightness of the LED can be changed
through software control or the hardware’s duty cycle fading function.

Figure 6.17. Steps of configuring PWM controller

1. Configuring the timer

Timers can be configured by calling ledc_timer_config(), when an ledc_timer_

config_t structure with the following parameters needs to be passed to the function:
• Speed mode (the value of this parameter must be LEDC_LOW_SPEED_MODE);
• Timer index timer_num;
• PWM frequency;
• PWM duty resolution.

94 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

PWM frequency is inversely proportional to duty resolution, as higher frequency results in
fewer available duty cycles for a given period and vice versa. This interrelationship may be
more important if the API is used for purposes other than changing the brightness of LEDs.

2. Configuring the channel

After configuring the timer, you also need to configure the required channel (one of ledc_
channel_t) by calling ledc_channel_config(). An ledc_channel_config_t struc-
ture with channel configuration parameters needs to be passed to the function.
Then the channel will start operating according to the ledc_channel_config_t struc-
ture and generate PWM signals on the selected GPIOs with the frequency specified in step 1
and the duty cycle specified in step 2. This process can be suspended at any time by calling
the ledc_stop() function.

3. Changing PWM signals

Once the channel starts operating and generating the PWM signal with a constant duty
cycle and frequency, there are a couple of ways to change this signal. For LED dimming, we
primarily change the duty cycle to vary the light color and brightness.
Changing PWM duty cycle using software
To set the duty cycle, use the dedicated function ledc_set_duty(). After that, call
ledc_update_duty() to activate the changes. To check the currently set value, use the
function ledc_get_duty().
Another way to set the duty cycle, as well as some other channel parameters, is by calling
ledc_channel_config().
The PWM duty cycle passed to the function depends on duty_resolution, and the
value ranges from 0 to 2duty_resolution –1. For example, if duty_resolution is
10, then the duty cycle values can range from 0 to 1023.
Changing PWM duty cycle using hardware
LEDCs provide the means to gradually change (fade) the duty cycle. To use this function-
ality, enable fading with ledc_fade_func_install() and then configure it by calling
one of the following functions.
1. esp_err_t ledc_set_fade_with_time(ledc_mode_t speed_mode,
2. ledc_channel_t channel,
3. uint32_t target_duty,
4. int max_fade_time_ms);
5.
6. esp_err_t ledc_set_fade_with_step(ledc_mode_t speed_mode,
7. ledc_channel_t channel,

Chapter 6. Driver Development 95

 8. uint32_t target_duty,
9. uint32_t scale,
10. uint32_t cycle_num);
11.
12. esp_err_t ledc_set_fade(ledc_mode_t speed_mode,
13. ledc_channel_t channel,
14. uint32_t duty,
15. ledc_duty_direction_t fade_direction,
16. uint32_t step_num,
17. uint32_t duty_cycle_num,
18. uint32_t duty_scale);

Finally, call ledc_fade_start() to initiate fading. If not required anymore, the fading
can be disabled with ledc_fade_func_uninstall().

4. Range of PWM frequency and duty resolution

The LED PWM controller is mainly used for driving LED dimming. It provides a large flex-
ibility of PWM duty cycle settings. For instance, the PWM frequency of 5 kHz can have
the maximum duty resolution of 13 bits. This means that the duty can be set anywhere
from 0% to 100% with a resolution of ⇠0.012% (213 = 8192 discrete levels of LED bright-
ness). Please note that these parameters depend on the clock signal clocking the LED PWM
controller timer which in turn clocks the channel.
The LEDC can be used for generating signals with higher frequencies that are sufficient to
clock other devices such as digital camera modules. In this case, the maximum frequency
can be 40 MHz with duty resolution of 1 bit. This means that the duty cycle is fixed at 50%
and cannot be adjusted.
The LEDC API will report an error when the configured frequency and duty resolution exceed
the range of LEDC’s hardware. For example, an attempt to set the frequency to 20 MHz and
the duty resolution to 3 bits will result in the following error reported on a serial monitor:
[E (196) ledc: requested frequency and duty resolution cannot be achieved, try
reducing freq_hz or duty_resolution. div_param=128]

In such a situation, either the duty resolution or the frequency must be reduced. For exam-
ple, setting the duty resolution to 2 bits can solve this problem and will make it possible to
set the duty cycle at 25% steps, i.e., at 25%, 50%, or 75%.
The LEDC driver will also capture and report attempts to configure frequency / duty resolu-
tion combinations that are below the supported minimum, e.g.:
[[E (196) ledc: requested frequency and duty resolution cannot be achieved, try
increasing freq_hz or duty_resolution. div_param=128000000]]

The duty resolution is normally set by ledc_timer_bit_t, with a range of 10 to 15 bits.

For smaller duty resolutions (from 10 down to 1), just enter the equivalent numeric directly.

96 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

6.5 Practice: Adding Drivers to Smart Light Project
There are two drivers to be developed in a smart light project – the button driver and the
LED dimming driver. With these two drivers, we can use a button to control LEDs.

6.5.1 Button Driver

When using ESP32-C3-DevKitM-1 to simulate smart light for development, we can find two
buttons on the board, namely the Boot button and the RST button. The RST button is used
for resetting and restarting ESP32-C3, while the Boot button functions as a regular button
once the firmware starts operating. In other words, the Boot button can be used to simulate
a light switch.
For this purpose, we introduce the button component as the button driver. You may read
the source code to learn about its development, as we will not expound on it in this book.
To add the driver to the smart light project, please follow the steps below.

1. Adding driver-related source files

Create a folder named components under the directory of the smart light project and put
the components used by the project into the folder.
Create a subfolder named button under components.
Then, create the source files and header files for the button driver in button, and edit the
code accordingly.
Specific code can be found from book-esp32c3-iot-projects/device firmware/
components/button.
In the main folder of the project, create a source file named app_driver.c to process all
the drivers. Meanwhile, create header files in the include folder under main. Add driver
operations and function declarations to corresponding files, such as driver initialization and
button event processing. The key code is as follows.
1. //Callback function for pressing the button
2. static void push_btn_cb(void *arg)
3. {
4. //Code Omitted
5. }
6. void app_driver_init()
7. {
8. //Initializing button driver
9. button_config_t btn_cfg = {
10. .type = BUTTON_TYPE_GPIO,
11. .gpio_button_config = {
12. .gpio_num = LIGHT_BUTTON_GPIO,

Chapter 6. Driver Development 97

13. .active_level = LIGHT_BUTTON_ACTIVE_LEVEL,
14. },
15. };
16. button_handle_t btn_handle = iot_button_create(&btn_cfg);
17. if (btn_handle) {
18. iot_button_register_cb(btn_handle, BUTTON_PRESS_UP, push_btn_cb);
19. }
20. //Code Omitted
21. }

2. Adding source files to the compiling system

First, edit the [Link] file under the project directory. Append the path of the
components in components to the search path with the following code:
1. //Code Omitted
2. set(EXTRA_COMPONENT_DIRS ${CMAKE_CURRENT_LIST_DIR}/../components)
3. //Code Omitted

Then, edit the [Link] file in the main folder. Add the source file app_driver.c
to the compiling system with the following code:
1. set(srcs "app_main.c"
2. "app_driver.c")
3. set(include_dirs "include")
4. idf_component_register(SRCS "${srcs}"
5. INCLUDE_DIRS "${include_dirs}")

Besides, the button component also has a [Link] file, which is used to add the
button driver source code to the compiling system. You may refer to book-esp32c3-iot
-projects/device firmware/components/button/[Link] for details.
Other components to be added have similar structures and will not be explained again in
subsequent chapters.

6.5.2 LED Dimming Driver

The LED lights used in our project have five color options: red, green, blue, warm (WW),
and cold (CW), so we need five PWM channels to control them. The target functions in-
clude turning on/off LED lights, and controlling their color, color temperature, brightness,
breathing, and fading. However, since we are using the ESP32-C3-DevKitM-1 in practice,
which only has R, G, and B channels, we can only change LEDs’ colors but not their color
temperature during development.
According to the requirements of the smart light project, the LED dimming driver is encap-
sulated and provided in the form of a light_driver component, which can be found at
book-esp32c3-iot-projects/device firmware/components/light driver.
Besides, to save the status of LED lights, the project also introduces an app_storage com-

98 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

ponent. Its underlayer adopts NVS, which stores key-value pairs in the main flash partition
by calling APIs in esp_partition.h. The component can be found at book-esp32c3-
iot-projects/device firmware/components/app storage.

1. light driver component

According to the requirements of this project, the light_driver component implements

functions such as initializing/deinitializing the LED dimming driver, turning on/off lights,
controlling their color, brightness, color temperature, etc. Table 6.1 lists the APIs provided
for the main application by the LED dimming driver in the light_driver component.

Table 6.1. APIs provided by the LED dimming driver in light driver

API Function
light_driver_init() Initialize light_driver

light_driver_deinit() Deinitialize light_driver

light_driver_config() Configure fade time and blink cycle of light_driver

light_driver_set_switch() Turn on/off LED lights

light_driver_get_switch() Get the on/off status of LED lights

light_driver_set_hue() Set the Hue

light_driver_get_hue() Get the Hue

light_driver_set_saturation() Set the Saturation

light_driver_get_saturation() Get the Saturation

light_driver_set_value() Set the Value (as in HSV)

light_driver_get_value() Get the Value (as in HSV)

light_driver_set_hsv() Set the three HSV components in one call

light_driver_get_hsv() Get the three HSV components in one call

light_driver_set_lightness() Set the Lightness (as in HSL)

light_driver_get_lightness() Get the Lightness (as in HSL)

light_driver_set_hsl() Set the three HSL components in one call

light_driver_get_hsl() Get the three HSL components in one call

light_driver_set_color_temperature() Set the color temperature

light_driver_get_color_temperature() Get the color temperature

light_driver_set_brightness() Set the brightness

Chapter 6. Driver Development 99

 Continuation of Table 6.1
API Function
light_driver_get_brightness() Get the brightness

light_driver_set_ctb() Set color temperature and brightness in one call

light_driver_get_ctb() Get color temperature and brightness in one call

light_driver_set_rgb() Set the three RGB components in one call

light_driver_breath_start() Set the color of LED breathing and start breathing

light_driver_breath_stop() Stop breathing

light_driver_blink_start() Set the colour of LED blinking and start blinking

light_driver_blink_stop() Stop blinking

2. app storage component

The app_storage component uses non-volatile storage (NVS) at its underlying layer. Table
6.2 shows the APIs provided for the main application by the app_storage component.

Table 6.2. APIs provided by the app storage component

API Function
app_storage_init() Initialize app_storage

app_storage_set() Store data in key-value pair format

app_storage_get() Get key-value pairs

app_storage_erase() Erase a specific key-value pair

3. Saving LED status

When the LED lights are powered up, we may expect them to be set to the color and bright-
ness of last use. To achieve this function, we need to save the status of the lights after each
control and load the latest status when the driver is initialized. This status saving function is
implemented in the light_driver component. Every time an API in the light_driver
component is called to modify the LEDs’ status, the new status will be saved, which will be
loaded when the initialization API is called.
1. //Save LED status
2. if (app_storage_get(LIGHT_STATUS_STORE_KEY, &g_light_status,
3. sizeof(light_ status_t)) ! = ESP_OK) {
4. //Code Omitted
5. }
6. //Load LED status

100 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

7. if (app_storage_set(LIGHT_STATUS_STORE_KEY, &g_light_status,
8. sizeof(light_ status_t)) ! = ESP_OK) {
9. //Code Omitted
10. }

4. Initializing the driver

When adding the LED dimming driver to the smart light project, you need to write the code
to initialize the driver in the app_main() function. To use such code, a light_driver_
config_t parameter should be provided, which specifies the ESP32-C3 GPIOs used by the
five PWM channels, the fade time, the breathing cycles, the PWM frequency, the clock source
of the LEDC, the PWM duty resolution, etc. The driver initialization code in app_main()
is defined as function app_driver_init(), and called as follows:
1. void app_driver_init()
2. {
3. //Code Omitted
4. //Initialize LED dimming driver
5. light_driver_config_t driver_config = {
6. .gpio_red= LIGHT_GPIO_RED,
7. .gpio_green = LIGHT_GPIO_GREEN,
8. .gpio_blue = LIGHT_GPIO_BLUE,
9. .gpio_cold = LIGHT_GPIO_COLD,
10. .gpio_warm = LIGHT_GPIO_WARM,
11. .fade_period_ms = LIGHT_FADE_PERIOD_MS,
12. .blink_period_ms = LIGHT_BLINK_PERIOD_MS,
13. .freq_hz = LIGHT_FREQ_HZ,
14. .clk_cfg = LEDC_USE_APB_CLK,
15. .duty_resolution = LEDC_TIMER_11_BIT,
16. };
17. ESP_ERROR_CHECK(light_driver_init(&driver_config));
18. //Code Omitted
19. }

5. Controlling LED status

After initializing the LED dimming driver, we can use the APIs provided by the light_
driver component to control the LED lights. Combined with the button driver, the LED
lights can be turned on/off through a button. Here, we will focus on controlling the on/off
status, the color, and the color temperature.
(1) On/off status
The following API can be used to control the on/off status of the LED lights.
1. //Turn on the light
2. light_driver_set_switch(true);

Chapter 6. Driver Development 101

3. //Turn off the light
4. light_driver_set_switch(false);

(2) Color
After initializing the LED dimming driver and turning on the lights, the color of the LEDs
can be controlled based on RGB, HSL, or HSV color spaces. When using LED dimming APIs,
pay attention to the parameter value range of these APIs:
• HSV: Hue360, Saturation100, Value100.
• HSL: Hue360, Saturation100, Lightness100.
• RGB: Red255, Green255, Blue255.
The following APIs are used for adjusting all three components of RGB, HSL, or HSV color
space in one call. You can also adjust only one component using APIs listed in Table 6.1.
1. //RGB color control
2. light_driver_set_rgb(uint8_t red, uint8_t green, uint8_t blue);
3. //HSL color control
4. light_driver_set_hsl(uint16_t hue, uint8_t saturation, uint8_t lightness);
5. //HSV color control
6. light_driver_set_hsv(uint16_t hue, uint8_t saturation, uint8_t value);

(3) Color temperature

In addition to controlling the on/off status and color, light_driver APIs can also be used
to control the color temperature (but not available for simulated lights). The following API
can be used to change the color temperature and brightness:
1. light_driver_set_ctb(uint8_t color_temperature, uint8_t brightness);

Source code
Please refer to book-esp32c3-iot-projects/device firmware/2 light dri-
vers for the complete code to add the LED dimming driver and the button driver to the
smart light project . You can also check the running results after compiling the code and
flashing it onto the development board.

6.6 Summary
ESP32-C3 has a rich set of peripheral interfaces, which can be used in different application
scenarios, such as screen display, sensor data collection, audio playback, etc. For smart light
applications, LED PWM controllers are usually used to implement LED dimming drivers.
After learning the driver development steps in this chapter, you should be able to program
on your own to control LED lights. In subsequent chapters, we will introduce commonly
used wireless communication technologies and protocols in IoT, and how to apply them in
the smart light project.

102 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Chapter Wi-Fi Configuration and
7 Connection

In this chapter, we’ll focus on the specifications of Wi-Fi network configuration and connec-
tion, from the basics of Wi-Fi and Bluetooth to the common methods for configuring Wi-Fi
network. Then, examples will be given to help you better understand its operating mecha-
nism, and ways of smart Wi-Fi configuration. Finally, we’ll lead you to a practice of smart
Wi-Fi configuration with the Smart Light project based on ESP32-C3.
Since the wireless communication technology has gone through a long history, many docu-
ments have delved deeply into this topic. Therefore, this chapter will only provide a brief
introduction. For details, please see the references listed at the end of this book.

7.1 Basics of Wi-Fi

This section will walk you through the Wi-Fi technology from the following aspects:
• What is Wi-Fi?
• How does Wi-Fi evolve?
• What do the Wi-Fi concepts mean?
• How does Wi-Fi connection work?

7.1.1 Introduction to Wi-Fi

Wi-Fi is a trademark of wireless communication technology owned by the Wi-Fi Alliance
(WFA), which supervises Wi-Fi certification. It is a family of wireless network protocols
based on the IEEE 802.11 standards. Products passing rigorous testings will be given the
Wi-Fi CERTIFIED™ seal to prove that they have met industry-agreed standards for interop-
erability, security, and a range of application specific protocols.
Compared with other wireless communication technologies, Wi-Fi boasts wider coverage,
better penetrating ability through walls, and higher throughput.

7.1.2 Evolution of IEEE 802.11

As Wi-Fi users, we should have heard about IEEE 802.11 more or less. But what does it
exactly mean?

104
IEEE is the abbreviation of the Institute of Electrical and Electronics Engineers. 802 is a
committee in the institute for networking standards, also known as the LMSC – LAN/MAN
Standards Committee. The committee covers such a big family of standards that it needs
to be divided into groups devoted to specific areas. Each group has its own number (the
one following “802”, separated by a dot), so 802.11 refers to the 11th group of committee
802, which develops the Medium Access Control (MAC) protocols and Physical Layer (PHY)
specifications of wireless local area networks (WLANs). IEEE 802.11 has experienced several
“amendments”, as shown in Table 7.1.

Table 7.1. Generations of IEEE 802.11

Frequency Channel Bandwidth Rate Modulation

Version Alias
Range (GHz) (MHz) (Mbit/s) Method

802.11a 5 20 54 OFDM2 —

802.11b 2.4 22 11 CCK3 /DSSS4 —

802.11g 2.4 20 54 OFDM —

72-600
802.11n 2.4, 5 20, 40 OFDM Wi-Fi 4
(MIMO1 :4×4)

433-1733
802.11ac 5 20, 40, 80, 80+80, 160 OFDM Wi-Fi 5
(MIMO:4×4)

600-2401
802.11ax 2.4, 5 20, 40, 80, 80+80, 160 OFDMA5 Wi-Fi 6
(MIMO:4×4)

Table notes:
1
MIMO: Multiple Input Multiple Output.
2
OFDM: Orthogonal Frequency Division Multiplexing.
3
CCK: Complementary Code Keying.
4
DSSS: Direct Sequence Spread Spectrum.
5
OFDMA: Orthogonal Frequency Division Multiple Access.

7.1.3 Wi-Fi Concepts

In this section, we’ll introduce the network technologies related to IEEE 802.11, including
the Open System Interconnection Reference Model (OSI/RM) and physical components of
IEEE 802.11.

Chapter 7. Wi-Fi Configuration and Connection 105

Open System Interconnection Reference Model (OSI/RM)
In OSI/RM, the computer network system is conceived as a seven-layer framework. Their
names and relationships are shown in Figure 7.1.

Figure 7.1. Architecture of OSI/RM

Physical components of IEEE 802.11

IEEE 802.11 architecture consists of four major physical components, as shown in Figure
7.2.

Figure 7.2. Physical components of IEEE 802.11

Wireless Medium (WM)

WMs refer to the physical layer where wireless MAC frame data is transmitted. Ini-
tially, two radio frequency (RF) physical layers and one infrared physical layer were
introduced, but the RF layers turned out to be more popular.

106 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Stations (STA)
Stations comprise all devices and equipments that are connected to the wireless LAN.
Battery-operated laptops and handheld computers are typical STAs, but “portable” is not
a must. In some cases, desktops are also connected to wireless LANs to avoid pulling
new cables.
Access Points (AP)
As a branch of STA, AP provides access to distribution services for associated STAs.
Distribution System (DS)
When several APs are connected to cover a larger area, they have to communicate with
each other to track the movements of mobile stations. This is conducted through the
distribution system, an external data network. It is responsible for transmitting data
frames to their destinations.
Building wireless networks
The physical components above constitute a wireless network. The basic building block
of an IEEE 802.11 network is the Basic Service Set (BSS). It comes in two categories, the
Independent BSS and the Infrastructure BSS, as shown in Figure 7.3.

Figure 7.3. Independent BSS and infrastructure BSS

Independent BSS
Stations in an independent BSS communicate with each other directly without AP.
Infrastructure BSS
In an infrastructure BSS, STAs must associate with an AP to obtain network services. APs
function as the control centre of the set. This is the most common network architecture.
Every STA needs to go through association and authorisation before joining a certain
BSS. Infrastructure BSS is the most common network architecture.
Network architectures are accompanied with identifications:

Chapter 7. Wi-Fi Configuration and Connection 107

 BSS Identification (BSSID)
Each BSS has a physical address for identification, called BSSID. For an Infrastructure
BSS, the BSSID is the MAC address of the AP. It comes with a factory default value and
can be changed according a fixed naming format.
Service Set Identification (SSID)
Each AP has an identifier for user identification. In most cases, one BSSID is associated
with one SSID. It is usually a readable string, which is what we call the Wi-Fi name.

7.1.4 Wi-Fi Connection

An STA first searches for nearby wireless networks through active or passive scanning, then
establishes a connection with an AP after authentication and association, and finally accesses
the wireless LAN. Figure 7.4 shows the process of Wi-Fi connection.

Figure 7.4. Process of Wi-Fi connection

1. Scanning
An STA can actively or passively scan wireless networks.
Passive scanning
Passive scanning refers to discovering wireless networks nearby through monitoring the
beacon frames periodically sent by an AP. It is recommended when users need to save
power.
Active scanning
During active scanning, the STA actively sends out probe requests and receives probe
responses from the AP. It is further devided into two modes based on the involvement of
SSID.
Active scanning without SSID
The STA periodically sends out probe requests through supported channels to search

108 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 for wireless networks. APs that receive the probe request will return probe responses,
which carry the information of available wireless networks. This enables an STA to
obtain all the available wireless services nearby.
Active scanning with specific SSIDs
If the STA needs to configure a wireless network to be connected or has accessed a
wireless network before, it will periodically send out probe requests with configuration
information or the SSID of the accessed wireless network. When an AP with specific
SSID receives the request, it will return a probe response. In this way, an STA can
actively access a specified wireless network.
For hidden APs, active scanning with specific SSID is recommended.
2. Authentication
When the STA finds an available wireless network, it will select one of the APs with match-
ing SSID according to certain connection strategy, such as selecting the one with strongest
signal or with matching MAC address. The next step is authentication. There is open au-
thentication and non-open authentication.
Open authentication
Essentially, open authentication requires no authentication or encryption. Any STA can
access the network. The AP does not verify STA’s identity in this process, as shown in
Figure 7.5.

Figure 7.5. Process of open authentication

The STA sends a request for authentication, and the AP returns the result. If the result
reads “Success”, then the authentication is completed.
Non-open authentication
Non-open authentication includes shared key authentication, Wi-Fi Protected Access (WPA),
and Robust Security Network (RSN).
Shared key
Shared key authentication is based on the Wired Equivalent Privacy (WEP) method. It
is a basic encryption technology with security flaws.

Chapter 7. Wi-Fi Configuration and Connection 109

 STAs and APs can only interpret the data transmitted between each other when they
have the same key configured. There are 64-bit keys and 128-bit keys. Users can
set up to four groups of different keys. Figure 7.6 shows the process of shared key
authentication.

Figure 7.6. Process of shared key authentication

The STA sends an authentication request to the AP. Then the AP generates a challenge
text and sends it to the STA. The STA uses its preconfigured keys to encrypt the text
and sends it back. The AP uses its preconfigured keys to decrypt the text and com-
pares it with the original text. If the two texts are identical, then the authentication is
completed.
Wi-Fi Protected Access (WPA)
WPA is an intermediate solution to replace WEP before the official release of IEEE
802.11i. It uses a new Message Integrity Check (MIC) algorithm to replace the CRC
algorithm in WEP. It also adopts the Temporal Key Integrity Protocol (TKIP) to generate
different keys for different MAC frames. TKIP is a transitional encryption protocol and
has proved of low security.
Robust Security Network (RSN)
The WFA calls RSN the WPA2. It adopts a new encryption method, the Counter Mode
with CBC-MAC Protocol (CCMP), a block security protocol based on the Advanced En-
cryption Standard (AES). We will expound on this later along with authorisation.
Wi-Fi Protected Access 3 (WPA3)
Although WPA2 consolidates Wi-Fi networks to a certain extent, new security vulnera-
bilities keep emerging, such as offline dictionary attacks, brute force attacks, and key
reinstallation attacks (KRACK). To this end, the WFA released the WPA3 in 2018, a new
generation of Wi-Fi encryption protocol that mitigates the risks in WPA2 and provides
new features. Compared with WPA2, WPA3 has the following advantages:

110 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 • The use of AES encryption is mandatory instead of TKIP.
• Management frames are protected.
• The more secure method, Simultaneous Authentication of Equals (SAE), is used to
replace the pre-shared key (PSK) authentication in WPA2.
First, SAE denies services for STAs that repeatedly try to connect to the AP, preventing
brute-force attacks or password cracking. Second, its forward secrecy function ensures
that the key will be changed frequently and automatically, so that even if the most
recent key is hacked, only a minimal amount of data will be exposed. Last, SAE con-
siders devices as peers. Either party can initiate a handshake and send authentication
information independently, cancelling the message exchange process, thus leaving no
opportunity for KRACKs.
• 192-bit security suite is used to strengthen password protection.
• HMAC-SHA-384 algorithm is used to export and confirm keys in the four-way hand-
shake phase.
• Galois-Counter Mode Protocol-256 (GCMP-256) is used to protect wireless traffic after
STAs go online.
• Galois Message Authentication Code-256 (GMAC-256) of GCMP is used to protect
multicast management frames.
• WPA3 introduces a Wi-Fi Enhanced Open authentication mode – the Opportunistic
Wireless Encryption (OWE) – which allows for connection without password, retaining
the facilitation for accessing open networks. It uses the Diffie-Hellman key exchange
algorithm to encrypt data on the Wi-Fi network, thereby protecting data exchange
between STAs and the Wi-Fi network.
3. Association
After the AP returns successful authentication result to the STA, the next step is association
to get full network access, as shown in Figure 7.7.
4. Authorisation
After scanning, authentication, and association, let’s focus on the last step – authorisation.
In this section, we’ll introduce the Extensible Authentication Protocol (EAP), and the key
agreement, the four-way handshake protocol.
Extensible Authentication Protocol (EAP)
EAP is the most basic security protocol for identity verification, which is not only a proto-
col, but also a protocol framework. Based on this protocol framework, various authenti-
cation methods are well supported. Supplicants send identity verification requests to the

Chapter 7. Wi-Fi Configuration and Connection 111

 Figure 7.7. Process of association

Authenticator through EAP over LAN (EAPOL), and get allowed to use the network once
the verification succeeds. Figure 7.8 shows the architecture of EAP.
This book only touches the basics about EAP. To learn more, please refer to RFC 3748.

Figure 7.8. EAP architecture

• Supplicant: the entity that initiates an authentication request. For wireless networks,
an STA is a Supplicant.
• Authenticator: the entity that responds to an authentication request. For wireless
networks, an AP is an Authenticator.
• Backend Authentication Server (BAS): In some cases, such as in enterprise appli-
cations, Authenticator does not directly handle authorisation. Instead, it sends the
authentication request to the BAS. This is how the EAP extends its range of applica-
tion.
• Authentication, Authorization, and Accounting (AAA): another EAP-based proto-
col. The entity implementing this protocol is a certain type of BAS, for example, the
RADIUS server.
• EAP server: This is what actually handles authorisation. If there is no BAS, the Au-
thenticator plays as the EAP server, otherwise the BAS will serve the purpose.

112 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Key agreement
Robust Secure Network Association (RSNA) is a set of procedures defined in IEEE 802.11
to ensure wireless network security. It consists of data encryption and integrity verifi-
cation. RSNA uses the above mentioned TKIP and CCMP. The Temporary Key (TK) used
in TKIP and CCMP comes from the key derivation function defined by RSNA. Based on
IEEE 802.1X, RSNA also defines the Four-Way Handshake to generate keys for unicast
data encryption, and the Group Key Handshake for multicast data encryption.
But why do we need to derive keys? In the WEP encryption mode, all STAs use the same
WEP key for encryption, resulting in low security, while RSNA requires different STAs to
use different keys for encryption after associating with APs. Does this mean that the AP
needs to set different passwords for different STAs? Obviously, the answer is no. In real
life, we use the same password to associate different STAs with the same AP.
Then how can different STAs use different passwords? The password we set in an STA is
called Pairwise Master Key (PMK). It comes from the PSK, namely the password set in the
wireless router at home. It is set without any authentication server, and the corresponding
setting is WPA/WPA2-PSK. Figure 7.9 shows how a PMK is generated from the PSK.

Figure 7.9. Generation of PMK from PSK

In WPA2-PSK, PMK is identical with PSK. But in WPA3, it generates new PMK through the
SAE method based on the PMK from WPA2, to ensure that every STA has a unique PMK
at different stages. Figure 7.10 shows how PMK is generated through SAE.
SAE treats the Supplicant and the Authenticator as peers. Either of them can initiate
authentication. The two parties exchange data with each other to prove their knowledge
of the key and generate PMK. SAE includes two phases, Commit and Confirm. In the
Commit phase, both parties send SAE Commit frames for deducing the PSK. Then in
the Confirm phase, they send SAE Confirm frames to verify the PSK. Once verification
succeeds, PMK will be generated and the association will proceed.

Chapter 7. Wi-Fi Configuration and Connection 113

 Figure 7.10. Generation of PMK with SAE

• Commit
The sender uses the Hunting and Pecking algorithm to generate a Password Element
(PWE) based on PSK and the MAC addresses of the sender and receiver. Then the scalar
integer and element coordinates are generated by the sender based on PWE and a random
value through elliptic curve operation. Upon receiving the commit frame, the receiver
verifies the frame, and uses both the local and received frame content to generate a Key
Confirmation Key (KCK) and PMK. The KCK will generate and verify the frame content in
the Confirm phase.
• Confirm
Both parties generate a verification code from the KCK, local and peer Scalars, and local
and peer PWEs through the same hash algorithm. If the codes turn out to be identical,
the verification passes.
After the STA and AP obtain PMK, they will start key derivation. During this process, the
AP and different STAs generate different keys, which are configured into hardware for
encryption/decryption. Since the AP and STAs need to re-generate these keys every time
they are associated, they are named Pairwise Transient Keys (PTK). AP and STA use the
EAPOL-Key frame to exchange Nonce and other messages, when the Four-Way Handshake
comes into play. The process is shown in Figure 7.11.
(1) The Authenticator generates a Nonce (ANonce), and sends it to the Supplicant in
EAPOL-Key (Message 1).
(2) Supplicant performs key derivation based on the ANonce, its own Nonce (SNonce)
and PMK, and Authenticator’s MAC address . Then it sends the Authenticator another
EAPOL-Key message (Message 2) that contains the SNonce. Message 2 also carries an
MIC value encrypted by KCK. Once the Authenticator gets the SNonce in Message 2, it

114 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 7.11. Process of Four-Way Handshake

performs calculations similar to that of the Supplicant to verify whether the message
returned is correct. If it is incorrect, which means the Supplicant’s PMK is wrong, the
handshake will be terminated.
(3) If the Supplicant’s PMK is correct, the Authenticator will also perform key derivation.
Later, the Authenticator sends the third EAPOL-Key message (Message 3) to the Sup-
plicant. This message carries the Group Transient Key (GTK, used to update the group
key encrypted by KEK) and MIC (encrypted by KCK). When the Supplicant receives
Message 3, it will check whether the PMK of the AP is correct by calculation.
(4) The Supplicant sends the last EAPOL-Key message (Message 4) to the Authenticator
for confirmation. Then both parties will use it to encrypt data.
So far, the Supplicant and Authenticator have completed key derivation and pairing. They
can eventually communicate with each other.

7.2 Basics of Bluetooth

This section will introduce Bluetooth from the following aspects:
• What is Bluetooth?
• How does Bluetooth evolve?
• What do the Bluetooth concepts mean?
• How does Bluetooth connection work?

Chapter 7. Wi-Fi Configuration and Connection 115

7.2.1 Introduction to Bluetooth
Bluetooth is a short-range wireless communication technology originally conceived by Eric-
sson in 1994. The goal of Bluetooth is to facilitate transmission and sharing of information
over a short distance without cable connections among mobile devices, embedded devices,
computer peripherals, and household appliances. Compared with other wireless communi-
cation technologies, Bluetooth boasts high security and easy connection.

NOTE: Why “Bluetooth”?

The word “Bluetooth” dates back more than a millennia to the Danish King Harald Blue-
tooth. King Harald is credited with the first unification of Scandinavia. Legend has it
that King Harald liked blueberries so much that his teeth were stained blue. So he was
called Bluetooth. In 1998, Intel, Nokia, Ericsson, and IBM established a Special Interest
Group (SIG) called Bluetooth. The word Bluetooth quickly gained popularity and became
synonymous with the short-range wireless communication technology.

Bluetooth adopts decentralised network structure, fast frequency hopping, and short packet
technology to support point-to-point and point-to-multipoint communication. It works in
the 2.4 GHz ISM (Industrial, Scientific, Medical) band, which is commonly used worldwide.
Bluetooth technology can be divided into two categories, Bluetooth Classic and Bluetooth
Low Energy.
Bluetooth Classic
Bluetooth Classic (BR/EDR) generally refers to modules that support the Bluetooth proto-
col below version 4.0, and are used for the transmission of large amounts of data such as
voice and music. Bluetooth Classic protocols have different profiles of personal area net-
works for different scenarios. Commonly used profiles are Advanced Audio Distribution
Profile (A2DP) for audio communication, Hands-Free Profile/Head-Set Profile (HFP/HSP)
for hands-free devices, Serial Port Profile (SPP) for text serial port transparent transmis-
sion, and Human Interface Device (HID) for wireless input/output devices.
Bluetooth Low Energy
Bluetooth Low Energy (LE) is a new type of ultra-low power wireless communication
technology, designed for low-cost, less complicated wireless body and personal area net-
works. It is worth mentioning that Bluetooth LE chips can be powered by button cells.
Together with microsensors, you can use the chips to build embedded or wearable sensors
and applications of sensor networks.
In terms of protocols, Bluetooth 1.1, 1.2, 2.0, 2.1, and 3.0 apply to Bluetooth Classic, Blue-
tooth 4.0 supports both Bluetooth Classic and Bluetooth LE, and all later versions employ
Bluetooth LE.

116 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

7.2.2 Bluetooth Concepts
This section introduces the network technologies related to Bluetooth, including the core
architecture and components in Bluetooth specifications.
Core architecture
The core system of Bluetooth is composed of Host, Controller, and Host Controller In-
terface (HCI). Host is used for application development, while Controller is for message
sending and receiving, physical connection management, and other basic features which
are implemented by dedicated Bluetooth chip manufacturers.
The original design is to run the Host and the Controller independently on two chips or
even systems, and they can communicate through the HCI. In this way, it is easier to
replace and upgrade either module. Although there are many chips that put both the
Host and the Controller together, they still follow this architecture, except that the HCI is
changed from a hardware communication port to a software one.
The Bluetooth LE protocol stack includes Physical Layer (PHY), Link Layer (LL), Logical
Link Control and Adaptation Protocol (L2CAP), Attribute Protocol (ATT), Security Man-
ager Protocol (SMP), Generic Attribute Profile (GATT), and Generic Access Profile (GAP),
as shown in Figure 7.12.

Figure 7.12. Protocol stack layers of Bluetooth LE

• Physical Layer (PHY) specifies the wireless frequency band and modulation mode
used by Bluetooth LE. How the PHY performs determines the power consumption,
sensitivity, and selectivity of the Bluetooth LE chip and influences other radio frequency
indicators.
• Link Layer (LL) only sends or receives data, leaving data analysis to GAP or ATT at the
upper layer. LL is at the core of the Bluetooth LE protocol stack, as it decides which
radio frequency channel to choose for communication, how to identify data packets

Chapter 7. Wi-Fi Configuration and Connection 117

 transmitted through the air, when to send data packets, how to ensure data integrity,
and how to manage and control links, how to receive and retransmit ACKs, etc.
• Host Controller Interface (HCI) provides a means of communication between the
Host and the Controller. This layer can be implemented either by a hardware interface
such as UART or USB in dual-chip architectures, or through a software API in single-
chip architectures.
• Logical Link Control and Adaptation Protocol (L2CAP) provides connection-oriented
and connectionless data services to upper layer protocols with protocol multiplexing
capability, segmentation and reassembly operation, and group abstractions. It also
permits per-channel flow control and retransmission.
• Attribute Protocol (ATT) defines data for user commands and command operations,
such as reading or writing certain data. Bluetooth LE introduces the concept of at-
tributes, which are used to describe data in a piece. Besides, ATT also defines the ATT
commands that data can use. It is the layer that you will most frequently deal with.
• Security Manager Protocol (SMP) is responsible for the encryption and security of
Bluetooth LE connections, without affecting user experience.
• Generic Attribute Profile (GATT) standardises the data content in attributes, and use
the concept of groups to classify and manage attributes. Although the BLE protocol
stack can operate without GATT, its interoperability will be compromised. It is GATT
and a rich set of profiles that frees Bluetooth LE from the compatibility problem faced
by other wireless protocols such as ZigBee.
• Generic Access Profile (GAP) defines effective data packets in LL, offering an easiest
way to analyse LL payload. It is limited to features such as broadcasting, scanning,
and initiating connections.
Roles of Bluetooth
Bluetooth can be an Advertiser, a Scanner, or an Initiator. A master device plays the role
of Initiator and Scanner, while a slave device plays the role of Advertiser.
Bluetooth communication refers to the communication between two or more Bluetooth
devices. It only occurs between masters and slaves, as slave devices cannot communicate
directly.
Master mode
Master (or “central”) devices scan for slaves and initiate connection. In theory, if Blue-
tooth LE Mesh is not used to enable many-to-many device communication, only piconets
can be established among devices.
A device with Bluetooth technology can switch between master mode and slave mode.

118 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 It normally works in slave mode and waits for master devices to connect. When needed,
it switches to master mode and calls other devices. To initiate a call in master mode,
a Bluetooth device needs to know the Bluetooth address and pairing password of the
other device, and start calling after pairing successfully.
Slave mode
Slave (or “peripheral”) devices advertise and wait for connections. Once connected,
slaves can exchange data with the master.
In summary, a master can search for slaves and actively connects with them, while a slave
cannot initiate any connection but to wait to be connected.
Building Bluetooth networks
Now that we’ve learned about master and slave, let’s take a look at how to build Bluetooth
networks. According to topological structures, Bluetooth networks can be divided into
piconets, scatternets, and mesh networks.
Piconet
Every time a Bluetooth wireless link is formed, it is within a context of piconet. A pi-
conet consists of two or more devices that occupy the same physical channel, which
means the devices are synchronised according to a common clock and frequency hop-
ping sequence. Figure 7.13 shows the piconet topology.

Figure 7.13. Piconet topology

Scatternet
A scatternet is formed when multiple piconets overlap. Figure 7.14 shows the scatternet
topology.
Each piconet that constitutes a scatternet maintains its own master. The master of one
piconet may act as the slave of another piconet at the same time. In Figure 7.14, the
mobile phone is the master of the left piconet as well as the slave of the right piconet.
Mesh
Bluetooth Mesh was born after Bluetooth 4.0. It is a Bluetooth LE network used to
establish many-to-many device communication. It allows the creation of large-scale

Chapter 7. Wi-Fi Configuration and Connection 119

 Figure 7.14. Scatternet topology

networks, where dozens, hundreds, or even thousands of Bluetooth mesh devices can
transmit data with each other. Bluetooth mesh is not the focus of this book, so you only
need to know its definition for now.

7.2.3 Bluetooth Connection

Bluetooth first searches for nearby devices through advertising or scanning, then establishes
a connection, and finally form a network for data transmission.
1. Slave advertising
Usually, the peripheral device (slave) advertises itself and waits for the central device (mas-
ter) to discover it and establish GATT connection for more exchange. In some cases, the
peripheral only disseminates its own information to multiple central devices, without the
need for connection.
To be discovered by a master, a slave will periodically send out advertising packets at an
interval of t. We call every advertising packet sent an “advertising event”, so t is also called
the advertising event interval, as shown in Figure 7.15.

Figure 7.15. Slave advertising interval

120 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Advertising events occur once in a while, and each event lasts for a period. The Bluetooth
chip only enables the radio frequency module to send packets during the event, hence the
relatively high power consumption. At other times, the chip goes idle, so the average power
consumption is quite low.
Each advertising event contains three packets for the same message to be advertised on
channel 37, 38, and 39 simultaneously. The process of slave advertising event is shown in
Figure 7.16.

Figure 7.16. Slave advertising event

2. Master scanning
Scanning refers to the process when a master tries to find other Bluetooth LE devices within
a certain range using advertising channels. Different from advertising, no interval or channel
is set for scanning. The master may customise its own settings.
Passive scanning
In passive scanning, the scanner only listens to advertising packets without sending any
data to the advertiser., as shown in Figure 7.17.
Once the scanning parameters are set, the master may send commands in the protocol
stack to start scanning. During the process, if the controller receives an advertising packet
that meets the filter policy or other constraints, it will report an event to the master. In
addition to the advertiser’s address, the event also includes the data and the received
signal strength indication (RSSI) of the advertising packet. Developers can estimate the
signal path loss based on the RSSI and the transmission power of the advertising packet.
This feature can be used to develop anti-lost trackers and positioning solutions.
Active scanning
In active scanning, the master can capture not only the advertising packets sent by slaves
but also the scan response packets, and distinguish the two types. See Figure 7.18 for the
process of active scanning.

Chapter 7. Wi-Fi Configuration and Connection 121

 Figure 7.17. Passive scanning

Figure 7.18. Active scanning

After the controller receives any data, it will report an event to the master, containing the
advertising type of the LL packet. The master can thereby decide whether to connect or
scan the slave, and distinguish advertising packets from scan response packets.
3. Master Connection
(1) The peripheral device starts advertising. Within the T IFS after sending an advertising
packet, it enables radio frequency to receive packets from the central device. (T IFS:
Inter Frame Space, the time interval between two packet transmissions on the same
channel)

122 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(2) The central device scans for advertising. Within the T IFS after receiving the advertis-
ing packet, if it enables scan response, it will reply to the peripheral device.
(3) Once the peripheral device receives scan response, it will return an ACK packet and
prepare to receive data.
(4) If the central device does not receive the ACK packet, it will continue sending scan
responses until it times out. During this period, only one ACK packet being received is
enough to establish the connection.
(5) Now that the two devices are connected, they start communicating. The central device
will send data packets to the peripheral at connection intervals, starting from the time
when the advertising packet is received. The data packets are used to synchronise
the clocks of the two devices and establish communication in master-slave mode. The
process is as follows:
a. Every time the peripheral device receives a packet from the central device, it resets
the starting point to synchronise with the central device (service synchronsied with
the client).
b. Bluetooth LE communication is established in master-slave mode. The central de-
vice becomes the master, and the peripheral device becomes the slave. A slave can
only send data back to the master within a specified time after the master sends a
packet to it.
c. Connection is established.
d. The peripheral device automatically stops advertising, and it can no longer be
found by other devices.
e. During the interval between packet transmissions by the central device, the periph-
eral device may send multiple advertising packets.
The communication sequence is shown in Figure 7.19.

Figure 7.19. Communication sequence

Chapter 7. Wi-Fi Configuration and Connection 123

To be disconnected, the central device only needs to stop sending packets. It can write
the MAC address of the peripheral device into flash, SRAM, or other storage devices to
keep monitoring the address, and reestablish communication when it receives advertising
packets from the peripheral again. In order to save power, the slave will not send advertising
packets if there is no data to be transmitted, and the two parties will be disconnected due to
connection timeout. At this time, the central device needs to start monitoring, so that when
the slave needs to send data, they can connect again.

7.3 Wi-Fi Network Configuration

With the development of IoT, more and more devices get connected via Internet. However,
such devices do not have rich HCIs as smartphones and tablets. To connect them to the Inter-
net, users cannot directly enter the SSID and password of the router. How to empower such
devices, connecting them to the Internet or LANs using the router? This is one of the impor-
tant targets of Wi-Fi devices. This section will cover some common network configuration
methods.

7.3.1 Wi-Fi Network Configuration Guide

Network configuration is to provide SSID and password to Wi-Fi devices, so that they can
connect to a specified AP and join its Wi-Fi network.
The final goal here is to send the SSID and password of the AP to the Wi-Fi device in
different ways, and connect the device to the specified Wi-Fi network to join the LAN or
Internet. Figure 7.20 shows the process of Wi-Fi network configuration.

Figure 7.20. Process of Wi-Fi configuration

The IoT devices waiting for connection also need to be associated with an account, so here
come some new concepts:
• Network configuration in a narrow sense: A Wi-Fi device obtains the AP information
(SSID, password, etc.) and connects to the AP.
• Binding: Associating user’s application accounts with the configured device.
• Network configuration in a broad sense: Network configuration in a narrow sense
+ binding.
This section will focus on network configuration in a narrow sense, thus omitting the binding
process. At present, the most popular methods to configure networks are SoftAP, SmartCon-
fig, and Bluetooth.

124 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

7.3.2 SoftAP
1. Introduction
SoftAP is a traditional method. First, the IoT device to be configured establishes an AP. The
user connects a smartphone, tablet, or other devices with HCIs to this AP, and sends informa-
tion about the network providing device. Then, the IoT device looks for the corresponding
network and connects with it. Figure 7.21 shows the steps of SoftAP network configuration.

Figure 7.21. Steps of SoftAP network configuration

The SoftAP mechanism connects devices directly to the LAN without routers, thereby pre-
venting router compatibility issues. This makes it easier to successfully configure the net-
work compared with SmartConfig. But the downside is that there is an extra step for con-
nection, as we need to manually switch to the IoT SoftAP in the Wi-Fi list. If we want to
access cloud services, we still need a router. Some smartphones may automatically switch
APs, but with iOS 11.0 or previous versions, we need to do the extra settings manually.
2. Configuration
Figure 7.22 indicates how to configure networks via SoftAP.
In-depth introduction to SoftAP will be given later together with Wi-Fi programming.

7.3.3 SmartConfig
1. Introduction
SmartConfig allows smartphones to fill the SSID and password in the unencrypted header
of the MAC packet according to a certain encoding format, and send them in segments to
the IoT device in multiple times by broadcasting and multicasting. Generally, we need to
install an application on the smartphone for protocol interaction between the two parties.
The steps of SmartConfig network configuration are shown in Figure 7.23.
After the receiver enables SmartConfig, the IoT device starts monitoring data on the router
from channel 1. Once detecting data packets that meet the rules, it stops switching and
stays on the current channel to receive all the data. Otherwise, the IoT device automatically
switches to the subsequent channels until channel 13, and start over from channel 1.
The frame format of MAC layer in IEEE 802.11 allows for clear identification of LL payload
data, which includes the header and data of the network layer. This makes it possible to
immediately extract and calculate the length of the payload data as soon as the MAC frames

Chapter 7. Wi-Fi Configuration and Connection 125

are received. The payload data here is usually the password. Figure 7.24 shows the packet
structure of SmartConfig network configuration.

Figure 7.22. Network configuration via SoftAP

126 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 7.23. Steps of SmartConfig network configuration

Figure 7.24. Packet structure of SmartConfig network configuration

Table 7.2 explains the fields of the data packet of SmartConfig network configuration.

Table 7.2. Fields of the data packet of SmartConfig network configuration

Data Frame Description

DA Target MAC Address

SA Source MAC Address

LENGTH Payload Data Length

LLC LLC Head

SNAP 3 B for Manufacturer Code and 2 B for Protocol Type

DATA Payload Data

FCS Frame Check Sequence

The sender usually uses the following methods to send data.

UDP broadcasting
The MAC frame format of IEEE 802.11 ensures that the DA, SA, LENGTH, LLC, SNAP,
and FCS fields are always visible to wireless signal monitors to acquire valid information,
regardless of whether the channels are encrypted. When broadcasting, the sender is lim-
ited by the operating system, leaving only the LENGTH field at its disposal. However, by
specifying a length-encoded communication protocol, a LENGTH field is enough for the
sender to transmit the data needed.

Chapter 7. Wi-Fi Configuration and Connection 127

UDP multicasting
The multicast address is a reserved class D address, with a range of [Link] to 239.255.
255.255. The mapping between IP and MAC addresses is accomplished by setting the
first 25 bits of the MAC address to 01.00.5E, while the last 23 bits of the MAC address
corresponding to the bits of the IP address. As a result, the sender can encode data in the
last 23 bits of the multicast IP and transmit it through the multicast packet for the receiver
to decode.
SmartConfig offers user-friendly, smooth experience, but it places stringent requirements
on the compatibility of smartphones and routers. For example, some routers may disable
broadcast/multicast packet forwarding by default, preventing devices from receiving packets
forwarded by the router. In other cases, different frequency bands used by smartphones and
IoT devices can also result in configuration failure. If a smartphone is connected to a router
using a 5 GHz frequency band, a device using the 2.4 GHz band may not be able to receive
data. Such uncontrollable factors can significantly reduce overall compatibility and make it
hard to successfully configure the network.
2. Configuration
The SmartConfig mechanisms developed by Espressif are:
• ESP-TOUCH V2: UDP broadcast and multicast encoding.
• ESP-TOUCH: UDP broadcast encoding.
• AIRKISS: WeChat mini program.

Source code
In-depth introduction to SmartConfig will be given later together with Wi-Fi pro-
gramming. Visit [Link] to find the example code for
examples/wifi/smart config.

7.3.4 Bluetooth
1. Introduction
If the IoT device to be configured features Bluetooth, the network binding information can
be sent via Bluetooth channel.
The principle behind the network configuration by Bluetooth is similar to that by SoftAP,
except that the communication method used for transmitting Wi-Fi information is changed
from Wi-Fi (AP mode) to Bluetooth. The IoT device to be configured creates a Bluetooth pro-
file. The user then connects smartphones, tablets, or other devices with HCIs to it through
a Bluetooth channel, and sends the information needed for network configuration. After
receiving the information, the IoT device looks for corresponding AP and connects with it.

128 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The steps of Bluetooth network configuration are shown in Figure 7.25.

Figure 7.25. Steps of Bluetooth network configuration

The advantage of Bluetooth network configuration is that it eliminates the compatibility

issues related to routers, producing higher connection rate. It can also discover and connect
devices directly, so there is no need to turn on the device and connect to its own AP. However,
the compatibility between the Bluetooth module and the mobile phone may affect network
configuration. Additionally, using Bluetooth modules will increase the cost of the device.
2. Configuration
ESP32-C3 chip features both Wi-Fi and Bluetooth LE, thus supporting different network
configuration methods. When it comes to Bluetooth network configuration, ESP32-C3 offers
a comprehensive solution called BluFi. Figure 7.26 indicates how to configure networks via
BluFi.

Figure 7.26. Network configuration via BluFi

Source code
In-depth introduction to Bluetooth network configuration will be given later together with
Wi-Fi programming. Visit [Link] to find the example code
for examples/bluetooth/blufi.

Chapter 7. Wi-Fi Configuration and Connection 129

7.3.5 Other Methods
1. Direct network configuration
Direct network configuration refers to sending SSID and password directly to IoT devices
through peripheral interfaces such as UART, SPI, SDIO, and I2C according to a certain com-
munication protocol. It is also known as wired network configuration. Once the IoT device
receives the SSID and password, it connects to the AP and then returns the connection result
through the master interface.
Moreover, some devices come with pre-set Wi-Fi information, such as SSID and password.
When such devices are started in specified Wi-Fi environment, they can automatically con-
nect to the corresponding AP. Such devices are typically used in large-scale networks, factory
testing, or industrial scenarios. The steps of direct network configuration are shown in Fig-
ure 7.27.

Figure 7.27. Steps of direct network configuration

This method adopts a software solution and is easy to implement. It is well-suited for
devices with Wi-Fi chips or connected by transmission lines of other protocols. However,
transmission lines must be pre-installed between systems.
The Espressif AT (ESP-AT) command firmware provided by Espressif can be directly used in
mass-produced IoT applications. Developers can easily join wireless networks by running
the Wi-Fi commands. For details, please refer to ESP-AT User Guide.
2. RouterConfig
RouterConfig is based on Wi-Fi Protected Setup (WPS), a standard introduced by the Wi-Fi
Alliance to address the complex process of configuring wireless network encryption and au-
thentication settings. The goal of WPS is to simplify Wi-Fi security and network management
for users. The standard offers two methods, Personal Identification Number (PIN) method
and Push Button Configuration (PBC) method. Figure 7.28 shows the steps of RouterConfig.

Figure 7.28. Steps of RouterConfig

The process is relatively straightforward, but it requires both the router and the device to

130 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

support WPS. Unfortunately, many users neglect encryption security settings due to the cum-
bersome steps involved, which can lead to serious security issues. As a result, an increasing
number of routers are abandoning or disabling support for WPS by default. The method has
become less popular in recent years.

Source code
ESP-IDF, the official IoT development framework by Espressif, provides an example of this
network configuration solution. The process there is quite simple. Visit [Link]
com/espressif/esp-idf to see the example in examples/wifi/wps.

3. ZeroConfig
ZeroConfig is a method of using one connected device to configure the network for another
one. This method does not involve smartphones, as other devices like smart speakers can be
used instead.
To initiate the process, the device to be connected sends its MAC address to the networked
device through a custom message. The networked device then responds by sending back its
saved router SSID and password via another custom message. After connecting, the device
can perform further configuration such as external network binding. The steps of ZeroConfig
network configuration are shown in Figure 7.29.

Figure 7.29. Steps of ZeroConfig

Since the networked device stores the SSID and password of the router, users do not need to
enter them manually. The configuration will be easier, thus providing better user experience.
However, this method cannot be widely adopted, as there must be networked devices con-
nected with the router. At the same time, because mobile applications have limited access,
it is impossible to assemble or receive Wi-Fi management frames through third-party pro-
grams. Therefore, smartphones can only be used to implement this method under certain
circumstances.

Chapter 7. Wi-Fi Configuration and Connection 131

4. Phone AP network configuration
Phone AP network configuration refers to setting a smartphone as an AP with a unique
name and password, connecting the IoT device to the AP and sending network binding
information. Figure 7.30 shows the steps of phone AP network configuration.

Figure 7.30. Steps of phone AP network configuration

Phone AP network configuration does not require the IoT device to support AP mode, so
users do not have to do much development work on the device. It can be used with Smart-
Config (simultaneously), making it a good candidate for backup network configuration.
However, the user experience provided is barely satisfying. Many users struggle with setting
the AP name of the smartphone or even enabling the phone AP. Particularly on iOS de-
vices, the application cannot automatically create an AP, so users have to manually modify
the device name and enable the AP. As a result, this method is not suitable for consumer
devices.
In addition to the configuration methods above, Espressif also supports Wi-Fi Easy Connect,
also known as Device Provisioning Protocol (DPP). For more information, please visit https:
//bookc3/[Link]/esp-dpp.

7.4 Wi-Fi Programming

This section provides an overview of Wi-Fi APIs, covering how to use the APIs, to establish
STA connection, and to connect in a smart way.
When developing a Wi-Fi application, the most efficient way is to adapt a similar example
for your own requirements. Therefore, if you want to create a robust application, we suggest
that you read this section and do the practices before getting started with your own project.

7.4.1 Wi-Fi Components in ESP-IDF

1. Features
Wi-Fi components can be used to configure and monitor the Wi-Fi network connection of
ESP32-C3. The following features are supported.
• STA mode: aka station mode or Wi-Fi client mode. ESP32-C3 is connected to the AP
in this mode.
• AP mode: aka SoftAP mode or access point mode. The AP is connected to ESP32-C3
in this mode.

132 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 • AP-STA coexistence mode: ESP32-C3 is connected to another AP as an AP.
• Security standards for the modes above: WPA, WPA2, WPA3, WEP, etc.
• Scanning for APs, including active and passive scanning.
• Promiscuous mode for monitoring IEEE 802.11 Wi-Fi packets.
2. APIs
esp_wifi.h defines the APIs for Wi-Fi components, as shown in Table 7.3.

Table 7.3. APIs for Wi-Fi components

Function Name Description

Initialise resources for the Wi-Fi driver, such as Wi-Fi
esp_wifi_init()
control structures and Wi-Fi tasks
Free resources allocated in esp_wifi_init() and
esp_wifi_deinit()
stop Wi-Fi tasks
esp_wifi_set_mode() Set the WiFi operating mode for ESP32-C3
esp_wifi_get_mode() Get the WiFi operating mode of ESP32-C3
esp_wifi_start() Start Wi-Fi according to current configuration
esp_wifi_stop() Stop Wi-Fi according to current configuration
esp_wifi_connect() Connect ESP32-C3 to the AP
esp_wifi_disconnect() Disconnect ESP32-C3 from the AP
esp_wifi_scan_start() Scan for all available APs
esp_wifi_scan_stop() Stop the scan in progress
esp_wifi_scan_get_ap_num() Get the number of APs found by ESP32-C3
esp_wifi_scan_get_ap_records() Get the information about APs found by ESP32-C3
esp_wifi_set_config() Set the configuration of the ESP32-C3 STA or AP
esp_wifi_get_config() Get the configuration of the ESP32-C3 STA or AP

3. Programming model
The ESP32-C3 Wi-Fi programming model is depicted in Figure 7.31.
The Wi-Fi driver can be considered a black box that knows nothing about upper-layer code,
such as TCP stacks, application tasks, and event tasks. The application task (code) generally
calls Wi-Fi driver APIs to initialise Wi-Fi and handles Wi-Fi events when necessary. The Wi-Fi
driver receives API calls, handles them, and posts events in the application.
Wi-Fi event handling is based on the esp_event library. Events are sent by the Wi-Fi driver

Chapter 7. Wi-Fi Configuration and Connection 133

 Figure 7.31. ESP32-C3 Wi-Fi programming model

to the default event loop. Applications may handle these events in callbacks registered using
esp_event_handler_register(). Wi-Fi events are also handled by the esp_netif
component to provide a set of default behaviors. For example, when a Wi-Fi station connects
to an AP, esp_netif will automatically start the Dynamic Host Configuration Protocol
(DHCP) client by default.

7.4.2 Exercise: Wi-Fi Connection

1. Put ESP32-C3 into STA mode, and connect to an AP.
When operating in STA mode, ESP32-C3 can connect to an AP as an STA.
The BSS based on the central AP allows for multiple STAs to build a wireless network, with
the AP forwarding all communications within the network. In this mode, the device is able
to access both the external and internal network directly, using the Internet Protocol (IP)
address assigned by the AP. Figure 7.32 explains Wi-Fi STA mode.

Figure 7.32. Wi-Fi STA mode

2. Use ESP-IDF components to connect devices to routers.

Figure 7.33 shows how to use ESP-IDF components to connect devices to routers.
(1) Initialisation. See 1.1, 1.2, and 1.3 in Figure 7.33.
a. Initialise LwIP. Create an LwIP core task and initialise LwIP-related work.
1. ESP_ERROR_CHECK(esp_netif_init());

134 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Figure 7.33. Using ESP-IDF components to connect devices to routers

Chapter 7. Wi-Fi Configuration and Connection 135

b. Initialise event. As introduced before, Wi-Fi event handling is based on the esp_event
library, assisted by the esp_netif component. The code to initialise the event is as follows:
1. ESP_ERROR_CHECK(esp_event_loop_create_default());
2. esp_netif_create_default_wifi_sta();
3. esp_event_handler_instance_t instance_any_id;
4. esp_event_handler_instance_t instance_got_ip;
5. ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT,
6. ESP_EVENT_ANY_ID,
7. &event_handler,
8. NULL,
9. &instance_any_id));
10. ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT,
11. IP_EVENT_STA_GOT_IP,
12. &event_handler,
13. NULL,
14. &instance_got_ip));

c. Initialise Wi-Fi. Create the Wi-Fi driver task, and initialise the Wi-Fi driver. The code to
initialise Wi-Fi is as follows:
1. wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
2. ESP_ERROR_CHECK(esp_wifi_init(&cfg));

(2) Configuration. Once the Wi-Fi driver is initialised, you can start configuration. At this
stage, the driver is in STA mode, so you may call esp_wifi_set_mode(WIFI_MODE_STA)
to put ESP32-C3 into STA mode. Refer to the code below:
1. wifi_config_t wifi_config = {
2. .sta = {
3. .ssid = EXAMPLE_ESP_WIFI_SSID,
4. .password = EXAMPLE_ESP_WIFI_PASS,
5. },
6. };
7. ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
8. ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config));

(3) Startup. Call esp_wifi_start() to start the Wi-Fi driver.

1. ESP_ERROR_CHECK(esp_wifi_start());

The Wi-Fi driver posts WIFI_EVENT_STA_START to the event task; then, the event task
will do some routine work and call the application event callback function.
The application event callback function relays WIFI_EVENT_STA_START to the application
task, and then we call esp_wifi_connect().
(4) Connection. Once esp_wifi_connect() is called, the Wi-Fi driver will start the
internal scan/connection process.
If the internal scan/connection is successful, WIFI_EVENT_STA_CONNECTED will be gen-

136 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

erated. In the event task, the DHCP client will be started and trigger the DHCP process.
Refer to the code below:
1. static void event_handler(void* arg, esp_event_base_t event_base,
2. int32_t event_id, void* event_data)
3. {
4. if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
5. esp_wifi_connect();
6. } else if
7. (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
8. if (s_retry_num < EXAMPLE_ESP_MAXIMUM_RETRY) {
9. esp_wifi_connect();
10. s_retry_num++;
11. ESP_LOGI(TAG, "retry to connect to the AP");
12. } else {
13. xEventGroupSetBits(s_wifi_event_group, WIFI_FAIL_BIT);
14. }
15. ESP_LOGI(TAG, "connect to the AP fail");
16. } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
17. ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
18. ESP_LOGI(TAG, "got ip:" IPSTR, IP2STR(&event->ip_info.ip));
19. s_retry_num = 0;
20. xEventGroupSetBits(s_wifi_event_group, WIFI_CONNECTED_BIT);
21. }
22. }

(5) Getting IP. Once the DHCP client is initialised, the “getting IP” phase will begin. If the
IP address is successfully received from the DHCP server, IP_EVENT_STA_GOT_IP will be
triggered and commonly handled in event task.
In the application event callback, IP_EVENT_STA_GOT_IP is relayed to the application
task. For LwIP-based applications, this marks a special event which means that everything
is ready for the application to perform subsequent tasks. But remember not to start the
socket-related work before receiving the IP.
(6) Disconnection. Wi-Fi connection may fail because of active disconnection, wrong pass-
word, AP not found, etc. In this case, WIFI_EVENT_STA_DISCONNECTED will arise and
provide the reason for the failure, such as esp_wifi_disconnect() being called to ac-
tively disconnect.
1. ESP_ERROR_CHECK(esp_wifi_disconnect());

(7) IP Changed. If the IP address is changed, IP_EVENT_STA_GOT_IP will be triggered

with ip_change set to true.
(8) Cleanup, including breaking Wi-Fi connection, stopping and unloading the Wi-Fi driver,
etc. The code is as follows:

Chapter 7. Wi-Fi Configuration and Connection 137

1. ESP_ERROR_CHECK(esp_event_handler_instance_unregister(IP_EVENT,
2. IP_EVENT_STA_GOT_IP,
3. instance_got_ip));
4. ESP_ERROR_CHECK(esp_event_handler_instance_unregister(WIFI_EVENT,
5. ESP_EVENT_ANY_ID,
6. instance_any_id));
7. ESP_ERROR_CHECK(esp_wifi_stop());
8. ESP_ERROR_CHECK(esp_wifi_deinit());
9. ESP_ERROR_CHECK(esp_wifi_clear_default_wifi_driver_and_handlers(
10. station_netif));
11. esp_netif_destroy(station_netif);

7.4.3 Exercise: Smart Wi-Fi Connection

1. SoftAP
The wifi_provisioning components provided by ESP32-C3 can transmit SSID and pass-
word of the AP through SoftAP or Bluetooth LE, and then use them to connect to the AP.
(1) APIs
The APIs for wifi_provisioning are defined in esp-idf/components/wifi provi
sioning/include/wifi provisioning/manager.h, as shown in Table 7.4.

Table 7.4. APIs for wifi provisioning components

API Description
Initialise provisioning manager interface ac-
wifi_prov_mgr_init()
cording to current configuration
wifi_prov_mgr_deinit() Release provisioning manager interface
wifi_prov_mgr_is_provisioned() Check the provisioning status of ESP32-C3
wifi_prov_mgr_start_provisioning() Start the provisioning service
wifi_prov_mgr_stop_provisioning() Stop the provisioning service
wifi_prov_mgr_wait() Wait for the provisioning service to finish
Disable auto stopping of the provisioning ser-
wifi_prov_mgr_disable_auto_stop()
vice upon completion
Create an endpoint and allocate internal re-
wifi_prov_mgr_endpoint_create()
sources for it
wifi_prov_mgr_endpoint_register() Register a handler for the created endpoint
wifi_prov_mgr_endpoint_unregister() Unregister the handler for the created endpoint
Get the state of the Wi-Fi STA during
wifi_prov_mgr_get_wifi_state()
provisioning
Get the reason code for Wi-Fi STA disconnection
wifi_prov_mgr_get_wifi_disconnect_reason()
during provisioning

138 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(2) Program structure
• Initialisation:
1. wifi_prov_mgr_config_t config = {
2. .scheme = wifi_prov_scheme_softap,
3. .scheme_event_handler = WIFI_PROV_EVENT_HANDLER_NONE
4. };
5.
6. ESP_ERR_CHECK(wifi_prov_mgr_init(config));

• Checking the provisioning status:

1. bool provisioned = false;
2. ESP_ERROR_CHECK(wifi_prov_mgr_is_provisioned(&provisioned));
• Starting provisioning service:
1. const char *service_name = "my_device";
2. const char *service_key = "password";
3.
4. wifi_prov_security_t security = WIFI_PROV_SECURITY_1;
5. const char *pop = "abcd1234";
6.
7. ESP_ERR_CHECK(wifi_prov_mgr_start_provisioning(security, pop, service_name,
8. service_key));

• Releasing resources for provisioning.

Once the provisioning service is complete, the main application will release the re-
sources for provisioning and start executing its own logic. There are two ways to do
this. The simpler way is to call wifi_prov_mgr_wait(). See the code below:
1. //Wait for the provisioning service to finish
2. wifi_prov_mgr_wait();
3.
4. //Release the resources for provisioning
5. wifi_prov_mgr_deinit();

The other way is to use the callback function of the event. See the code below:
1. static void event_handler(void* arg, esp_event_base_t event_base,
2. int event_id, void* event_data)
3. {
4. if (event_base == WIFI_PROV_EVENT && event_id == WIFI_PROV_END) {
5. //Release the resources for provisioning upon completion
6. wifi_prov_mgr_deinit();
7. }
8. }

(3) Functional verification

To get started, install ESP SoftAP Provisioning on your phone. Next, turn on the Wi-Fi and
power on the device. Ensure that the output log by the serial port (see Figure 7.34) contains

Chapter 7. Wi-Fi Configuration and Connection 139

information beginning with PROV_.

NOTE
You may download the APP at [Link]

Figure 7.34. Output log by the serial port

a. Startup
Open the application on your phone and tap “Start Provisioning”. Then you will find the
device PROV DAED2CXXXXX on the screen (refer to Figure 7.35).

Figure 7.35. Startup Figure 7.36. SoftAP connection

140 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

b. Connection
Tap “Connect” to navigate to the Wi-Fi setting interface. Select to connect the device
PROV DAED2CXXXXX. If connected, you will see the screen as Figure 7.36.
The output log is as follows:
I (102906) wifi:station: [Link] join, AID=1, bgn, 40U
I (103056) esp_netif_lwip: DHCP server assigned IP to a station, IP is: [Link]
I (124286) wifi:station: [Link] leave, AID = 1, bss_flags is 134259, bss:0x3fca7844
I (124286) wifi:new: <1,0>, old: <1,1>, ap: <1,1>, sta: <0,0>, prof:1
I (149036) wifi:new: <1,1>, old: <1,0>, ap: <1,1>, sta: <0,0>, prof:1
I (149036) wifi:station: [Link] join, AID=1, bgn, 40U
I (149246) esp_netif_lwip: DHCP server assigned IP to a station, IP is: [Link]

c. Provisioning
Tap “Provision Network” to enter the provisioning screen shown in Figure 7.37.
d. Completion
Tap “Provision” to enter the completion screen shown in Figure 7.38.

Figure 7.37. Provisioning Figure 7.38. Completion

The output log is as follows:

I (139471) app: Received Wi-Fi credentials
SSID : myssid
Password : mypassword
.
.
.
I (144091) app: Connected with IP Address:[Link]
I (144091) esp_netif_handlers: sta ip: [Link], mask: [Link], gw: [Link]
I (144091) wifi_prov_mgr: STA Got IP
I (144101) app: provisioningsuccessful
I (144101) app: Hello World!
I (145101) app: Hello World!
.
.

Chapter 7. Wi-Fi Configuration and Connection 141

 .
I (146091) wifi_prov_mgr: Provisioning stopped
I (146101) app: Hello World!
I (147101) app: Hello World!
I (148101) app: Hello World!

2. SmartConfig
The SmartConfig component provided by ESP32-C3 can transmit the SSID and password of
the AP through promiscuous mode, and then use them to connect to the AP.
(1) APIs
The APIs for SmartConfig are defined in esp-idf/components/esp wifi/include/
esp smartconfig.h, as shown in Table 7.5.

Table 7.5. APIs for SmartConfig

API Description

esp_smartconfig_get_version() Get the version of the current SmartConfig

esp_smartconfig_start() Start SmartConfig

esp_smartconfig_stop() Stop SmartConfig

esp_esptouch_set_timeout() Set the timeout of SmartConfig process

esp_smartconfig_set_type() Set the protocol type of SmartConfig

esp_smartconfig_fast_mode() Set the mode of SmartConfig

esp_smartconfig_get_rvd_data() Get the reserved data of ESPTouch v2

(2) Program structure

• Wi-Fi event handling
This module takes care of Wi-Fi connection, disconnection, reconnection, scanning, etc., as
detailed in the sections before. Additionally, when the WIFI_EVENT_STA_START event
occurs, it will also create a SmartConfig task.
• NETIF event handling
This module helps acquire the IP address. Details are provided in the sections before. When
the IP_EVENT_STA_GOT_IP event occurs, the connection flag will be set.
• SmartConfig event handling
The received request determines how the event is handled and processed. SmartConfig
events are shown in Table 7.6.

142 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Table 7.6. SmartConfig events

Event Description
SC_EVENT_SCAN_DONE Scan to obtain the information about nearby APs
SC_EVENT_FOUND_CHANNEL Get the channel of the target AP
SC_EVENT_GOT_SSID_PSWD Enter STA mode to get the SSID and password of the target AP
SC_EVENT_SEND_ACK_DONE Set the SmartConfig completion flag

• SmartConfig tasks
The code for SmartConfig tasks is as follows.
1. static void smartconfig_example_task (void *param)
2. {
3. EventBits_t uxBits;
4. ESP_ERROR_CHECK(esp_smartconfig_set_type(SC_TYPE_ESPTOUCH));
5. smartconfig_start_config_t cfg = SMARTCONFIG_START_CONFIG_DEFAULT();
6. ESP_ERROR_CHECK(esp_smartconfig_start(&cfg));
7. while (1) {
8. uxBits = xEventGroupWaitBits (s_wifi_event_group,
9. CONNECTED_BIT | ESPTOUCH_DONE_BIT,
10. true,
11. false,
12. portMAX_DELAY);
13. if (uxBits & CONNECTED_BIT) {
14. ESP_LOGI (TAG, "WiFi Connected to ap");
15. }
16. if (uxBits & ESPTOUCH_DONE_BIT) {
17. ESP_LOGI (TAG, "smartconfig over");
18. esp_smartconfig_stop();
19. vTaskDelete (NULL);
20. }
21. }
22. }

As demonstrated in the code above, a SmartConfig task primarily performs three functions.
First, it sets the SmartConfig type, such as ESP-TOUCH and ESP-TOUCH V2. Second, after
the configuration, it enables SmartConfig by calling esp_smartconfig_start(). Finally,
it checks the event group in a loop. Upon receiving the SC_EVENT_SEND_ACK_DONE event,
it stops SmartConfig by calling esp_smartconfig_stop().
• Main program
It creates an event group to set the flag when a relevant event is triggered, and then ini-
tialises Wi-Fi.

Chapter 7. Wi-Fi Configuration and Connection 143

(3) Functional verification
To get started, install Espressif Esptouch on your phone. Then turn on the Wi-Fi and power
on the device. You will see the output log by the serial port as follows:
I (1084) wifi:mode : sta ([Link])
I (1084) wifi:enable tsf
I (1134) smartconfig: SC version: V3.0.1
I (5234) wifi:ic_enable_sniffer
I (5234) smartconfig: Start to find channel...
I (5234) smartconfig_example: Scan done

NOTE
You may download the APP at [Link]

Connect your phone to Wi-Fi, and enter the password to start configuration. The SmartCon-
fig interface is shown in Figure 7.39.

Figure 7.39. SmartConfig configuration

The output log is as follows:

I (234592) smartconfig: TYPE: ESPTOUCH
I (234592) smartconfig: T|PHONE M[Link]
I (234592) smartconfig: T|AP M[Link]
I (234592) sc: SC_STATUS_GETTING_SSID_PSWD
I (239922) smartconfig: T|pswd: 123456789
I (239922) smartconfig: T|ssid: IOT_DEMO_TEST
I (239922) smartconfig: T|bssid: [Link]
I (239922) wifi: ic_disable_sniffer

144 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

I (239922) sc: SC_STATUS_LINK
I (239932) sc: SSID:IOT_DEMO_TEST
I (239932) sc: PASSWORD:123456789
I (240062) wifi: n:1 0, o:1 0, ap:255 255, sta:1 0, prof:1
I (241042) wifi: state: init -> auth (b0)
I (241042) wifi: state: auth -> assoc (0)
I (241052) wifi: state: assoc -> run (10)
I (241102) wifi: connected with IOT_DEMO_TEST, channel 1
I (244892) event: ip: [Link], mask: [Link], gw: [Link]
I (244892) sc: WiFi Connected to ap
I (247952) sc: SC_STATUS_LINK_OVER
I (247952) sc: Phone ip: [Link]
I (247952) sc: smartconfig over

3. Bluetooth
The BluFi components of ESP32-C3 help transmit the SSID and password through Bluetooth
LE, which can be used to connect to the AP.
(1) APIs
The APIs for BluFi components are defined in esp blufi api.h, as shown in Table 7.7.

Table 7.7. APIs for BluFi components

API Description
esp_blufi_register_callbacks() Register BluFi callback events
esp_blufi_profile_init() Initialise BluFi profile
esp_blufi_profile_deinit() Deinitialise BluFi profile
esp_blufi_send_wifi_conn_report() Send Wi-Fi connection reports
esp_blufi_send_wifi_list() Send the Wi-Fi list
esp_blufi_get_version() Get the version of the current BluFi profile
esp_blufi_close() Disconnect the device
esp_blufi_send_error_info() Send BluFi error messages
esp_blufi_send_custom_data() Send custom data

(2) Program structure

• Wi-Fi event handling: taking care of Wi-Fi connection, disconnection, reconnection,
scanning, etc., as detailed in the sections before.
• NETIF event handling: acquiring IP address. Details are provided in the sections before.
• BluFi event handling: determined by the received request. BluFi events are shown in
Table 7.8.

Chapter 7. Wi-Fi Configuration and Connection 145

 Table 7.8. BluFi events

Event Description
Initialise BluFi features, name the device, and send specified
ESP_BLUFI_EVENT_INIT_FINISH
broadcast data

ESP_BLUFI_EVENT_DEINIT_FINISH Handle deinit configuration events

ESP_BLUFI_EVENT_BLE_CONNECT Connect to Bluetooth LE and put the device into safe mode

ESP_BLUFI_EVENT_BLE_DISCONNECT Set Bluetooth LE to disconnect and reconnect

ESP_BLUFI_EVENT_SET_WIFI_OPMODE Put ESP32-C3 into operating mode

Disconnect from the original Wi-Fi and connect to the speci-

ESP_BLUFI_EVENT_REQ_CONNECT_TO_AP
fied Wi-Fi

ESP_BLUFI_EVENT_REQ_DISCONNECT_FROM_AP Disconnect from the AP currently connected to ESP32-C3

ESP_BLUFI_EVENT_REPORT_ERROR Send error messages

Get Wi-Fi status, including the current Wi-Fi mode and

ESP_BLUFI_EVENT_GET_WIFI_STATUS
whether it is connected

ESP_BLUFI_EVENT_RECV_SLAVE_DISCONNECT_BLE Notify BluFi that the GATT connection is closed

ESP_BLUFI_EVENT_RECV_STA_BSSID Enter STA mode and get the BSSID of the target AP

ESP_BLUFI_EVENT_RECV_STA_SSID Enter STA mode and get the SSID of the target AP

ESP_BLUFI_EVENT_RECV_STA_PASSWD Enter STA mode and get the password of the target AP

ESP_BLUFI_EVENT_RECV_SOFTAP_SSID Enter SoftAP mode and get the custom AP SSID

ESP_BLUFI_EVENT_RECV_SOFTAP_PASSWD Enter SoftAP mode and get the custom AP password

Set the maximum number of connected devices in SoftAP

ESP_BLUFI_EVENT_RECV_SOFTAP_MAX_CONN_NUM
mode

ESP_BLUFI_EVENT_RECV_SOFTAP_AUTH_MODE Enter authentication mode in SoftAP mode

ESP_BLUFI_EVENT_RECV_SOFTAP_CHANNEL Set the channel in SoftAP mode

Obtain the SSID list, channel, and STA MAC address scanned
ESP_BLUFI_EVENT_GET_WIFI_LIST
over the air

ESP_BLUFI_EVENT_RECV_CUSTOM_DATA Print the received data and trim it to fit the application

• Main program: initialising Wi-Fi, initialising and enabling Bluetooth controller, initial-
ising and enabling Bluetooth protocol, obtaining Bluetooth address and BluFi version,
processing Bluetooth GAP events, and creating BluFi events.
(3) Functional verification
To get started, install EspBlufi on your phone. Turn on the Wi-Fi and power on the device.

146 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

You will see the output log by the serial port as follows:
I (516) phy_init: phy_version 500,985899c,Apr 19 2021,[Link]
I (696) wifi:set rx active PTI: 0, rx ack PTI: 12, and default PTI: 1
I (908) wifi:mode : sta ([Link])
I (908) wifi:enable tsf
W (706) BTDM_INIT: esp_bt_controller_mem_release not implemented, return OK
I (706) BTDM_INIT: BT controller compile version [9c99115]
I (716) coexist: coexist rom version 9387209
I (726) BTDM_INIT: Bluetooth MAC: [Link]
I (746) BLUFI_EXAMPLE: BD ADDR: [Link]
I (1198) BLUFI_EXAMPLE: BLUFI VERSION 0102
I (1198) BLUFI_EXAMPLE: BLUFI init finish

NOTE
You may download the APP at [Link]

a. Startup
Open the application on your phone and pull down to refresh. You will see the information
about nearby Bluetooth devices on the screen as shown in Figure 7.40.
b. Connection
Select the ESP32-C3 module BLUFI DEVICE to get details about the device. Tap “Connect”
to connect with Bluetooth. If connected, you will see the interface as Figure 7.41.

Figure 7.40. EspBlufi startup Figure 7.41. Bluetooth connected

The output log is as follows:

I (32736) BLUFI_EXAMPLE: BLUFI ble connect

Chapter 7. Wi-Fi Configuration and Connection 147

c. Provisioning
Tap “Provision network” in Figure 7.41 to enter the provisioning interface shown in Figure
7.42.
d. STA connection
Tap “OK” in Figure 7.42 to configure the network. If the configuration succeeds, you will see
the STA connected interface shown in Figure 7.43. Details about STA connection in Wi-Fi
mode will be displayed at the bottom of the screen, including the BSSID and SSID of the AP
and the connection status.

Figure 7.42. Provisioning Figure 7.43. STA connected

The output log is as follows:

I (63756) BLUFI_EXAMPLE: BLUFI Set WIFI opmode 1
I (63826) BLUFI_EXAMPLE: Recv STA SSID NETGEAR
I (63866) BLUFI_EXAMPLE: Recv STA PASSWORD 12345678
I (63936) BLUFI_EXAMPLE: BLUFI requset wifi connect to AP
I (65746) wifi:new: <8,2>, old: <1,0>, ap: <255,255>, sta: <8,2>, prof:1
I (66326) wifi:state: init -> auth (b0)
I (67326) wifi:state: auth -> init (200)
I (67326) wifi:new: <8,0>, old: <8,2>, ap: <255,255>, sta: <8,2>, prof:1
I (69516) wifi:new: <10,0>, old: <8,0>, ap: <255,255>, sta: <10,0>, prof:1
I (69516) wifi:state: init -> auth (b0)

148 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

I (69566) wifi:state: auth -> assoc (0)
I (69626) wifi:state: assoc -> run (10)
I (69816) wifi:connected with NETGEAR, aid = 1, channel 10, BW20, bssid = [Link]
[Link]
I (69816) wifi:security: WPA2-PSK, phy: bgn, rssi: -48
I (69826) wifi:pm start, type: 1
I (69826) wifi:set rx beacon pti, rx_bcn_pti: 14, bcn_timeout: 14, mt_pti: 25000,
mt_time: 10000
I (69926) wifi:BcnInt:102400, DTIM:1 W (70566) wifi:idx:0 (ifx:0, [Link]
7d), tid:0, ssn:2, winSize:64
I (71406) esp_netif_handlers: sta ip: [Link], mask: [Link], gw:
[Link]

7.5 Practice: Wi-Fi Configuration in Smart Light Project

In this section, we will start by programming for Wi-Fi connection based on the LED dimming
driver project, and then give an example of smart Wi-Fi configuration with the Smart Light
project.

7.5.1 Wi-Fi Connection in Smart Light Project

After learning the basics of Wi-Fi connection, we may put it into practice based on ESP32-C3,
and encapsulate the Wi-Fi features according to application requirements, so as to provide
APIs for Wi-Fi initialisation and Wi-Fi connection initialisation.
1. Driver initialisation
This API specifies parameters for ESP32-C3, such as GPIO pins, fading time, breathing cycle,
PWM frequency, clock source of the PWM controller, and PWM duty cycle resolution. For
details, please refer to Chapter 5.
1. app_driver_init();

2. NVS initialisation
Before initialising Wi-Fi, it is necessary to initialise the NVS library as the Wi-Fi component
needs to acquire and store certain parameters. The API is as follows:
1. nvs_flash_init();

3. Wi-Fi initialisation
This API handles LwIP and Wi-Fi events, and initialises Wi-Fi drivers.
1. wifi_initialize();

4. Wi-Fi connection initialisation

This API implements Wi-Fi configuration, starts the Wi-Fi driver, and waits for Wi-Fi connec-
tion to complete.
1. wifi_station_initialize();

Chapter 7. Wi-Fi Configuration and Connection 149

Source code
To code for Wi-Fi connection based on the LED dimming driver project, refer to
book-esp32c3-iot-projects/device firmware/3 wifi connection.

You may compile and run the code on the development board. The output is as follows:
I (397) wifi station: Application driver initialization
I (397) gpio: GPIO[9]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown:
0| Intr:0
I (427) wifi station: NVS Flash initialization
I (427) wifi station: Wi-Fi initialization
I (547) wifi station: Wi-Fi Station initialization
I (727) wifi station: wifi_station_initialize finished.
I (6427) wifi station: connected to ap SSID:espressif password:espressif
[00] Hello world!

7.5.2 Smart Wi-Fi Configuration

Now, we will turn to Wi-Fi configuration based on ESP32-C3. Similar to Wi-Fi connection,
we will encapsulate the smart Wi-Fi configuration features according to application require-
ments, in order to provide APIs for initialising smart Wi-Fi configuration.
After initialising the provisioning, the program will check its status. If the device has been
provisioned, the program will complete Wi-Fi connection using the router information; oth-
erwise, it will output a QR code for you to start provisioning.
1. wifi_prov_mgr_initialize();

To integrate the code for Bluetooth network configuration into the project in section 7.5.1,
please refer to book-esp32c3-iot-projects/device firmware/4 network config.
With the ESP BLE Provisioning App, you may compile and run the code on the development
board. The output is as follows.

NOTE
You may download the APP at [Link]

If the device has not been provisioned, you will see the log shown in Figure 7.44.
If the device has been provisioned, you will see the following log:
I (399) wifi station: Application driver initialization
I (399) gpio: GPIO[9]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown:
0| Intr:0
I (429) wifi station: NVS Flash initialization
I (429) wifi station: Wi-Fi initialization
I (549) wifi station: Wi-Fi Provisioning initialization
I (549) wifi station: Already provisioned, starting Wi-Fi STA
I (809) wifi station: wifi_station_initialize finished.
I (1939) wifi station: got ip:[Link]

150 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 7.44. Log output if device not provisioned

7.6 Summary
In this chapter, we first introduced two important technologies for network configuration
of IoT devices, Wi-Fi and Bluetooth. Then we covered some concepts and mechanisms
of Wi-Fi network configuration, including SoftAP, SmartConfig, Bluetooth, direct network
configuration, RouterConfig, ZeroConfig, and phone AP network configuration. We also
analysed the code for SoftAP, SmartConfig, and Bluetooth network configuration, combined
with Wi-Fi programming. Finally, we tried out smart Wi-Fi configuration with the Smart
Light project.

Chapter 7. Wi-Fi Configuration and Connection 151

 Chapter
Local Control
8

In Chapter 7, we learnt about basics of Wi-Fi and Bluetooth, as well as several common
network configuration methods. Through the introduction and practice in Chapter 7, you
should be able to configure devices and connect them to Wi-Fi routers. On this basis, this
chapter will introduce how to implement local control of devices based on Wi-Fi and Blue-
tooth, and realize local control with ESP32-C3. It is intended to explain the definition and
process of local control, along with some common local control communication protocols,
and help you to build your own local control framework for smart lights based on ESP32-C3.

8.1 Introduction to Local Control

This section first introduces what local control is and its usage conditions, scenarios, and
advantages. Then it will expound on the device discovery function and data communication
protocols involved in local control, and how to choose the data transmission medium for
local control. After reading this section, you will have a full understanding of local control
of devices.
As the name suggests, local control refers to operating controlled devices within a certain
distance through a series of methods such as hardware switches, touch buttons, infrared
remote control, smartphones, and computer networks. It is ubiquitous in our daily life,
such as setting air conditioners through infrared remote controls, controlling voice-activated
equipment through voice commands, and turning on household lighting through switches
or smartphone apps. The concept and technology of local control have become deeply
integrated into every aspect of our daily lives.
You may have noticed that some of the examples listed above are performed through hard-
ware circuit switches or wireless communication technologies such as infrared remote con-
trol, while others involve voice recognition, and data communication technologies such as
the Internet of Things. In this book, we will focuse on the data communication technology
of IoT, and help you build your own local control framework to control the ESP32-C3 smart
lights.
Within the Internet of Things, each device needs to transmit commands through certain data
communication method. Some common ways are as follows:

152
 • Using Wi-Fi or Ethernet. Generally, devices based on Wi-Fi and Ethernet natively run
the TCP/IP protocol stack, which greatly reduces the workload of protocol adaptation
and development. When performing local communication, they also need a gateway or
Wi-Fi router.
• Using short-range wireless communication technologies such as Bluetooth and Zig-
Bee, which is suitable for data transmission between low-speed and low-power devices.
According to the functional characteristics of ESP32-C3, we will introduce two commonly
used local control technologies in this chapter, namely through Wi-Fi or Bluetooth within a
LAN.
The network topology centered on Wi-Fi is shown in Figure 8.1. The command sending
devices (smartphone or PC) should be in the same LAN as the controlled device, and send
data to the device through Wi-Fi. But for Bluetooth control, there is no need of Wi-Fi routers,
as data can be directly transmitted between the smartphone and the controlled device via
Bluetooth.

Figure 8.1. Network topology centered on Wi-Fi within a LAN

Using Bluetooth is simpler than using Wi-Fi as Bluetooth does not require Wi-Fi routers.
However, in practice, if the IoT device wants to access cloud platforms, it needs Wi-Fi routers
to connect to the Internet and then the cloud platform. Moreover, smartphones are mostly
connected to Wi-Fi routers. Therefore, an LAN usually includes Wi-Fi routers, which makes
it convenient to use Wi-Fi for local control. If the IoT device does not need access to cloud
platforms, Bluetooth can be a good option for local control. You can select one of the
methods according to whether your device needs access to cloud platforms.
• If yes, it is recommended to choose Wi-Fi, as it supports multiple smartphones control-
ling one device at the same time, and its transmission bandwidth is larger than that
of Bluetooth. You can use Bluetooth only for network configuration, and then stop its
protocol stack to save ESP32-C3’s resources.

Chapter 8. Local Control 153

• If no, you can use Bluetooth instead of Wi-Fi to exchange data between the smartphone
and controlled device.

8.1.1 Application of Local Control

Most IoT devices are connected to cloud platforms which forward commands from smart-
phones to implement remote control. Such control depends on the Internet provided by
Wi-Fi routers to maintain the link between the controlled device and cloud platforms. But
once the Wi-Fi routers disconnect from the Internet, remote control will be paralyzed. At
this point, local control will be a good supplement for sending commands, thus preventing
the IoT devices from a full-out breakdown due to network exceptions.
As shown in Figure 8.1, a local control framework based on Wi-Fi in a LAN consists of a
Wi-Fi router, a controlling device, and a controlled device. The controlling device can be a
smartphone or a computer that can run TCP/IP protocol stack. It should be connected to
the same Wi-Fi router as the controlled device, to ensure that they are in the same LAN for
data communication.
As for local control based on Bluetooth, Wi-Fi routers are not needed. Smartphones can
directly connect to the controlled device through Bluetooth and realize point-to-point data
transmission.

8.1.2 Advantages of Local Control

Local control only requires data to be transmitted within the LAN instead of through the
Internet. Therefore, it functions with shorter delay and faster response. Moreover, as the
data only interacts within the LAN composed of the controlled device, a Wi-Fi router and
the controlling devices, there is lower probability of data being stolen or modified, hence
enhancing data privacy and security. In addition, local control saves Internet bandwidth. It
is not vulnerable to Internet disconnection. As long as the two parties are in the same LAN
or linked through Bluetooth, the control can be maintained. For some products without
access to cloud platforms, local control is the only means for smartphones to take charge of
them.
Considering these advantages, local control is increasingly favored by IoT companies. More
and more SDKs and products support local control functions. For example, the ESP Rain-
Maker, a complete IoT platform developed by Espressif, includes a smartphone app to im-
plement local control.

8.1.3 Discovering Controlled Devices through Smartphones

For local control based on Wi-Fi wireless transmission media, data runs on the TCP/IP pro-
tocol stack. The following two issues should be solved since the smartphone is not directly

154 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

connected to the controlled device. How does the smartphone find the controlled device,
and how does the smartphone communicate with the controlled device?
How does the smartphone find the controlled device, that is, how does the smartphone
know the IP address of the controlled device? Since all data is transmitted based on the IP
layer, obtaining the IP address of the controlled device is a prerequisite for subsequent data
communication. You may consider: “I can log in to the Wi-Fi router interface and check the
IP address of the controlled device directly on the Wi-Fi router interface, right?” Yes, you
can certainly obtain the IP address of the controlled device in this way. However, manually
querying IP addresses completely goes against the original intention of IoT technology to
bring convenience. Thus, a technology is required to automatically discover the controlled
device. This part will be discussed in detail in section 8.2.
For the local control frameworks based on Bluetooth control, you can learn from the Blue-
tooth scanning introduced in Chapter 7 that the Bluetooth of the controlled device will
broadcast its own Bluetooth information, and the smartphone only needs to scan the Blue-
tooth of the controlled device. Discovering the controlled device through Bluetooth is much
simpler than Wi-Fi. After the smartphone connects to the Bluetooth of the controlled device,
it can send data to the device. In addition, Bluetooth transmission does not depend on the
TPC/IP protocol stack, as it has its own transmission protocol. This part will be introduced
in detail in section 8.3.

8.1.4 Data Communication Between Smartphones and Devices

How does smartphones communicate with controlled devices?
When using Wi-Fi wireless transmission media, a smartphone can communicate with the
controlled device through TCP/IP protocol or UDP protocol after obtaining the IP address of
the controlled device. Generally speaking, as the receiver, the controlled device receives con-
trol commands sent by the smartphone in local control; and the smartphone,as the sender,
sends control commands to the controlled device. Therefore, the controlled device plays the
role of a server; and the smartphone plays the role of a client, allowing multiple clients to
send control commands to the server. This part will be introduced in detail in section 8.3.

8.2 Common Local Discovery Methods

In section 8.1.4, we mentioned how to discover controlled devices in the LAN using Wi-Fi
wireless transmission media. In the TCP/IP protocol stack, discovering the controlled device
means obtaining the IP address of the controlled device.
In a LAN, how to obtain the peer’s IP address is a problem worth studying. A common pro-
tocol for obtaining the peer’s IP address is the Reverse Address Resolution Protocol (RARP).
This protocol sends query packets with knowing the peer’s MAC address, and the gateway

Chapter 8. Local Control 155

server parses its own ARP table to obtain the IP address of the queried MAC device. If you
are familiar with LAN, you may immediately associate with the Address Resolution Protocol
(ARP), which is a protocol that sends query packets with knowing the peer’s IP address. The
peer device or gateway device replies with the MAC address corresponding to the IP address
after querying its own ARP table. ARP and RARP are a pair of protocols that mutually re-
solve network layer addresses and data link layer addresses. However, these two protocols
both need to know the peer’s network layer address or data link layer address. This feature
brings difficulty to IoT applications as the network layer address and data link layer address
of a device in the LAN are difficult to be obtained. Thus, this subsection will introduce the
local discovery technology that is truly suitable for IoT.
Local discovery is to discover information about nodes in the LAN, including address infor-
mation for communication with nodes, application service information supported by nodes,
and customised information. For example, mDNS (Multicast DNS, which will be introduced
in subsection 8.2.4) is a commonly-used local discovery protocol. The principle of local dis-
covery is to send a message, and the peer will inform the sender of its device information
after receiving the message. Now the problem that needs to be solved is how to ensure that
the peer can receive the message sent by the sender.
In fact, if you understand the classification of IP addresses, you will know that in addition
to the more commonly-used point-to-point communication (unicast), there are also one-to-
many (multicast) and one-to-all (broadcast) communications. IP addresses can be divided
into unicast addresses, multicast addresses, and broadcast addresses. Unicast needs to know
the peer’s IP address, so it is not suitable for local discovery. Multicast and broadcast do not
need to know the peer’s IP address. They send messages to specific addresses, and the peer
can receive the messages as long as it monitors this address. Thus, multicast and broadcast
are suitable for discovering devices in the LAN, and the peer can receive the message sent
by the sender with these two technologies.

8.2.1 Broadcast
Broadcast refers to sending messages to all possible receivers in the network. There are two
major applications of broadcast:
• Locating a host in the local network.
• Reducing the packet flow in the local network, so that one message can notify all hosts
in the local network.
Common broadcast application messages include:
Address Resolution Protocol (ARP)
It can be used to broadcast an ARP request in the local network: “Please tell me what the
MAC address of the device with IP address a.b.c.d is”. ARP broadcast is MAC broadcast on

156 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 layer 2 (link layer), not the IP broadcast on layer 3 (network layer).
Dynamic Host Configuration Protocol (DHCP)
If there is a DHCP server on the local network, the DHCP client sends a DHCP request for
the destination IP address (usually [Link]), and the DHCP server on the same
network can receive the request and reply with an assigned IP address.
Broadcast mainly uses the UDP protocol (see subsection 8.3.3 for details) instead of the TCP
protocol (see subsection 8.3.1 for details) as it is more suitable for unicast.

1. Broadcast addresses

Broadcast addresses include MAC broadcast addresses on layer 2 (link layer) ([Link]
FF:FF) and IP broadcast addresses on layer 3 (network layer) ([Link]), here-
inafter referred to as layer 2 addresses and layer 3 addresses. This section mainly introduces
layer 3 addresses. Generally, when the layer 3 address of a message is all 255, the layer 2
address is usually all FF. This is because a message with a layer 3 address full of 255 means
that all devices in the local network will receive this message. If the layer 2 address of the
message is not all FF, the message will be discarded during the layer 2 address processing
of the receiving device. For the receiving device, if the layer 2 address of the message is
not a broadcast address, nor the local MAC address and multicast MAC address (such as
[Link]XX:XX:XX), it will be discarded and not processed. Therefore, generally, if the
layer 3 address is a broadcast address, so is the layer 2 address.
IPv4 addresses consist of a subnet ID and a host ID. For example, for a device with an
IP address of [Link] and a subnet mask of [Link], its subnet ID and host
ID are calculated from the IP address and subnet mask. In this example, the subnet ID is
[Link] and the host ID is 4. When the subnet ID and host ID are all 255, it is a broadcast
address; it is also a broadcast address when only the host ID is all 255. For example, if you
have a subnet of 192.168.1/24, then [Link] is the broadcast address of this subnet.
You may wonder, what is the difference between a broadcast address with a subnet ID and
host ID of all 255 and a broadcast address with only a host ID of 255?
The broadcast range of the first address is larger than that of a specific subnet. For example,
a Wi-Fi router has two subnets, 192.168.1/24 and 192.168.2/24. If a host with IP ad-
dress of [Link] in the subnet 192.168.1/24 sends a message to the destination address
[Link], the Wi-Fi router will only forward the message to the host in the subnet
192.168.1/24, and will not forward it to the host in the subnet 192.168.2/24. If the host
sends a message to the destination address [Link], the Wi-Fi router will forward
the message to hosts in both subnets. Therefore, the broadcast address with a host ID of
255 is also called a subnet-directed broadcast address. By using a subnet-directed broadcast

Chapter 8. Local Control 157

address, you can send messages to a specified subnet, so that these messages will not be
sent to the subnets that do not need them in the LAN, thus saving network resources.

2. Implementing a broadcast sender using socket

Source code
For the source code of the function esp send broadcast(), please refer to book-esp
32c3-iot-projects/test case/broadcast discovery.

The function esp_send_broadcast() sends UDP broadcast packets with the data “Are
you Espressif IOT Smart Light” to the LAN, and then waits for the peer to reply. This function
uses the standard interface of Berkeley sockets, also known as BSD socket. Berkeley socket
is a common network interface in UNIX systems, which not only supports different network
types, but also is a communication mechanism between internal processes. The TCP/UDP
network programming covered in this book applies Berkeley sockets. If you are interested,
you can read the UNIX Network Programming (Volume 1): Socket Networking API published
by Posts & Telecommunications Press to learn more about Berkeley socket programming. In
this book, we will only briefly introduces how to use socket programming.
In this section, we will introduce how to use socket(AF_INET, SOCK_DGRAM, 0) to
create a UDP socket, and then use setsockopt() to enable socket support for broadcast-
ing. Then we will set the destination address for broadcasting to all 255 and the port to
3333, and call sendto()to send the message. You can determine whether the data is sent
successfully according to the return value of the sendto() function. The code is as below:
1. esp_err_t esp_send_broadcast(void)
2. {
3. int opt_val = 1;
4. esp_err_t err = ESP_FAIL;
5. struct sockaddr_in from_addr = {0};
6. socklen_t from_addr_len = sizeof(struct sockaddr_in);
7. char udp_recv_buf[64 + 1] = {0};
8.
9. //Create an IPv4 UDP socket
10. int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
11. if (sockfd == -1) {
12. ESP_LOGE(TAG, "Create UDP socket fail");
13. return err;
14. }
15.
16. //Set SO_BROADCAST socket option, and use it to send broadcast
17. int ret = setsockopt(sockfd, SOL_SOCKET, SO_BROADCAST, &opt_val,
18. sizeof(int));
19. if (ret < 0) {

158 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

20. ESP_LOGE(TAG, "Set SO_BROADCAST option fail");
21. goto exit;
22. }
23.
24. //Set broadcast destination address and port
25. struct sockaddr_in dest_addr = {
26. .sin_family = AF_INET,
27. .sin_port = htons(3333),
28. .sin_addr.s_addr = htonl(INADDR_BROADCAST),
29. };
30.
31. char *broadcast_msg_buf = "Are you Espressif IOT Smart Light";
32.
33. //Call sendto() to send broadcast data
34. ret = sendto(sockfd, broadcast_msg_buf, strlen(broadcast_msg_buf), 0,
35. (struct sockaddr *)&dest_addr,
36. sizeof(struct sockaddr));
37. if (ret < 0) {
38. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
39. } else {
40. ESP_LOGI(TAG, "Message sent successfully");
41. ret = recvfrom(sockfd, udp_recv_buf, sizeof(udp_recv_buf) - 1, 0,
42. (struct sockaddr *)&from_addr,
43. (socklen_t *)&from_addr_len);
44. if (ret > 0) {
45. ESP_LOGI(TAG, "Receive udp unicast from %s:%d, data is %s",
46. inet_ntoa (((struct sockaddr_in *)&from_addr)->sin_addr),
47. ntohs(((struct sockaddr_in *)& from_addr)->sin_port),
48. udp_recv_buf);
49. err = ESP_OK;
50. }
51. }
52. exit:
53. close(sockfd);
54. return err;
55. }

3. Implementing a broadcast receiver using socket

Source code
For the source code of the function esp receive broadcast(), please refer to book-
esp32c3-iot-projects/test case/broadcast discovery.

The function esp_receive_broadcast() implements reception of broadcast packets

and unicast replies. The implementation logic of a receiver is same as that of the sender.

Chapter 8. Local Control 159

First create a UDP socket, and set the source address and port number of the message to be
listened. Generally, it is used as a server. The source address of the message is set to [Link],
which means that the source address of the message is not verified. Call bind() to bind
the socket, and then use recvfrom() to receive the message. When a broadcast packet
carrying the data “Are you Espressif IOT Smart Light” is received, the IP address and port
number of the peer are saved in from_addr, which will be sent to the peer in the form of
unicast. The code is as below:
1. esp_err_t esp_receive_broadcast(void)
2. {
3. esp_err_t err = ESP_FAIL;
4. struct sockaddr_in from_addr = {0};
5. socklen_t from_addr_len = sizeof(struct sockaddr_in);
6. char udp_server_buf[64+1] = {0};
7. char *udp_server_send_buf = "ESP32-C3 Smart Light https 443";
8.
9. //Create an IPv4 UDP socket
10. int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
11. if (sockfd == -1) {
12. ESP_LOGE(TAG, "Create UDP socket fail");
13. return err;
14. }
15.
16. //Set broadcast destination address and port
17. struct sockaddr_in server_addr = {
18. .sin_family = AF_INET,
19. .sin_port = htons(3333),
20. .sin_addr.s_addr = htonl(INADDR_ANY),
21. };
22.
23. int ret = bind(sockfd, (struct sockaddr *)&server_addr,
24. sizeof(server_addr));
25. if (ret < 0) {
26. ESP_LOGE(TAG, "Bind socket fail");
27. goto exit;
28. }
29.
30. //Call recvfrom()to receive broadcast data
31. while (1) {
32. ret = recvfrom(sockfd, udp_server_buf, sizeof(udp_server_buf) - 1, 0,
33. (struct sockaddr *)&from_addr,
34. (socklen_t *)&from_addr_len);
35. if (ret > 0) {
36. ESP_LOGI(TAG, "Receive udp broadcast from %s:%d, data is %s",
37. inet_ntoa (((struct sockaddr_in *)&from_addr)->sin_addr),
38. ntohs(((struct sockaddr_in *)& from_addr)->sin_port),

160 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

39. udp_server_buf);
40. //Upon reception of broadcast request, send data communication port of peer through unicast
41. if (!strcmp(udp_server_buf, "Are you Espressif IOT Smart Light")){
42. ret = sendto(sockfd, udp_server_send_buf, strlen(udp_server_send_buf),
43. 0, (struct sockaddr *)&from_addr, from_addr_len);
44. if (ret < 0) {
45. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
46. } else {
47. ESP_LOGI(TAG, "Message sent successfully");
48. }
49. }
50. }
51. }
52. exit:
53. close(sockfd);
54. return err;
55. }

4. Running result

Add the sender and receiver code to the Wi-Fi Station example to ensure they are connected
to the same Wi-Fi router. The log of broadcast sending is as follows:
I (774) wifi:mode : sta ([Link])
I (774) wifi: enable tsf
I (774) wifi station: wifi_init_sta finished
I (784) wifi:new: <6,0>, old: <1,0>, ap: <255,255>, sta: <6,0>, prof:1
I (794) wifi:state: auth -> assoc (0)
I (814) wifi:state: assoc -> run (10)
I (834) wifi: connected with myssid, aid = 1, channel 6, BW20, bssid = [Link]
[Link]
I (834) wifi:security: WPA2-PSK, phy: bgn, rssi: -23
I (834) wifi: pm start, type: 1
I (884) wifi: AP’s beacon interval = 102400 us, DTIM period = 1
I (1544) esp netif handlers: sta ip: [Link], mask: [Link], gw: 192.
168.3.1
I (1544) wifi station: got ip:[Link] I (1544) wifi station: connected to ap
SSID: myssid password: 12345678
I (1554) wifi station: Message sent successfully
I (1624) wifi station: Receive udp unicast from [Link]:3333, data is ESP32
-C3 Smart Light https 443

The log of broadcast reception is as follows:

I (1450) wifi:new: <6,0>, old: <1,0>, ap: <255,255>, sta: <6,0>, prof:1
I (2200) wifi:state: init -> auth (b0)
I (2370) wifi:state: auth -> assoc (0)
I (2380) wifi:state: assoc -> run (10)
I (2440) wifi: connected with myssid, aid = 2, channel 6, BW20, bssid = [Link]

Chapter 8. Local Control 161

[Link]
I (2450) wifi:security: WPA2-PSK, phy: bgn, rssi: -30
I (2460) wifi: pm start, type: 1
I (2530) wifi: AP’s beacon interval = 102400 us, DTIM period = 1
I (3050) esp_netif_handlers: sta ip: [Link], mask: [Link], gw: 192.
168.3.1
I (3050) wifi station: got ip:[Link]
I (3050) wifi station: connected to ap SSID: myssid password: 12345678
W (17430) wifi: <ba-add>idx:0 (ifx:0, [Link]), tid:5, ssn:0,
winSize:64
I (26490) wifi station: Receive udp broadcast from [Link]:60520, data is
Are you Espressif IOT Smart Light
I (26500) wifi station: Message sent successfully
I (382550) wifi station: Receive udp broadcast from [Link]:63439, data is
Are you Espressif IOT Smart Light
I (382550) wifi station: Message sent successfully

The log of broadcast sending indicates that the sender sent a UDP broadcast packet with
data “Are you Espressif IOT Smart Light”. The broadcast receiving log indicates that the
receiver listens to the broadcast packet of the local network and replies with a unicast packet
carrying the data “ESP32-C3 Smart Light https 443” upon receiving a packet carrying “Are
you Espressif IOT Smart Light”. In this way, local devices can be discovered. After receiving
the unicast reply from the receiver, the sender can confirm the IP address of the peer and
know the application protocol and port number for subsequent data communication from
the carried data.
The broadcast protocol of the local network can complete the device discovery function.
However, broadcasting the discovery request to all devices on the local network will cause a
certain burden on the local network and host. Therefore, discovering devices by broadcast-
ing is not a good choice.

8.2.2 Multicast
Multicast refers to sending messages to interested receivers. Compared with the two “ex-
tremes” of unicast and broadcast addressing schemes (either one or all), multicast provides
a compromise solution. As the name implies, multicast mainly emphasises the concept of
groups, that is, a host can send a message to a group address, and all hosts that join this
group can receive the message. This is somewhat like subnet-directed broadcast, but more
flexible than it, because a host can join or leave a certain group at any time, thus reducing
the burden on the local network and hosts.
Internet Group Management Protocol (IGMP) is a protocol responsible for managing IP mul-
ticast members, which is used to establish and maintain multicast group membership rela-
tionship between an IP host and its directly adjacent multicast Wi-Fi routers. For multicast,

162 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Wi-Fi routers need to support the IGMP protocol.

1. Multicast addresses

The destination addresses of multicast messages use a class D IP address. The first byte
starts with binary numbers 1110, and it ranges from [Link] to [Link]. Since
the multicast IP address identifies a group of hosts, the multicast IP address can only be used
as the destination address, not the source address, which is always a unicast address.
A multicast group is a group identified by a specific multicast address. When members inside
or outside the group send a message to this multicast address, group members identified
by the multicast address can receive the message. Multicast groups can be permanent or
temporary. Among multicast addresses, multicast addresses officially assigned are called
permanent multicast groups, while those that are neither reserved nor permanent are called
temporary multicast groups. The numbers of hosts in permanent and temporary multicast
groups are dynamic and may even be zero.
Multicast addresses are classified as follows:
• [Link] ⇠ [Link]: Reserved multicast addresses (permanent multicast groups).
The address [Link] is not allocated, and the others are used for routing protocols.
• [Link] ⇠ [Link]: Public multicast addresses, which can be used on the Internet.
• [Link] ⇠ [Link]: Multicast addresses available to users (temporary multi-
cast groups), which are valid throughout the network.
• [Link] ⇠ [Link]: Multicast addresses for local management, which are
valid only within specific local ranges.

2. Implementing a multicast sender using socket

Source code
For the source code of the function esp join multicast group(), please refer to
book-esp32c3-iot-projects/test case/multicast discovery.

The implementation of multicast sending is more complex than that of broadcast sending.
Multicast sending requires setting the sending interface of the multicast packets. If you
need to receive packets from a certain multicast group, you also need to join the multicast
group. The function esp_join_multicast_group() implements the setting of the mul-
ticast group sending interface and the function of joining the multicast group. The function
esp_send_multicast() implements the creation, binding, configuration of destination
address and port of regular UDP sockets, and sending and receiving functions. In addition,
TTL settings are also added to ensure that the multicast group can only be performed in the
LAN of this route. The code is as below:

Chapter 8. Local Control 163

1. #define MULTICAST_IPV4_ADDR "[Link]"
2. int esp_join_multicast_group(int sockfd)
3. {
4. struct ip_mreq imreq = { 0 };
5. struct in_addr iaddr = { 0 };
6. int err = 0;
7.
8. //Configure sending interface of multicast group
9. esp_netif_ip_info_t ip_info = { 0 };
10. err = esp_netif_get_ip_info(esp_netif_get_handle_from_ifkey("WIFI_STA_DEF"),
11. &ip_info);
12. if (err ! = ESP_OK) {
13. ESP_LOGE(TAG, "Failed to get IP address info. Error 0x%x", err);
14. goto err;
15. }
16. inet_addr_from_ip4addr(&iaddr, &ip_info.ip);
17. err = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_IF, &iaddr, sizeof(struct in_addr));
18. if (err < 0) {
19. ESP_LOGE(TAG, "Failed to set IP_MULTICAST_IF. Error %d", errno);
20. goto err;
21. }
22.
23. //Configure the address of monitoring multicast group
24. inet_aton(MULTICAST_IPV4_ADDR, &imreq.imr_multiaddr.s_addr);
25.
26. //Configure the socket to join the multicast group
27. err = setsockopt(sockfd, IPPROTO_IP, IP_ADD_MEMBERSHIP,
28. &imreq, sizeof(struct ip_mreq));
29. if (err < 0) {
30. ESP_LOGE(TAG, "Failed to set IP_ADD_MEMBERSHIP. Error %d", errno);
31. }
32. err:
33. return err;
34. }
35.
36. esp_err_t esp_send_multicast(void)
37. {
38. esp_err_t err = ESP_FAIL;
39. struct sockaddr_in saddr = {0};
40. struct sockaddr_in from_addr = {0};
41. socklen_t from_addr_len = sizeof(struct sockaddr_in);
42. char udp_recv_buf[64 + 1] = {0};
43.
44. //Create an IPv4 UDP socket
45. int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
46. if (sockfd == -1) {
47. ESP_LOGE(TAG, "Create UDP socket fail");

164 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

48. return err;
49. }
50.
51. //Bind socket
52. saddr.sin_family = PF_INET;
53. saddr.sin_port = htons(3333);
54. saddr.sin_addr.s_addr = htonl(INADDR_ANY);
55. int ret = bind(sockfd, (struct sockaddr *)&saddr, sizeof(struct sockaddr_in));
56. if (ret < 0) {
57. ESP_LOGE(TAG, "Failed to bind socket. Error %d", errno);
58. goto exit;
59. }
60.
61. //Set multicast TTL to 1, limiting the multicast packet to one route
62. uint8_t ttl = 1;
63. ret = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_TTL, &ttl, sizeof(uint8_t));
64. if (ret < 0) {
65. ESP_LOGE(TAG, "Failed to set IP_MULTICAST_TTL. Error %d", errno);
66. goto exit;
67. }
68.
69. //Join the multicast group
70. ret = esp_join_multicast_group(sockfd);
71. if (ret < 0) {
72. ESP_LOGE(TAG, "Failed to join multicast group");
73. goto exit;
74. }
75.
76. //Set multicast destination address and port
77. struct sockaddr_in dest_addr = {
78. .sin_family = AF_INET,
79. .sin_port = htons(3333),
80. };
81. inet_aton(MULTICAST_IPV4_ADDR, &dest_addr.sin_addr.s_addr);
82. char *multicast_msg_buf = "Are you Espressif IOT Smart Light";
83.
84. //Call sendto() to send multicast data
85. ret = sendto(sockfd, multicast_msg_buf, strlen(multicast_msg_buf), 0,
86. (struct sockaddr *)&dest_addr, sizeof(struct sockaddr));
87. if (ret < 0) {
88. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
89. } else {
90. ESP_LOGI(TAG, "Message sent successfully");
91. ret = recvfrom(sockfd, udp_recv_buf, sizeof(udp_recv_buf) - 1, 0,
92. (struct sockaddr *)&from_addr,
93. (socklen_t *)&from_addr_len);
94. if (ret > 0) {

Chapter 8. Local Control 165

95. ESP_LOGI(TAG, "Receive udp unicast from %s:%d, data is %s",
96. inet_ntoa(((struct sockaddr_in *)&from_addr)->sin_addr),
97. ntohs(((struct sockaddr_in *)&from_addr)->sin_port),
98. udp_recv_buf);
99. err = ESP_OK;
100. }
101. }
[Link]:
103. close(sockfd);
104. return err;
105.}

3. Implementing a multicast receiver using socket

Source code
For the source code of the function esp recv multicast(), please refer to book-esp
32c3-iot-projects/test case/multicast discovery.

Implementing a multicast receiver is similar to implementing a multicast sender, which re-

quires specifying the interface of multicast packets and the multicast group to be joined.
The function esp_recv_multicast() implements the creation, binding, configuration of
destination address and port of regular UDP sockets, and sending and receiving functions.
In addition, since multicast needs to be sent in this example, Time To Live (TTL) is set. The
code is as below:
1. esp_err_t esp_recv_multicast(void)
2. {
3. esp_err_t err = ESP_FAIL;
4. struct sockaddr_in saddr = {0};
5. struct sockaddr_in from_addr = {0};
6. socklen_t from_addr_len = sizeof(struct sockaddr_in);
7. char udp_server_buf[64+1] = {0};
8. char *udp_server_send_buf = "ESP32-C3 Smart Light https 443";
9.
10. //Create an IPv4 UDP socket
11. int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
12. if (sockfd == -1) {
13. ESP_LOGE(TAG, "Create UDP socket fail");
14. return err;
15. }
16.
17. //Bind socket
18. saddr.sin_family = PF_INET;
19. saddr.sin_port = htons(3333);
20. saddr.sin_addr.s_addr = htonl(INADDR_ANY);
21. int ret = bind(sockfd, (struct sockaddr *)&saddr, sizeof(struct sockaddr_in));

166 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

22. if (ret < 0) {
23. ESP_LOGE(TAG, "Failed to bind socket. Error %d", errno);
24. goto exit;
25. }
26.
27. //Set multicast TTL to 1, limiting the multicast packet to one route
28. uint8_t ttl = 1;
29. ret = setsockopt(sockfd, IPPROTO_IP, IP_MULTICAST_TTL, &ttl, sizeof(uint8_t));
30. if (ret < 0) {
31. ESP_LOGE(TAG, "Failed to set IP_MULTICAST_TTL. Error %d", errno);
32. goto exit;
33. }
34.
35. //Join the multicast group
36. ret = esp_join_multicast_group(sockfd);
37. if (ret < 0) {
38. ESP_LOGE(TAG, "Failed to join multicast group");
39. goto exit;
40. }
41.
42. //Call recvfrom() to receive multicast data
43. while (1) {
44. ret = recvfrom(sockfd, udp_server_buf, sizeof(udp_server_buf) - 1, 0,
45. (struct sockaddr *)&from_addr,
46. (socklen_t *)&from_addr_len);
47. if (ret > 0) {
48. ESP_LOGI(TAG, "Receive udp multicast from %s:%d, data is %s",
49. inet_ntoa (((struct sockaddr_in *)&from_addr)->sin_addr),
50. ntohs(((struct sockaddr_in *)& from_addr)->sin_port),
51. udp_server_buf);
52. //Upon reception of multicast request, send data communication port of peer through unicast
53. if (!strcmp(udp_server_buf, "Are you Espressif IOT Smart Light")) {
54. ret = sendto(sockfd, udp_server_send_buf, strlen(udp_server_send_buf),
55. 0, (struct sockaddr *)&from_addr, from_addr_len);
56. if (ret < 0) {
57. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
58. } else {
59. ESP_LOGI(TAG, "Message sent successfully");
60. }
61. }
62. }
63. }
64. exit:
65. close(sockfd);
66. return err;
67. }

Chapter 8. Local Control 167

4. Running result

Add the sender and receiver code to the Wi-Fi Station example to ensure they are connected
to the same Wi-Fi router. The log of multicast sending is as follows:
I (752) wifi :mode : sta ([Link])
I (752) wifi:enable tsf
I (752) wifi station: wifi init sta finished.
I (772) wifi:new: <6,0>, old: <1,0>, ap: <255,255>, sta: <6,0>, prof:1
I (772) wifi:state: init -> auth (b0)
I (792) wifi:state: auth -> assoc (0)
I (802) wifi:state: assoc -> run (10)
I (822) wifi:connected with myssid, aid = 2, channel 6, BW20, bssid = [Link]
[Link]
I (822) wifi:security: WPA2-PSK, phy: bgn, rssi: -17
I (822) wifi: pm start, type: 1
I(882) wifi:AP’s beacon interval = 102400 us, DTIM period = 1
I (1542) esp_netif_handlers: sta ip: [Link], mask: [Link], gw: 192.
168.3.1
I (1542) wifi station: got ip:[Link] I (1542) wifi station: connected to ap
SSID: myssid password: 123456 8
I (1552) wifi station: Message sent successfully
I (1632) wifi station: Receive udp unicast from [Link]:3333, data is ESP32
-C3 Smart Light https 443

The log of multicast reception is as follows:

I (806) wifi:state: init -> auth (b0)
I (816) wifi:state: auth -> assoc (0)
I (836) wifi:state: assoc -> run (10)
I (966) wifi:connected with myssid, aid = 1, channel 6, BW20, bssid = [Link]
[Link]
I (966) wifi:security: WPA2-PSKI phy: bgn, rssi: -29
I (976) wifi:pm start, type: 1
I (1066) wifi:AP’s beacon interval = 102400 us, DTIM period = 1
I (2056) esp_netif_handlers: sta ip: [Link], mask: [Link], gw: 192.
168.3.1
I (2056) wifi station: got ip:[Link]
I (2056) wifi station: connected to ap SSID: myssid password: 12345678
W (18476) wifi: <ba-add>idx:0 (ifx:0, [Link]), tid:0, ssn:4, winSize: 64
W (23706) wifi: <ba-add>idx:1 (ifx:0, [Link]), tid:5, ssn:0, winSize: 64
I (23706) wifi station: Receive udp multicast from [Link]:3333, data is Are
you Espressif IOT Smart Light

Similar to broadcasting logs, the sender sends packets of specific data, and the receiver
informs the sender of the application protocol and port number of data communication.

168 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

8.2.3 Comparison Between Broadcast and Multicast
The comparison between broadcast and multicast is shown in Table 8.1. It can be seen
that multicast has smaller bandwidth overhead, and devices in the LAN can join or leave
multicast groups of interest or pre-determined to receive and send data, which is more
flexible. For broadcast, all devices in the LAN will receive the packet, which will increase
burden on other devices in the LAN and also increase burden on the LAN bandwidth.

Table 8.1. Comparison between broadcast and multicast

Comparison Broadcast Multicast

Packets are sent to all hosts Packets are sent only to their
Principle
connected to the network. intended recipients in the network.

Transmission One-to-all One-to-many

Management No need for group management Need group management

May cause network band-

Network Controllable network bandwidth
width waste and congestion

Rate Slow Fast

8.2.4 Multicast Application Protocol mDNS for Local Discovery

In computer networks, the Multicast DNS (mDNS) protocol resolves host names to IP ad-
dresses in small networks that do not include local name servers. This is a zero-configuration
server. mDNS has basically the same programming interface, packet format, and operation
mode as the traditional domain name system (DNS).
mDNS was first proposed by Bill Woodcock and Bill Manning in the IETF in 2000. It was
finally published as a standard protocol in RFC 6762 by Stuart Cheshire and Marc Krochmal
in 2013, and implemented by Apple Bonjour and the open source Avahi software packages.
It is included in most Linux distributions (excerpted from Wikipedia).

1. Introduction to mDNS protocol

mDNS is a domain name resolution protocol for local networks, which uses port 5353 and
multicast address [Link]. It is an application protocol running on UDP. Unlike tra-
ditional DNS protocols, mDNS does not require a DNS server to perform domain name
resolution, which can avoid configuratingdomain name servers on local networks.
After a host with mDNS service enabled joins a LAN, it will first multicast a message to the
multicast address [Link] of the LAN, “Who am I? What is my IP address? What are the

Chapter 8. Local Control 169

services and port numbers I provide?”. After receiving the message, other hosts with mDNS
service enabled on the LAN will record the message and respond with “Who is it? What is
its IP address? What is the service and port number it provides?”. If a host wants to query
the mDNS domain name, it will first query its own cache information. If it is not found, it
will multicast a query to the LAN to ask for the IP address, services, and port numbers of the
domain name.
Then how can a host distinguish whether a domain name is from DNS or mDNS when
querying a domain name?
mDNS domain names differ from DNS domain names by the suffix “.local”.

2. Using mDNS component based on ESP-IDF

NOTE: mDNS component

ESP-IDF provides the mDNS component, which helps you develop applications. You may
refer to the mDNS service in the ESP-IDF Programming Guide for relevant interfaces.
For mDNS component, please visit [Link]
components/mdns. For mDNS service, please visit [Link]

This section mainly introduces how to use the mDNS component for developing devices to
be discovered.
1. esp_err_t esp_mdns_discovery_start(void)
2. {
3. char *host_name = "my_smart_light";
4. char *instance_name = "esp32c3_smart_light";
5.
6. //Initialise the mDNS component
7. if (mdns_init() ! = ESP_OK) {
8. ESP_LOGE(TAG, "mdns_init fail");
9. return ESP_FAIL;
10. }
11.
12. //Set host name (the DNS domain name tag to be queried by other hosts)
13. if (mdns_hostname_set(host_name) ! = ESP_OK) {
14. ESP_LOGE(TAG, "mdns_hostname_set fail");
15. goto err;
16. }
17. ESP_LOGI(TAG, "mdns hostname set to: [%s]", host_name);
18.
19. //Set mDNS instance name to be discovered by mDNS LAN
20. if (mdns_instance_name_set(instance_name) ! = ESP_OK) {
21. ESP_LOGE(TAG, "mdns_instance_name_set fail");
22. goto err;

170 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

23. }
24.
25. //Set service TXT field data (optional)
26. mdns_txt_item_t serviceTxtData[1] = {
27. {"board", "esp32c3"}
28. };
29.
30.//Add HTTP service; port 80 corresponds to mDNS service. The second parameter (application layer
[Link]) and the third parameter (transport layer protocol) need to correspond to each other.
32. if (mdns_service_add(instance_name, "_http", "_tcp", 80,
33. serviceTxtData, 1) ! = ESP_OK) {
34. ESP_LOGE(TAG, "mdns_instance_name_set fail");
35. goto err;
36. }
37.
38. //Set service TXT field data
39. if (mdns_service_txt_item_set("_http", "_tcp", "path", "/foobar") ! =ESP_OK){
40. ESP_LOGE(TAG, "mdns_service_txt_item_set fail");
41. goto err;
42. }
43. return ESP_OK;
44. err:
45. mdns_free();
46. return ESP_FAIL;
47. }

The above code implements the mDNS service with the domain name my_smart_light
and the node esp32c3_smart_light. Your other hosts can query the node esp32c3_
smart_light through the mDNS service. The smart light host will reply with its own
domain name (my_smart_light), corresponding IP address, provided service (HTTP),
corresponding server port (80), and TXT node field (path=/foobar board=esp32c3).

Source code
For complete code of the example, please refer to book-esp32c3-iot-projects/
test case/mdns discovery.

You can use the Windows command dns-sd -L esp32c3_smart_light_http to query

the information of the host in the LAN. The command is as follows:
c:\Users> dns-sd -L esp32c3 smart light http
Lookup esp32c3_smart_light._http._tcp.local
[Link].682 esp32c3_smart_light._http._tcp.local. can be reached at my_smart_
[Link].:80 (interface 6)
path=/foobar board=esp32c3

Chapter 8. Local Control 171

NOTE: “Bonjour”
“Bonjour” is a network configuration software that supports zero-configuration network-
ing service and can automatically discover computers, devices, and services on the IP
network. It needs to be installed before using the command dns-sd. You can download
Bonjour at [Link]

You can also use the Linux command avahi-browse -a --resolve to query service
information of all mDNS hosts in the LAN. The command is as follows:
# avahi-browse -a --resolve
= enp1s0 IPv4 esp32c3_smart_light Web Site local
hostname = [my_smart_light.local]
address = [[Link]]
port = [80]
txt = ["board=esp32c3" "path=/foobar"]

8.3 Common Communication Protocols for Local Data

After introducing how to discover devices in the LAN, this section will introduce how to
control the devices. Taking the smart light as an example, the simplest control is to turn the
smart light on and off, which is essentially the GPIO pin level being pulled high or low at the
software level. Controlling the on and off of the smart light through other devices is nothing
more than providing commands to perform GPIO operations. So how are these commands
sent from a smartphone to the smart light? What is the format of these commands? What
protocols are used? This section will answer these questions one by one. This section
mainly introduces the transmission of data conforming to the TCP/IP protocol through Wi-
Fi wireless transmission media, and the transmission of data conforming to the Bluetooth
data communication protocol through the Bluetooth wireless transmission media.

8.3.1 Transmission Control Protocol (TCP)

TCP is one of the major protocols in the Internet protocol family. In the TCP/IP model, TCP
serves as the transport layer protocol, providing reliable data transmission for application
layer protocols such as HTTP, MQTT, FTP, etc. The TCP/IP model is shown in Figure 8.2.

1. Introduction to TCP

TCP is a connection-oriented, reliable, byte-stream-based communication protocol at the

transport layer, defined by RFC 793 of IETF.
• Connection-oriented. Before sending data using TCP, a connection must be established
between the sender and receiver, which is commonly referred to as a three-way hand-
shake.

172 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 8.2. TCP/IP model

• Reliable. When sending data using TCP, the receiver’s receipt can be guaranteed. If data
is lost, the lost data will be retransmitted. TCP can also ensure that the receiver receives
data in order.
• Byte-stream-based. When sending data using TCP, the application layer data is first
written into the TCP buffer. Then, TCP controls the transmission of data in a byte-
stream-based manner, which is independent of the length of the message written by the
application layer. Therefore, it is a byte-stream-based protocol.
The process of TCP sending upper-layer application data to the receiver is as follows:
(1) The upper-layer application program writes the application data into the TCP buffer.
(2) The TCP buffer packages the data into a TCP message and sends it to the network
layer.
(3) The receiver receives the TCP message and puts it into the TCP buffer.
(4) After a certain amount of data is received, the data is sorted and reorganised before
being reported to the application layer.
The process of sending and receiving data using TCP is shown in Figure 8.3.

2. Creating a TCP server using socket

Source code
For the source code of the function esp create tcp server(), please refer to book-
esp32c3-iot-projects/test case/tcp socket.

The function esp_create_tcp_server() can create a TCP server, including creating a

TCP socket, configuring and binding the port, listening, receiving data, and sending data.
Compared with TCP clients, UDP servers and clients, the code flow of TCP servers is more

Chapter 8. Local Control 173

 Figure 8.3. Data sending and receiving process using TCP

complicated, and involves two socket functions, listen and accept, which are unique to
TCP servers. The code is as below:
1. esp_err_t esp_create_tcp_server(void)
2. {
3. int len;
4. int keepAlive = 1;
5. int keepIdle = 5;
6. int keepInterval = 5;
7. int keepCount = 3;
8. char rx_buffer[128] = {0};
9. char addr_str[32] = {0};
10. esp_err_t err = ESP_FAIL;
11. struct sockaddr_in server_addr;
12.
13. //Create a TCP socket
14. int listenfd = socket(AF_INET, SOCK_STREAM, 0);
15. if (listenfd < 0) {
16. ESP_LOGE(TAG, "create socket error");
17. return err;
18. }

174 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

19. ESP_LOGI(TAG, "create socket success, listenfd : %d", listenfd);
20.
21. //Enable SO_REUSEADDR, allowing the server to bind the connected address
22. int opt = 1;
23. int ret = setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &opt,
24. sizeof(opt));
25. if (ret < 0) {
26. ESP_LOGE(TAG, "Failed to set SO_REUSEADDR. Error %d", errno);
27. goto exit;
28. }
29.
30. //Bind the server to an interface with all-zero IP address and port number 3333
31. server_addr.sin_family = AF_INET;
32. server_addr.sin_addr.s_addr = INADDR_ANY;
33. server_addr.sin_port = htons(3333);
34. ret = bind(listenfd, (struct sockaddr *) &server_addr, sizeof(server_addr));
35. if (ret < 0) {
36. ESP_LOGE(TAG, "bind socket failed, socketfd: %d, errno : %d",
37. listenfd, errno);
38. goto exit;
39. }
40. ESP_LOGI(TAG, "bind socket success");
41. ret = listen(listenfd, 1);
42. if (ret < 0) {
43. ESP_LOGE(TAG, "listen socket failed, socketfd : %d, errno : %d",
44. listenfd, errno);
45. goto exit;
46. }
47. ESP_LOGI(TAG, "listen socket success");
48. while (1) {
49. struct sockaddr_in source_addr;
50. socklen_t addr_len = sizeof(source_addr);
51.
52. //Wait for new TCP connection, and return the communicating socket with the peer
53. int sock = accept(listenfd, (struct sockaddr *)&source_addr, &addr_len);
54. if (sock < 0) {
55. ESP_LOGE(TAG, "Unable to accept connection: errno %d", errno);
56. break;
57. }
58.
59. //Enable TCP keep-alive function to prevent zombie clients
60. setsockopt(sock, SOL_SOCKET, SO_KEEPALIVE, &keepAlive, sizeof(int));
61. setsockopt(sock, IPPROTO_TCP, TCP_KEEPIDLE, &keepIdle, sizeof(int));
62. setsockopt(sock, IPPROTO_TCP, TCP_KEEPINTVL,&keepInterval, sizeof(int));
63. setsockopt(sock, IPPROTO_TCP, TCP_KEEPCNT, &keepCount, sizeof(int));
64. if (source_addr. sin_family == PF_INET) {
65. inet_ntoa_r(((struct sockaddr_in *)&source_addr)->sin_addr,

Chapter 8. Local Control 175

66. addr_str, sizeof(addr_str) - 1);
67. }
68. ESP_LOGI(TAG, "Socket accepted ip address: %s", addr_str);
69. do {
70. len = recv(sock, rx_buffer, sizeof(rx_buffer) - 1, 0);
71. if (len < 0) {
72. ESP_LOGE(TAG, "Error occurred during receiving: errno %d", errno);
73. } else if (len == 0) {
74. ESP_LOGW(TAG, "Connection closed");
75. } else {
76. rx_buffer[len] = 0;
77. ESP_LOGI(TAG, "Received %d bytes: %s", len, rx_buffer);
78. }
79. } while (len > 0);
80. shutdown(sock, 0);
81. close(sock);
82. }
83. exit:
84. close(listenfd);
85. return err;
86. }

The above code creates a TCP server and listens to the application data on port 3333. The
socket option SO_REUSEADDR allows the server to bind to the address of an already es-
tablished connection, which is useful for the code on the server side. The socket option
SO_KEEPALIVE enables the TCP keep-alive function, which can detect some abnormally
disconnected clients and prevent them from occupying server processes. Socket options
TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT correspond to the idle time since
the last data sent by the peer, the interval time for sending TCP keep-alive messages, and
the maximum number of retries for sending messages, respectively. For example, if a TCP
client sets TCP_KEEPIDLE to 5, it means that if there is no data communication between
the client and the server within 5 seconds, the client needs to send a TCP keep-alive message
to the server; if the client sets TCP_KEEPINTVL to 5, it means that if the client sends a TCP
keep-alive message to the server and the server does not reply within 5 seconds, the client
needs to resend the message to the server; if the client sets TCP_KEEPCNT to 3, it means
that the client can retry sending the TCP keep-alive message to the server for a maximum of
3 times.

3. Creating a TCP client using socket

Source code
For the source code of the function esp create tcp client(), please refer to book-
esp32c3-iot-projects/test case/tcp socket.

176 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The function esp_create_tcp_client() can create a TCP connection between a TCP
client and a server, including creating of a TCP socket, configuring the destination address
and port, connecting, and sending data. The code is as below:
1. #define HOST_IP "[Link]"
2. #define PORT 3333
3.
4. esp_err_t esp_create_tcp_client(void)
5. {
6. esp_err_t err = ESP_FAIL;
7. char *payload = "Open the light";
8. struct sockaddr_in dest_addr;
9. dest_addr.sin_addr.s_addr = inet_addr(HOST_IP);
10. dest_addr.sin_family = AF_INET;
11. dest_addr.sin_port = htons(PORT);
12.
13. //Create a TCP socket
14. int sock = socket(AF_INET, SOCK_STREAM, 0);
15. if (sock < 0) {
16. ESP_LOGE(TAG, "Unable to create socket: errno %d", errno);
17. return err;
18. }
19. ESP_LOGI(TAG, "Socket created, connecting to %s:%d", HOST_IP, PORT);
20.
21. //Connect to the TCP server
22. int ret = connect(sock, (struct sockaddr *)&dest_addr, sizeof(dest_addr));
23. if (ret ! = 0) {
24. ESP_LOGE(TAG, "Socket unable to connect: errno %d", errno);
25. close(sock);
26. return err;
27. }
28. ESP_LOGI(TAG, "Successfully connected");
29.
30. //Send TCP data
31. ret = send(sock, payload, strlen(payload), 0);
32. if (ret < 0) {
33. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
34. goto exit;
35. }
36. err = ESP_OK;
37. exit:
38. shutdown(sock, 0);
39. close(sock);
40. return err;
41. }

After the client establishes a TCP connection with the server, it sends the TCP data “Open the

Chapter 8. Local Control 177

light” to the server. In addition to using TCP sockets, the client can also use TCP debugging
tools to simulate the client for TCP connection.
Based on the above TCP client and server code, and combined with the requirement of
controlling the smart light through a smartphone, you can implement code of TCP server on
the smart light device and code of TCP client on the smartphone. After establishing a TCP
connection between the smartphone and the smart light, the smartphone can send data. For
example, in the above code, the TCP client sends the data “Open the light”; and after the
smart light receives the data, it can turn on the light by pulling up the GPIO pin level of the
smart light.

8.3.2 HyperText Transfer Protocol (HTTP)

HTTP is an application protocol based on the transport layer. It is the data communication
foundation of the World Wide Web (WWW or Web), which specifies the format and method
of data transmission between clients and servers. Clients can use HTTP to obtain the on/off
status of the smart light (GET) or turn the smart light on and off (POST) through HTTP re-
quests, and each operation will have a response from the peer. Therefore, HTTP is completer
and more reasonable in applications than simple TCP.

1. Introduction to HTTP

HTTP is a standard for requests and responses between clients (users) and servers (web-
sites). The client establishes a TCP connection with the server through a web browser, web
crawler or other tools, and then sends requests to read server data, upload data or forms
to the server, and read the response status of the server, such as “HTTP/1.1 200 OK”, as
well as the returned content (such as requested files, error messages or other information).
Resources requested through HTTP are identified by uniform resource identifiers (URIs).
In versions 0.9 and 1.0 of HTTP, the TCP connection is closed after each request and re-
sponse. In version 1.1 of HTTP, a mechanism for maintaining the connection was intro-
duced, allowing a connection to repeat multiple requests and responses, reducing TCP hand-
shake time and network overhead before each data request.
Common HTTP request methods include:
• GET: Request the specified URI resource.
• POST: Submit data to the specified URI resource and request the server to process it
(such as submitting a form or uploading a file).
• DELETE: Request the server to delete the resource identified by the URI.
For local control of smart lights, you can use the GET method to obtain their status, and use
the POST method to control them.

178 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2. Creating an HTTP server using ESP-IDF component

Source code
For the source code of the function esp start webserver(), please refer to book-esp
32c3-iot-projects/test case/https server.

The function esp_start_webserver() can create an HTTP server. The callback func-
tions corresponding to the GET and POST operations on the server side are defined as
esp_light_get_handler() and esp_light_set_handler() respectively, and must
be registered through the function httpd_register_uri_handler() after calling the
function httpd_start() on the server side.
1. char buf[100] = "{\\"status\\": true}";
2. //Callback function of the HTTP GET request
3. esp_err_t esp_light_get_handler(httpd_req_t *req)
4. {
5. //Send data in JSON containing the status of smart lights to the client
6. httpd_resp_send(req, buf, strlen(buf));
7. return ESP_OK;
8. }
9.
10. //Callback function of the HTTP POST request
11. esp_err_t esp_light_set_handler(httpd_req_t *req)
12. {
13. int ret, remaining = req->content_len;
14. memset(buf, 0 , sizeof(buf));
15. while (remaining > 0) {
16. //Read HTTP request data
17. if ((ret = httpd_req_recv(req, buf, remaining)) <= 0) {
18. if (ret == HTTPD_SOCK_ERR_TIMEOUT) {
19. continue;
20. }
21. return ESP_FAIL;
22. }
23. remaining -= ret;
24. }
25. ESP_LOGI(TAG, "%.*s", req->content_len, buf);
26.
27. //TODO: Read and parse the data; then control the smart light
28. return ESP_OK;
29. }
30.
31. //Callback function corresponding to GET
32. static const httpd_uri_t status = {
33. .uri = "/light",
34. .method = HTTP_GET,

Chapter 8. Local Control 179

35. .handler = esp_light_get_handler,
36. };
37.
38. //Callback function corresponding to POST
39. static const httpd_uri_t ctrl = {
40. .uri = "/light",
41. .method = HTTP_POST,
42. .handler = esp_light_set_handler,
43. };
44.
45. esp_err_t esp_start_webserver()
46. {
47. httpd_handle_t server = NULL;
48. httpd_config_t config = HTTPD_DEFAULT_CONFIG();
49. config.lru_purge_enable = true;
50.
51. //Start the HTTP server
52. ESP_LOGI(TAG, "Starting server on port: ’%d’", config. server_port);
53. if (httpd_start(&server, &config) == ESP_OK) {
54. //Set the callback function corresponding to the HTTP URI
55. ESP_LOGI(TAG, "Registering URI handlers");
56. httpd_register_uri_handler(server, &status);
57. httpd_register_uri_handler(server, &ctrl);
58. return ESP_OK;
59. }
60. ESP_LOGI(TAG, "Error starting server!" );
61. return ESP_FAIL;
62. }

The above code implements an HTTP server for querying and setting the status of the smart
light. When accessing [Link] through a browser, the browser will return
{"status": true} (as shown in Figure 8.4) or {"status": false} to indicate the
status of the smart light.

Figure 8.4. Using HTTP to query the status of the smart light

Press F12 on the current page to enter the Console. Enter the following command and press
“Enter” to send a POST request.

180 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

$ var xhr = new XMLHttpRequest();
$ [Link]("POST", "[Link]/light", true);
$ [Link]("{\"status\": false}");

Figure 8.5 shows how to use HTTP to set the status of the smart light.

Figure 8.5. Using HTTP to set the status of the smart light

At this point, the server will receive the HTTP POST request {"status": false}. The
log of using HTTP to set the status of the smart light is as follows:
I (773) wifi:mode:sta ([Link])
I (773) wifi:enable tsf
I (773) wifi station: wifi init sta finished.
I (793) wifi:new: <6,0>, old: <1,0>, ap: <255,255>, sta: <6,0>, prof:1
I (793) wifi:state: init -> auth (be)
I (813) wifi:state: auth -> assoc (0)
I (823) wifi:state: assoc -> run (10)
I (873) wifi:connected with myssid, aid = 1, channel 6, BW20, bssid =
[Link]
I (873) wifi:security: WPA2-PSK, phy: bgn, rssi: -21
I (883) wifi:pm start, type: 1
I (943) wifi:AP’s beacon interval = 102400 us, DTIM period = 1
I (1543) esp netif handlers: sta ip: [Link], mask: [Link], gw: 192.
168.3.1
I (1543) wifi station: got ip:[Link]
I (1543) wifi station: connected to ap SSID: myssid password: 12345678
I (1553) wifi station: Starting server on port: ’80’
I (1563) wifi station: Registering URI handlers
W (11393) wifi:<ba-add>idx:0 (ifx:0,[Link]), tid:7, ssn:4, winSize:64
I (11413) wifi station: {"status": false}

Refreshing the current page at this time can continue to query the status of the smart light,
and the previously set status will be displayed, as shown in Figure 8.6.

Figure 8.6. Displaying the modified status of the smart light

Chapter 8. Local Control 181

8.3.3 User Datagram Protocol (UDP)
Subsections 8.3.1 and 8.3.2 respectively introduce TCP and HTTP, both of which are char-
acterised by reliable transmission. This subsection will introduce another protocol at the
transport layer, UDP. Unlike TCP, UDP is an unreliable transmission protocol. Common
application protocols based on UDP include DNS, TFTP, and SNMP.

1. Introduction to UDP

UDP is a simple datagram-oriented communication protocol, which is located at the trans-

port layer like TCP. UDP was designed by David P. Reed in 1980 and defined in RFC 768
(excerpted from Wikipedia). UDP is an unreliable transmission protocol. After data is sent
through UDP, the underlying layer does not retain the data to prevent loss during trans-
mission. UDP itself does not support error correction, queue management, or congestion
control, but supports checksums.
UDP is a connectionless protocol. It does not need to establish a connection before sending
data, unlike TCP. Data can be sent directly to the peer without establishing a connection.
Because no connection needs to be established during data transmission, there is no need to
maintain connection status, including sending and receiving status.
UDP is only responsible for transmission, so applications that use this protocol need to
do more control over how data is sent and processed, such as how to ensure that peer’s
applications receive the data correctly and in order.
Compared with TCP, UDP cannot guarantee the safe and reliable transmission of data. You
may wonder why the UDP protocol is still used. The connectionless nature of UDP results in
less network and time overhead than TCP. The unreliable transmission of UDP (mainly the
inability to guarantee retransmission after packet loss) is more suitable for applications such
as streaming media, real-time multiplayer games, and IP voice, where losing a few packets
will not affect the application. On the other hand, if TCP is used for retransmission, it will
greatly increase network latency.

2. Creating a UDP server using socket

Source code
For the source code of the function esp create udp server(), please refer to book-
esp32c3-iot-projects/test case/udp socket.

Creating a UDP server using socket is similar to creating a multicast group receiver as in-
troduced in subsection 8.2.2. Both involve creating a UDP socket, configuring the bound
port, and receiving and sending data. The function esp_create_udp_server() sets the

182 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

SO_REUSEADDR option, allowing the server to bind the address of the already established
connection. The code is as below:
1. esp_err_t esp_create_udp_server(void)
2. {
3. char rx_buffer[128];
4. char addr_str[32];
5. esp_err_t err = ESP_FAIL;
6. struct sockaddr_in server_addr;
7. //Create a UDP socket
8. int sock = socket(AF_INET, SOCK_DGRAM, 0);
9. if (sock < 0) {
10. ESP_LOGE(TAG, "create socket error");
11. return err;
12. }
13. ESP_LOGI(TAG, "create socket success, sock : %d", sock);
14. //Enable SO_REUSEADDR, allowing the server to bind connected address
15. int opt = 1;
16. int ret = setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
17. if (ret < 0) {
18. ESP_LOGE(TAG, "Failed to set SO_REUSEADDR. Error %d", errno);
19. goto exit;
20. }
21. //Bind the server to an interface with all-zero IP address and port number 3333
22. server_addr.sin_family = AF_INET;
23. server_addr.sin_addr.s_addr = INADDR_ANY;
24. server_addr.sin_port = htons(PORT);
25. ret = bind(sock, (struct sockaddr *) &server_addr, sizeof(server_addr));
26. if (ret < 0) {
27. ESP_LOGE(TAG, "bind socket failed, socketfd: %d, errno : %d", sock, errno);
28. goto exit;
29. }
30. ESP_LOGI(TAG, "bind socket success");
31. while (1) {
32. struct sockaddr_in source_addr;
33. socklen_t addr_len = sizeof(source_addr);
34. memset(rx_buffer, 0, sizeof(rx_buffer));
35. int len = recvfrom(sock, rx_buffer, sizeof(rx_buffer) - 1, 0,
36. (struct sockaddr *)&source_addr, &addr_len);
37. // Reception error
38. if (len < 0) {
39. ESP_LOGE(TAG, "recvfrom failed: errno %d", errno);
40. break;
41. } else { //Data is received
42. if (source_addr. sin_family == PF_INET) {
43. inet_ntoa_r(((struct sockaddr_in *)&source_addr)->sin_addr,
44. addr_str, sizeof(addr_str) - 1);

Chapter 8. Local Control 183

45. }
46. //String ends with NULL
47. rx_buffer[len] = 0;
48. ESP_LOGI(TAG, "Received %d bytes from %s:" , len, addr_str);
49. ESP_LOGI(TAG, "%s", rx_buffer);
50. }
51. }
52. exit:
53. close(sock);
54. return err;
55. }

3. Creating a UDP client using socket

Source code
For the source code of the function esp create udp client(), please refer to book-
esp32c3-iot-projects/test case/udp socket.

With the function esp_create_udp_client(), the UDP client can send data, including
creating UDP sockets, configuring destination addresses and ports, calling socket interface
sendto() to send data. The code is as below:
1. esp_err_t esp_create_udp_client(void)
2. {
3. esp_err_t err = ESP_FAIL;
4. char *payload = "Open the light";
5. struct sockaddr_in dest_addr;
6. dest_addr.sin_addr.s_addr = inet_addr(HOST_IP);
7. dest_addr.sin_family = AF_INET;
8. dest_addr.sin_port = htons(PORT);
9.
10. //Create a UDP socket
11. int sock = socket(AF_INET, SOCK_DGRAM, 0);
12. if (sock < 0) {
13. ESP_LOGE(TAG, "Unable to create socket: errno %d", errno);
14. return err;
15. }
16.
17. //Send data
18. int ret = sendto(sock, payload, strlen(payload), 0,
19. (struct sockaddr *)&dest_addr, sizeof(dest_addr));
20. if (ret < 0) {
21. ESP_LOGE(TAG, "Error occurred during sending: errno %d", errno);
22. goto exit;
23. }
24. ESP_LOGI(TAG, "Message send successfully");

184 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

25. err = ESP_OK;
26. exit:
27. close(sock);
28. return err;
29. }

UDP clients do not need to establish a connection with the server, and can directly send data
to the server. Since UDP creates unreliable connections, the data sent, such as “Open the
light,” may be lost, causing the peer to fail to receive it. Therefore, when writing code for
the client and server, some logic should be added to the application layer code to ensure
that data is not lost. For example, when the client sends “Open the light” to the server, the
server returns “Open the light OK” after receiving it successfully. If the client receives the
data within 1 second, it means that the data has been sent to the server correctly. If the
client does not receive it within 1 second, it needs to send the data “Open the light” again.

8.3.4 Constrained Application Protocol (CoAP)

With the rapid development of IoT technology, a series of protocols have been created for
IoT devices. Most IoT devices have limited resources, such as RAM, flash, CPU, network
bandwidth, etc. More memory and network bandwidth are often required if they want to
use TCP and HTTP protocols for data transmission. If UDP can be used for data transmis-
sion, is there an application protocol similar to HTTP? The answer is yes, CoAP is designed
according to the REST architecture of HTTP.

1. Introduction to CoAP

CoAP is a protocol similar to web applications in IoT devices. It is defined in RFC 7252
and can be used for resource-constrained IoT devices, allowing those resource-constrained
devices called nodes to communicate with a wider range of the Internet using similar pro-
tocols. CoAP is designed for devices on the same constrained network (such as low-power,
lossy networks), between devices and general nodes on the Internet, and between devices
on different constrained networks connected by the Internet.
CoAP is based on the request and response model, similar to HTTP, which can make up for
the shortcomings of unreliable transmission of UDP and ensure that data is not lost or disor-
dered. The server’s resources are identified by URLs (such as coap://[IP]/id/light_
status) to access the status of a smart light. The client accesses the server’s resources
through the URL of a resource and operate the server’s resources through four request meth-
ods (GET, PUT, POST, and DELETE).
CoAP also has the following features:
• Both the client and server can independently send requests to each other.
• Supports reliable data transmission.

Chapter 8. Local Control 185

• Supports multicast and broadcast, enabling one-to-many data transmission.
• Supports communication with low power consumption and non-persistent connections.
• Compared with HTTP, its header is lighter.

2. Creating a CoAP server using ESP-IDF component

The following code shows how to create a CoAP server using ESP-IDF component, which
provides GET and PUT operations for resource retrieval and modification in CoAP. CoAP pro-
tocol operations are generally fixed, and you only need to focus on your own resource URI
paths and the operations you need to provide. The function coap_resource_init() can
be used to set the URI for resource access, and the function coap_register_handler()
can be used to register GET and PUT callback functions corresponding to the resource URI.

Source code
For the source code of the functions coap resource init() and coap register
handler(), please refer to book-esp32c3-iot-projects/test case/coap.
1. static char buf[100] = "{\"status\": true}";
2.
3. //Callback function of GET method in CoAP
4. static void esp_coap_get(coap_context_t *ctx, coap_resource_t *resource,
5. coap_session_t *session, coap_pdu_t *request,
6. coap_binary_t *token, coap_string_t *query,
7. coap_pdu_t *response)
8. {
9. coap_add_data_blocked_response(resource, session, request, response,
10. token, COAP_MEDIATYPE_TEXT_PLAIN, 0,
11. strlen(buf), (const u_char *)buf);
12. }
13.
14. //Callback function of PUT method in CoAP
15. static void esp_coap_put(coap_context_t *ctx,
16. coap_resource_t *resource,
17. coap_session_t *session,
18. coap_pdu_t *request,
19. coap_binary_t *token,
20. coap_string_t *query,
21. coap_pdu_t *response)
22. {
23. size_t size;
24. const unsigned char *data;
25. coap_resource_notify_observers(resource, NULL);
26.
27. //Read the received CoAP protocol data
28. (void)coap_get_data(request, &size, &data);

186 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

29. if (size) {
30. if (strncmp((char *)data, buf, size)) {
31. memcpy(buf, data, size);
32. buf[size] = 0;
33. response->code = COAP_RESPONSE_CODE(204);
34. } else {
35. response->code = COAP_RESPONSE_CODE(500);
36. }
37. } else { //A size of 0 indicates a receiving error
38. response->code = COAP_RESPONSE_CODE(500);
39. }
40. }
41.
42. static void esp_create_coap_server(void)
43. {
44. coap_context_t *ctx = NULL;
45. coap_address_t serv_addr;
46. coap_resource_t *resource = NULL;
47. while (1) {
48. coap_endpoint_t *ep = NULL;
49. unsigned wait_ms;
50.
51. //Create a CoAP server socket
52. coap_address_init(&serv_addr);
53. serv_addr.addr.sin6.sin6_family = AF_INET6;
54. serv_addr.addr.sin6.sin6_port = htons(COAP_DEFAULT_PORT);
55.
56. //Create CoAP ctx
57. ctx = coap_new_context(NULL);
58. if (!ctx) {
59. ESP_LOGE(TAG, "coap_new_context() failed");
60. continue;
61. }
62.
63. //Set the CoAP protocol node
64. ep = coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_UDP);
65. if (!ep) {
66. ESP_LOGE(TAG, "udp: coap_new_endpoint() failed");
67. goto clean_up;
68. }
69.
70. //Set CoAP protocol resource URI
71. resource = coap_resource_init(coap_make_str_const("light"), 0);
72. if (!resource) {
73. ESP_LOGE(TAG, "coap_resource_init() failed");
74. goto clean_up;
75. }

Chapter 8. Local Control 187

76.
77. //Register callback functions of GET and PUT methods corresponding to CoAP resource URI
78. coap_register_handler(resource, COAP_REQUEST_GET, esp_coap_get);
79. coap_register_handler(resource, COAP_REQUEST_PUT, esp_coap_put);
80.
81. //Set CoAP GET resource observable
82. coap_resource_set_get_observable(resource, 1);
83.
84. //Add resource to CoAP ctx
85. coap_add_resource(ctx, resource);
86. wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;
87. while (1) {
88. //Wait to receive CoAP data
89. int result = coap_run_once(ctx, wait_ms);
90. if (result < 0) {
91. break;
92. } else if (result && (unsigned)result < wait_ms) {
93. //Decrease waiting time
94. wait_ms -= result;
95. } else {
96. //Reset waiting time
97. wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;
98. }
99. }
100. }
101.clean_up:
102. coap_free_context(ctx);
103. coap_cleanup();
104.}

The above code creates a CoAP server, and provides the GET method to query the status of
the smart light and the PUT method to set the status of the smart light. You can use the
Chrome browser to install CoAP to debug the client Copper plugin and simulate the CoAP
client.
Open the Chrome plugin Copper, enter the URL coap://[ip]/light, and press “Enter”
to connect to the server. Figure 8.7 shows the connection of the CoAP plugin.
After the connection is successful, click the “GET” button in the upper left corner to get the
status and display {"status": true}, the query status of CoAP plugin is shown in Figure
8.8.
Click the “PUT” button in the upper left corner, and modify the data in “Payload” → “Out-
going” to {"status": false} to set the status of the smart light to false. Figure 8.9
shows the configuration status of the CoAP plugin.
At this time, click the “GET” button in the upper left corner again to get the status, which

188 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 8.7. Connection of CoAP plugin

Figure 8.8. Query status of CoAP plugin

Figure 8.9. Setting status of CoAP plugin

Chapter 8. Local Control 189

displays {"status": false}. Figure 8.10 shows the query setting status of the CoAP
plugin.

Figure 8.10. Query setting status of CoAP plugin

8.3.5 Bluetooth Protocol

1. Introduction to Bluetooth protocol

Chapter 7 introduces the protocol and architecture of Bluetooth. The Bluetooth protocol
defines message formats and process for completing specific functions, such as link control,
security services, service information exchange and data transmission. This section only
introduces the attribute protocol (ATT) of the Bluetooth protocol specification. Bluetooth
data exists in the form of attributes, and each attribute consists of four elements.
Attribute handle
Just as memory addresses are used to find contents in memory, attribute handles can also
help find the corresponding attribute. For example, the first attribute handle is 0x0001,
the second attribute handle is 0x0002, and so on, up to a maximum of 0xFFFF.
Attribute UUID
Each data represents specific property. For example, a smart light has two basic attributes,
one for setting the on/off status, and the other for reading the on/off status.
Attribute value
Attribute value is the information that each attribute carries, while the other three ele-
ments are to enable the peer to obtain the attribute value much easier. For example, for
a smart light, the attribute value for setting the on/off status can be set to “1” to turn on
the light, or to “0” to turn off the light; the attribute value for reading the on/off status
can be “1” for the “on” status or “0” for the “off” status.

190 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Attribute permissions
Each attribute has corresponding access restrictions for its own attribute values, such as
some attributes are readable, some are writable, and some are readable and writable.
The party that owns the data can control the attribute permissions of local data through
attribute permissions. For example, the switch attribute permission of the smart light can
be set as writable but not readable, and the attribute permission for reading the switch
status of the smart light can be set as read-only and not writable.
Table 8.2 lists the Bluetooth attributes for the basic functions of a smart light.

Table 8.2. Bluetooth attributes for basic functions of smart light

Attribute Attribute
Attribute UUID Attribute permissions
handle value

0x0001 Set the on/off status 1/0 Writable but not readable

0x0002 Read the on/off status 1/0 Readable but not writable

The device that stores the data (i.e., attributes) is usually called the server, and the device
that receives data from other devices is called the client. For a smart light and a smartphone,
the smart light is server, and the smartphone is the client. The following are common
operations between a server and a client:
(1) The client sends data to the server.
Data is transmitted by writing data to the server. There are two types of write oper-
ations: one is write request, and the other is write command. The main difference
between the two is that the former requires a response (write response) from the
peer, while the latter does not. For a smart light, the command to turn on/off the light
sent by the smartphone is a write operation, and this is a write request which requires
the smart light to respond. Such response is not a simple ACK response. The result of
the action of turning on/off the light needs to be returned to the smartphone to inform
it of the current status of the smart light.
(2) The server sends data to the client.
The updated data is sent from the server to the client mainly in the form of server
indication or notification. Similar to write operations, the main difference between
indication and notification is that the former requires the other device to respond
(confirm) after receiving the data indication. For a smart light, if it is turned on/off
through a physical switch button, its status needs to be reported to the smartphone
through indications or notifications, and the smartphone will display the latest status.

Chapter 8. Local Control 191

 (3) The client reads data from the server actively.
Generally, the client obtains values of corresponding attributes from the server through
read operation. In the case mentioned above where a smart light is turned on/off
through a physical switch button, except for waiting to be notified by the server, the
smartphone can also obtain the real-time status through read operations.
Then let’s consider which way is better to get the status of the smart light. Active reading
takes time whenever the smartphone initiates a read operation, while indication or notifica-
tion saves the time for repetitive data transmission. It seems that the latter option is faster
but if the smart light is not connected to the phone when sending the notification, its status
will not be updated. This can be fixed by updating the status as soon as the phone becomes
connected to the smart light; otherwise, it is recommended to use the read operation.

2. Creating a Bluetooth server using ESP-IDF component

Source code
For the source code of Bluetooth, please refer to esp-idf/examples/provisioning/
legacy/ble prov. For the example code of customised configuration, please refer to
esp-idf/examples/provisioning/legacy/custom config.

The following example uses the protocomm component to implement the smart light server,
and the customised configuration uses the custom-proto protocol. As mentioned earlier, to
implement the on/off control and status query of the smart light, two attributes need to be
defined. The code is as below:
1. static esp_err_t wifi_prov_config_set_light_handler(uint32_t session_id,
2. const uint8_t *inbuf,
3. ssize_t inlen,
4. uint8_t **outbuf,
5. ssize_t *outlen,
6. void *priv_data)
7. {
8. CustomConfigRequest *req;
9. CustomConfigResponse resp;
10. req = custom_config_request_unpack(NULL, inlen, inbuf);
11. if (!req) {
12. ESP_LOGE(TAG, "Unable to unpack config data");
13. return ESP_ERR_INVALID_ARG;
14. }
15. custom_config_response_init(&resp);
16. [Link] = CUSTOM_CONFIG_STATUS_ConfigFail;
17. if (req->open_light) {//Turn on the smart light
18. //Pull up GPIO level according to the status
19. ESP_LOGI(TAG, "Open the light");

192 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

20. } else {
21. //Pull down GPIO level according to the status
22. ESP_LOGI(TAG, "Close the light");
23. }
24.
25. //Set response status and smart light status according to the light’s execution result
26. [Link] = CUSTOM_CONFIG_STATUS_ConfigSuccess;
27. custom_config_request_free_unpacked(req, NULL);
28. resp.light_status = 1; //Respond according to the light status
29. *outlen = custom_config_response_get_packed_size(&resp);
30. if (*outlen <= 0) {
31. ESP_LOGE(TAG, "Invalid encoding for response");
32. return ESP_FAIL;
33. }
34. *outbuf = (uint8_t *) malloc(*outlen);
35. if (*outbuf == NULL) {
36. ESP_LOGE(TAG, "System out of memory");
37. return ESP_ERR_NO_MEM;
38. }
39.
40. custom_config_response_pack(&resp, *outbuf);
41. return ESP_OK;
42. }
43.
44. static int wifi_prov_config_get_light_handler(uint32_t session_id,
45. const uint8_t *inbuf,
46. ssize_t inlen,
47. uint8_t **outbuf,
48. ssize_t *outlen,
49. void *priv_data)
50. {
51. CustomConfigResponse resp;
52. custom_config_response_init(&resp);
53. [Link] = CUSTOM_CONFIG_STATUS_ConfigSuccess;
54. resp.light_status = 1; //Respond according to the light status
55. *outlen = custom_config_response_get_packed_size(&resp);
56. if (*outlen <= 0) {
57. ESP_LOGE(TAG, "Invalid encoding for response");
58. return ESP_FAIL;
59. }
60. *outbuf = (uint8_t *) malloc(*outlen);
61. if (*outbuf == NULL) {
62. ESP_LOGE(TAG, "System out of memory");
63. return ESP_ERR_NO_MEM;
64. }
65. custom_config_response_pack(&resp, *outbuf);
66. return ESP_OK;

Chapter 8. Local Control 193

67. }
68.
69. static esp_err_t app_prov_start_service(void)
70. {
71. //Create protocomm
72. g_prov->pc = protocomm_new();
73. if (g_prov->pc == NULL) {
74. ESP_LOGE(TAG, "Failed to create new protocomm instance");
75. return ESP_FAIL;
76. }
77.
78. //Attribute value
79. protocomm_ble_name_uuid_t nu_lookup_table[] = {
80. {"prov-session", 0x0001},
81. {"prov-config", 0x0002},
82. {"proto-ver", 0x0003},
83. {"set-light", 0x0004}, //Set the state of the smart light
84. {"get-light", 0x0005}, //Get the status of the smart light
85. };
86.
87. //Bluetooth configuration
88. protocomm_ble_config_t config = {
89. .service_uuid = {
90. /* LSB <---------------------------------------
91. * ---------------------------------------> MSB */
92. 0xb4, 0xdf, 0x5a, 0x1c, 0x3f, 0x6b, 0xf4, 0xbf,
93. 0xea, 0x4a, 0x82, 0x03, 0x04, 0x90, 0x1a, 0x02,
94. },
95. .nu_lookup_count=sizeof(nu_lookup_table)/sizeof(nu_lookup_table[0]),
96. .nu_lookup = nu_lookup_table
97. };
98.
99. uint8_t eth_mac[6];
100. esp_wifi_get_mac(WIFI_IF_STA, eth_mac);
101. snprintf(config.device_name,
102. sizeof(config.device_name),
103. "%s%02X%02X%02X",
104. ssid_prefix,
105. eth_mac[3],
106. eth_mac[4],
107. eth_mac[5]);
108.
109. //Release BT memory as only Bluetooth LE protocol stack is used.
110. esp_err_t err = esp_bt_controller_mem_release(ESP_BT_MODE_CLASSIC_BT);
111. if (err) {
112. ESP_LOGE(TAG, "bt_controller_mem_release failed %d", err);
113. if (err ! = ESP_ERR_INVALID_STATE) {

194 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

114. return err;
115. }
116. }
117. //Start protocomm Bluetooth LE protocol stack
118. if (protocomm_ble_start(g_prov->pc, &config) ! = ESP_OK) {
119. ESP_LOGE(TAG, "Failed to start BLE provisioning");
120. return ESP_FAIL;
121. }
122. //Set protocomm version verification endpoint for the protocol
123. protocomm_set_version(g_prov->pc, "proto-ver", "V0.1");
124. //Set protocomm security type for the endpoint
125. if (g_prov->security == 0) {
126. protocomm_set_security(g_prov->pc, "prov-session",
127. &protocomm_security0, NULL);
128. } else if (g_prov->security == 1) {
129. protocomm_set_security(g_prov->pc, "prov-session",
130. &protocomm_security1, g_prov->pop);
131. }
132. //Add an endpoint for Wi-Fi configuration
133. if(protocomm_add_endpoint(g_prov->pc, "prov-config",
134. wifi_prov_config_data_handler,
135. (void *) &wifi_prov_handlers) ! =ESP_OK){
136. ESP_LOGE(TAG, "Failed to set provisioning endpoint");
137. protocomm_ble_stop(g_prov->pc);
138. return ESP_FAIL;
139. }
140. //Add an endpoint for setting smart light status
141. if (protocomm_add_endpoint(g_prov->pc, "set-light",
142. wifi_prov_config_set_light_handler,
143. NULL) ! = ESP_OK) {
144. ESP_LOGE(TAG, "Failed to set set-light endpoint");
145. protocomm_ble_stop(g_prov->pc);
146. return ESP_FAIL;
147. }
148. //Add an endpoint for getting smart light status
149. if (protocomm_add_endpoint(g_prov->pc, "get-light",
150. wifi_prov_config_get_light_handler,
151. NULL) ! = ESP_OK) {
152. ESP_LOGE(TAG, "Failed to set get-light endpoint");
153. protocomm_ble_stop(g_prov->pc);
154. return ESP_FAIL;
155. }
156. ESP_LOGI(TAG, "Provisioning started with BLE devname : ’%s’",
157. config.device_name);
158. return ESP_OK;
159.}

Chapter 8. Local Control 195

The above example provides two attributes: set-light and get-light, and the cor-
responding attribute handles are 0x0004 and 0x0005, respectively. When the smartphone
sends a command to set the light, the wifi_prov_config_set_light_handler() call-
back function will be executed to handle the on/off action and inform the smartphone of
the current status of the smart light. When the smartphone sends a read command, the
wifi_prov_config_get_light_handler() callback function will be executed to in-
form the smartphone of the current status of the smart light. You can use the Bluetooth
debugging assistant of the smartphone to scan the devices connected to Bluetooth, and un-
derstand the function of each service more intuitively through the services provided by the
Bluetooth device.
The above example implements local control via Bluetooth based on the protocomm com-
ponent, and the data structure is relatively complex. If you are an experienced developer,
you can try to use the ideas of the above example to implement local control. In addition,
this book provides the most basic server example based on Bluetooth for beginners. You
can refer to the example code in subsection 8.5.3 to understand the process of local control
using Bluetooth.

8.3.6 Summary of Data Communication Protocols

Both UDP and TCP protocols in the transport layer can directly serve as communication
protocols for application data. Table 8.3 lists the differences between UDP and TCP.

Table 8.3. Differences between TCP and UDP

Comparison TCP UDP

Reliable transmission; Unreliable transmission;
Reliability supports retransmission, flow con- does not support retransmission,
trol and congestion control flow control or congestion control
Connection-oriented, with three
No connection;
handshakes for connection estab-
Connection direct data transmission;
lishment and four handshakes for
short connection
disconnection; long connection
One-to-one unicast,
Connection
One-to-one connection one-to-all broadcast,
object
and one-to-many multicast
Header
20 B 8B
overhead

196 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Continuation of Table 8.3
Comparison TCP UDP
Depends on network environment;
Fast, independent of network envi-
Transmission retransmission occurs in case of
ronment, and only responsible for
rate packet loss, lowering transmission
transmitting data to the network
rate.
Suitable for real-time transmission,
Application Suitable for reliable transmission,
e.g., VoIP telephony, video tele-
scenario e.g., file transfer.
phony, streaming media, etc.

For data communication of local control, TCP can be selected from the perspective of the
transport layer as it can ensure the data is accurate. When using UDP, the smartphone app
will send the command to turn on the light. The command may be discarded due to network
environment issues, and ESP32-C3 may not receive the command. While for TCP, even if
the data packet is discarded, the underlying layer of the smartphone app will resend the
command.
However, a drawback to sending data using a pure transport layer protocol is that you need
to develop business logic of upper-layer applications. Therefore, this section also introduces
the application protocols HTTP and CoAP based on TCP and UDP.
Both HTTP and CoAP are network transmission protocols based on the REST model, which
are used to send requests and respond to requests. The only difference is that one is based
on TCP and the other is based on UDP, and each inherits the relevant characteristics of the
transport layer protocol. Table 8.4 lists the differences between HTTP and CoAP.

Table 8.4. Differences between HTTP and CoAP

Comparison HTTP CoAP

Transport
TCP UDP
layer
May contain a large amount of mes-
Header Packet headers are binary com-
sage header data with high over-
overhead pressed for low overhead
head
Power Long connection, Short connection,
consumption high power consumption low power consumption
Resource
Not support Support
discovery

Chapter 8. Local Control 197

 Continuation of Table 8.4
Comparison HTTP CoAP
Request Generally triggered by the client; Both the client and the server can
method no active trigger by the server. actively trigger requests.
Application Suitable for devices with good per- Suitable for devices with poor per-
scenario formance and large memory formance and small memory

Compared to HTTP, CoAP is more suitable for IoT devices with limited resources. For the
device has more resources and better performance, HTTP has more functions than CoAP.
After comparing the communication protocols within the TCP/IP protocol family, we will
compare these protocols with the Bluetooth protocol. The most intuitive difference between
them is that Bluetooth is a point-to-point protocol, while the TCP/IP is an end-to-end proto-
col that may go through routers. Therefore, in terms of response speed, although Bluetooth
and Wi-Fi are both wireless transmission technologies on the 2.4 GHz channel, Bluetooth is
faster than Wi-Fi in data communication between smartphones and ESP32-C3. The packet
size of Bluetooth is smaller than that of application data using TCP/IP protocol stack, and
the power consumption of Bluetooth is naturally lower than that of Wi-Fi. The Bluetooth
protocol supports resource discovery and does not require local discovery because Bluetooth
is a point-to-point connection, which is very suitable for local control. However, since most
IoT products currently need to connect to the cloud, Wi-Fi functionality is essential. Many
IoT products can use only Wi-Fi or only Bluetooth for network configuration. If the IoT
product does not need to connect to the cloud, Bluetooth can be used for local control only.
If the IoT product needs to connect to the cloud, it needs to use Wi-Fi for cloud connection
and local control.

8.4 Guarantee of Data Security

As we all know, TCP and UDP, as well as the application protocols HTTP and CoAP that
run on top of them, transmit data in plaintext. This can lead to data being intercepted or
tampered with during transmission. If sensitive information such as passwords or account
numbers is included in the data, irreparable losses may occur. Therefore, it is necessary
to encrypt the data transmitted in plaintext. For data transmitted via Bluetooth, since the
Bluetooth is a point-to-point protocol, the data will not leak onto the network and the prob-
ability of it being intercepted is very low. In addition, the Bluetooth protocol itself encrypts
user data. Therefore, this section mainly discusses the data encryption of TCP/IP.
Encryption is used to ensure confidentiality and integrity of transmitted data. Common
encryption systems usually encode data before transmission. For example, in previous wars,

198 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

telegrams were encoded and both the sender and receiver had the same codebook. The
receiver used the numbers or letters in the codebook to replace the words or sentences in
the telegram. Even if the telegram content was intercepted by a third party, the third party
could not decipher the true content of the telegram in a short time. However, this method
has a flaw that the telegram content is still susceptible to being deciphered, which is just a
matter of time. In addition, to prevent the telegram from being deciphered, the receiver and
sender need to periodically change the codebook. This may also lead to the codebook being
leaked and the telegram content being deciphered.
The telegram example above is a common encryption algorithm—a usage scenario of sym-
metric encryption. In the symmetric encryption algorithm, the same algorithm is used for
encryption and decryption, and their keys are also the same. Symmetric encryption has the
advantages of open algorithm, small computational complexity, fast encryption speed, and
high encryption efficiency. However, before data transmission, the sender and receiver must
agree on the key, and in order to ensure that the data is not deciphered, both parties must
also periodically update the key, which makes key management a burden for both parties.
Common symmetric encryption algorithms include AES, DES, and RC4. Figure 8.11 shows
the process of symmetric encryption.

Figure 8.11. Process of symmetric encryption

In this section, we will introduce the algorithm that is opposite to symmetric encryption,
asymmetric encryption. Both parties in asymmetric encryption have a pair of public key
and private key. Data is encrypted using the public key, and decrypted using the private
key. Because different keys are used for encryption and decryption, this encryption algo-
rithm is called asymmetric encryption. Compared with symmetric encryption, asymmetric
encryption is more secure. Because asymmetric encryption is more complex than symmetric
encryption, it takes longer time to decrypt, and it is difficult for third parties to directly de-
cipher the data. Because the asymmetric encryption algorithm has high complexity and the
private key used for decryption is not transmitted on the network, which can only be ob-
tained by the recipient, this greatly improves data security. Common asymmetric encryption
algorithms include RSA, Diffie-Hellman, DSA, etc.
The advantage of asymmetric encryption is its security. User A can keep the private key and
transmit the public key to user B through the network. Even if user C obtains the public

Chapter 8. Local Control 199

key, user C cannot decipher the data because user C does not have user A’s private key. In
this way, user A and user B can confidently transmit their respective public keys through the
network. Remember, the public key is used for encryption, and the private key is used for
decryption. Figure 8.12 shows the process of asymmetric encryption.

Figure 8.12. Process of asymmetric encryption

Asymmetric encryption seems very secure, but have you ever thought about this question:
what if user C replaces all the public keys sent to user A and user B with its own corre-
sponding private key’s public key? User A does not know whether this public key belongs
to user B, so when user A sends data, it will use user C’s public key for encryption. At this
time, user C can steal the ciphertext data and decrypt it using the corresponding private key.
Therefore, it is crucial to ensure the legitimacy of the public key. In reality, the legitimacy of
the public key can be ensured through a Certificate Authority (CA). CA also works based on
asymmetric encryption algorithms. With CA, user B will first give its public key and some
other information to CA. CA encrypts this data using its private key, and the encrypted data
is called user B’s digital certificate. The public key transmitted by user B to user A is the
digital certificate encrypted by the CA. After receiving the digital certificate, user A will use
the digital certificate published by CA (which contains CA’s public key) to decrypt user B’s
digital certificate and obtain user B’s public key.

8.4.1 Introduction to Transport Layer Security (TLS)

TLS is a protocol based on TCP and serves the application layer. Its predecessor is the Secure
Socket Layer (SSL) protocol. Through the TLS protocol, the packets of the application layer
can be encrypted and delivered to the TCP layer for transmission.

1. What does TLS do?

The TLS protocol mainly solves the following three network problems:
• Guarantee data confidentiality. All data is transmitted encrypted to ensure protection
against unauthorised access or data theft by third parties.
• Guarantee data integrity. All data is protected by a verification mechanism, so any tam-
pering will be immediately detected by both parties involved in the communication.

200 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

• Guarantee the authentication and identity verification of both parties involved in data
communication. Certificate authentication can be employed by both parties in the com-
munication to ensure the legitimacy of their identities.

2. How does TLS work?

The TLS protocol can be divided into two parts. The record layer uses the key negotiated
by the client and the server to encrypt and transmit data. The handshake layer negotiates
between the client and the server to determine a set of key strings for data transmission
encryption. The TLS protocol model is shown in Figure 8.13, where the handshake layer
includes four sub-protocols: handshake protocol, change cipher spec protocol, application
data protocol, and alert protocol.

Figure 8.13. TLS protocol model

Record layer is responsible for all the underlying data exchanged at the transport layer
and can encrypt data. Each TLS record begins with a short header, which includes the
Content Type (or subprotocol), Protocol Version, and Length fields. The underlying data is
segmented (or merged), compressed, added with a message authentication code, encrypted,
and then converted into the data part of the TLS record. Figure 8.14 shows the structure of
a TLS record packet.
Handshake layer has four sub-protocols, which are introduced in the list below.
Handshake protocol
Responsible for generating the shared key required for the communication process and
performing identity authentication. Note that the handshake protocol does not use ci-
pher suites directly. Instead, it relies on public key cryptography or Diffie-Hellman key
exchange to establish secure communication and prevent data from being eavesdropped
or intercepted.
Change cipher spec protocol
Responsible for the synchronisation of password switching, and is used after the hand-
shake protocol. During the handshake process, the ‘null’ cipher suite, which means no

Chapter 8. Local Control 201

 Figure 8.14. TLS record packet

encryption, is used. After the handshake is completed, the negotiated cipher suite is used
for securing the subsequent data transfer.
Application data protocol
Used by the communicating parties for data transmission. The transmission process is car-
ried out through the application data protocol and TLS record protocol of the handshake
layer.
Alert protocol
Used to notify the other party when an error occurs, such as an exception during the hand-
shake process, a message authentication code error, or data that cannot be decompressed.
The algorithm used during TLS encryption is introduced in the list below.
• Hash function verifies data integrity. Common encryption algorithms include MD5,
SHA, etc.
• Symmetric encryption algorithm encrypts the application data. Common encryption
algorithms include AES, RC4, DES, etc.
• Asymmetric encryption algorithm for identity authentication and key agreement.
Common encryption algorithms include RSA, DH, etc.
When using TLS, the client and server use asymmetric encryption algorithm to authenticate
identity and negotiate the key of symmetric encryption algorithm, and then use symmet-
ric encrypted data and data digest for data communication. Figure 8.15 shows the TLS
handshake process.

202 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 8.15. TLS handshake process

(1) Client Hello. The client sends the highest version of the supported TLS protocol and all
the cipher suites it supports, which are used to send information such as the random
number for generating the session key to the server.
(2) Server Hello. After receiving the Client Hello message sent by the client, the server
selects the TLS protocol version and a cipher suite according to the protocol version
and cipher suite sent by the client, and returns them to the client.
(3) (Optional) Send Certificate. The server sends its own server-side certificate to the
client, which is used by the client to verify the legitimacy of the server.
(4) (Optional) Request Certificate. When the server needs to verify the client’s certificate,
the server will send a certificate request message to the client if mutual authentication
is selected.
(5) Server Hello Done. The server informs the client that the server has sent all the hand-
shake messages, and the server will wait for the client to send messages.
(6) (Optional) Response Certificate. If mutual authentication is selected, the client will
send its certificate to the server. Then the server will verify the identity of the client.
(7) Client Key Exchange. The client uses the server’s public key to encrypt the client’s
public key and key seed before sending them to the server.

Chapter 8. Local Control 203

 (8) (Optional) Certificate Verify. If mutual authentication is selected, the client uses the
local private key to generate a digital signature and sends it to the server for authenti-
cation through the received client public key.
(9) Create Secret Key. The communicating parties generate the communication key based
on information such as the key seed.
(10) Change Cipher Spec. The client notifies the server that the communication method
has been switched to encrypted mode.
(11) Finished. The client is ready for encrypted communication.
(12) Change Cipher Spec. The server notifies the client that the communication method
has been switched to the encrypted mode.
(13) Finished. Prepare for encrypted communication on the server side.
(14) Encrypted/Decrypted Data. Both parties use the client key to encrypt/decrypt the
communication content through a symmetric encryption algorithm.
(15) Closed Connection. After the communication is over, either party sends a message to
disconnect the TLS connection.

3. Creating an HTTP+TLS server with ESP-IDF

HTTPS, namely HTTP over SSL, encrypts HTTP data through the SSL or TLS protocol. Com-
pared with HTTP, HTTPS can prevent data from being stolen or changed during transmis-
sion, thus ensuring data integrity. Section 8.3.2 introduces how to use ESP-IDF to create an
HTTP server. In fact, creating an HTTPS server is similar. Call httpd_ssl_start() to
start the HTTP+TLS service, and call httpd_register_uri_handler() to register the
corresponding callback function.

Source code
For the source code of functions httpd ssl start() and httpd register uri hand
ler(), please refer to book-esp32c3-iot-projects/test case/https server.
1. static esperr_t root_get_handler(httpd_req_t *req)
2. {
3. httpd_resp_set_type(req, "text/html");
4. httpd_resp_send(req, "<h1>Hello Secure World! </h1>", HTTPD_RESP_USE_STRLEN);
5. return ESP_OK;
6. }
7.
8. static const httpd_uri_t root = {
9. .uri = "/",
10. .method = HTTP_GET,

204 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

11. .handler = root_get_handler
12. };
13.
14. esp_err_t esp_create_https_server(void)
15. {
16. httpd_handle_t server = NULL;
17. ESP_LOGI(TAG, "Starting server");
18. httpd_ssl_config_t conf = HTTPD_SSL_CONFIG_DEFAULT();
19. //Configure CA certificate and private key for the server
20. extern const unsigned char cacert_pem_start[] asm("_binary_cacert_pem_start");
21. extern const unsigned char cacert_pem_end[] asm("_binary_cacert_pem_end");
22. conf.cacert_pem = cacert_pem_start;
23. conf.cacert_len = cacert_pem_end - cacert_pem_start;
24. extern const unsigned char prvtkey_pem_start[] asm("_binary_prvtkey_pem_start");
25. extern const unsigned char prvtkey_pem_end[] asm("_binary_prvtkey_pem_end");
26. conf.prvtkey_pem = prvtkey_pem_start;
27. conf.prvtkey_len = prvtkey_pem_end - prvtkey_pem_start;
28. //Start the HTTP+TLS server
29. esp_err_t ret = httpd_ssl_start(&server, &conf);
30. if (ESP_OK ! = ret) {
31. ESP_LOGI(TAG, "Error starting server!" );
32. return ESP_FAIL;
33. }
34. //Set URI callback function
35. ESP_LOGI(TAG, "Registering URI handlers");
36. httpd_register_uri_handler(server, &root);
37. return ESP_OK;
38. }

The above code provides an example of how to create an HTTPS server. Before using this
code, please manually create a CA certificate and a private key in the main directory using
the following command:
$ openssl req -newkey rsa:2048 -nodes -keyout [Link] -x509 -days 3650 -out
[Link] -subj "/CN=ESP32 HTTPS server example"

Then modify the [Link] file to compile the certificate into the code.
1. idf_component_register(SRCS "station_example_main.c"
2. INCLUDE_DIRS "."
3. EMBED_TXTFILES "[Link]"
4. "[Link]")

In addition, you also need to go to [Link] menuconfig → Component config →

ESP HTTPS server, and configure CONFIG_ESP_HTTPS_SERVER_ENABLE.
Enter [Link] device IP]:443/ in the Chrome browser. The CA certificate on
the server side is not issued by a certification authority, thus it is not trusted. Therefore, you
will see the screen shown in Figure 8.16.

Chapter 8. Local Control 205

 Figure 8.16. Interface of untrusted HTTPS connection

Users need to click the “Advanced” button to allow this untrusted connection. Figure 8.17
shows the interface of a successful HTTPS connection.

Figure 8.17. Interface of successful HTTPS connection

Should you encounter a “Header fields are too long for server to interpret” message, just go
to [Link] menuconfig → Component config → HTTP Server → Max HTTP
Request Header Length, and increase HTTPD_MAX_REQ_HDR_LEN. Figure 8.18 shows
the interface where the HTTPS connection fails.

Figure 8.18. Interface of HTTPS connection failure

8.4.2 Introduction to Datagram Transport Layer Security (DTLS)

DTLS is a UDP-based protocol that serves the application layer. TLS protocol cannot guaran-
tee the security of data transmitted by UDP. Therefore, the DTLS protocol has been extended
on the existing TLS protocol architecture to support UDP, and becomes a version of TLS pro-
tocol that supports data packet transmission. DTLS 1.0 is based on TLS 1.1, and DTLS 1.2 is
based on TLS 1.2. The encryption algorithm, certificate, and encryption process of the DTLS

206 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

protocol are basically the same as those of the TLS protocol, thus will not be described in
this section.

1. Differences Between DTLS and TLS

The working principle of the DTLS protocol is basically the same as that of the TLS protocol,
except for the following differences:
• In the handshake stage, DTLS protocol has added the Cookie mechanism. The DTLS
protocol has added a Cookie mechanism in version 1.0, which is used by the server to
verify the client, and can avoid DoS attacks. When the client sends the Client Hello
message to the server, the server does not directly reply to the Server Hello message to
carry out the handshake process. Instead, the server replies the Hello Verify Request
message, which carries the Cookie value, to the client. When the client receives the
message, it will write the Cookie value into the Client Hello message and resend it to the
server. After receiving it, the server checks the local Cookie list to determine whether a
handshake is required.
• DLTS supports the retransmission mechanism. Since the UDP protocol itself does not
support retransmission like the TCP protocol, the DTLS protocol introduces a retrans-
mission mechanism. Taking the above Client Hello message as an example, after the
client sends the Client Hello message, the client will start a timer to receive the Hello
Verify Request message replied by the server; if the server does not reply within a certain
period of time, the client will resend the Client Hello message. Similarly, once a mes-
sage is sent, the server will activate a timer to monitor for timeouts and determine if the
message needs to be resent.
• DLTS supports orderly reception. UDP does not guarantee the order of delivered packets.
In contrast, DTLS protocol has added a message_seq field in the handshake message.
The receiver will provide a receiving buffer to receive out-of-order messages (similarly
to TCP), and process the messages in order according to the message_seq field.
• DLTS supports packet size limitation. UDP is a packet-oriented protocol, and TCP is a
stream-oriented protocol. TCP supports packet fragmentation and reassembly. However,
when a UDP message exceeds the maximum transmission unit (MTU) of the link layer, it
may be forcibly fragmented at the IP layer. The receiver then needs to process the frag-
mented packet based on the IP header and reassemble the original data. If one packet is
lost, the entire UDP message will be invalid. Therefore, in DTLS protocol, the handshake
messages are segmented on top of UDP. This is done by adding the fragment_offset
field and fragment_length field to the handshake message, which represent the offset
of this message relative to the beginning of the message and the length of this message,
respectively.

Chapter 8. Local Control 207

2. Creating a CoAP+DTLS server with ESP-IDF

The following example introduces how to create a CoAP+DTLS server. This example is ac-
tually the same as the CoAP example introduced in Section 8.3.4, except that two functions
are added to support the DTLS protocol. The function coap_context_set_psk() is used
to set the PSK encryption key in the DTLS protocol, and can also use certificate (PKI) for the
DTLS protocol handshake. The coap_new_endpoint(ctx, &serv_addr, COAP_PRO
TO_DTLS) function indicates that the node supports the DTLS protocol.

Source code
For the complete example code of coap context set psk(), please refer to book-
esp32c3-iot-projects/test case/coap. For instructions on how to use PKI,
please see esp-idf/examples/protocols/coap server.
1. static char psk_key[] = "esp32c3_key";
2. static void esp_create_coaps_server(void)
3. {
4. coap_context_t *ctx = NULL;
5. coap_address_t serv_addr;
6. coap_resource_t *resource = NULL;
7. while (1) {
8. coap_endpoint_t *ep = NULL;
9. unsigned wait_ms;
10.
11. //Create a CoAP server socket
12. coap_address_init(&serv_addr);
13. serv_addr.addr.sin6.sin6_family = AF_INET6;
14. serv_addr.addr.sin6.sin6_port = htons(COAP_DEFAULT_PORT);
15.
16. //Create CoAP ctx
17. ctx = coap_new_context(NULL);
18. if (!ctx) {
19. ESP_LOGE(TAG, "coap_new_context() failed");
20. continue;
21. }
22.
23. //Add PSK encryption key
24. coap_context_set_psk(ctx, "CoAP",
25. (const uint8_t *)psk_key,
26. sizeof(psk_key) - 1);
27.
28. //Set CoAP node
29. ep = coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_UDP);
30. if (!ep) {
31. ESP_LOGE(TAG, "udp: coap_new_endpoint() failed");
32. goto clean_up;

208 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

33. }
34.
35. //Add DTLS node and port
36. if (coap_dtls_is_supported()) {
37. serv_addr.addr.sin6.sin6_port = htons(COAPS_DEFAULT_PORT);
38. ep = coap_new_endpoint(ctx, &serv_addr, COAP_PROTO_DTLS);
39. if (!ep) {
40. ESP_LOGE(TAG, "dtls: coap_new_endpoint() failed");
41. goto clean_up;
42. } else {
43. ESP_LOGI(TAG, "MbedTLS (D)TLS Server Mode not configured");
44. }
45. }
46.
47. //Set CoAP resource URI
48. resource = coap_resource_init(coap_make_str_const("light"), 0);
49. if (!resource) {
50. ESP_LOGE(TAG, "coap_resource_init() failed");
51. goto clean_up;
52. }
53.
54. //Register callback functions for GET and PUT method corresponding to CoAP resource URI
55. coap_register_handler(resource, COAP_REQUEST_GET, esp_coap_get);
56. coap_register_handler(resource, COAP_REQUEST_PUT, esp_coap_put);
57.
58. //Set CoAP GET resource visible
59. coap_resource_set_get_observable(resource, 1);
60.
61. //Add resource to CoAP ctx
62. coap_add_resource(ctx, resource);
63. wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;
64. while (1) {
65. //Wait to receive CoAP data
66. int result = coap_run_once(ctx, wait_ms);
67. if (result < 0) {
68. break;
69. } else if (result && (unsigned)result < wait_ms) {
70. //Decrease waiting time
71. wait_ms -= result;
72. } else {
73. //Reset waiting time
74. wait_ms = COAP_RESOURCE_CHECK_TIME * 1000;
75. }
76. }
77. }
78. clean_up:
79. coap_free_context(ctx);

Chapter 8. Local Control 209

80. coap_cleanup();
81. }

8.5 Practice: Local Control in Smart Light Project

The local control component (esp_local_ctrl) of ESP-IDF enables you to control Espres-
sif chips via Wi-Fi+HTTPS or Bluetooth LE easily. With this component, you can access
application-defined properties, which can be read from or written to through a set of con-
figurable handlers. This section mainly introduces the local control module based on Wi-Fi.
Taking the smart light as an example, the local control module can be configured as follows:
• Configure the local device to discover mDNS protocol.
• Configure the local HTTPS server and certificate for data communication.
• Configure the smart lights.
The previous sections have introduced the control of ESP32-C3 via Bluetooth LE. When
using Bluetooth LE, the TCP/IP protocol stack is not involved, and local device discovery is
not necessary. Bluetooth has its own resource discovery service.

8.5.1 Creating a Wi-Fi-based Local Control Server

The following sample code implements the Wi-Fi-based local control server. The local con-
trol is based on HTTP for data communication, and the data is encrypted using the TLS
protocol. Additionally, the sample adds the mDNS module for device discovery.

Source code
For the complete example code, please refer to book-esp32c3-iot-projects/test
case/local control.
1. #define PROPERTY_NAME_STATUS "status"
2. static char light_status[64] = "{\"status\": true}";
3.
4. //Property type definition, used with scripts
5. enum property_types {
6. PROP_TYPE_TIMESTAMP = 0,
7. PROP_TYPE_INT32,
8. PROP_TYPE_BOOLEAN,
9. PROP_TYPE_STRING,
10. };
11.
12. //Get attribute value
13. esp_err_t get_property_values(size_t props_count,
14. const esp_local_ctrl_prop_t props[],
15. esp_local_ctrl_prop_val_t prop_values[],
16. void *usr_ctx)
17. {

210 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

18. int i = 0;
19. for (i = 0; i < props_count; i ++) {
20. ESP_LOGI(TAG, "Reading property : %s", props[i].name);
21. if (!strncmp(PROPERTY_NAME_STATUS,
22. props[i].name,
23. strlen(props[i].name))) {
24. prop_values[i].size = strlen(light_status);
25. prop_values[i].data = &light_status;//prop_values[i].data is
26. //just a pointer, and cannot be assigned.
27. break;
28. }
29. }
30. if (i == props_count) {
31. ESP_LOGE(TAG, "Not found property %s", props[i].name);
32. return ESP_FAIL;
33. }
34. return ESP_OK;
35. }
36.
37. //Set property value
38. esp_err_t set_property_values(size_t props_count,
39. const esp_local_ctrl_prop_t props[],
40. const esp_local_ctrl_prop_val_t prop_values[],
41. void *usr_ctx)
42. {
43. int i = 0;
44. for (i = 0; i < props_count; i ++) {
45. ESP_LOGI(TAG, "Setting property : %s", props[i].name);
46. if (!strncmp(PROPERTY_NAME_STATUS,
47. props[i].name,
48. strlen(props[i].name))) {
49. memset(light_status, 0, sizeof(light_status));
50. strncpy(light_status,
51. (const char *)prop_values[i].data,
52. prop_values[i].size);
53. if (strstr(light_status, "true")) {
54. app_driver_set_state(true); //Turn on the smart light
55. } else {
56. app_driver_set_state(false); //Turn off the smart light
57. }
58. break;
59. }
60. }
61. if (i == props_count) {
62. ESP_LOGE(TAG, "Not found property %s", props[i].name);
63. return ESP_FAIL;
64. }

Chapter 8. Local Control 211

65. return ESP_OK;
66. }
67. #define SERVICE_NAME "my_esp_ctrl_device"
68. void esp_local_ctrl_service_start(void)
69. {
70. //Initialise the HTTPS server-side configuration
71. httpd_ssl_config_t https_conf = HTTPD_SSL_CONFIG_DEFAULT();
72. //Load the server certificate
73. extern const unsigned char cacert_pem_start[] asm("_binary_cacert_pem_ start");
74. extern const unsigned char cacert_pem_end[] asm("_binary_cacert_pem_end");
75. https_conf.cacert_pem = cacert_pem_start;
76. https_conf.cacert_len = cacert_pem_end - cacert_pem_start;
77. //Load server-side private key
78. extern const unsigned char prvtkey_pem_start[] asm("_binary_prvtkey_pem_ start");
79. extern const unsigned char prvtkey_pem_end[] asm("_binary_prvtkey_pem_end");
80. https_conf.prvtkey_pem = prvtkey_pem_start;
81. https_conf.prvtkey_len = prvtkey_pem_end - prvtkey_pem_start;
82. esp_local_ctrl_config_t config = {
83. .transport = ESP_LOCAL_CTRL_TRANSPORT_HTTPD,
84. .transport_config = {
85. .httpd = &https_conf
86. },
87. .proto_sec = {
88. .version = PROTOCOM_SEC0,
89. .custom_handle = NULL,
90. .pop = NULL,
91. },
92. .handlers = {
93.
94. //User-defined processing function
95. .get_prop_values = get_property_values,
96. .set_prop_values = set_property_values,
97. .usr_ctx = NULL,
98. .usr_ctx_free_fn = NULL
99. },
100.
101. //Set the maximum number of attributes
102. .max_properties = 10
103. };
104.
105. //Initialise local discovery
106. mdns_init();
107. mdns_hostname_set(SERVICE_NAME);
108.
109. //Start the local control service
110. ESP_ERROR_CHECK(esp_local_ctrl_start(&config));
111. ESP_LOGI(TAG, "esp_local_ctrl service started with name : %s", SERVICE_NAME);

212 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

112. esp_local_ctrl_prop_t status = {
113. .name = PROPERTY_NAME_STATUS,
114. .type = PROP_TYPE_STRING,
115. .size = 0,
116. .flags = 0,
117. .ctx = NULL,
118. .ctx_free_fn = NULL
119. };
120. //Add attribute value
121. ESP_ERROR_CHECK(esp_local_ctrl_add_property(&status));
122.}

The above sample code implements the discovery of a device (domain name: my_esp_ctrl
_device.local) through the local discovery protocol (mDNS), establishes an HTTPS local
control connection, and allows the client to set and query attribute values via a registered
endpoint.
Users can enable the transmission security protection for local control via the following
options:
• PROTOCOM SEC0: specifies the end-to-end encryption algorithm used.
• PROTOCOM SEC1: specifies that data is exchanged as plain text.
• PROTOCOM SEC CUSTOM: customises security requirements.
Each attribute must have a unique name (a string), type (e.g., int, bool, or string), flag (e.g.,
read-only, or readable and writable), and size. If the property value is expected to be of
variable length (e.g., if the property value is a string or byte stream), the size should be
kept at 0. For fixed-length property value data types, such as int, float, etc., setting the size
field to the correct value helps esp_local_ctrl perform internal checks on parameters
received via write requests.
You can process it by matching props[i].name with the corresponding property name,
and further checking the flag and type of the property to determine if the property satis-
fies the corresponding flag and type requirements.
The default endpoints are shown in Table 8.5.

Table 8.5. Default endpoints

Endpoint name
URI (HTTPS Server + mDNS) Description
(BLE + GATT Server)
[Link] esp ctrl device.
esp local ctrl/version For retrieving version strings
local/esp local ctrl/version
[Link] esp ctrl device. For sending/receiving
esp local ctrl/control
local/esp local ctrl/control control messages

Chapter 8. Local Control 213

8.5.2 Verifying Local Control Functionality using Scripts
After introducing how to create a local control module, this section will further introduce
how to use scripts for verification. Here, we use the official example esp_local_ctrl as
an example for verification.
1. Create a certificate for TLS handshake between the client and the server.
a. Generate a rootCA that will be used to sign the server-side certificate, and the client
will use it to verify the server-side certificate during the SSL handshake. A passphrase
needs to be set to encrypt the generated [Link].
$ openssl req -new -x509 -subj "/CN=root" -days 3650 -sha256 -out [Link]
-keyout [Link]

b. Generate a certificate signing request and its private key [Link] for the server.
$ openssl req -newkey rsa:2048 -nodes -keyout [Link] -days 3650 -out
[Link] -subj "/CN=my esp ctrl [Link]"

c. Use the previously generated rootCA to process the server-side certificate signing re-
quest and generate the signing certificate [Link]. The passphrase set earlier for
the encrypted [Link] must be entered in this step.
$ openssl x509 -req -in [Link] -CA [Link] -CAkey [Link]
-CAcreateserial -out [Link] -days 500 -sha256

Among the generated certificates, [Link] and [Link] are compiled into
the server, and [Link] is suitable for client-side scripts for server-side verifica-
tion. The directory of the certificate can be set in the script esp_local_ctrl.py.
1. def get_transport(sel_transport, service_name, check_hostname):
2. ...
3. example_path = [Link][’IDF_PATH’] + ’/examples/protocols/esp_local_ctrl’
4. cert_path = example_path + ’/main/certs/[Link]’
5. ...

2. Use the following command to connect to the local control server via script. If sec_ver
is 0, it means that PROTOCOM_SEC0 is set on the server.
$ python esp local [Link] --sec ver 0

The script will automatically get the property value, i.e.:

Connecting to my_esp_ctrl_device.local

==== Starting Session ====

==== Session Established ====

==== Available Properties ====

S.N. Name Type Flags Value

[1] status STRING {"status": true}

214 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

3. According to the script prompt, enter the attribute number “1”, and set the attribute
value to {"status": false}. Then the script will automatically start querying and
find that the property value has been changed.

Select properties to set (0 to re-read, ’q’ to quit) : 1

Enter value to set for property (status) : {"status": false}
==== Available Properties ====
S.N. Name Type Flags Value
[1] status STRING {"status": false}

Select properties to set (0 to re-read, ’q’ to quit) :

8.5.3 Creating a Bluetooth-based Local Control Server

The following sample code creates a Bluetooth-based local control server, which can be
used to transmit data. You can refer to the gatt_server sample. The following example
implements a Bluetooth server. You can compile and flash this example, then use a Blue-
tooth debugging tool on your smartphone to scan and connect to a Bluetooth device named
ESP32C3-LIGHT. Once connected, you can access the Bluetooth services provided by the
device.

Source code
For the complete code of the gatt server example, please refer to book-esp32c3-
iot-projects/test case/gatt server.

This example provides two Bluetooth services: one for getting the device status (UUID: FF01)
and another for setting the device status (UUID: EE01), as shown in Figure 8.19.
The running log is as follows.
I (387) GATTS_DEMO: NVS Flash initialization
I (387) GATTS_DEMO: Application driver initialization
I (397) gpio: GPIO[9]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1|
Pulldown: 0| Intr:0
W (437) BTDM_INIT: esp_bt_controller_mem_release not implemented, return OK
I (437) BTDM_INIT: BT controller compile version [501d88d]
I (437) coexist: coexist rom version 9387209
I (437) phy_init: phy_version 500,985899c,Apr 19 2021,[Link]
I (617) system_api: Base MAC address is not set
I (617) system_api: read default base MAC address from EFUSE
I (617) BTDM_INIT: Bluetooth MAC: [Link]
I (637) GATTS_DEMO: REGISTER_APP_EVT, status 0, app_id 0
I (647) GATTS_DEMO: CREATE_SERVICE_EVT, status 0, service_handle 40
I (647) GATTS_DEMO: SERVICE_START_EVT, status 0, service_handle 40
I (647) GATTS_DEMO: ADD_CHAR_EVT, status 0, attr_handle 42, service_handle 40
I (657) GATTS_DEMO: ADD_DESCR_EVT, status 0, attr_handle 43, service_handle 40
I (667) GATTS_DEMO: REGISTER_APP_EVT, status 0, app_id 1

Chapter 8. Local Control 215

 Figure 8.19. Bluetooth services in the example

I (677) GATTS_DEMO: CREATE_SERVICE_EVT, status 0, service_handle 44

I (677) GATTS_DEMO: SERVICE_START_EVT, status 0, service_handle 44
I (687) GATTS_DEMO: ADD_CHAR_EVT, status 0, attr_handle 46, service_handle 44
I (697) GATTS_DEMO: ADD_DESCR_EVT, status 0, attr_handle 47, service_handle 44
I (6687) GATTS_DEMO: ESP_GATTS_CONNECT_EVT, conn_id 0, remote [Link]
I (6687) GATTS_DEMO: CONNECT_EVT, conn_id 0, remote [Link]
I (6987) GATTS_DEMO: ESP_GATTS_MTU_EVT, MTU 500
I (6987) GATTS_DEMO: ESP_GATTS_MTU_EVT, MTU 500
I (7347) GATTS_DEMO: update connection params status = 0, min_int = 16, max_int
= 32,conn_int = 24,latency = 0, timeout = 400
I (15117) GATTS_DEMO: GATT_READ_EVT, conn_id 0, trans_id 3, handle 42
I (23037) GATTS_DEMO: GATT_WRITE_EVT, conn_id 0, trans_id 4, handle 46
I (23037) GATTS_DEMO: GATT_WRITE_EVT, value len 1, value :
I (23037) GATTS_DEMO: 00
I (23037) app_driver: Light OFF
I (30987) GATTS_DEMO: GATT_WRITE_EVT, conn_id 0, trans_id 5, handle 46
I (30987) GATTS_DEMO: GATT_WRITE_EVT, value len 1, value :
I (30987) GATTS_DEMO: 01
I (30987) app_driver: Light ON

216 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

8.6 Summary
In this chapter, we first presented an overview of the framework model, applicable condi-
tions, and application scenarios of local control. We also compared it to remote control, en-
abling you to assess the suitability of local control functionality based on your unique project
requirements. Local discovery plays a pivotal role in local control as it governs the ability
of smartphones to search for devices within the LAN. This allows smartphones to retrieve
device characteristics and facilitates subsequent control of the identified devices. Therefore,
we also delved into the operational mode of the local discovery protocols, specifically in
terms of the principle layer. We also conducted a comparative analysis of the characteristics
of the two modes: broadcast and multicast, shedding light on their similarities and differ-
ences. The most commonly used local discovery protocol is mDNS, which provides you with
the flexibility to implement local discovery functionality in your IoT projects. You can also
leverage the resource discovery technology inherent in the Bluetooth protocol directly when
utilising Bluetooth for control purposes.
Then we introduced the most critical data communication protocols and corresponding data
encryption algorithms in local control. The fundamental protocols for data communication
in local control are TCP and UDP . While you can directly utilise these protocols for local con-
trol, it is generally not recommended due to certain limitations. TCP and UDP are classified
as transport layer protocols and do not inherently carry application format data, in contrast
to protocols like HTTP and CoAP, which incorporate an application layer on top of the trans-
port layer. Furthermore, it’s important to note that transport layer protocols such as TCP and
UDP do not support direct encryption of data using protocols like TLS or DTLS. Therefore,
relying solely on the transport layer protocol for data transmission may not guarantee the
security of the transmitted data. Hence, it is recommended that you utilise protocols such
as HTTP with TLS or CoAP with DTLS for data communication in local control scenarios.
Finally, we introduced how to implement the complete local control function based on the
esp_local_ctrl component in ESP-IDF. esp_local_ctrl supports local control based
on Wi-Fi and Bluetooth, with data communication protocols that include HTTPS for Wi-Fi
and Bluetooth protocols for Bluetooth. You can get started with local control development
with esp_local_ctrl component. Additionally, the esp_local_ctrl component does
not support local network device discovery functionality. You need to implement device dis-
covery functionality using the mDNS module. The esp_local_ctrl component is widely
used in ESP-IDF, and you can find provisioning and local communication features in the
wifi_provisioning component for network configuration and local communication.
Of course, there are various protocols and implementation methods for local control. If
the esp_local_ctrl component does not meet your requirements, you can follow the
example codes in sections 8.2, 8.3, and 8.4 to build your own local control framework.

Chapter 8. Local Control 217

 Chapter
Cloud Control
9

After reading the local control introduced in Chapter 8, you should know how to design the
local control function for your IoT projects. However, this function is far from enough, be-
cause a complete IoT project aims to connect all things and local control has a geographical
limitation: the smartphone must be in the same local area network (LAN) as the controlled
device. If you want to remotely control the IoT devices at home through your smartphone,
you will need the remote control function.
This chapter mainly introduces how to remotely control devices based on ESP32-C3. The
purpose is to help you understand what remote control and its process are, what protocol
is involved, how to build an MQTT server locally to simulate the cloud server, and how to
build a product model through ESP RainMaker for remote control of a device.

9.1 Introduction to Remote Control

What is remote control? As the name suggests, remote control refers to the behaviour of one
device (such as smartphones, computers, or other network devices) controlling another de-
vice through a wide area network (WAN). It is not restricted by region. For example, you can
control smart lights at home through your smartphone in the workplace. In general, both
the remotely-controlling device and the remotely-controlled device need to be connected to
the cloud server, and the commands sent by the controlling device are transmitted to the
controlled device over the cloud server.
Similar to local control (covered in Chapter 8), remote control is also a way of data commu-
nication, but it is over WAN other than LAN. In local control, the server can be the controlled
device itself, or a host in LAN; the controlling device (such as mobile phones or computers)
must be in the same LAN as the server, which is a limitation. In remote control, the server
is generally a cloud server (several large-scale cloud server providers are Alibaba Cloud,
Amazon Cloud, Tencent Cloud, etc.), the controlling device and the controlled device need
to be connected to the cloud server, and the data forwarding and storage are handled by the
cloud server.
The advantage of remote control is that the control is flexible and can break through the
limitation of space. However, compared with local control, it requires cloud services and

218
network traffic, thus more costly. Moreover, it usually has higher latency, resulting in a
greater risk of leaking data.
As the implementation principle and components of ESP RainMaker covered in Section 3.2
indicate, in remote control, both the controlling device (smartphone) and the controlled
device (such as ESP32-C3) are directly connected to the cloud server, which facilitates the
transfer of data between the devices. As a result, it is essential to have a thorough under-
standing of how these devices communicate with the cloud server.
Remote control costs more than local control as it requires cloud servers, but it is more
convenient to remotely view the operating status of the controlled device. Both have their
advantages and disadvantages. At present, most of the IoT devices on the market can be
connected to various clouds. For instance, the products of Xiaomi, Alibaba, and JD are
connected to their own cloud platform. The user only needs to download the corresponding
app and perform the provision and binding to view and control their IoT devices.
If your smartphone and the controlled device are on the same LAN, local control is a better
option. Otherwise, remote control has to be used. Local control has its own use scenarios
and advantages. The advantages of both should be fully utilized to develop the most suitable
IoT control technology.

9.2 Cloud Data Communication Protocols

Section 9.1 described what remote control was. As the topological structure of remote con-
trol shows, the smartphone and the controlled device are not directly connected. They are
connected to the cloud server, and the data sent by both are forwarded by the cloud. Then,
what is the protocol for connecting the device to the cloud? What is the protocol for data
communication? Only by figuring out these protocols can you have a basic understanding
of remote control.
At present, common protocols for connecting devices to the cloud are the MQTT protocol
and HTTP protocol. This chapter will only cover the former as the latter has been introduced
in Chapter 8.

9.2.1 MQTT Introduction

MQTT (Message Queue Telemetry Transport) is a server-client publish-subscribe messaging
transmission protocol. It is open, simple, lightweight, standardized, and easy to implement.
These characteristics make it a standard IoT transmission protocol that is ideal for resource-
constrained devices. The protocol was released by IBM in 1999. At present, it has been
developed to v5.x, and ESP-IDF supports v3.1.1. The two versions have significant differ-
ences and are not compatible with each other. Most cloud platforms currently still rely on
the older v3.x version. Therefore, in this chapter, we will be focusing on MQTT v3.x.

Chapter 9. Cloud Control 219

The MQTT protocol runs over the TCP protocol. It has the following features:
• The publish/subscribe pattern which supports one-to-many message distribution and
decoupling of applications.
• A messaging transport that is agnostic to the content of the payload.
• Three qualities of service (QoS) for message delivery.
• Small transport overhead and protocol exchanges minimised to reduce network traffic.
• Will messages to notify interested parties when an abnormal disconnection occurs.

9.2.2 MQTT Principles

The MQTT protocol is based on client-server communication. It defines three roles: pub-
lisher, broker, and subscriber. The publisher and the subscriber serve as the Client, which
can both publish and subscribe messages. The broker acts as the Server. Figure 9.1 shows
the architecture of the protocol.

Figure 9.1. Architecture of MQTT protocol

(1) Client: a device running MQTT applications, such as smartphones and controlled de-
vices. It can work as a publisher or subscriber. A Client always connects to the Server over
the network. It can:
• Publish application messages that other Clients might be interested in.
• Subscribe to request application messages that it is interested in receiving.
• Unsubscribe to remove a request for application messages.
• Disconnect from the Server.
(2) Server: a broker that acts as an intermediary between Clients publishing application
messages and Clients requesting to subscribe to them, such as cloud platforms and cloud
servers. It can:

220 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 • Accept network connections from Clients.
• Accept application messages published by Clients.
• Process subscribe and unsubscribe requests from Clients.
• Forward application messages that match Client subscriptions.
(3) Subscription: comprising a Topic Filter and a maximum QoS. It is associated with
a single Session, while a Session may contain multiple Subscriptions. Each Subscription
within a session has a different Topic Filter.
(4) Topic: the label attached to an application message. It is matched against the Subscrip-
tions known to the Server. The Server sends a copy of the application message to each Client
that has a matching Subscription.
(5) Topic Filter: an expression contained in a Subscription, indicating an interest in one or
more topics. A Topic Filter can include wildcard characters to represent single or multiple
characters.
(6) Session: a stateful interaction between a Client and a Server from the start to the end
of a connection. Some Sessions last only as long as the Network Connection.
(7) Publish/Subscribe: the core of the MQTT protocol. It allows communication between
subscribers and publishers without knowledge of each other’s IP address or port number.
Direct connection is not even necessary, and subscribers and publishers can operate with-
out knowing each other’s existence. Message exchanges between them are done by the
broker, which filters all the published messages and then distributes them to the matching
subscribers.
Both subscribers and publishers are concerned about the topic of the message. For example,
a smartphone wants to check the status of a smart light A. In this case, the smartphone
can act as a subscriber to subscribe to the message with the topic A/light_state from
the broker. Smart light A can act as a publisher. When its state changes, it publishes a
status message with the same topic to the broker. Then, the broker filters subscribers who
have subscribed to the topic and publishes the status message to them. In this way, the
smartphone can query the status of smart light A.

9.2.3 MQTT Message Format

As defined by the MQTT protocol, an MQTT control packet consists of three parts: fixed
header, variable header, and payload.
(1) Fixed header, present in all MQTT control packets.
As shown in Figure 9.2, the packet type takes 4 bits.
There are 14 types of control packets in total, as listed in Table 9.1.

Chapter 9. Cloud Control 221

 Figure 9.2. Fixed header of MQTT control packets

Table 9.1. Types of MQTT control packets

Name Value Direction of Flow Description

Reserved 0 Forbidden Reserved
CONNECT 1 Server ( Client Client request to connect to Server
CONNACK 2 Server ) Client Connection acknowledgement
PUBLISH 3 Server , Client Publish message
PUBACK 4 Server , Client Publish of QoS 1 message acknowledged
PUBREC 5 Server , Client Publish received (assured delivery part 1)
PUBREL 6 Server , Client Publish release (assured delivery part 2)
Publish of QoS 2 message complete
PUBCOMP 7 Server , Client
(assured delivery part 3)
SUBSCRIBE 8 Server ( Client Subscribe request of Client
SUBACK 9 Server ) Client Subscribe acknowledgement
UNSUBSCRIBE 10 Server ( Client Unsubscribe request
UNSUBACK 11 Server ) Client Unsubscribe acknowledgement
PINGREQ 12 Server ( Client PING request
PINGRESP 13 Server ) Client PING response
DISCONNECT 14 Server ( Client Client is disconnecting

There are three MQTT QoS levels: QoS 0, QoS 1, and QoS 2.
a. QoS 0: delivered once at most.
The transmission of messages is completely dependent on the underlying TCP/IP net-
work. As the MQTT protocol does not define response and retry, the message will either
reach the server only once or not at all. The flow of MQTT QoS 0 is shown in Figure 9.3.
b. QoS 1: delivered once at least.
The acknowledgement of message receipt is provided by the PUBACK message. If the
communication link or the sending device is abnormal, or the message is not received

222 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 9.3. Flow of MQTT QoS 0

within the specified time, the sender will redeliver the message, and in the fixed header
of the MQTT Control packet set the duplicate flag (DUP). The flow of MQTT QoS 1 is
shown in Figure 9.4.

Figure 9.4. Flow of MQTT QoS 1

c. QoS 2: delivered only once.

This is the highest quality of service where message loss and duplication are unacceptable
and increased overhead is incurred. The flow of MQTT QoS 2 is shown in Figure 9.5.
Bits [3–0] of the fixed header contain flags specific to each type of control packets. Except
the PUBLISH type, the flag bits of other types are taken by the system. For types without
flags, the bits are reserved. If invalid flags are received, the receiver must close the network
connection. The bits [3–0] in byte 1 of the PUBLISH packet header are as follows:
a. DUP (bit3): duplicate delivery.
“0” means that this is the first time that the Client or Server requests to send a PUBLISH

Chapter 9. Cloud Control 223

 Figure 9.5. Flow of MQTT QoS 2

message. “1” indicates that this may be a duplicate delivery of an earlier message. The
DUP flag of QoS 0 messages must be set to 0.
b. QoS (bits [2–1]): determining the number of message delivery.
Table 9.2 shows how to represent QoS values in bits [2–1].

Table 9.2. Representation of QoS values in bits [2–1]

QoS Value Bit2 Bit1 Description

0 0 0 Delivered once at most
1 0 1 Delivered once at least
2 1 0 Delivered only once
— 1 1 Reserved

c. RETAIN (bit0): determining the need for message retaining.

If this flag is set to 1 in a PUBLISH message sent by the Client to the Server, the Server
must store this message and its QoS to distribute them later to subscribers with a match-
ing topic. Each Client subscribing to a topic pattern that matches the topic of the retained

224 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 message receives the retained message immediately after they subscribe. The RETAIN
flag is usually used for will messages. For example, after a device is unexpectedly dis-
connected, the broker will send the will message to the smartphone, where the device
will be displayed as offline.
The second and subsequent bytes indicate the remaining length, indicating the number
of remaining bytes within the current packet, including data in the variable header and
the payload. The remaining length is encoded using a variable-length encoding scheme
that uses a single byte for values up to 127. For larger values, the least significant seven
bits of each byte encode the data, and the most significant bit is used to indicate whether
there are following bytes in the representation. Thus, each byte encodes 128 values and a
“continuation bit”. The maximum number of bytes in the remaining length field is 4B. The
number of remaining length bytes is shown in Table 9.3.

Table 9.3. Number of remaining length bytes

Byte Minimum Value Maximum Value

1 0 (0x00) 127 (0x7F)
2 128 (0x80, 0x01) 16383 (0xFF, 0x7F)
3 16384 (0x80, 0x80, 0x01) 2097151 (0xFF, 0xFF, 0x7F)
4 2097152 (0x80, 0x80, 0x80, 0x01) 268435455 (0xFF, 0xFF, 0xFF, 0x7F)

(2) Variable header.

Some types of MQTT Control Packets contain a variable header component. It resides be-
tween the fixed header and the payload. The content of the variable header varies depending
on the packet type. The Packet Identifier field of the variable header is present in multiple
types of packets, such as PUBLISH (when QoS¿0), PUBACK, PUBREC, PUBREL, PUBCOMP,
SUBSCRIBE, SUBACK, UNSUBSCRIBE, and UNSUBACK.
(3) Payload: the third part of an MQTT control packet, containing the message content.
It exists in five types of packets, CONNECT, SUBSCRIBE, SUBACK, UNSUBSCRIBE, and
PUBLISH.
a. CONNECT: Client ID, topic, message, user name, and password.
b. SUBSCRIBE: a series of topics to subscribe to and the QoS.
c. SUBACK: the server’s acknowledgment and reply to the topic (that the Client requests
to subscribe) and the QoS.
d. UNSUBSCRIBE: the topics to unsubscribe.
e. PUBLISH: the to-be-published application message, which can be zero-length.

Chapter 9. Cloud Control 225

9.2.4 Protocol Comparison
Chapter 8 introduced protocols such as TCP, HTTP, UDP, and CoAP, which can facilitate
local control. In addition, they can also be used for remote control.
MQTT vs. TCP
MQTT is an application protocol based on the TCP protocol. Both can be used for re-
mote data communication. For socket programming, TCP requires users to develop their
own application protocols, which have limited usability in the current environment of IoT
interconnection. On the other hand, MQTT is a standardized lightweight protocol for
IoT and is widely used by most cloud servers, such as Alibaba Cloud and Amazon Cloud,
making it advantageous for product integration.
MQTT vs. HTTP
Both are client-server application protocols based on TCP. However, compared to MQTT,
HTTP has a much larger overhead in message size, and it is generally difficult for an HTTP
server to initiate data push to clients, which may not meet the requirements of remote
control in IoT. In cases where only one-way transmission from clients to the server is
needed, HTTP protocol can be used.
MQTT vs. CoAP
Similar to HTTP, CoAP adopts the REST model where the server creates resources in URI
format and clients access these resources using GET, PUT, POST, and DELETE methods.
Besides, CoAP also has a similar protocol style to HTTP, but it requires fewer device re-
sources and network overheads, making it suitable for IoT. However, CoAP may not be a
good choice for remote control. If smartphones send control commands for remote con-
trol, the architecture may require CoAP + Web + Database + App. When CoAP protocol is
used, control commands must pass through the Database before reaching the device, be-
cause CoAP is connectionless. When smartphones send control commands, the server will
first store the control commands in the Database, and the device will request the server
via GET method to check if there are any control commands, and then decide whether to
operate. On the other hand, MQTT is connection-oriented, and the server will forward
the control commands from smartphones to all subscribed devices without storing them.
Only MQTT client + MQTT server + App is needed to implement remote control, making
MQTT more advantageous in terms of deployment.

9.2.5 Setting Up MQTT Broker on Linux and Windows

Some commonly used MQTT brokers are Mosquitto, EMQTT, and HiveMQ. HiveMQ is not
open-source and has a fee, so it may not be suitable for local testing. EMQTT has powerful
features, such as viewing data traffic on a web interface, and can be used on most cloud
servers, with both free and paid custom versions available.

226 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

This section focuses on how to use Mosquitto to set up an MQTT broker on Windows or
Linux. Mosquitto is an open-source (EPL/EDL licensed) message broker that implements
MQTT protocol versions 5.0, 3.1.1, and 3.1. It is considered a lightweight open-source
software. The Mosquitto project provides a C language library for implementing MQTT
clients and popular command-line MQTT clients mosquitto_pub and mosquitto_sub.
Besides, Mosquitto can also be used as an MQTT broker. For more information, please refer
to its official website.
1. Setting up MQTT broker on Linux
All terminal commands in this section must be run in the user role. The $ symbol represents
the command prompt.
(1) Download [Link] from [Link]
(2) Extract Mosquitto.
$ tar -zxvf [Link]

Check if the installation is successful using mosquitto --help.

$ cd mosquitto-2.0.12/src
$ mosquitto --help
mosquitto version 2.0.12
mosquitto is an MQTT v5.0/v3.1.1/v3.1 broker.
Usage: mosquitto [-c config_file] [-d] [-h] [-p port]
-c : specify the broker config file.
-d : put the broker into the background after starting.
-h : display this help.
-p : start the broker listening on the specified port.
Not recommended in conjunction with the -c option.
-v : verbose mode - enable all logging types. This overrides
any logging options given in the config file.
See [Link] for more information.

(3) Start MQTT broker and test in the MQTT client.

a. Start MQTT.
$ mosquitto

b. Use mosquitto_sub to subscribe to topic.

$ mosquitto sub -t ‘test/topic’ -v

c. Open a new terminal and use mosquitto_pub to publish data.

$ mosquitto pub -t ‘test/topic’ -m ‘hello world’

d. In the original terminal where the topic was subscribed, view the received data.
$ mosquitto sub -t ‘test/topic’ -v
test/topic hello world

Chapter 9. Cloud Control 227

2. Setting up MQTT broker on Windows
(1) Download the 32-bit or 64-bit MQTT installation package based on your computer’s
architecture. Double-click to install it.
(2) Open a command prompt window, navigate to the directory where Mosquitto is installed,
and start the Mosquitto broker.
cd C:\Program Files\mosquitto\

(3) Use mosquitto_sub.exe to subscribe to topic.

C:\Program Files\mosquitto>mosquitto [Link] -t ‘test/topic’ -v

(4) Use mosquitto_pub.exe to publish data.

C:\Program Files\mosquitto>mosquitto [Link] -t ‘test/topic’ -m ‘hello world’

9.2.6 Setting Up MQTT Client Based on ESP-IDF

The component used in ESP-IDF to implement MQTT client is ESP-MQTT, which has the
following features:
• Support for MQTT, MQTT over TLS, MQTT over WebSocket, and MQTT over Web-
Socket, and TLS
• Easy to set up with URI
• Multiple clients in one application
• Support for subscribing, publishing, authentication, last will messages, keep-alive pings,
and QoS messages
The following code is based on ESP-IDF and creates a connection to a local MQTT broker.
For complete code, please go to ESP-IDF project on GitHub and navigate to the directory
esp-idf/examples/protocols/mqtt/tcp.
1. //MQTT event handler function
2. static void mqtt_event_handler(void *handler_args, esp_event_base_t base,
3. int32_t event_id, void *event_data)
4. {
5. esp_mqtt_event_handle_t event = event_data;
6. esp_mqtt_client_handle_t client = event->client;
7. int msg_id;
8. switch ((esp_mqtt_event_id_t)event_id) {
9. case MQTT_EVENT_CONNECTED:
10. ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
11. //Subscribe to topic /topic/test
12. msg_id = esp_mqtt_client_subscribe(client, "/topic/test", 0);
13. ESP_LOGI(TAG, "sent subscribe successful, msg_id=%d", msg_id);
14. break;
15. case MQTT_EVENT_DISCONNECTED:
16. ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");

228 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

17. break;
18. case MQTT_EVENT_SUBSCRIBED:
19. ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
20. break;
21. case MQTT_EVENT_UNSUBSCRIBED:
22. ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
23. break;
24. case MQTT_EVENT_PUBLISHED:
25. ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
26. break;
27. case MQTT_EVENT_DATA:
28. ESP_LOGI(TAG, "MQTT_EVENT_DATA");
29. ESP_LOGI(TAG, "TOPIC=%.*s\r\n", event->topic_len, event->topic);
30. ESP_LOGI(TAG, "DATA=%.*s\r\n", event->data_len, event->data);
31. break;
32. case MQTT_EVENT_ERROR:
33. ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
34. break;
35. default:
36. ESP_LOGI(TAG, "Other event id:%d", event->event_id);
37. break;
38. }
39. }
40.
41. #define CONFIG_BROKER_URL "mqtt://[Link]/"
42.
43. static void esp_mqtt_start(void)
44. {
45. //Configure MQTT URI
46. esp_mqtt_client_config_t mqtt_cfg = {
47. .uri = CONFIG_BROKER_URL,
48. };
49.
50. //Initialise MQTT client
51. esp_mqtt_client_handle_t client = esp_mqtt_client_init(&mqtt_cfg);
52.
53. //Register event handler function
54. esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt_event_handler, NULL);
55.
56. //Start MQTT client
57. esp_mqtt_client_start(client);
58. }

The client device connects to the MQTT broker and subscribes to the topic /topic/test.
After another MQTT client publishes the message hello world to the topic /topic/test,
the following log will show up in the device:
I (2598) wifi station: MQTT_EVENT_CONNECTED

Chapter 9. Cloud Control 229

I (2598) wifi station: sent subscribe successful, msg_id=25677
I (2648) wifi station: MQTT_EVENT_SUBSCRIBED, msg_id=25677
I (314258) wifi station: MQTT_EVENT_DATA
I (314258) wifi station: TOPIC=/topic/test
I (314258) wifi station: DATA=hello world

9.3 Ensuring MQTT Data Security

Data transmitted using the MQTT protocol is in plain text, which means it can be intercepted
if not encrypted. In Chapter 8.4.1, the TLS protocol is introduced as a means to ensure
that data can only be decrypted by the communicating parties, thereby safeguarding data
security and legitimacy.
Similarly, TLS can also be used for encryption in cloud communication over MQTT. Since it
has already been covered in Chapter 8.4.1, this section only introduces what the certificates
in the TLS handshake mean and what functions they perform, how to generate certificates
locally, and how to set up a mutual authentication TLS environment based on the local
MQTT broker.

9.3.1 Meaning and Function of Certificates

1. Introduction
Certificates, also known as public-key certificates (PKC), contain personal information such
as user name, organization, email, the user’s public key, and the digital signature of a cer-
tification authority or certifying authority (CA). You can think of a certificate as a personal
identity card, with the public key serving as the card number, identifying which individual
it represents. A CA is like a police station that issues the card. It can be an international
organization, government entity, for-profit company, or general individual.
2. Certificate generation
(1) User A generates a private-public key pair locally using an asymmetric encryption al-
gorithm.
(2) User A submits the locally generated public key and certificate information file to a CA
for digital signing and certificate generation.
(3) The CA generates a private-public key pair locally, which is the [Link] mentioned in
a later example. The CA uses its private key to digitally sign User A’s public key and
issues the certificate.
(4) User B obtains the CA’s public key, which is publicly available, and uses it to verify the
legitimacy of the digital signature in User A’s certificate. If the verification is successful,
it is confirmed that the public key in User A’s certificate belongs to User A.

230 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 (5) To send data to User A, User B only needs to encrypt the data using the public key in
User A’s certificate and send it. User A then decrypts the data with its own private key.
So far, we have covered the generation of User A’s certificate and the data communication
process between User A and User B. However, it only involves one-way authentication of
TLS certificate—User A is like the server side, User B is like the client side, and this process
can be considered as the client’s verification of the server’s certificate. A similar process is
followed for the server’s verification of the client’s certificate.
3. Certificate function
As indicated by the above certificate generation process, certificates are a means to verify
the legitimacy of the peer device. Only when it is legitimate can the transmitted data can be
secured without the risk of being leaked.
4. Certificate standard
Certificates adopt the common format X.509. All certificates comply with the ITU-T X.509
international standard. The structure of X.509 certificates is described and encoded using
Abstract Syntax Notation One (ASN1).
A certificate typically consists of the following fields:
• Version Number: the version number of the specification. The current version is 3,
corresponding to the value 0x2.
• Serial Number: the unique serial number maintained by the CA and assigned to each
certificate for tracking and revocation. The maximum size is 20 bytes.
• Signature Algorithm: the algorithm used for digital signatures.
• Validity: the validity period of the certificate, including start and end dates.
• Subject: the identifier information of the certificate holder, namely the personal infor-
mation mentioned above.
• Subject Public Key Info: protected information related to the public key, including the
public key algorithm and subject unique identifier.
5. Certificate format
Privacy Enhanced Mail (PEM) is a common format for X.509 certificates. PEM files are
usually seen with the extensions .crt or .cer (for certificates), .key (for private keys),
and .csr (for certificate signing request).
The PEM file is a text file that usually contains headers, footers, and the content blocks
encoded in Base64.

Chapter 9. Cloud Control 231

9.3.2 Generating Certificates Locally
OpenSSL is an open-source Secure Socket Layer (SSL) cryptographic library that provides
functions for algorithms, key and certificate encapsulation management, and SSL protocol
implementation. It consists of three parts: an SSL protocol library, command-line tools for
applications, and cryptographic algorithm libraries. The following examples demonstrate
how use it to generate certificates and keys on Linux.
1. Generating private keys for the certificate
This command generates a private key (2048 bits) for the certificate. The public key can be
extracted from it.
$ openssl genrsa -out [Link] 2048
Generating RSA private key, 2048 bit long modulus
...............................................................................
............................................+++
.......+++
e is 65537 (0x10001)

This command generates a private key (2048 bits) for the server certificate.
$ openssl genrsa -out [Link] 2048
Generating RSA private key, 2048 bit long modulus
................+++
........................+++
e is 65537 (0x10001)

This command generates a private key (2048 bits) for the client certificate.
$ openssl genrsa -out [Link] 2048
Generating RSA private key, 2048 bit long modulus
...............................................+++
........................................................+++
e is 65537 (0x10001)

The recommended minimum key length for RSA algorithm is 2048 bits. If the key length is
1024 bits, mbedtls will reject TLS negotiation due to low security.
2. Generating CSRs for the certificate
This command generates a certificate sign request (CSR) that is required by the CA cer-
tificate. Enter the required information as prompted. The Organization Name can be
entered as desired because this is only for local use.
$ openssl req -out [Link] -key [Link] -new
You are about to be asked to enter information that will be incorporated into
your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,

232 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

If you enter ‘.’, the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:IOT Certificate Test
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

Please enter the following ‘extra’ attributes

to be sent with your certificate request
A challenge password []:
An optional company name []:

This command generates a CSR that is required by the server certificate. Note that the
Common Name field should be filled with the domain name or IP address of the server.
$ openssl req -out [Link] -key [Link] -new
You are about to be asked to enter information that will be incorporated into
your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter ‘.’, the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:MQTT Server
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:[Link]
Email Address []:

Please enter the following ‘extra’ attributes

to be sent with your certificate request
A challenge password []:
An optional company name []:

3. Generating CA certificate, server certificate, and client certificate

This command generates the CA certificate [Link].
$ openssl x509 -req -in [Link] -out [Link] -sha256 -days 5000 -signkey [Link]
Signature ok
subject=/C=CN/ST=Some-State/O=IOT Certificate Test
Getting Private key

This command generates the server certificate [Link].

Chapter 9. Cloud Control 233

$ openssl x509 -req -in [Link] -out [Link] -sha256 -CAcreateserial
-days 5000 -CA [Link] -CAkey [Link]
Signature ok
subject=/C=CN/ST=Some-State/O=MQTT Server/CN=[Link]
Getting CA Private Key

This command generates the client certificate [Link].

$ openssl x509 -req -in [Link] -out [Link] -sha256 -CAcreateserial
-days 5000 -CA [Link] -CAkey [Link]
Signature ok
subject=/C=CN/ST=Some-State/O=MQTT Client/CN=[Link]
Getting CA Private Key

NOTE
Do not use the SHA1 algorithm, as mbedtls may reject TLS negotiation due to low security.

9.3.3 Configuring MQTT Broker

Section 9.2.5 described how to set up an MQTT broker on Windows or Linux using Mosquitto,
and this section will introduce how to do it over TLS.
Firstly, in the root directory of mosquitto, open the configuration file [Link],
and add the absolute path to the files that are generated in Section 9.3.2, including the CA
certificate ([Link]), the server certificate ([Link]), and the private key of the server
certificate ([Link]). The command is as follows:
$ port 8883
certfile {absolute path}/[Link]
keyfile {absolute path}/[Link]
cafile {absolute path}/[Link]
require_certificate true
use_identity_as_username true

Then, restart Mosquitto and load the configuration file.

$ mosquitto -c [Link] -v
1635927859: mosquitto version 1.6.3 starting
1635927859: Config loaded from [Link].
1635927859: Opening ipv4 listen socket on port 8883.
1635927859: Opening ipv6 listen socket on port 8883.

9.3.4 Configuring MQTT Client

In Section 9.2.5, we have covered how to set up an MQTT client based on ESP-IDF using
MQTT over TCP, but this approach cannot guarantee data security. So, in this section, we
will introduce a more secure approach, which is setting up the client using MQTT over TLS.

234 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

[Link] const uint8_t client_cert_pem_start[] asm("_binary_client_crt_start");
2. extern const uint8_t client_cert_pem_end[] asm("_binary_client_crt_end");
3. extern const uint8_t client_key_pem_start[] asm("_binary_client_key_start");
4. extern const uint8_t client_key_pem_end[] asm("_binary_client_key_end");
5. extern const uint8_t server_cert_pem_start[] asm("_binary_ca_crt_start");
6. extern const uint8_t server_cert_pem_end[] asm("_binary_ca_crt_end");
7.
8. #define CONFIG_BROKER_URL "mqtts://[Link]/"
9.
10. //Configure MQTT URI
11. esp_mqtt_client_config_t mqtt_cfg = {
12. .uri = CONFIG_BROKER_URL,
13. .client_cert_pem = (const char *)client_cert_pem_start,
14. .client_key_pem = (const char *)client_key_pem_start,
15. .cert_pem = (const char *)server_cert_pem_start,
16. };

(1) Load the client certificate ([Link]), the client private key ([Link]), and
the CA certificate that authenticates the server ([Link]).
(2) Change the previous MQTT connection to mqtts.
(3) Use the default port number 8883.
(4) Modify the [Link] file to load the certificates into the firmware during
compilation. The code is as follows:
1. target_add_binary_data(${CMAKE_PROJECT_NAME}.elf "main/[Link]" TEXT)
2. target_add_binary_data(${CMAKE_PROJECT_NAME}.elf "main/[Link]" TEXT)
3. target_add_binary_data(${CMAKE_PROJECT_NAME}.elf "main/[Link]" TEXT)

After compilation and flashing, connect the device to Wi-Fi. Then, both the client and
server will show a successful connection log, and the client will subscribe to the topic
/topic/test. The server’s log is as follows:
1635927859: mosquitto version 1.6.3 starting
1635927859: Config loaded from [Link].
1635927859: Opening ipv4 listen socket on port 8883.
1635927859: Opening ipv6 listen socket on port 8883.
1635927867: New connection from [Link] on port 8883.
1635927869: New client connected from [Link] as ESP32_2465F1 (p2, c1, k120,
u’[Link]’).
1635927869: No will message specified.
1635927869: Sending CONNACK to ESP32_2465F1 (0, 0)
1635927869: Received SUBSCRIBE from ESP32_2465F1
1635927869: /topic/test (QoS 0)
1635927869: ESP32_2465F1 0 /topic/test
1635927869: Sending SUBACK to ESP32_2465F1

Now, use mosquitto_pub to send “hello world” to the topic /topic/test. Check if the
device can receive it. The command is as follows:

Chapter 9. Cloud Control 235

$ mosquitto pub -h [Link] -p 8883 -t \/topic/test" -m ‘hello world’
--cafile [Link] --cert [Link] --key [Link]

Here is the log on the device side:

I (1600) esp_netif_handlers: sta ip: [Link], mask: [Link], gw: [Link]
I (1600) wifi station: got ip:[Link]
I (1600) wifi station: connected to ap SSID:myssid password:12345678
I (1610) wifi station: Other event id:7
W (1630) wifi:<ba-add>idx:0 (ifx:0, [Link]), tid:0, ssn:4, winSize:64
I (4110) wifi station: MQTT_EVENT_CONNECTED
I (4120) wifi station: sent subscribe successful, msg_id=42634
I (4140) wifi station: MQTT_EVENT_SUBSCRIBED, msg_id=42634
I (10290) wifi station: MQTT_EVENT_DATA
I (10290) wifi station: TOPIC=/topic/test
I (10290) wifi station: DATA=hello world

9.4 Practice: Remote Control through ESP RainMaker

After reading the previous chapters, you should have a basic understanding of Wi-Fi con-
figuration and MQTT protocol. In this section, we will delve deeper into the Smart Light
project discussed in this book. We will utilise the ESP RainMaker to provide the smart light
with more functions, including remote control, local control, OTA upgrade, and scheduling.
Besides, we will enable third-party applications Alexa and Google Home to control the smart
light using Skill and facilitate voice control using voice assistants such as Alexa and Google
Assistant.
A standard voice assistant allows you to switch a smart light on or off, as well as adjust its
brightness. If the smart light supports colour and colour temperature adjustments, you can
send specific voice commands to alter these settings. Moreover, you can use voice assistant-
enabled speakers such as Echo and Nest to discover and control the light.

9.4.1 ESP RainMaker Basics

Before delving into the functions of ESP RainMaker, this section first explains some funda-
mental concepts that will be mentioned in the description of the ESP RainMaker framework
(backend and frontend). The ESP RainMaker framework is illustrated in Figure 9.6.

Figure 9.6. ESP RainMaker framework

236 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Node
It refers to the device model that represents the physical device (such as ESP32-C3) in the
cloud. Each node has a unique identifier, namely, node ID. It is the smallest operational
unit and a representation of the physical device in the ESP RainMaker framework.
Node attribute
It is used to better describe and define the functions of nodes. ESP RainMaker has set
default metadata for the node, including fw_version and model. The name and type
that are set when a node is created also belong to the default metadata. You can also add
your own information to the metadata to better describe the node.
Device
It is a logical entity that the user can control, such as a switch, smart light, temperature
sensor, or fan. Unlike a node, a device is the smallest unit that can be operated at the user
level.
Device attribute
Similar to node attribute, it is used to better describe and define functions of devices.
Service
In the ESP RainMaker framework, a service is a very similar entity to a device. The main
difference is that the service does not require operations from the user. For example, the
OTA upgrade service has some states that do not require any operation and management
from the user.
Parameter
It is used to implement functions of devices and services, such as the power status, bright-
ness, and colour of a smart light, and status updates during OTA upgrades.
The concepts of nodes, devices, parameters, and services in the ESP RainMaker framework
can aptly describe the form and functions of the product. For example, to create a smart
light with controllable power status, brightness, colour, and scheduled switching, the light is
represented by a node and a device, the power status, brightness, and colour are controlled
by parameters, and the scheduling function is achieved by a service.

9.4.2 Node and Cloud Backend Communication Protocol

The communication between the node and cloud backend is encrypted using TLS over
MQTT, and their identities are mutually authenticated using X.509 certificates. The private
key used for the node connection is automatically generated on the node.
During the first Wi-Fi provisioning, ESP32-C3 obtains a certificate through Assisted Claim-
ing, and saves it in its flash. The process for ESP32-C3 to use Assisted Claiming is shown in
Figure 9.7.

Chapter 9. Cloud Control 237

 Figure 9.7. ESP32-C3 Assisted Claiming

(1) ESP32-C3 generates an RSA2048 private key, uses its MAC address as the initial node
ID, and then sends relevant messages to the smartphone app.
(2) During the first provisioning, the app and the Claiming Service authenticate each
other’s identity. Once the authentication is successful, the receiving server issues a
node ID, which is then forwarded by the app to ESP32-C3.
(3) ESP32-C3 generates a CSR with the CN field set as the node ID. Then, the CSR is
forwarded by the app to the Claiming Service.
(4) The Claiming Service verifies the CSR and issues the certificate, which is then for-
warded by the app to ESP32-C3.
The node ID serves not only as a means of identifying a node during certificate application
but also as a way to map to a user and filter MQTT messages. For example, a node can only
subscribe to topics with a specific prefix (node/<node_id>/*) and publish messages to
those topics.

238 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

ESP RainMaker defines some default messages, including configuration messages, control
messages, status messages, initial status messages, mapping messages, OTA upgrade mes-
sages, and warning messages. These messages are packaged in JSON and sent to the cloud
backend over MQTT.
The configuration message is published by nodes through node/<node_id>/config. It
contains information about the node itself, its attributes, devices, device attributes, services,
and parameters. Here is an example.
1. //Configuration message for led_light
2. {
3. "node_id": "xxxxxxxxxx", //Node ID
4. "config_version": "2020-03-20", //Configuration version
5. "info": { //Node information
6. "name": "ESP RainMaker Device",
7. "fw_version": "1.0",
8. "type": "Lightbulb",
9. "model": "led_light"
10. },
11. "devices": [ //Devices of this node
12. {
13. "name": "Light",
14. "type": "[Link]",
15. "primary": "Power",
16. "params": [ //Device parameters
17. {
18. "name": "Name",
19. "type": "[Link]",
20. "data_type": "string",
21. "properties": ["read", "write"]
22. },
23. {
24. "name": "Power",
25. "type": "[Link]",
26. "data_type": "bool",
27. "properties": ["read", "write"],
28. "ui_type": "[Link]"
29. },
30. ......
31. ]
32. }
33. ],
34. "services": [ //Node services
35. {
36. "name": "OTA",
37. "type": "[Link]",
38. "params": [

Chapter 9. Cloud Control 239

39. {
40. "name": "Status",
41. "type": "[Link].ota_status",
42. "data_type": "string",
43. "properties": ["read"]
44. }
45. ......
46. ]
47. }
48. ]
49. }

The smartphone app can obtain the unique identifier of the product by parsing node_id, the
device information and number by parsing devices, the services by parsing the services,
and the read and write permissions of the app by parsing properties. If ui_type of
params in devices is set, the app will display the corresponding UI. For more information
on the use of standard parameters, standard devices, and standard UI, please refer to Section
9.4.7.
The downlink control message is used the app and third-party applications to control nodes.
It contains device parameters that need to be updated. To receive such messages, a node
needs to subscribe to the topic node/<node_id>/remote. Here is an example of control
messages.
1. {
2. "Light": {
3. "Power": false
4. }
5. }

A node can actively report its status messages through the topic node/<node_id>/params
/local. The cloud backend will cache the parameters in the message and push them to the
clients that have enabled the push function.
The mapping message is used to map nodes to users. An unmapped node needs to be
mapped to a user first to ensure that only that user has access to it. The mapping request
occurs during the Wi-Fi configuration phase. During Wi-Fi provisioning, the device receives
the user ID and security key from the smartphone. Once the node is connected to the cloud
backend, it will concatenate the user ID and security key with its own node ID and send
them to the cloud backend. Here is an example of mapping messages.
1. {
2. "node_id": "112233AABBCC",
3. "user_id": "02e95749-8d9d-4b8e-972c-43325ad27c63",
4. "secret_key": "9140ef1d-72be-48d5-a6a1-455a27d77dee"
5. }

After receiving the above message, the cloud backend will check whether it receives the

240 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

same security key from the app. If yes, it will map the user to the device. The mapping
process is shown in Figure 9.8.

Figure 9.8. Mapping process

The OTA upgrade message is used to implement OTA upgrades for nodes. It uses three MQTT
topics: node/<node_id>/otafetch, node/<node_id>/status, and node/<node_
id>/otaurl. These topics respectively report OTA upgrade status, distribute OTA upgrade
firmware, and query OTA upgrade tasks. The code is as follows:
1. //Distribute OTA upgrade firmware
2. {
3. "url": "<ota_image_url>",
4. "ota_job_id": "<ota_job_id>",
5. "file_size": "<num_bytes>"
6. }
7.
8. //Query OTA upgrade tasks
9. {

Chapter 9. Cloud Control 241

10. "node_id": "<node_id>",
11. "fw_version": "<fw_version>"
12. }
13.
14. //Report OTA upgrade status
15. {
16. "ota_job_id": "<ota_job_id>",
17. "status": "<in-progress/success/fail>",
18. "additional_info": "<additional_info>"
19. }

The warning message is a type of push messages used to notify and remind users. A node
can publish warning messages via the topic node/<node_id>/alert. After receiving a
warning message, the app pushes it to the smartphone notification bar. All the data in the
cloud backend have push properties, and the use of this topic explicitly marks the data as a
notification that needs to be actively pushed to the smartphone’s notification bar. Here is an
example of warning messages.
1.
2. "[Link]": "alert"
3.

9.4.3 Communication between Client and Cloud Backend

ESP RainMaker offers two client tools: app and CLI, both of which are implemented using
the RESTful API. This section briefly explains how to use the CLI tool that comes with the
device SDK to communicate with the cloud backend.
The CLI tool is a Python-based submodule of the esp-rainmaker repository, under the esp-
rainmaker/cli directory. To use it, please refer to Chapter 4 to set up the ESP-IDF envi-
ronment and export the ESP-IDF environment variables. You can verify whether the ESP-IDF
and Python environments are ready by running the following commands:
# Print ESP-IDF version
$ [Link] --version
ESP-IDF v4.3.2

# Print Python version

$ python3 --version
Python 3.6.9

A similar Shell output to the above indicates that the ESP-IDF environment is ready. Note
that the CLI tool depends on Python 3.x, and the older versions need upgrading.
After the ESP-IDF environment is ready, use pip to install the Python dependencies of the
CLI tool with the following commands:
$ cd your RainMaker path/esp-rainmaker/cli
$ pip install -r [Link]

242 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Collecting argparse
Using cached [Link] (23 kB)
...
...
...
Installing collected packages: cryptography, argparse
Attempting uninstall: cryptography
Found existing installation: cryptography 2.9.2
Uninstalling cryptography-2.9.2:
Successfully uninstalled cryptography-2.9.2
Successfully installed argparse-1.4.0 cryptography-2.4.2
WARNING: You are using pip version 21.1.2; however, version 21.3.1 is available.

Once the environment is set up, you can use the CLI tool to communicate with the cloud
backend. All the commands supported by the CLI are listed in Table 9.4, and you can view
the usage of each command by running python3 [Link] {h. Additionally, you
can use the parameter -h together with each command to view more help information.

Table 9.4. CLI commands

Command Description
signup Sign up for ESP RainMaker
login Login to ESP RainMaker
logout Logout current (logged-in) user
forgotpassword Reset the password
getnodes List all nodes associated with the user
getnodeconfig Get node configuration
getnodestatus Get online/offline status of the node
setparams Set node parameters
getparams Get the last parameter of the node in the cloud
removenode Remove user node mapping
provision Provision the node to join Wi-Fi network
getmqtthost Get the address of the MQTT host that the node connects to
claim Perform host driven claimming to the node and get the MQTT cerficate
test Test whether the node has been mapped to the user
otaupgrade Distribute OTA upgrade information
getuserinfo Get detailed information of the logged-in user
sharing Share the node

Chapter 9. Cloud Control 243

The claim command in the CLI tool is for host driven claiming, which is no longer supported
in ESP32-C3. Instead, the more convenient Self Claiming is supported in ESP32-C3.
Before using any other command, you need to first run the signup command to sign up for
an ESP RainMaker account:
$ cd your RainMaker path/esp-rainmaker/cli
$ cd python3 [Link] signup someone@[Link]
Choose a password
Password :
Confirm Password :
Enter verification code sent on your Email.
Verification Code : 973854
Signup Successful
Please login to continue with ESP Rainmaker CLI

Check the verification code in your email, as shown in Figure 9.9.

Figure 9.9. Verification code in email

Then, log in.

Execute the login command, and the Shell will open a web page, as shown in Figure 9.10.
Enter your account and password in the box.

244 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 9.10. Log in to ESP RainMaker using web browser

Alternatively, you can enter the command login together with the parameter -email to
directly log in using CLI:
$ python3 [Link] login --email someone@[Link]
Password:
Login Successful

9.4.4 User Roles

As mentioned earlier, user-node mapping aims to ensure that each node is controlled by a
unique user. There are two types of users in ESP RainMaker: admin users and end users.
Admin user:
a user who owns the MQTT certificate of a given node or gets the certificate through
claiming service. Admin users can access nodes via the ESP RainMaker dashboard, push
OTA firmware updates, and use ESP Insight for remote monitoring, but cannot read or
write node parameters.
End user:
a user who has the control permissions to a given node. End users can set and obtain node
parameters and configuration but cannot view nodes via the ESP RainMaker dashboard.
They are subdivided into primary users and secondary users.
Primary user:
a user who last performs the user-node mapping. Primary users can access node con-
figuration, read/write node parameters, and add/remove/view other secondary users.

Chapter 9. Cloud Control 245

 Secondary user:
any user who gets access to a node via node sharing. Secondary users can access node
configuration and read/write node parameters, but cannot add/remove/view other sec-
ondary users.

9.4.5 Basic Services

ESP RainMaker services are practical examples that integrate specific functions to facilitate
secondary development and enrich the functionality of nodes. For example, the scheduling
service provides devices with the offline timing/countdown function; the system service
offers the remote reboot and factory reset functions; the time & time zone service enables
the time zone switching function; the OTA upgrade service provides the remote update
function; and the local control service allows fast, stable, and secure LAN communication.
These services can be quickly implemented with simple configuration.
1. Time & time zone service
Fetching time is one of the most critical tasks for an IoT device after connecting to the
Internet, especially when the scheduling service is enabled. In ESP RainMaker, there are
two important concepts: time and time zone.
Time is typically fetched using Simple Network Time Protocol (SNTP). The ESP RainMaker
SDK provides an abstraction layer over the SNTP component in ESP-IDF, making it easy for
you to synchronise and check time. The code is as follows:
1. /*Initialise time synchronisation. This will call the SNTP component internally and
2. set the SNTP server through sntp_server_name passed by esp_rmaker_time_config_t*/
3. esp_err_t esp_rmaker_time_sync_init(esp_rmaker_time_config_t *config);
4.
5. //Check if time has been synchronised by comparison with the standard time 1546300800
6. bool esp_rmaker_time_check(void);
7.
8. //Wait for time to be synchronised
9. esp_err_t esp_rmaker_time_wait_for_sync(uint32_t ticks_to_wait);

As countries and regions in different longitudes have different local times and time zones,
the TZ environment variable and the tz_set() function are provided by ESP-IDF to set the
time zone. RainMaker provides an abstraction layer over this and provides multiple ways of
setting it. For example:
(1) Using the C API.
1. //Set time zone using the timezone region string
2. esp_err_t esp_rmaker_time_set_timezone(const char *tz);
3.
4. //Set time zone using the POSIX format
5. esp_err_t esp_rmaker_time_set_timezone_posix(const char *tz_posix);

(2) Modifying Default Timezone in menuconfig. To use this method, you need to have

246 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

some basic understanding of ESP-IDF. For more information, please refer to Chapter 4. The
configuration for this method is as follows:
(Top) → Component config → ESP RainMaker Common
Espressif IoT Development Framework Configuration
...
(Asia/Shanghai) Default Timezone
...

(3) Setting time zone directly on the client side via the time zone service. To enable this
service, call the following function on the device side:
1. esp_err_t esp_rmaker_timezone_service_enable(void);

2. Scheduling service
The scheduling service performs periodic modifications to device parameters. For example,
if you need to turn on a light at 7pm and turn it off at 11pm every day, this service can spare
you from manually turning it on and off. After configured, this service runs independently
on the device and does not rely on the network, which means that the device can perform
the configured operation correctly even when the device is disconnected from the network.
To enable this service, call the following function on the device side:
1. esp_err_t esp_rmaker_schedule_enable(void);

3. OTA upgrade service

ESP RainMaker provides the OTA upgrade service to update firmware. You only need to call
a simple API to enable it. There are two methods of performing OTA upgrade.
(1) OTA upgrade using parameters. This is the simplest way for developers to upgrade
firmware OTA. You only need to upload the firmware to any secure web server and provide
the URL to the node. This method can be triggered from the ESP RainMaker CLI client. The
otaupgrade command in the CLI tool is used to complete the upgrade, as shown below:
1. esp_rmaker_ota_config_t ota_config = {
2. .server_cert = ESP_RMAKER_OTA_DEFAULT_SERVER_CERT,
3. };
4. esp_rmaker_ota_enable(&ota_config, OTA_USING_PARAMS);

(2) OTA upgrade using MQTT topics. This is a more advanced method available to admin
users to upgrade firmware OTA. They need to upload the firmware to the dashboard and
create a task there to enable the OTA upgrade. The device will receive the OTA upgrade
URL and report the upgrade progress over MQTT. The code is as follows:
1. esp_rmaker_ota_config_t ota_config = {
2. .server_cert = ESP_RMAKER_OTA_DEFAULT_SERVER_CERT,
3. };
4. esp_rmaker_ota_enable(&ota_config, OTA_USING_TOPICS);

Chapter 9. Cloud Control 247

4. Local control service
Besides remote control, ESP RainMaker also enables the client to locally control the nodes
that are on the same Wi-Fi network as the client. This makes the entire process of control
and response faster and more reliable. ESP-IDF provides a component called ESP Local
Control, which uses mDNS-based discovery and HTTP-based control. It is now integrated
into the ESP RainMaker SDK.
Local control does not require adding a service to the node configuration message. It protects
data using asymmetric encryption algorithms and transmits the Proof of possession (PoP) to
the app through the local control service. The smartphone app completes encryption using
the PoP.
# Enable local control
CONFIG_ESP_RMAKER_LOCAL_CTRL_ENABLE=y

# Enable local control encryption

CONFIG_ESP_RMAKER_LOCAL_CTRL_SECURITY_1=y
5. System service
ESP RainMaker presets a set of system services for factory reset and remote reboot. Smart-
phone apps can use these services to erase the network configuration information on the
device and unmap users from the devices. To enable this service, call the following API on
the device side:
1. esp_err_t esp_rmaker_system_service_enable(esp_rmaker_system_serv_config_t *config)

9.4.6 Smart Light Example

The RainMaker SDK is built on top of ESP-IDF and provides simple APIs for building applica-
tions based on the ESP RainMaker specification. This section will explain and run the smart
light example. The code is as follows:
1. esp_rmaker_device_t *light_device;
2. //Callback function to handle commands received from ESP RainMaker
3. static esp_err_t write_cb(const esp_rmaker_device_t *device,
4. constesp_rmaker_param_t *param,
5. const esp_rmaker_param_val_t val,
6. void *priv_data,
7. esp_rmaker_write_ctx_t *ctx)
8. {
9. if (ctx) {
10. ESP_LOGI(TAG,
11. "Received write request via : %s",
12. esp_rmaker_device_cb_src_to_str(ctx->src));
13. }
14. const char *device_name = esp_rmaker_device_get_name(device);

248 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

15. const char *param_name = esp_rmaker_param_get_name(param);
16. if (strcmp(param_name, ESP_RMAKER_DEF_POWER_NAME) == 0) {
17. ESP_LOGI(TAG,
18. "Received value = %s for %s - %s",
19. [Link].b? "true" : "false",
20. device_name,
21. param_name);
22. app_light_set_power([Link].b);
23. } else if (strcmp(param_name, ESP_RMAKER_DEF_BRIGHTNESS_NAME) == 0) {
24. ESP_LOGI(TAG,
25. "Received value = %d for %s - %s",
26. [Link].i,
27. device_name,
28. param_name);
29. app_light_set_brightness([Link].i);
30. } else if (strcmp(param_name, ESP_RMAKER_DEF_HUE_NAME) == 0) {
31. ESP_LOGI(TAG,
32. "Received value = %d for %s - %s",
33. [Link].i,
34. device_name,
35. param_name);
36. app_light_set_hue([Link].i);
37. } else if (strcmp(param_name, ESP_RMAKER_DEF_SATURATION_NAME) == 0) {
38. ESP_LOGI(TAG,
39. "Received value = %d for %s - %s",
40. [Link].i,
41. device_name,
42. param_name);
43. app_light_set_saturation([Link].i);
44. } else {
45. //Omit parameters that do not need processing
46. return ESP_OK;
47. }
48. esp_rmaker_param_update_and_report(param, val);
49. return ESP_OK;
50. }
51.
52. void app_main()
53. {
54. //Initialise the driver layer
55. app_driver_init();
56.
57. //Initialise the NVS partition
58. esp_err_t err = nvs_flash_init();
59. if (err == ESP_ERR_NVS_NO_FREE_PAGES || err ==
60. ESP_ERR_NVS_NEW_VERSION_FOUND) {

Chapter 9. Cloud Control 249

61. ESP_ERROR_CHECK(nvs_flash_erase());
62. err = nvs_flash_init();
63. }
64. ESP_ERROR_CHECK( err );
65.
66. //Initialise Wi-Fi
67. app_wifi_init();
68.
69. //Initialise ESP RainMaker Agent
70. esp_rmaker_config_t rainmaker_cfg = {
71. .enable_time_sync = false,
72. };
73. esp_rmaker_node_t *node = esp_rmaker_node_init(&rainmaker_cfg,
74. "ESP RainMakerDevice",
75. "Lightbulb");
76. if (!node) {
77. ESP_LOGE(TAG, "Could not initialise node. Aborting!!!");
78. vTaskDelay(5000/portTICK_PERIOD_MS);
79. abort();
80. }
81.
82. //Create the device and add parameters
83. light_device = esp_rmaker_lightbulb_device_create("Light",
84. NULL,
85. DEFAULT_POWER);
86. esp_rmaker_device_add_cb(light_device, write_cb, NULL);
87. esp_rmaker_device_add_param(light_device,
88. esp_rmaker_brightness_param_create(
89. ESP_RMAKER_DEF_BRIGHTNESS_NAME,
90. DEFAULT_BRIGHTNESS));
91. esp_rmaker_device_add_param(light_device, esp_rmaker_hue_param_creat(
92. ESP_RMAKER_DEF_HUE_NAME,
93. DEFAULT_HUE));
94. esp_rmaker_device_add_param(light_device,
95. esp_rmaker_saturation_param_create(
96. ESP_RMAKER_DEF_SATURATION_NAME,
97. DEFAULT_SATURATION));
98. esp_rmaker_node_add_device(node, light_device);
99.
100. //Enable OTA upgrade
101. esp_rmaker_ota_config_t ota_config = {
102. .server_cert = ota_server_cert,
103. };
104. esp_rmaker_ota_enable(&ota_config, OTA_USING_PARAMS);
105.
106. //Enable time & time zone service

250 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

107. esp_rmaker_timezone_service_enable();
108.
109. //Enable scheduling service
110. esp_rmaker_schedule_enable();
111.
112. //Enable ESP Insight
113. app_insights_enable();
114.
115. //Enable ESP RainMaker Agent
116. esp_rmaker_start();
117.
118. //Enable Wi-Fi
119. err = app_wifi_start(POP_TYPE_RANDOM);
120. if (err != ESP_OK) {
121. ESP_LOGE(TAG, "Could not start Wifi. Aborting!!!");
122. vTaskDelay(5000/portTICK_PERIOD_MS);
123. abort();
124. }
125.}

Firstly, the above example initialises the hardware driver by configuring GPIO and initialising
peripherals. Next, the NVS partition is initialised in preparation for reading data from the
flash. The partition table [Link] is as follows:
1. # Name, Type, SubType, Offset, Size, Flags
2. # Note: Firmware partition offset needs to be 64K aligned, initial 36K (9
3. sectors) are reserved for bootloader and partition table
4. sec_cert, 0x3F, , 0xd000, 0x3000, ,
5. nvs, data, nvs, 0x10000, 0x6000,
6. otadata, data, ota, , 0x2000
7. phy_init, data, phy, , 0x1000,
8. ota_0, app, ota_0, 0x20000, 1600K,
9. ota_1, app, ota_1, , 1600K,
10. fctry, data, nvs, 0x340000, 0x6000

As shown in the above partition table, there are two NVS partitions in this example project:
nvs and fctry. The former is used to store network configuration and local scheduling
information and the latter is used to store certificate information.
Then, Wi-Fi is initialised. This step must be performed before the esp_rmaker_node_
init() function is called. If the fctry partition does not contain a certificate, Assisted
Claiming will be enabled because the MAC address can be used as the initial node ID when
Wi-Fi is initialised. Afterward, the device model is created and the callback function is
added. All cloud downlink data will be transmitted through this callback function, and
the ESP RainMaker core task is started. Finally, Wi-Fi is enabled. If the device has not
been connected to Wi-Fi, the provisioning application will be automatically started. The
application is implemented using the wifi_provisioning component in ESP-IDF and

Chapter 9. Cloud Control 251

started by calling app_wifi_start() in the ESP RainMaker SDK.
Run [Link] to compile and flash the led_light project and [Link] monitor to open
the monitor, and then you will see the following log:
I (30) boot: ESP-IDF v4.3.2-dirty 2nd stage bootloader
...
...
...
I (488) cpu_start: Starting scheduler.
I (493) gpio: GPIO[9]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1|
Pulldown: 0| Intr:3
I (503) coexist: coexist rom version 9387209
I (503) pp: pp rom version: 9387209
I (503) net80211: net80211 rom version: 9387209
I (523) wifi:wifi driver task: 3fca4d8c, prio:23, stack:6656, core=0
I (523) system_api: Base MAC address is not set
I (523) system_api: read default base MAC address from EFUSE
...
...
...
I (623) esp_rmaker_work_queue: Work Queue created.
I (623) esp_claim: Initialising Assisted Claiming. This may take time.
W (633) esp_claim: Generating the private key. This may take time.
I (110533) esp_rmaker_node: Node ID ----- 7CDFA161BE38
I (21213) esp_rmaker_node: Node ID ----- 7CDFA1C21DA0
I (21213) esp_rmaker_ota: OTA state = 2
I (21213) esp_rmaker_ota_using_params: OTA enabled with Params
I (21223) esp_rmaker_time_service: Time service enabled
I (21223) esp_rmaker_time: Initializing SNTP. Using the SNTP server: [Link]
I (21233) app_insights: Enable CONFIG_ESP_INSIGHTS_ENABLED to get Insights.
I (21243) esp_rmaker_core: Starting RainMaker Work Queue task
I (21253) esp_rmaker_work_queue: RainMaker Work Queue task started.
I (21253) esp_claim: Waiting for assisted claim to finish.
...
...
...

I (21623) app_wifi: If QR code is not visible, copy paste the below URL in a browser.
[Link]
e0","pop":"827e49ae","transport":"ble"}
I (21633) app_wifi: Provisioning Started. Name : PROV_8a20e0, POP : 827e49ae

252 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Use the smartphone app to scan the QR code. If no certificate is present in the fctry
partition, Assisted Claiming will be enabled, as shown in Figure 9.11.

Figure 9.11. Interface for enabling Assisted Claiming

...
...
I (444493) esp_claim: Assisted Claiming Started.
I (447603) esp_rmaker_core: New Node ID ----- nq8xT6p53BZHTm6k8AZqN
I (472813) esp_claim: Assisted Claiming was Successful.

After the certificate is obtained, the device enters the network configuration phase. The
smartphone app will send the selected SSID and password to the device, which will then
attempt to connect to the router and the cloud, as shown in Figure 9.12.
...
...
I (491113) esp_rmaker_user_mapping: Received request for node details
I (491113) esp_rmaker_user_mapping: Got user_id = 764865be-e49f-49d1-afa1-696d6
a7e3233, secret_key = a3c89473-514f-4aa4-a190-a9aa38e7a9d8
I (491123) esp_rmaker_user_mapping: Sending status SUCCESS
I (491753) app_wifi: Received Wi-Fi credentials
SSID : Xiaomi_32BD
Password : 12345678
I (495173) wifi:new:<11,0>, old:<1,0>, ap:<255,255>, sta:<11,0>, prof:1
I (495753) wifi:state: init -> auth (b0)
I (495793) wifi:state: auth -> assoc (0)
I (495833) wifi:state: assoc -> run (10)
I (495973) wifi:connected with Xiaomi_32BD, aid = 2, channel 11, BW20, bssid =
[Link]
I (495973) wifi:security: WPA2-PSK, phy: bgn, rssi: -25
I (495983) wifi:pm start, type: 1

Chapter 9. Cloud Control 253

I (495983) wifi:set rx beacon pti, rx_bcn_pti: 14, bcn_timeout: 14, mt_pti:
25000, mt_time: 10000
I (496043) wifi:BcnInt:102400, DTIM:1
W (496573) wifi:<ba-add>idx:0 (ifx:0, [Link]), tid:0, ssn:2, winSize:64
I (497503) app_wifi: Connected with IP Address:[Link]
I (497503) esp_netif_handlers: sta ip: [Link], mask: [Link], gw:
[Link]
I (497503) wifi_prov_mgr: STA Got IP
I (497503) app_wifi: Provisioning successful
I (497513) esp_mqtt_glue: Initialising MQTT
I (500973) esp_mqtt_glue: MQTT Connected

After completing the configuration and successfully connecting to the cloud, the device will
send a user-node association message, and the app will continue to check the association
status, as shown in Figure 9.13.

Figure 9.12. Device con- Figure 9.13. Check-

necting to router and cloud ing association status

I (45959) esp_rmaker_user_mapping: User Node association message published

successfully.
After the association, you can check this node using CLI.

254 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

9.4.7 RainMaker App and Third-Party Integrations
In Section 9.4.6, we have completed the device provisioning and user-node mapping, en-
abling control of a smart light through the app. As a result, the smart light icon and UI
interface are now displayed on the app’s homepage. The standard parameters, devices, and
UIs mentioned earlier are defined by ESP RainMaker and form the basis of its standard
framework. By using this framework, the app can accurately manage each device’s parame-
ters and supported services. These standard items, which are listed in the tables below, are
also applicable to third-party platforms.
(1) Standard UI device types

Table 9.5. Standard UI device types

Name Type Params GVA Alexa Image

Switch [Link] Name, Power* SWITCH SWITCH

Name, Power*,
Brightness, Color
Lightbulb [Link] Temperature, LIGHT LIGHT
Hue, Saturation,
Intensity

Name, Power*,
Brightness, Color
Light [Link] Temperature, LIGHT LIGHT —
Hue, Saturation,
Intensity

Name, Power*,
Fan [Link] FAN FAN
Speed, Direction

Temperature [Link]- Name, Tempera-

— TEMPERATURE SENSOR
Sensor sensor ture*

Outlet [Link] Name, Power OUTLET SMARTPLUG

Plug [Link] Name, Power OUTLET SMARTPLUG —

Socket [Link] Name, Power OUTLET SMARTPLUG —

Lock [Link] Name, Lock State LOCK SMARTLOCK

Chapter 9. Cloud Control 255

 Continuation of Table 9.5

Name Type Params GVA Alexa Image

Internal [Link]-
Name BLINDS INTERIOR BLIND —
Blinds internal

External [Link]-
Name BLINDS EXTERIOR BLIND —
Blinds externa

Garage Door [Link]-door Name GARAGE GARAGE DOOR —

[Link]-door-
Garage Lock Name GARAGE SMARTLOCK —
lock

Speaker [Link] Name SPEAKER SPEAKER —

Air [Link]-
Name AC UNIT AIR CONDITIONER —
Conditioner conditioner

Thermostat [Link] Name THERMOSTAT THERMOSTAT

TV [Link] Name TV TV —

Washer [Link] Name WASHER WASHER —

Other [Link] — — OTHER

(2) Standard UI types

The standard UI type added to a parameter is displayed as the corresponding UI shown
in Table 9.6 in the ESP RainMaker app.

Table 9.6. Standard UI types

Name Type Data Types Requirements Sample

Text (Default) [Link] All N/A

Toggle Switch [Link] bool N/A

Slider [Link] int, float Bounds (min, max)

Param type =
Brightness Slider [Link] int
[Link]

256 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Continuation of Table 9.6

Name Type Data Types Requirements Sample

CCT Slider [Link] int Param type = [Link]

Param type =
Saturation Slider [Link] int
[Link]

Hue Slider [Link]-slider int Param type = [Link]

Hue Circle [Link]-circle int Param type = [Link]

Push button (Big) [Link]-btn-big bool N/A

Bounds (min/max) for Int

Dropdown [Link] int/string
Valid strs for String

Trigger
[Link] bool N/A
(Android only)

Hidden
[Link] bool N/A Param will be hidden
(Android only)

(3) Standard parameter types

They are mapped to the parameters of corresponding names and UIs in the Alexa and
Google Home apps.

Table 9.7. Standard parameter types

Name Type Data Types UI Type Properties Min, Max, Step

Power [Link] bool [Link] Read, Write N/A

Brightness [Link] int [Link] Read, Write 0, 100, 1

CCT [Link] int [Link] Read, Write 2700, 6500, 100

Hue [Link] int [Link] Read, Write 0, 360, 1

Saturation [Link] int [Link] Read, Write 0, 100, 1

Chapter 9. Cloud Control 257

 Continuation of Table 9.7

Name Type Data Types UI Type Properties Min, Max, Step

Intensity [Link] int [Link] Read, Write 0, 100, 1

Speed [Link] int [Link] Read, Write 0, 5, 1

Direction [Link] int [Link] Read, Write 0, 1, 1

Temperature [Link] float N/A Read N/A

OTA URL [Link] url string N/A Write N/A

OTA Status [Link] status string N/A Read N/A

OTA Info [Link] info string N/A Read N/A

Timezone [Link] string N/A Read, Write N/A

Timezone POSIX [Link] posix string N/A Read, Write N/A

Schedules [Link] array N/A Read, Write, Persist N/A

Reboot [Link] bool N/A Read, Write N/A

[Link]-
Factory-Reset bool N/A Read, Write N/A
reset

Wi-Fi-Reset [Link]-reset bool N/A Read, Write N/A

Toggle Controller [Link] bool Any type applicable Read, Write N/A

Range Controller [Link] int/float Any type applicable Read, Write App Specific

Mode Controller [Link] string [Link] Read, Write N/A

Setpoint [Link]-
int/float Any type applicable Read/Write N/A
Temperature temperature

Lock State [Link] bool Any type applicable Read/Write N/A

[Link]-
Blinds Position int [Link] Read/Write 0, 100, 1
position

[Link]-
Garage Position int [Link] Read/Write 0, 100, 1
position

0, 2, 1

[Link]/ 0:invalid
Light Mode [Link]-mode int Read/Write
[Link]
1:HSV

2:CCT

AC Mode [Link]-mode string [Link] Read/Write N/A

258 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 (4) Standard service types
They are only used to quickly create services in the ESP RainMaker SDK.

Table 9.8. Standard service types

Name Type Params

OTA [Link] OTA URL, OTA Status, OTA Info
Schedule [Link] Schedules
Time [Link] TZ, TZ-POSIX
System [Link] Reboot, Factory-Reset, Wi-Fi-Reset

On the Skill page of Alexa or the Google Service Compatibility page of Google Home, sync
your ESP RainMaker devices. Link to your ESP RainMaker account. Then, you can use the
two apps to control the devices and use voice commands to operate them.
Figure 9.14 displays the ESP RainMaker devices in Alexa. They can be controlled using voice
commands such as “Alexa, please turn on the light”.

NOTE
Learn more about Alexa Skill at [Link]
Maker/dp/B0881W7RPV/.

Figure 9.14. ESP RainMaker devices in Alexa

Chapter 9. Cloud Control 259

Figure 9.15 displays the ESP RainMaker devices in the Google Home. They can be controlled
using voice commands such as “Hey Google, please turn off the light”.

Figure 9.15. ESP RainMaker devices in Google Home

ESP RainMaker builds an intermediate layer in the cloud backend. This layer maps the
standard parameter types and device types that are built into firmware to the formats that
Alexa Skill and Google Assistant can understand. Therefore, device types in ESP RainMaker,
such as smart lights and switches, are mapped to similar device types in Alexa Skill and
Google Assistant, and their parameters, such as switch, brightness, and color, are mapped
to corresponding capabilities and traits. If you only set brightness, you will get a smart light
with adjustable brightness in Alexa and Google Home. If you also include color and CCT,
you can adjust its color and color temperature.

9.5 Summary
In this chapter, we introduced remote control and the MQTT protocol. It is a commonly used
protocol in remote control and connection of IoT devices to the cloud. It is now adopted
by many mainstream cloud platforms, such as Amazon Cloud, Alibaba Cloud, Baidu Cloud,
Tencent Cloud, and ESP RainMaker introduced in this chapter. This simple and lightweight
protocol provides reliable network services for IoT devices in low-bandwidth and unstable
network environments.
Besides, we also covered how to build an MQTT broker locally to simulate the cloud plat-
form, and how to generate server and client certificates for the TLS protocol handshake to
ensure data security.

260 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

In the practice in this chapter, we took the development of a smart light product as an
example to complete the remote control of devices using the ESP RainMaker platform over
MQTT. Self Claiming is used to obtain certificates. Multiple sets of MQTT topics are used
for device control, user-node mapping, and device status. The built-in basic services can
perform the scheduling operation and OTA upgrade. With the completed cloud connection
function of ESP RainMaker, smart lights can quickly be given voice control capabilities.
ESP RainMaker’s capabilities are not limited to this. Data collection and analysis, device-to-
device linkage, and third-party scene triggers are all interesting functions yet not mentioned
in this chapter. With these cloud functions, we can roughly calculate power consumption by
counting the online/offline time and frequency of smart lights and found out how they work
together with other hardware. These functions can be achieved using the open RESTful API.
Chapter 10 will introduce the use of the RESTful API and use them to develop a smartphone
app.

Chapter 9. Cloud Control 261

 Chapter
Smartphone App Development
10

In Chapter 9, we introduced how to control devices through the cloud using Wi-Fi technolo-
gies, as well as how to communicate with the cloud using MQTT protocol and TLS protocol
to ensure data security and validity.
In this chapter, we will learn to develop our own smartphone application to wirelessly con-
trol the smart lights. It should enable users to connect smart home devices with the control
system based on Wi-Fi, Bluetooth, and other wireless communication technologies, so as
to control these devices and transfer data easily even if they are thousands of miles away.
It depends on network and data and obtains detailed information of devices through the
smartphone.
In daily life, we may simply send commands to the built-in Wi-Fi module in home appliances
through wireless network to implement corresponding control, such as switching on/off a
smart light, or setting its color or brightness.

10.1 Introduction to Smartphone App Development

In the Practice section of Chapter 9, we explained how to use ESP RainMaker client to
wirelessly control the Smart Light project. In this chapter, we will turn to its smartphone
app and present its development process in detail.
The app project is available for both iOS and Android. It is an end-to-end solution provided
by Espressif to remotely control and monitor IoT devices based on Espressif chips without
any configuration required in the cloud.

Source code
The source code of the iOS app is stored in the book-esp32c3-iot-projects/
phone app/app ios directory. The source code of the Android app is stored in the
book-esp32c3-iot-projects/phone app/app android directory.

Don’t worry if you have not developed any Android or iOS apps before. In this chapter,
we will elaborate on how to develop a smartphone app starting from creating a new app
project. After you have a basic understanding of Android and iOS application development,

262
we will dive deeper into the main features of this open-source project.

10.1.1 Overview of Smartphone App Development

The smartphone app for controlling smart lights includes both iOS and Android versions.
The iOS version is developed in Xcode and written in Swift, whereas the Android version
is developed in Android Studio and programmed in Java. In this chapter, we will first
make prototype diagrams according to requirement documents and API documents, then
design interfaces and interactions, and finally implement the design for each interface based
on prototype diagrams. We will use Git to manage codes, get updated versions, compare
versions, and commit changes.
The implementation of the smartphone app requires permission to access the smartphone’s
camera, local network, location, Bluetooth, etc. It is based on network request, ESP Pro-
vision SDK for provisioning, data parsing, pop-ups, and other third-party frameworks, and
involves the development of functions such as device list, scheduling, user centre, login and
registration.
In the following sections, we will introduce the project structure and the lifecycle of the
smartphone app in Android and iOS, so that you can develop apps more easily with a pre-
liminary understanding.

10.1.2 Structure of the Android Project

This section takes MyRainmaker App as an example to introduce the structure of an Android
project. There are two folders in the root directory, namely app and Gradle
Scripts. The app folder contains all the code and resources to develop the smartphone
app, and the Gradle Scripts folder contains scripts related to Gradle compilation. The
structure of this Android project is shown in Figure 10.1.
App folder
The app folder contains three subfolders: manifests, java, and res.
• manifests: Stores app configurations, including name, version, SDK, and permis-
sions.
• java: Mainly stores source code and test code.
• res: Stores all project resources.
Gradle Scripts folder
The Gradle Scripts folder contains [Link] (two files with the same name),
[Link], [Link], [Link],
[Link], and [Link].
• [Link]: Compiles the app with Gradle.
• [Link]: Configures the Gradle version.

Chapter 10. Smartphone App Development 263

 Figure 10.1. Structure of the Android project

• [Link]: Configures proguard rules to obfuscate the code.

• [Link]: Configures Gradle-related global properties.
• [Link]: Configures relevant Gradle scripts.
• [Link]: Configures the path to the SDK/NDK.

10.1.3 Structure of the iOS Project

This section takes MyRainmaker Apps an example to introduce the structure of an iOS
project. As the navigation view shows in Figure 10.2, the project includes a MyRainmaker
folder for source code, a MyRainmakerTests folder for unit test code, and a MyRainma-
kerUITests folder for UI test code.
MyRainmaker folder
This folder contains the AppDelegate, SceneDelegate, ViewController, Main,
Assets, LaunchScreen, and Info files.
• AppDelegate: The entry file of the entire app, which stores the app’s delegate class.
• SceneDelegate: New class added to Xcode 11 that handles scenes split from Ap-
pDelegate.
• ViewController: Host controller class that controls view display and handles touch
events, etc.
• Main: The main interface storyboard, which contains view controller scenes in the
app and describes the connection between multiple view controllers.
• Assets: The file that stores most images.
• LaunchScreen: Configures the app’s launch screen.
• Info: Configures app permissions, such as Bluetooth, location, and camera permis-
sions.

264 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 10.2. Structure of the iOS project

Test files
MyRainmakerTests, MyRainmakerUITests, and MyRainmakerUITestsLaunch-
Tests are all test classes. They are not commonly used in project development, and thus
are not explained in detail.

10.1.4 Lifecycle of an Android Activity

Android has four basic components – Activity, Service, ContentProvider, and BroadcastRe-
ceiver, among which Activity is used very frequently and handles almost all interface inter-
actions. Now, let’s explore the lifecycle of an activity following the Figure 10.3.
• onCreate() indicates that the activity is being created. It is the first method through
an activity’s lifecycle and is where you should do the initialization.
• onStart() indicates that the activity is being started and is visible to users.
• onRestart() indicates that the activity is being restarted, and should be called when
the activity changes from invisible to visible. For example, when users press the Home
button to switch to the desktop, or open a new activity, the current activity will be
stopped. When the current activity returns to the front, the onRestart() method
will be called.
• onResume() indicates that the activity has been created and users can operate and
interact on the interface.
• onPause() indicates that the activity is paused, and usually onStop() will be called
immediately after. If users quickly return to the current activity, onResume() will be

Chapter 10. Smartphone App Development 265

 Figure 10.3. Lifecycle of an activity

called.
• onStop() indicates that the activity is about to stop. It is no longer visible to users
and is running only in the background.
• onDestroy() indicates that the activity is about to be destroyed. This is the last
method executed in an activity’s lifecycle and is where you should reclaim or release
resources.

10.1.5 Lifecycle of iOS ViewController

The lifecycle of the ViewController refers to the lifecycle of the views it controls. When the
state of a view changes, the ViewController will automatically call a series of methods in
response to the change. The lifecycle of the ViewController is shown in Figure 10.4.
Each method is used as follows:
• init() initializes relevant, critical data.
• loadView() initializes the view. This method should not be called directly, but auto-
matically by the system.

266 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 10.4. Lifecycle of ViewController

• viewDidLoad() indicates that the view is loaded, but not yet been displayed on the
screen. By overriding this method, you can perform additional initialization on views,
such as removing some views, modifying constraints, loading data, etc.
• viewWillAppear() indicates that the view is about to be displayed on the screen.
You may use this method to change the orientation or style of the status bar.
• viewDidApper() indicates that the view has been displayed on the screen. You may
use this method to modify how the view is presented.
• viewWillDisappear() indicates that the view is about to disappear, be covered, or
be hidden.
• viewDidDisappear() indicates that the view has disappeared, been covered, or
been hidden.

Chapter 10. Smartphone App Development 267

10.2 Creating a New Smartphone App Project
Now that we have learned about the development of Android and iOS project, let’s start
creating a new app project. Since the provisioning function of the app requires a Blue-
tooth module, simulator does not meet the demand. Please prepare a smartphone for the
development and debugging of the app.
Before creating a new smartphone app project, you need to download the corresponding
development tools. IDE tools have integrated all the necessary environments, and do not
require setting environment variables and other tedious work.

10.2.1 Preparing for Android Development

The Android app may be developed on Linux, Mac, or Windows using Android Studio. It
should be targeted at Android 6.0 and above, and can be built with Java and Kotlin.
Kotlin is preferred for developing the Android app. As a JVM-oriented language, it is fully
compatible with Java, and offers more flexible syntax and powerful features. Back in 2017,
Google announced that it would support Kotlin on Android as a first-class language. With
experience in Java development, you can easily get started with Kotlin.

10.2.2 Creating a New Android Project

To create a new Android project, proceed as follows:
After downloading and installing Android Studio on your PC, open it and you should see the
interface as shown in Figure 10.5. Then click “New Project”.

Figure 10.5. Interface of Android Studio

Select “Empty Activity”, click “Next”, and a “New Project” dialog box as Figure 10.6 will pop

268 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

up. Specify the Name of your project (e.g., MyRainmaker) and the Package name, and set
the Save location, the Language (Kotlin), and the Minimum SDK (Android 6.0 and above).
Then click “Finish” to create a new project.
When you create a project for the first time, it may take some time to automatically down-
load all the dependencies, so please be patient.

Figure 10.6. The “New Project” dialog box

10.2.3 Adding Dependencies for MyRainmaker

Add repository to [Link] (Project Settings).
1. repositories {
2. ...
3. maven { url ’[Link] }
4. }

Add dependencies to [Link] (Module: [Link]).

1. dependencies {
2. implementation ’[Link]:eventbus:3.2.0’
3. implementation ’[Link]:
4. esp-idf-provisioning-android:lib-2.0.11’
5. implementation ’[Link]:rainmaker-proto-java:1.0.0’
6. implementation ’[Link]:protobuf-javalite:3.14.0’
7. implementation ’[Link]:tink-android:1.6.1’
8. }

Chapter 10. Smartphone App Development 269

Click “Sync Now” or the “ ” (Sync) button in the upper right corner to download the
dependencies, as shown in Figure 10.7.

Figure 10.7. Download dependencies

10.2.4 Permission Request in Android

Add the required permissions to the [Link] file.
1. //Location permissions
2. <uses-permission android:name="[Link].ACCESS_FINE_LOCATION" />
3. //Bluetooth permissions
4. <uses-permission android:name="[Link]" />
5. <uses-permission android:name="[Link].BLUETOOTH_ADMIN" />
6. //Network permissions
7. <uses-permission android:name="[Link]" />

Besides being decalred as static, location permissions should also be requsted in the activity
as follows:
1. registerForActivityResult([Link]())
2. { granted ->
3. //Result callback
4. if (granted) {
5. //Permission granted
6. } else {
7. //Permission denied
8. }
9. }.launch([Link].ACCESS_FINE_LOCATION)

Add the code above to the activity’s onCreate() method, so that the app will request
location permissions whenever being launched.

10.2.5 Preparing for iOS Development

The iOS app may be developed on computers runing macOS 10.12 and above using Xcode
(available on App Store). It should be target at iOS 11.0 and above, and can be built with
Swift and Objective-C.
Swift is preferred as it is faster, safer, and more interactive. It removes pointers and other

270 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

unsafe access in Objective-C, and switches from the smalltalk-style syntax used by Objective-
C to dot notation. All the sample code in this chapter is written in Swift.

10.2.6 Creating a New iOS Project

To create a new iOS project, proceed as follows:
After downloading and installing Xcode on your PC, open it, click “Create a new Xcode
project”, select “iOS” → “App” as shown in Figure 10.8, and click “Next”. You should see a
new prompt for your project details.

Figure 10.8. Select “iOS” → “App”

Figure 10.9. Set project details

Chapter 10. Smartphone App Development 271

Set the Product Name (e.g., MyRainmaker), Team, Organization Identifier, Interface, Life
Cycle, and Language (Swift), as shown in Figure 10.9. Click “Next”, and you should see a
prompt for the project’s location. Set the storage path and click “Create”.

10.2.7 Adding Dependencies for MyRainmaker

Open the terminal, navigate to the project directory, and execute the following command to
generate a Podfile.
% touch Podfile

Open the Podfile and add dependencies.

1. # Uncomment the next line to define a global platform for your project
2. platform :ios, ’12.0’
3.
4. target ’ESPRainMaker’ do
5. # Comment the next line if you’re not using Swift and don’t want to
6. use dynamic frameworks
7. use_frameworks!
8.
9. # Pods for ESPRainMaker
10.
11. pod ’MBProgressHUD’, ’˜> 1.1.0’
12. pod ’Alamofire’, ’˜> 5.0.0’
13. pod ’Toast-Swift’
14. pod ’ReachabilitySwift’
15. pod ’JWTDecode’, ’˜> 2.4’
16. pod ’M13Checkbox’
17. pod ’ESPProvision’
18. pod ’DropDown’
19. pod ’FlexColorPicker’
20.
21. end
22.
23. post_install do |installer|
24. .pods_project.[Link] do |target|
25. target.build_configurations.each do |config|
26. config.build_settings[’IPHONEOS_DEPLOYMENT_TARGET’] = ’12.0’
27. end
28. end
29. end

Execute the following command to download dependencies.

% pod install

After download, open the project folder and double-click [Link] to

open the project, as shown in Figure 10.10.

272 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 10.10. Double-click [Link]

The structure of the project is shown in Figure 10.11.

Figure 10.11. Structure of the project

10.2.8 Permission Request in iOS

Add the following permissions to [Link] in the MyRainmaker folder.
• key NSBluetoothAlways Usage Description and key NSBluetooth
Peripheral UsageDescription for Bluetooth permission.
• key NSCamera Usage Description for camera permission to scan QR codes.
• key NSLocationWhenInUseUsage Description for location permission. (It is
required for devices running iOS 13 and above to access SSID.)
• key NSLocalNetworkUsageDescription for local network permission. (It is re-
quired for devices running iOS 14 and above to communicate over local network.)

Chapter 10. Smartphone App Development 273

10.3 Analysis of the App’s Functional Requirements
In the previous sections, we have introduced how to create a new app project, along with its
structure and lifecycle. Now, to help you understand the development of app functionalities
more concretely, we have included the source code of the smartphone app project in our
GitHub, and you can import it into Android Studio/Xcode to run for for reference.
The main function of the smartphone app is to configure devices developed based on Espres-
sif’s chips and modules to a designated router, and send commands through the app to con-
trol these devices, such as smart lights and sensors. Another function is to set the device
status at a specified time using the scheduling module, for example, to turn on the water
heater on the way home, so that hot water will be available as soon as you arrive.

Figure 10.12. Functional requirements of the project

274 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

10.3.1 Analysis of the Project’s Functional Requirements
Before developing the app, you should first understand the functional modules and specific
functions to be implemented in this project. The smartphone app project in this chapter
mainly includes modules such as user management, provisioning, and device control (more
functions illustrated in Figure 10.12). The following sections will provide a module-by-
module breakdown.

10.3.2 Analysis of User Management Requirements

User registration and login are implemented in one interface and switched through the
toggle button. The critical part of this module is to allow third-party accounts, and the
network requests for registration, login, verification code acquisition, etc., and data parsing.
The analysis of user management requirements is shown in Figure 10.13.

Figure 10.13. Analysis of user management requirements

• To log in with a third-party account such as GitHub, Apple, or Google, the app will first
open a webpage in the browser and obtain the unique identifier of the account.
• Forgot password and verification code request can be implemented via corresponding
cloud APIs.
• Passwords entered will be displayed by default in ciphertext and can be switched to
plaintext for confirmation by clicking the toggle (eye icon).
• Email addresses entered will be validated using regular expressions.
• Documentation will include documents for the entire project and can be accessed from
various places in the app.

Chapter 10. Smartphone App Development 275

10.3.3 Analysis of Device Provisioning and Binding Requirements
There are two ways to provision a device. One is Bluetooth provisioning, where the app
connects and communicates with the device through Bluetooth, provides provisioning data
for the device, and allows it to join the network. Another way is SoftAP provisioning, where
the device starts a Wi-Fi hotspot for the smartphone to connect and communicate with each
other. Once the device is provisioned, enter PIN to bind the device over cloud. The analysis
of device provisioning and binding requirements is shown in Figure 10.14.

Figure 10.14. Analysis of device provisioning and binding requirements

• For Bluetooth provisioning, the app needs to implement Bluetooth scanning, connec-
tion, subscription, packet transmission and other functions.
• For SoftAP provisioning, the app needs to navigate to the system setting interface,
connect to the device’s Wi-Fi hotspot, and display information about the connected
Wi-Fi hotspot.
• The implementation of device binding after provisioning is the same for both ways and
can be reused.

10.3.4 Analysis of Remote-Control Requirements

Once the device gets provisioned and bound, users will be able to use the smartphone app
to monitor and control it remotely. In addition, users can also control multiple devices at the
same time and create groups to manage them. The analysis of remote-control requirements
is shown in Figure 10.15.
• In the smartphone app, all the provisioned devices will be displayed in a list and can
be turned on/off easily.

276 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

 Figure 10.15. Analysis of remote-control requirements

• By selecting a specified device, users will enter its independent control interface, which
varies for each device type. For example, it might be used to control switches, bright-
ness, and color.
• The device details interface will show device ID, type, version, location, etc., and allow
users to analyze and unbind devices.
• By creating groups, devices can be controlled and managed together.

10.3.5 Analysis of Scheduling Requirements

Figure 10.16. Analysis of scheduling requirements

The scheduling function is relatively simple, very similar to alarm clocks we use every day. It
mainly includes functions like creating schedules, list of schedule events, editing schedules,

Chapter 10. Smartphone App Development 277

enabling/disabling schedules, etc. The analysis of scheduling requirements is shown in
Figure 10.16. The details of a schedule refer to its event name, date, time, recurring pattern,
etc.

10.3.6 Analysis of User Centre Requirements

The user centre module mainly features user profile, notification, changing password, terms
of use, project documents, privacy policy, voice services, and logout. Note that the chang-
ing password and logout functions need to call the cloud API. The analysis of user centre
requirements is shown in Figure 10.17.

Figure 10.17. Analysis of user centre requirements

10.4 Development of User Management

After going through the analysis of the project’s functional requirements in Section 10.3, you
should already have an overall picture of modules and functions that need to be developed,
as well as the required frameworks and third-party libraries. In this section, we will put all
the modules and functions into code. Based on the new projects and permissions configured
before, now you need to know the classes designed for each interface and associations
between them, in order to achieve better operation through code. The code for each function
would be encapsulated to be reused and modularised.

10.4.1 Introduction to RainMaker APIs

RainMaker cloud supports two types of APIs: Unauthenticated and Authenticated.
Unauthenticated APIs do not have any authentication tokens in the HTTP header and will
receive access_token in the response when users log in successfully. Authenticated APIs
are marked in the Swagger file with a “lock” sign in the front. Their access_token needs

278 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

to be passed for authentification in the Authorization HTTP header.
For RainMaker API documentation, please refer to [Link]
com.
When smartphones communicate with the RainMaker cloud, the underlying protocol is Hy-
perText Transfer Protocol Secure (HTTPS). HTTPS can authenticate the server to protect
the privacy and integrity of the exchange data. The HTTPS body received by the RainMaker
cloud is in JSON format.

10.4.2 Initiating Communication via Smartphone

Android and iOS provide good native support for HTTPS and JSON.
• The Android system uses JSONObject and JSONArray to assemble and parse JSON
objects and arrays, and HttpURLConnection to initiate HTTPS requests.
• The iOS system uses NSJSONSerialization to assemble and parse JSON data, and
URLSession to initiate HTTPS requests.
Of course, you can also use third-party HTTPS and JSON libraries.

10.4.3 Account Registration

First, we need to implement the registration of a new account, which will be used to bind the
device in subsequent steps and control it remotely. In this project, the account is registered
via email address.

Figure 10.18. SIGN UP interface

Chapter 10. Smartphone App Development 279

There is a toggle button in the registration interface to switch between “SIGN IN” and “SIGN
UP”. For the SIGN UP interface, there are three input fields: Email, Password, and Confirm
Password. The content of the Password and Confirm Password fields can be shown or hidden
by toggling their visibility, so that users can check whether they have entered the correct
password. The password should contain at least one uppercase letter and a number.
Before clicking “Register”, users must read and agree to the Privacy Policy and Terms of Use.
Then, it will navigate to a verification interface, and a digital code will be sent to the email
address. Users need to enter the correct digital code to complete the registration procedure.
The SIGN UP interface is shown in Figure 10.18.
Figure 10.19 demonstrates the API for account registraion. For detailed information, please
refer to [Link]

Figure 10.19. API for account registration

The account registration function is implemented as follows:

Create a new account. Below shows the account registration API, where user_name refers
to the email address used for registration, and password to the password.
1. POST /v1/user
2. Content-Type: application/json
3.
4. {
5. "user_name": "username@[Link]",
6. "password": "password"
7. }

To create a new account in Android, use:

1. @POST
2. Call<ResponseBody> createUser(@Url String url, @Body JsonObject body);

Source code
For the source code of creating a new account in Android, please refer to
book-esp32c3-iot-projects/phone app/app android/app/src/main/java/
com/espressif/cloudapi/[Link].

To create a new account in iOS, use:

280 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

1. func createNewUser(name: String, password: String) {
2. [Link](endPoint: .createNewUser(url: [Link], name: name,
3. password: password), encoding: [Link]) { data, error in
4. [Link](data, withError: error) { umError in
5. [Link]?.verifyUser(withName: name, andPassword:
6.
7. }

Source code
For the source code of creating a new account in iOS, please refer to book-esp32c3-
iot-projects/phone app/app ios/ESPRainMaker/ESPRainMaker/
UserManagement/Interactors/[Link].

Verify the account after receiving the digital code. Below shows details of the API, where
user_name refers to the email address used for registration, and verification_code
to the digital code.
POST /v1/user
Content-Type: application/json

{
"user_name": "username@[Link]",
"verification_code": "verification_code"
}

To verify the account with digital code in Android, use:

1. @POST
2. Call<ResponseBody> confirmUser(@Url String url, @Body JsonObject body);

Source code
For the source code of verifying the account in Android, please refer to book-esp32c3-
iot-projects/phone app/app android/app/src/main/java/com/
espressif/cloudapi/[Link].

To verify the account with digital code in iOS, use:

1. func confirmUser(name: String, verificationCode: String) {
2. [Link](endPoint: .confirmUser(url: [Link], name: name,
3. verificationCode: verificationCode),
4. encoding: [Link]) { data, error in
5. [Link](data, withError: error) { umError in
6. [Link]?.userVerified(withError: umError)
7. }
8. }
9. }

Chapter 10. Smartphone App Development 281

Source code
For the source code of verifying the account in iOS, please refer to book-esp32c3-
iot-projects/phone app/app ios/ESPRainMaker/ESPRainMaker/
UserManagement/Interactors/[Link].

10.4.4 Account Login

After account registration, we can call the account login API to get the token for authenti-
cation and the basic profile.
The smartphone app project in this chapter supports login via third-party accounts such as
GitHub, Apple, and Google. So long as users have accounts of these three platforms, they
can log in directly in the app without registration.
If users have already registered new accounts, they can also log in to the app by entering
their email address and password.
If users forget their password, they can click “Forgot password?” under the “Sign in” button
to reset password.
At the bottom of the SIGN IN interface are the project-related documentation, privacy agree-
ment, terms of use, and the app version. The SIGN IN interface is shown in Figure 10.20.

Figure 10.20. SIGN IN interface

282 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The account login function is implemented as follows:
Request an access token. The API is shown in Figure 10.21 and details can be found at
[Link]

Figure 10.21. API for account login

POST /v1/login
Content-Type: application/json

{
"user_name": "username@[Link]",
"password": "password"
}

The server responds to the request as follows:

{
"status": "success",
"description": "Login successful",
"idtoken": "idtoken",
"accesstoken": "accesstoken",
"refreshtoken": "refreshtoken"
}

Among these fields, status tells whether the request is successful; description provides
details of the request; accesstoken is the token to be added to the HTTP request header
by all APIs requiring user permissions, in the format of Authorization:$acces-
stoken; idtoken and refreshtoken are not used for now and thus not explained here.
To request access token in Android, use:
1. @POST
2. Call<ResponseBody> login(@Url String url, @Body JsonObject body);

Source code
For the source code of requesting access token in Android, please refer to book-
esp32c3-iot-projects/phone app/app android/app/src/main/java/
com/espressif/cloudapi/[Link].

To request access token in iOS, use:

Chapter 10. Smartphone App Development 283

1. func loginUser(name: String, password: String) {
2. [Link](endPoint: .loginUser(url: [Link],
3. name: name, password: password),
4. encoding: [Link]) { data, error in|
5. [Link](data,
6. error: error) { _, umError in
7. [Link]?.loginCompleted(withError: umError)
8. }
9. }
10. }

Source code
For the source code of requesting access token in iOS, please refer to book-esp32c3-
iot-projects/phone app/app ios/ESPRainMaker/ESPRainMaker/
UserManagement/Interactors/[Link].

Get user profile. The API is shown in Figure 10.22 and details can be found at https:
//[Link]/#/User/getUser.

Figure 10.22. API to get user profile

GET /v1/user
Authorization: $accesstoken

In response to the “get user profile” request, the server returns:

{
"user_id": "string",
"user_name": "string",
"super_admin": true,
"picture_url": "string",
"name": "string",
"mfa": true,
"phone_number": "<+Mobile Number with country code>"
}

Among these fields, user_id is the user’s unique identifier and will be used later in provi-
sioning; user_name refers to the account; super_admin is returned true only when the
user is a super admin; picture_url points to the user’s profile picture; phone_number
is the user’s mobile phone number; name and mfa are not used in this project and thus not
explained here.
To get user profile in Android, use:
1. @GET
2. Call<ResponseBody> fetchUserDetails(@Url String url);

284 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Source code
For the source code of getting user profile in Android, please refer to book-esp32c3-
iot-projects/phone app/app android/app/src/main/java/com/
espressif/cloudapi/[Link].

To get user profile in iOS, use:

1. func fetchUserDetails() {
2. [Link] { accessToken, error in
3. if let token = accessToken, [Link] > 0 {
4. [Link](endPoint: .fetchUserDetails(url: [Link],
5. accessToken: token), encoding:
6. [Link]) { data, error in
7. [Link](data,
8. withError: error) { umError in
9. [Link]?.userDetailsFetched(error: umError)
10. return
11. }
12. }
13. } else {
14. [Link]?.userDetailsFetched(error: error)
15. }
16. }
17. }

Source code
For the source code of getting user profile in iOS, please refer to book-esp32c3-
iot-projects/phone app/app ios/ESPRainMaker/ESPRainMaker/
UserManagement/Interactors/[Link].

10.5 Development of Device Provisioning

As described in Section 10.4, we can get access token and user_id of the RainMaker
account through APIs for account login and getting user profile. The next step is to find
the device, connect it to the router, and activitate it on the cloud. The suitable provision-
ing library for the app is idf-provisioning, which is encapsulated based on ESP-IDF
provisioning.
For provisioning methods, please refer to [Link]
Figure 10.23 illustrates the data exchange between the smartphone and the device during
provisioning. This is also mentioned in Section 7.3.4 Bluetooth Provisioning.

Chapter 10. Smartphone App Development 285