Integration of Python-Based AI Models

with Simulation and Hardware Deployment

Ryotaro ABE

Electrical Device Development,

TS TECH Co.,Ltd.
 Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

All Rights Reserved. Copyright TS TECH CO.,LTD.

Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

3
 Corporate Profile
FY2024 consolidated sales
TS TECH Co., Ltd.
(Prime Market, Tokyo Stock Exchange
Name: TS TECH Securities Code: 7313 )
Consolidated
net sales
\441.7
billion
■ President Masanari Yasuda 【Sales breakdown】
Automobile business 93.1％
■ Established Dec. 5, 1960 Motorcycle business 1.8％
Other 5.1％
■ Head office Asaka-shi, Saitama

■ Capital stock 4.7 billion yen Interior

components
■ Total number of shares issued 136 million 8.2％

■ Lines of business
Manufacture and sale of automobile seats;
automobile interiors; motorcycle seats; Seats
and resin-based products for motorcycles 84.9％
■ Consolidated number of employees
14,719

4
 Lines of Business
Automobile business Motorcycle business
Business

▲Automobile
seats ▲▼Motorcycle seats
fields

▲Lighting
▲Door trim components

Honda Honda Suzuki

customers

Suzuki Yamaha Kawasaki

Main

Harley-Davidson
Volkswagen
5
Internal Structure of a Multifunction Seat
Electronic
Main components components
Frame Side airbag
Occupant posture sensor

Safety features

Electronic control unit (ECU)

(passenger seat)
Cushion (urethane, etc.)
Electronic
components Occupant weight sensor
(shown in (passenger seat)
Trim cover blue)
Seat position sensor
(driver’s seat)

Power slide

Comfort features
Power reclining
Power height adjuster
Air lumbar support
Ventilation system

*Image of seat features

6
Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

7
Proposing New Value for Cabin Space

Safety/Security HMI
・Intuitive Communication(five senses)
・Occupant Protection Evolution ・Arrangement-compatible operation
・Privacy

Environment Health
・Sensing
・Sustainable Materials
・Postural Improvement
・Easy Disassembly

Human
・Motion Sickness Reduction
・Recycling・Maintenance
・Hardware Evolution
・Coordinating User Experience Energy
Spatial Value ・Exploration of Human Nature
Saving
・Immersive Experience
with Five Senses ・Human-centered
・Mass Customization Heating/Cooling

Arrangement Feeling ・Energy Harvesting

・New Feeling/Posture
・Variable Space Including Stopping
for Use Cases
・New Way to Spend Time

Pursue “Human-centered” comfortable and attractive spaces

8
 Challenges for New Value

Automotive OEM
Interior Parts
Manufacturer
“Cabin Space”
with New Value

Startup Companies
Other Industries
We need …
Cooperation with companies with diverse development cultures

9
 Integrate functionality using MATLAB®/Simulink®
Interior Parts
mfr.

FUNC:1 OEM FUNC:6

Startup Other
FUNC:3 FUNC:4 FUNC:5
Companies Industries

FUNC:2 FUNC:7

MATLAB®/Simulink® = environment for the integration of “functions”

10
Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

11
 Development Background

Safety/Security HMI
・Intuitive Communication(five senses)
・Occupant Protection Evolution ・Arrangement-compatible operation
・Privacy

Environment Health
・Sensing
・Sustainable Materials
・Postural Improvement
・Easy Disassembly

Human
・Motion Sickness Reduction
・Recycling・Maintenance
・Hardware Evolution
・Coordinating User Experience Energy
Spatial Value ・Exploration of Human Nature
Saving
・Immersive Experience
with Five Senses ・Human-centered
・Mass Customization Heating/Cooling

Arrangement Feeling ・Energy Harvesting

・New Feeling/Posture
・Variable Space Including Stopping
for Use Cases
・New Way to Spend Time
We need …
technology to know “Human-centered”
the state of human comfortable and attractive spaces
12
Need for Driver Fatigue Estimation
Development of automated driving technology

・Adaptive Cruise Control

2020 2030
501,567 2,399,000
(actual) (predict)
・Traffic Jam Assist

ADAS(Lv2+) Market Forecast

Low-priced car
・Lane Change Assist

Reliance on Driver Assistance System

Be easy to drive! I can still drive!
Crash ! ! !
♪
Cognition Cause Mistrust of
Get tired Judgement
Operation
Automated driving & ADAS

For safe driving, it is necessary to estimate fatigue

and encourage rest breaks at appropriate times.
13
Overview of Fatigue Estimation Algorithm

Electro Cardio Gram(ECG)

R R
Feature Values
RRI
Feature
Value 1
Feature
P T Value 2
QS

・・・
Detect Feature Calculation Model
R-Peak (HRV analysis) Estimation
RRI
Fatigue Level
Fatigue

Vigor
ｔ

14
Challenges in Algorithm Development

90% ?

80% ?
Stack in a Improve

development loop 70% ?

Experiment Data Analysis

How much performance

Is required?

We need …

Model Run the model

in real time
Deliver actual services
… Can the current model provide value?

15
Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

16
Challenges in Implementation of Real Time System

Require C
x
Joint Development

eventually…
Port to C
ECU Verification
Port again
Deliverables
Modify
Porting to C takes time …

ECU

Port Create Auto Code Generation

Easier
Easy to Main
modify Environment

17
Flow to Real-time Implementation

①Porting from Python to MATLAB

②Consistency Check
Port

③Implementation in Simulink
④Operation Check Implementation

RRI AI
Extractor Model
⑤Model Modifications Simulink Model
for Code Generation
⑥Implementation in ECU Auto Code Generation

ECG RRI AI
Output
Acquisition Extractor Model (CAN)
ECU

18
Hardware Configuration

analog CAN
ECG CAN
Sensor Interface
PC
Raspberry Pi

19
Flow to Real-time Implementation

①Porting from Python to MATLAB

②Consistency Check
Port

③Implementation in Simulink
④Operation Check Implementation

RRI AI
Extractor Model
⑤Model Modifications Simulink Model
for Code Generation
⑥Implementation in ECU Auto Code Generation

ECG RRI AI
Output
Acquisition Extractor Model (CAN)
ECU

20
Porting from Python to MATLAB

std()

Same input but different output.

Why?
21
 Differences in Default Settings

∑ 𝑿𝒊 − 𝝁 𝟐 ∑ 𝑿𝒊 − 𝝁 𝟐
𝝈= 𝝈=
std() 𝒏 𝒏−𝟏
Standard Deviation Invariant
Standard Deviation

Fisher’s Definition Pearson’s Definition

kurtosis() Normal
=0 Normal
=3
Distribution Distribution

22
Consistency Check

Feature
Value 1

Feature
Value 2
・・・

・・・

Fatigue
Level
Python
MATLAB

23
Implementation in Simulink

Fatigue
RRI Extraction
Estimation
Feature
RRI Value 1

Feature
Value 2

・・・
・・・
MATLAB Fatigue
Simulink Level

24
Flow to Real-time Implementation

①Porting from Python to MATLAB

②Consistency Check
Port

③Implementation in Simulink
④Operation Check Implementation

RRI AI
Extractor Model
⑤Model Modifications Simulink Model
for Code Generation
⑥Implementation in ECU Auto Code Generation

ECG RRI AI
Output
Acquisition Extractor Model (CAN)
ECU

25
Modifying the Model – Variable-Sized Array

Heart Rate 75[bpm]

１ ２３４５６ ７ 71 72 73 74 75
ECG

・・・ 75 peaks
The # of peaks
Heart Rate 57[bpm] is not consistent.
1 2 3 4 5 54 55 56 57
ECG

・・・ 57 peaks

Store the peak detection time.

(Variable-Sized)

Can throw errors upon code generation

Specify
Maximum array size

26
 Modifying the Model – IO/Communication Model

Communication
Estimate Model Model

IO Model

Add models for hardware I/O.

The package dedicated to Raspberry Pi was used for this time.
27
 Auto Code Generation

Just Press!
C Code Generation

Analog
Signal Raspberry pi
Processing +
CAN HAT

A/D
Conversion

ECU
28
 Current Real-Time Estimation System

RRI AI Output
ECG (CAN)
Acquisition Extractor Model
ECU With Consulting

RRI AI Output
ECG (Bluetooth) Only Us
Acquisition Extractor Model
ECU

Update

Notify users
via tablet

29
Agenda

1. Our Company
2. Products We Aim for
3. Estimation of Fatigue Level
4. Implement Algorithm in ECU
5. Conclusion

30
 Conclusion

Conclusion
Realize a service that combines machine learning algorithms
built in Python with other functions in real-time using Simulink.

Future Work
Enhance development flow targeting vehicle OS.

MATLAB®/Simulink® Product Family Used

・MATLAB®
・Simulink®
・Statistics and Machine Learning Toolbox®
・Signal Processing Toolbox®
・DSP Toolbox®
・Wavelet Toolbox®
・Simulink Coder®
・Embedded Coder®
・Simulink Support Package for Raspberry Pi®

31