FEDERATED AND REINFORCEMENT LEARNING-BASED

COMPUTATIONAL OFFLOADING IN VEHICULAR EDGE

COMPUTING

A PROJECT REPORT

Submitted by

BHARATHVARSH SRINIVASAN - 2021115020

SRIVISHWA P - 2021115109
SURYA NARAYANAAN KC - 2021115113
SANTHOSH R - 2021115311

submitted to the Faculty of

INFORMATION AND COMMUNICATION ENGINEERING

in partial fulfillment for the award of the degree

of

BACHELOR OF TECHNOLOGY
in

INFORMATION TECHNOLOGY

DEPARTMENT OF INFORMATION SCIENCE AND TECHNOLOGY

COLLEGE OF ENGINEERING, GUINDY
ANNA UNIVERSITY
CHENNAI 600 025

DECEMBER 2024
 ii

ANNA UNIVERSITY
CHENNAI - 600 025
BONAFIDE CERTIFICATE

Certified that this project report titled ”Federated and

Reinforcement Learning-Based Computational Offloading in Vehicular
Edge Computing” is the bonafide work of Bharathvarsh Srinivasan
2021115020, Srivishwa P 2021115109, Surya Narayanaan KC 2021115113,
Santhosh R 2021115311 who carried out project work under my supervision.
Certified further that to the best of my knowledge and belief, the work reported
herein does not form part of any other thesis or dissertation on the basis of
which a degree or an award was conferred on an earlier occasion on this or any
other candidate.

PLACE: Dr. M.VIJAYALAKSHMI

DATE: PROFESSOR
PROJECT GUIDE
DEPARTMENT OF IST, CEG
ANNA UNIVERSITY
CHENNAI 600025

COUNTERSIGNED

Dr. SWAMYNATHAN
HEAD OF THE DEPARTMENT
DEPARTMENT OF INFORMATION SCIENCE AND TECHNOLOGY
COLLEGE OF ENGINEERING, GUINDY
ANNA UNIVERSITY
CHENNAI 600025
 iii

ABSTRACT

Autonomous driving systems require real-time detection and

processing capabilities to ensure safe navigation in complex environments.
Current approaches often struggle with balancing computational demands and
low-latency requirements, particularly when handling multiple detection tasks
like identifying vehicles, pedestrians, traffic signs, and signals. Resource
constraints on edge devices can lead to inefficiencies, especially for tasks that
involve multiple decisions and require intensive computation. To address these
challenges, this project focuses on optimizing task offloading within a Vehicular
Edge Computing (VEC) framework, using Reinforcement Learning (RL) to
determine which tasks are best handled locally on the edge and which should
be offloaded to the cloud for more computationally demanding processes.

The solution uses RL to optimize resource usage and reduce latency

by offloading complex tasks, like detecting multiple vehicles and pedestrians,
to the cloud, while handling simpler tasks such as traffic signal and sign
recognition locally on the edge. This approach provides a reliable framework
for safe and efficient autonomous driving, ensuring real-time processing while
making use of scalable, cloud-assisted solutions.
 iv

ACKNOWLEDGEMENT

It is our privilege to express my deepest sense of gratitude and sincere

thanks to Dr. M. VIJAYALAKSHMI, Professor, Project Guide, Department of
Information Science and Technology, College of Engineering, Guindy, Anna
University, for her constant supervision, encouragement, and support in my
project work. We greatly appreciate the constructive advice and motivation that
was given to help me advance my project in the right direction.

We are grateful to Dr. S. SWAMYNATHAN, Professor and Head,

Department of Information Science and Technology, College of Engineering
Guindy, Anna University for providing us with the opportunity and necessary
resources to do this project.

We would also wish to express my deepest sense of gratitude to the

Members of the Project Review Committee: Dr. J. INDUMATHI, Professor,
DR. S. SENDHIL KUMAR, Professor, Dr. P. GEETHA, Associate Professor,
Dr. D. NARASHIMAN, Teaching Fellow Department of Information Science
and Technology,College of Engineering Guindy, Anna University, for their
guidance and useful suggestions that were beneficial in helping me improve
my project.

We also thank the faculty member and non teaching staff members
of the Department of Information Science and Technology, Anna University,
Chennai for their valuable support throughout the course of our project work.

BHARATHVARSH SRINIVASAN - 2021115020

SRIVISHWA P - 2021115109
SURYA NARAYANAAN KC - 2021115113
SANTHOSH R - 2021115311
 v

TABLE OF CONTENTS

ABSTRACT iii
ACKNOWLEDGEMENT iv
LIST OF FIGURES viii
LIST OF TABLES ix

1 INTRODUCTION 1
1.1 AUTONOMOUS DRIVING 1
1.1.1 Sensor Technology 1
1.1.2 Computer Vision and Artificial Intelligence 2
1.1.3 Edge and Cloud Computing in Autonomous Driving 3
1.2 CHALLENGES IN AUTONOMOUS DRIVING 4
1.2.1 Dynamic Environments 4
1.2.2 Real-Time Processing 4
1.2.3 Task Complexity and Offloading 4
1.2.4 Connectivity Dependence 5
1.2.5 Safety and Regulatory Compliance 5
1.3 MOTIVATION 5
1.3.1 Safety 6
1.3.2 Mobility Solutions 6
1.3.3 Traffic Efficiency 6
1.3.4 Environmental Impact 6
1.4 OBJECTIVES 7
1.5 PROBLEM STATEMENT 7
1.6 PROPOSED SOLUTION 8
1.7 TESTING IN SIMULATION PLATFORMS 9
1.8 ORGANIZATION OF THE REPORT 10

2 LITERATURE SURVEY 11
2.1 SIAMESE NETWORKS FOR DEFECT DETECTION 11
2.1.1 Basic Architecture and Working Principles 11
2.1.2 Twin and Triple Siamese Networks 12
2.2 OPTIMIZATION TECHNIQUES FOR SIAMESE
NETWORKS 13
2.2.1 Parameter Reduction and Feature Enhancement 13
 vi

2.2.2 Loss Functions and Distance Metrics 13

2.3 PERFORMANCE ANALYSIS AND COMPARATIVE
STUDIES 14
2.3.1 Accuracy and Efficiency Metrics 14
2.3.2 Training Efficiency and Sample Requirements 15
2.4 APPLICATIONS IN SPECIALIZED DOMAINS 15
2.4.1 Manufacturing and Industrial Inspection 15
2.4.2 Small Sample Recognition Challenges 16
2.5 SUMMARY 16

3 SYSTEM DESIGN 18
3.1 SYSTEM ARCHITECTURE 18
3.1.1 Data Acquisition 19
3.1.2 Data Preprocessing 19
3.1.3 Siamese Network Model 20
3.1.4 Training Process 21
3.1.5 Evaluation Method 22
3.1.6 Deployment Strategy 22
3.2 IMPLEMENTATION 23
3.2.1 Dataset Implementation 23
3.2.2 Model Implementation 23
3.2.3 Training Implementation 24
3.2.4 Model Testing and Deployment 25
3.2.5 Real World Application 25

4 SYSTEM DESIGN 27
4.1 SETTING UP THE ENVIRONMENT 27
4.1.1 RSU Deployment in CARLA Simulator 27
4.1.2 Edge Server Configuration 28
4.1.3 Cloud Server Configuration 28
4.1.4 Socket Programming for Communication 29
4.1.5 Data Transmission and Storage 29
4.2 SIGNAL AND SIGN BOARD DETECTION 30
4.3 VEHICLES AND PEDESTRIAN DETECTION 32
4.4 OFFLOADING RL 34
4.5 DECISION AGENT 37
 vii

5 IMPLEMENTATION RESULTS 41
5.1 SETTING UP THE ENVIRONMENT MODULE 41
5.2 SIGNAL AND SIGN DETECTION MODULE 43
5.3 VEHICLE AND PEDESTRIAN DETECTION MODULE 47
5.4 OFFLOADING RL MODULE 49
5.5 DECISION AGENT 50

6 CONCLUSION AND FUTURE WORK 53

6.1 FUTURE WORK 53

REFERENCES 54
 viii

LIST OF FIGURES

3.1 Architecture of the Manufacturing Defect Detection System 18

3.2 Defect-free Product 19
3.3 Defective Product 19
3.4 Steps involved in Training the model 21

5.1 CARLA Environment 42

5.2 Cloud Server Configuration 42
5.3 Signal detection in the Environment 44
5.4 Detection of speed limit 60 45
5.5 Detection of stop sign 46
5.6 Detection of speed limit 90 46
5.7 vehicle detection in the environment 48
5.8 Pedestrian detection in the environment 48
5.9 output of Offloading RL agent 49
5.10 Decision agent controlling the autonomous car 51
5.11 Decision agent controlling for left turn 52
 ix

LIST OF TABLES
 1

CHAPTER 1

INTRODUCTION

Autonomous Driving (AD) is bringing transformative changes to the

transportation sector by aiming to reduce accidents, improve traffic flow, and
offer mobility to individuals who cannot drive. This technology combines
advanced sensors, artificial intelligence (AI), and machine learning (ML)
to enable vehicles to operate in complex environments. However, even
with substantial progress in recent years, achieving a level of full autonomy
that ensures safe and reliable navigation in real-world scenarios remains a
significant hurdle. This section delves into the essential concepts, major
challenges, underlying motivations, and potential solutions designed to enhance
autonomous driving systems through advancements in computing frameworks
and AI.

1.1 AUTONOMOUS DRIVING

Autonomous driving refers to the ability of a vehicle to operate and

perform tasks without human intervention. The vehicle relies on a combination
of sensors, computing platforms, and algorithms to sense the environment and
make decisions. Key aspects of autonomous driving include:

1.1.1 Sensor Technology

Autonomous vehicles rely on various sensor technologies to perceive

the environment in real-time. These sensors include:
 2

• LiDAR (Light Detection and Ranging): LiDAR sensors generate

high-resolution 3D maps of the vehicle’s surroundings, providing
depth information to detect obstacles and measure distances.

• Radar (Radio Detection and Ranging): Radar systems detect

objects by bouncing radio waves off them, providing data for velocity
estimation and detecting objects in low visibility conditions such as
fog or rain.

• Cameras: Cameras capture images of the vehicle’s environment

and are used for object recognition, traffic sign detection, and
lane-keeping assistance.

• Ultrasonic Sensors: These are used for close-range object detection,

such as parking assistance and collision avoidance.

• GPS (Global Positioning System): GPS provides the vehicle with

precise location data, essential for navigation and path planning.

These sensors work together to deliver a detailed understanding of

the surrounding environment, ensuring the vehicle navigates safely.

1.1.2 Computer Vision and Artificial Intelligence

Computer vision and AI algorithms are the backbone of autonomous

vehicles. These systems process sensor data to identify objects such as
pedestrians, vehicles, road signs, and traffic signals. AI, particularly deep
learning algorithms, can recognize patterns and predict future events by learning
from large datasets.

The critical components of AI in autonomous driving include:

 3

• Object Detection and Recognition: Algorithms identify objects

and classify them in real-time, which is essential for making driving
decisions.

• Decision Making: AI models make real-time decisions based on

sensor input and environmental data, such as stopping at traffic lights
or adjusting speed in response to traffic flow.

• Reinforcement Learning (RL): RL algorithms learn from the

vehicle’s actions to improve performance over time, enabling the
vehicle to adapt to changing road conditions.

1.1.3 Edge and Cloud Computing in Autonomous Driving

Autonomous driving systems require significant computational

resources to process large volumes of sensor data and run machine learning
algorithms in real-time.

• Edge Computing: Edge computing refers to processing data locally

on the vehicle, enabling faster decision-making. Edge servers
handle lightweight tasks such as traffic sign detection and obstacle
avoidance to reduce latency.

• Cloud Computing: For tasks that require significant computing

power, like handling large datasets or running complex algorithms,
the data is offloaded to cloud servers that can handle the load more
efficiently. This approach saves both time and resources.However,
it does come with challenges, such as potential delays in network
communication and the risk of losing data.
 4

A hybrid approach that balances edge and cloud computing is crucial

to meet the real-time processing requirements of autonomous systems.

1.2 CHALLENGES IN AUTONOMOUS DRIVING

The development and deployment of autonomous vehicles are

accompanied by several challenges that need to be addressed to ensure safe and
reliable operation.

1.2.1 Dynamic Environments

Autonomous vehicles must be capable of navigating in highly

dynamic environments, which include complex road networks, unpredictable
pedestrian behavior, and fluctuating traffic conditions. AVs must adjust to
sudden changes, such as a pedestrian crossing the street or another vehicle
making an unexpected lane change. These unpredictable elements require
the vehicle to make split-second decisions, which are challenging for current
systems to handle reliably.

1.2.2 Real-Time Processing

Autonomous driving systems need to process large volumes of data

from various sensors in real time. Quick and accurate data processing is crucial
for making fast driving decisions. Any delay could result in accidents or failure
to detect a potential threat. This necessitates efficient algorithms and powerful
computing platforms capable of handling demanding computational tasks.
 5

1.2.3 Task Complexity and Offloading

While some tasks, such as traffic sign detection can be performed

locally on the vehicle’s edge computing system, others, such as deep
learning-based pedestrian detection and vehicle tracking, require cloud-level
processing power. Efficiently deciding which tasks to process locally and
which to offload to the cloud, while considering factors like network latency,
computational resources, and safety, is a critical challenge for autonomous
driving systems.

1.2.4 Connectivity Dependence

For autonomous vehicles that rely on cloud computing for

resource-heavy tasks, continuous and reliable connectivity is crucial.
Fluctuating or poor network conditions can introduce delays in data
transmission, leading to slower response times and, in the worst case, system
failure. Ensuring seamless integration between edge and cloud systems is
essential to maintain system reliability.

1.2.5 Safety and Regulatory Compliance

Autonomous vehicles must meet safety standards and regulations to

operate legally. Ensuring these systems can handle edge cases and follow local
traffic laws across different regions adds extra complexity. Failing to comply
with safety regulations could lead to accidents, legal problems, and a loss of
public confidence in autonomous driving technology.
 6

1.3 MOTIVATION

The motivation for this research is rooted in the potential benefits

autonomous driving can offer, both socially and economically. The key
motivating factors are:

1.3.1 Safety

Human error is a major factor in many traffic accidents. Autonomous

driving technology seeks to minimize these accidents by eliminating human
mistakes, like distracted driving, impaired driving, and fatigue. With the help
of AI and sensor technology, autonomous vehicles can make more informed
decisions, lowering the risk of accidents and potentially saving lives.

1.3.2 Mobility Solutions

Autonomous driving offers the potential for increased mobility,

particularly for individuals who are unable to drive due to age, disability, or
other factors. Autonomous vehicles can provide an alternative transportation
option for these individuals, improving their quality of life and increasing
independence.

1.3.3 Traffic Efficiency

Autonomous vehicles have the potential to enhance traffic efficiency

by reducing congestion, optimizing routes, and minimizing the need for
parking. Through communication with other vehicles and infrastructure, they
can coordinate their movements, helping to avoid traffic jams and improve traffic
flow in urban areas.
 7

1.3.4 Environmental Impact

The widespread adoption of autonomous vehicles could lead to

reductions in fuel consumption and greenhouse gas emissions. Through
optimized driving patterns, autonomous vehicles could reduce fuel usage and
emissions, contributing to a cleaner and more sustainable environment.

1.4 OBJECTIVES

The primary goal of this research project is to develop an efficient

task offloading framework for autonomous driving systems using Vehicular
Edge Computing (VEC). The specific objectives are:

• Design and implement a hybrid task offloading framework that

balances real-time edge processing and cloud-based processing for
complex tasks.

• Develop a Reinforcement Learning (RL) model to dynamically

decide when and which tasks should be offloaded based on real-time
data.

• Minimize latency for lightweight tasks like traffic sign detection

using edge computing.

• Maximize computational resources for heavy tasks such as object

detection and predictive modeling by offloading them to cloud
servers.

• Evaluate the proposed framework using a simulation environment

like CARLA to test its performance under various traffic scenarios.
 8

1.5 PROBLEM STATEMENT

Despite the advances in autonomous driving technology, there are

several challenges that hinder the development of fully autonomous systems:

• Edge Computing Limitations: Real-time processing requirements

for AV systems are constrained by the limited computing resources
available on vehicles.

• Cloud Latency: Delays in task offloading to cloud servers can result

in slower decision-making, which is critical in time-sensitive driving
situations.

• Scalability: Autonomous systems must adapt to varying traffic

conditions, infrastructure, and environmental factors, posing
scalability challenges.

• Safety and Regulatory Compliance: Ensuring that autonomous

systems adhere to traffic laws and regulations while operating safely
under all conditions.

1.6 PROPOSED SOLUTION

To address these challenges, the project proposes a hybrid task

offloading solution using Vehicular Edge Computing (VEC) and Reinforcement
Learning (RL). The proposed solution involves:

• Task Offloading Framework: A framework that dynamically

offloads tasks based on computational demands, available resources,
and task complexity.
 9

• Reinforcement Learning (RL): An RL model to continuously

improve the task offloading decision-making process based on
real-time data and historical experience.

• Simulation Testing: Use of the CARLA simulation environment to

test the proposed framework in a variety of traffic and environmental
scenarios.

• Latency Minimization: Optimization of edge computing for

real-time, low-latency tasks while using the cloud for complex
computational tasks.

1.7 TESTING IN SIMULATION PLATFORMS

Simulation platforms like CARLA are invaluable for testing

autonomous driving systems in safe, controlled environments. CARLA provides
a rich, realistic simulation of urban environments, allowing for the testing of AV
systems under various conditions, such as:

• Urban Intersections: Testing how autonomous vehicles navigate

complex intersections with pedestrians, cyclists, and other vehicles.

• Obstacle Avoidance: Simulating scenarios where the vehicle needs

to avoid sudden obstacles, such as an animal crossing the road.

• Adverse Weather Conditions: Testing vehicle performance under

conditions such as rain, fog, and snow, which can affect sensor
performance.

• Traffic Scenarios: Simulating traffic congestion, accidents, and

emergency situations to evaluate the vehicle’s decision-making
capabilities.
 10

1.8 ORGANIZATION OF THE REPORT

This report is structured as follows:

• Chapter 2: Literature Review - Summarizes existing research

and methodologies in autonomous driving and Vehicular Edge
Computing.

• Chapter 3: System Architecture - Describes the project’s

architecture, algorithms, and design components.

• Chapter 4: System Design - Describes the project’s flow and full

design.

• Chapter 5: Implementation - Details the tools, frameworks, and

programming techniques used in the project.

• Chapter 6: Conclusion and Future Work - Summarizes the project

outcomes and suggests directions for future research.
 11

CHAPTER 2

LITERATURE SURVEY

2.1 SIAMESE NETWORKS FOR DEFECT DETECTION

Siamese networks have emerged as a powerful deep learning

architecture for similarity-based tasks, particularly in defect detection
applications. Unlike traditional convolutional neural networks (CNNs), Siamese
networks employ two or more identical subnetworks with shared weights to
learn and compare feature representations between input pairs. This approach is
especially valuable in scenarios with limited training data, where conventional
deep learning methods might struggle to generalize effectively. In defect
detection contexts, Siamese networks excel at learning discriminative features
that can distinguish between defective and non-defective samples through
similarity metrics rather than direct classification.

2.1.1 Basic Architecture and Working Principles

The fundamental architecture of a Siamese network consists of

twin networks with identical weights and parameters. As demonstrated by
Mohammad et al. [1], the network processes a pair of inputs through parallel
feature extraction pathways, computing a similarity measure between their
feature representations. The model learns to map inputs to a feature space where
similar items are positioned closer together and dissimilar items farther apart.
This distance-based learning approach makes Siamese networks particularly
suitable for defect detection tasks, where the goal is to identify deviations from
normal patterns rather than classify into predefined categories.
 12

The core components of a Siamese network architecture for defect

detection typically include:

1. Feature extraction networks (usually CNN-based)

2. Distance calculation between feature vectors

3. Similarity scoring mechanism

4. Loss function designed for comparative learning.

Mohammad et al. [1] implemented two different Siamese network

configurations for handwritten digit recognition. Their work demonstrated how
Siamese networks efficiently learn similarity measures to detect the distance
between data points in vector space. The configurations included various
combinations of dense layers, batch normalization, and specialized distance
functions to compute similarities between image pairs.

2.1.2 Twin and Triple Siamese Networks

Twin Siamese networks, as described by Mohammad et al. [1],

consist of two parallel networks with shared weights that encode two input
images. Each convolutional layer uses a single channel and filters of variable
sizes. The output feature maps pass through ReLU activation functions before
undergoing max pooling operations.

Triple Siamese Networks extend this concept with three parallel

networks sharing identical weights. While Twin Siamese networks have
demonstrated excellent performance compared to other metric learning
methods, Triple networks offer enhanced optimization capabilities. These
 13

networks are particularly valuable when dealing with complex defect patterns
that require more nuanced feature comparisons.

2.2 OPTIMIZATION TECHNIQUES FOR SIAMESE

NETWORKS

2.2.1 Parameter Reduction and Feature Enhancement

Several optimization techniques have been proposed to enhance the

performance of Siamese networks for defect detection tasks. Zhang et al. [2]
introduced a Siamese Group Chunking (GC) Capsule Network (SGCCN) that
addresses challenges related to limited sample sizes, which is often a constraint
in defect detection scenarios where defective samples may be rare. Their
approach employed GC blocks as feature extractors for the primary capsules,
using separate filters to learn unique representations of features, thereby
enhancing feature extraction capabilities while reducing model parameters.

A significant improvement in Zhang et al.’s [2] work was the

implementation of an adjusted cosine similarity metric, which captured both
directional and absolute numerical differences between feature vectors. This
modification substantially enhanced the network’s robustness in recognition
tasks with limited training data. Experimental results demonstrated that their
SGCCN reduced parameter usage by 57.65% while increasing recognition
accuracy by 7.67% compared to conventional Siamese capsule networks.
 14

2.2.2 Loss Functions and Distance Metrics

The choice of loss function and distance metric significantly impacts

the performance of Siamese networks in defect detection tasks. Mohammad et
al. [1] explored constructive loss to optimize their Siamese network models.
Their experiments showed that Binary Cross Entropy (BCE) loss is effective for
training Siamese networks to distinguish between similar and dissimilar pairs.

For distance calculation between feature vectors, Euclidean distance

is commonly employed in Siamese networks for defect detection. Mohammad
et al. [1] implemented a Lambda layer merged with Euclidean distance to
find similarities between pairs of images. The similarity values range from
0.0 (maximum similarity) to 1.0 (minimum similarity), providing a clear
quantitative measure for defect identification.

2.3 PERFORMANCE ANALYSIS AND COMPARATIVE

STUDIES

2.3.1 Accuracy and Efficiency Metrics

Comparative studies have demonstrated the superiority of Siamese

networks in defect detection tasks compared to traditional classification
methods. Mohammad et al. [1] compared their Siamese network
implementations with SVM, MLP, and CNN approaches using the MNIST
dataset. Their proposed Model-2 achieved 98.41% accuracy with a loss value
of 0.0157 and Root Mean Square Error (RMSE) of 0.48408. This performance
was comparable to CNN’s 99.31% accuracy but with significantly reduced
execution time—26.29 minutes for the Siamese network versus 44.02 minutes
for CNN.
 15

Zhang et al. [2] reported that their SGCCN achieved 92.67%

recognition rate on their validation set and 89.33% for untrained individuals.
These results highlight the potential of optimized Siamese networks to
generalize well to new, previously unseen defects—a critical capability in
industrial quality control applications.

2.3.2 Training Efficiency and Sample Requirements

One of the key advantages of Siamese networks for defect detection

is their ability to learn effectively from limited training data. Both Mohammad
et al. [1] and Zhang et al. [2] demonstrated that Siamese networks can achieve
high accuracy with relatively few training examples per class. This property is
particularly valuable in manufacturing contexts where collecting large datasets
of defective samples may be impractical or costly.

Mohammad et al. [1] found that their Siamese network models

achieved optimal performance with as few as 10-20 epochs, after which very
little improvement was observed. This rapid convergence further contributes to
the computational efficiency of Siamese networks compared to traditional CNN
approaches.

2.4 APPLICATIONS IN SPECIALIZED DOMAINS

2.4.1 Manufacturing and Industrial Inspection

Siamese networks have been applied to various defect detection

scenarios in manufacturing and industrial inspection. The architecture’s ability
to learn similarity metrics makes it well-suited for identifying deviations from
normal production samples. By training on pairs of defective and non-defective
 16

samples, these networks can effectively learn the discriminative features that
distinguish quality products from those with defects.

The similarity computation approach demonstrated by Mohammad

et al. [1] can be adapted to industrial contexts where defect detection is framed
as a comparison between a reference (defect-free) sample and a test sample. The
resulting similarity score provides a quantitative measure of product quality and
can be used to automate quality control processes.

2.4.2 Small Sample Recognition Challenges

Zhang et al. [2] addressed the challenge of small sample recognition,

which is directly relevant to defect detection scenarios where defective samples
may be rare. Their SGCCN approach demonstrates how Siamese networks can
be optimized to perform well even with limited training data per class. By
utilizing GC blocks as feature extractors and implementing an adjusted cosine
similarity metric, their model achieved robust recognition performance despite
the constraints of small sample sizes.

This capability is particularly valuable in defect detection

applications where collecting extensive datasets of specific defect types may
be challenging due to their rare occurrence in production environments.

2.5 SUMMARY

The reviewed research explores various Siamese network

architectures and optimization techniques for similarity-based recognition
tasks, which are directly applicable to defect detection. Mohammad et al. [1]
demonstrated the effectiveness of Siamese networks for similarity computation
and prediction, achieving high accuracy with reduced computational
 17

requirements compared to traditional approaches. Zhang et al. [2] introduced

enhancements through their SGCCN architecture, addressing the challenges
of limited sample sizes while improving recognition accuracy and reducing
parameter count.

Our system advances these efforts by applying Siamese networks

specifically to defect detection using the architecture shown in the diagram. The
model processes pairs of images through a ResNet18-based feature extractor,
calculates the absolute difference between feature vectors, and computes a
similarity score through a fully connected layer trained with Binary Cross
Entropy loss. This approach not only optimizes defect detection accuracy but
also enhances the system’s ability to identify previously unseen defect types, a
critical capability in industrial quality control applications.
 18

CHAPTER 3

SYSTEM DESIGN

3.1 SYSTEM ARCHITECTURE

Figure 3.1: Architecture of the Manufacturing Defect Detection System

The architecture of the proposed system is shown in Figure 3.1

 19

3.1.1 Data Acquisition

The data required for this system consists of leather product

images categorized into ”good” (defect-free) and ”bad” (with defects) samples.
These images are collected from manufacturing facilities and organized into
appropriate directories for processing. The collected input data includes various
products textures, colors, and defect types to ensure comprehensive training
of the model.Example images representing the dataset is shown in Figure 3.2
Figure 3.3.

Figure 3.2: Defect-free

Figure 3.3: Defective Product
Product

3.1.2 Data Preprocessing

The collected input data undergoes preprocessing using various

image transformation techniques implemented through PyTorch’s transforms
module. The preprocessing pipeline includes:

• Resizing: Resizing images to a standard dimension (224x224 pixels)

• Flipping: Random horizontal flipping for data augmentation

• Rotation: Random rotation (up to 15 degrees) to introduce variance

 20

• Color adjustments: Color jitter adjustments for brightness, contrast,

and saturation

• Normalization: Normalization of pixel values to the range [-1, 1]

These preprocessing steps enhance the model’s ability to generalize

across different lighting conditions, orientations, and image qualities while
ensuring consistent input dimensions for the neural network.

3.1.3 Siamese Network Model

The Siamese neural network is the core component of the defect

detection system. The model architecture consists of:

• A shared feature extractor based on a pre-trained ResNet18 model

• A fully connected neural network that computes similarity between

image pairs

• Sigmoid activation function to produce similarity scores between 0

and 1

The Siamese network processes pairs of images (reference and test

images) and determines whether they belong to the same class (both good or
both defective) or different classes. This approach allows the system to learn
discriminative features that distinguish between defective and non-defective
leather products.
 21

3.1.4 Training Process

The training process utilizes the BCELoss (Binary Cross-Entropy

Loss) function to optimize the model parameters. The Adam optimizer with a
carefully selected learning rate (0.00005) and weight decay (1e-5) is employed
to update the model weights during training. The system trains the model for 50
epochs, monitoring the loss value to ensure proper convergence.

Figure 3.4: Steps involved in Training the model

 22

The training data is organized into pairs consisting of:

• Same-class pairs: good-good or bad-bad (labeled as 1)

• Different-class pairs: good-bad (labeled as 0)

This pairwise training approach enables the model to learn the

similarity metric space effectively.

3.1.5 Evaluation Method

The evaluation of the model is performed using accuracy metrics,

specifically calculating the percentage of correctly classified image pairs. The
system converts the continuous similarity scores (between 0 and 1) to binary
predictions using a threshold of 0.5. These predictions are then compared
against the ground truth labels to compute the overall accuracy of the model.

3.1.6 Deployment Strategy

The trained model is saved as a PyTorch state dictionary (.pth file),

making it easy to deploy in various environments. The deployment strategy
involves:

• Loading the saved model in inference mode

• Processing incoming leather product images through the same

preprocessing pipeline

• Computing similarity scores against reference images

• Classifying products as defective or non-defective based on threshold

values
 23

3.2 IMPLEMENTATION

The implementation consists of four key modules. First, the dataset

module handles the creation of image pairs and data loading functionalities.
Second, the model module defines the Siamese network architecture using
ResNet18 as the backbone. Third, the training module implements the training
loop, loss computation, and optimizer updates. Finally, the evaluation module
provides functionality to assess the model’s performance and save the trained
model for future use.

3.2.1 Dataset Implementation

The ManufacturingDefectDataset class extends PyTorch’s Dataset

class and implements the following functionalities:

• Loading of good and bad leather sample images from specified

directories

• Creation of image pairs with appropriate labels (1 for same class, 0

for different class)

• On-the-fly image transformation and preprocessing during batch

retrieval

• Integration with PyTorch’s DataLoader for efficient batching and

shuffling

3.2.2 Model Implementation

The SiameseNetwork class implements a two-branch neural network

with shared weights:
 24

• Utilizes a pre-trained ResNet18 model as the feature extractor

• Modifies the network by replacing the final fully connected layer

with an identity mapping

• Adds a custom fully connected network for computing the similarity

between extracted features

• Implements batch normalization and ReLU activations to improve

training stability

• Uses sigmoid activation to produce similarity scores between 0 and

1

3.2.3 Training Implementation

The training implementation establishes a robust framework for

optimizing the Siamese network model. The system first determines the
available computational resources, automatically selecting GPU acceleration
when available to significantly reduce training time. The data pipeline
is configured with the custom ManufacturingDefectDataset class, which
efficiently handles the creation of image pairs and applies appropriate
transformations. This dataset is then processed through PyTorch’s DataLoader
with carefully tuned batch sizes to balance memory requirements and
computational efficiency.

The Siamese network model initialization incorporates the

pre-trained ResNet18 architecture to leverage transfer learning benefits,
drastically reducing the required training data and accelerating convergence.
The Binary Cross-Entropy loss function is specifically chosen for its
appropriateness in similarity-based binary classification tasks, while the
 25

Adam optimizer with a low learning rate (0.00005) and weight decay (1e-5)
ensures stable convergence while mitigating overfitting.

The core training loop implements a methodical epoch-based

approach, where each batch undergoes forward propagation through the
Siamese branches, loss computation based on the similarity scores, and
backward propagation to update model weights. Training metrics are
systematically logged after each epoch, providing continuous visibility into the
learning progress and enabling early detection of potential training issues such
as vanishing gradients or loss plateaus.

3.2.4 Model Testing and Deployment

The system includes a straightforward approach to testing and

deploying the trained model. Once training is complete, the model is saved
as a .pth file that contains all the learned weights. This file can be easily
loaded on different computers for use in the leather manufacturing facility. The
testing process involves running new leather images through the model to detect
defects. This happens in real-time as products move through the production
line. The system displays results clearly, showing whether a leather sample is
defective or good, along with a confidence score. For manufacturing facilities
with limited computing resources, the model can run efficiently on standard
computers without requiring expensive GPU hardware.

3.2.5 Real World Application

The real-world application of the Siamese network focuses on

practical implementation in manufacturing environments. The trained model
is integrated into the quality control process, where it analyzes leather products
as they come off the production line. A simple camera setup captures images
 26

of each leather piece, which are then processed by the model. The system
is designed to be user-friendly, allowing quality control personnel to view
detection results on a dashboard without needing technical expertise. When
a defect is detected, the system alerts operators and logs the information for
quality tracking purposes. This integration helps reduce manual inspection time
while maintaining high-quality standards in leather manufacturing.
 27

CHAPTER 4

SYSTEM DESIGN

4.1 SETTING UP THE ENVIRONMENT

The primary objective of this module is to create a strong and scalable

simulation framework for testing and validating computational offloading
strategies in vehicular edge computing. The CARLA simulator is used to
replicate realistic driving environments, while an edge server and cloud server
setup manage task processing and data in a hybrid system. This configuration
bridges the gap between virtual simulations and real-world applications,
allowing for thorough testing of reinforcement learning-based computational
offloading strategies.

4.1.1 RSU Deployment in CARLA Simulator

Road-Side Units (RSUs) play a key role in enabling

vehicle-to-infrastructure (V2I) communication within the CARLA simulator.
These RSUs are strategically placed across various locations in the simulated
environment to provide optimal coverage of dynamic vehicular activities. They
are designed to detect and process important vehicular data, including traffic
density, vehicle positions, and speed. With the help of CARLA’s advanced
simulation capabilities, RSUs can monitor and respond to real-time traffic
conditions.

The RSUs are integrated with CARLA’s API to ensure smooth

communication between vehicles and infrastructure. This integration allows
real-time data, such as traffic signal detection and pedestrian identification, to
 28

be transmitted efficiently. The placement and operation of the RSUs in CARLA

were carefully adjusted to mirror realistic urban and highway environments,
improving the accuracy and relevance of the simulations.

4.1.2 Edge Server Configuration

An edge server environment was set up using a separate system

to simulate the role of an edge server in vehicular edge computing (VEC)
frameworks. This system was designed to process data from RSUs in near
real-time, reducing the computational load on individual vehicles.

The server setup included the installation of necessary libraries and

frameworks such as Python, Flask, and socket programming modules. It was
optimized for low latency and high throughput to manage demanding tasks
like traffic signal recognition and pedestrian detection. The flexibility of the
setup allows for easy reconfigurations to test various network topologies and
computational offloading strategies.

4.1.3 Cloud Server Configuration

To handle computationally intensive tasks and provide scalable

processing power, a cloud server was set up using an EC2 instance on Amazon
Web Services (AWS). The EC2 instance was configured with Ubuntu 22.04,
chosen for its compatibility with the tools and libraries used in the project.
This cloud server serves as the higher-level computing resource in the vehicular
edge computing (VEC) hierarchy, complementing the edge server’s real-time
capabilities.

The cloud server was integrated with the edge server using secure
and efficient communication protocols. The Flask framework on the edge
 29

server facilitates the transmission of task data to the cloud server, while the
EC2 instance handles these tasks using pre-trained models for detection and
prediction.

4.1.4 Socket Programming for Communication

Establishing seamless communication between the CARLA

simulator, the edge server, and the cloud server was a crucial step in the system
setup. Socket programming in Python was used to build a reliable client-server
architecture. In this setup, the client (RSU) within CARLA gathers and sends
vehicular data to the edge server, which then forwards computationally heavy
tasks to the cloud server.

The architecture utilized TCP sockets to ensure stable and reliable

data transmission with minimal packet loss. The communication protocols
were designed to manage the high-frequency data streams generated by the
CARLA simulator and processed by the edge and cloud servers. Features such
as persistent connections, error handling, and data serialization with JSON were
incorporated to ensure efficient communication, even in fluctuating network
conditions.

4.1.5 Data Transmission and Storage

Once the connection was established, the RSUs generated real-time

vehicular data, including parameters like location, velocity, and surrounding
traffic conditions. This data was transmitted to the edge server over the
established socket-based communication channel. On the edge server, Flask
was used as a lightweight framework to manage incoming data streams.
 30

For tasks offloaded to the cloud server, the edge server encapsulated
the data and transmitted it to the EC2 instance using secure channels. The
received data was processed and stored in a structured format for further
analysis. A file-based storage approach was initially used on both servers to
enable rapid prototyping, with plans for database integration in later stages.

4.2 SIGNAL AND SIGN BOARD DETECTION

This module focuses on detecting traffic signals and signboards in

the vehicular environment using YOLOv8 for object detection. Images from
the CARLA simulator are processed to identify and classify different traffic
elements. The detection process is critical for enabling autonomous vehicles
to make real-time decisions. By utilizing YOLOv8, the system achieves high
accuracy and speed in detecting traffic signals and signboards across various
conditions.

Input

• Images generated from the CARLA simulator, captured from the virtual
cameras mounted on the vehicles or RSUs.

• These images include a variety of traffic scenes featuring signals,

signboards, and other road elements.

Assumptions

• The CARLA simulator generates images in a consistent format and

resolution suitable for YOLOv8 processing.

• The environment contains well-defined traffic signals and signboards that

conform to standard shapes and dimensions.
 31

• Lighting conditions in the simulator are varied but do not include extreme
conditions such as complete darkness or blinding light.

Algorithm

Algorithm 4.1 Signal and Signboard Detection

Require: Input image I from CARLA simulator
Ensure: Detected objects with bounding boxes and labels
1: Step 1: Data Preprocessing
2: Normalize I and resize it to the YOLOv8 input dimensions
3: Step 2: Feature Extraction
4: Pass I through YOLOv8 convolutional layers to extract features
5: Step 3: Object Detection
6: Identify bounding boxes and classify objects using anchor-based detection
7: Apply Non-Maximum Suppression (NMS) to remove overlapping boxes
8: Step 4: Label Assignment
9: Assign labels to detected elements based on classification results
10: Output: Detected bounding boxes with associated labels

Time Complexity:

• Feature Extraction: O(n2 ), where n is the size of the input image.

• Object Detection: O(m · k), where m is the number of grid cells, and k is
the number of anchor boxes.

Space Complexity: The space complexity is O(w · h · c), where w, h, and c

represent the width, height, and number of channels of the intermediate feature
maps.

Process

Images from the CARLA simulator are processed using the YOLOv8
model. These images are first preprocessed to match the model’s input size.
 32

YOLOv8 uses its convolutional neural network to extract features and identifies
objects within the image. Non-maximum suppression (NMS) eliminates
overlapping bounding boxes, ensuring only the best fit is retained. The detected
elements are then classified into predefined categories, such as ”Stop Sign” or
”Traffic Light.” Python programming and libraries like OpenCV and Ultralytics’
YOLOv8 are employed to implement this detection pipeline.

Expected Output

Bounding boxes for detected objects with corresponding labels, such

as ”front-red-signal,” ”Stop,” or ”limit-60”

4.3 VEHICLES AND PEDESTRIAN DETECTION

This module focuses on detecting vehicles and pedestrians within

the simulated CARLA environment. YOLOv8 is employed for real-time
object detection due to its ability to handle high-speed image analysis with
remarkable accuracy. Detecting vehicles and pedestrians is critical for ensuring
safety and facilitating navigation in autonomous driving systems. This module
complements signal and signboard detection by providing dynamic, real-time
tracking of moving objects within the environment.

Input

• Real-time image frames captured from vehicle-mounted and roadside

cameras in the CARLA simulator.

• Frames featuring vehicles, pedestrians, and complex road scenarios, such

as crowded intersections.
 33

Assumptions

• Vehicles and pedestrians in the simulated environment are diverse in size

and orientation but adhere to recognizable patterns for detection.

• The CARLA simulator captures images with sufficient resolution to detect

small and distant objects.

• YOLOv8’s pre-trained weights are fine-tuned on datasets containing

vehicles and pedestrian labels for improved accuracy.

Algorithm

Algorithm 4.2 Vehicles and Pedestrian Detection

Require: Input image I from CARLA simulator
Ensure: Detected vehicles and pedestrians with bounding boxes and labels
1: Step 1: Image Preprocessing
2: Resize and normalize I to YOLOv8’s input dimensions
3: Convert I to a compatible tensor format for inference
4: Step 2: Feature Extraction
5: Process I through YOLOv8 to extract hierarchical feature maps
6: Step 3: Object Localization
7: Apply YOLOv8’s anchor boxes to identify potential regions of interest
(ROIs)
8: Step 4: Object Classification
9: Classify detected objects as ”Vehicle” or ”Pedestrian” using the model’s
neural network
10: Step 5: Optimization
11: Use Non-Maximum Suppression (NMS) to remove redundant bounding
boxes
12: Output the labeled ROIs as detected objects

Time Complexity:

• Feature Extraction and Localization: O(n2 ), where n is the image

dimension.
 34

• Object Classification: O(k · m), where k is the number of ROIs, and m is

the number of classes (e.g., vehicle, pedestrian).

Space Complexity: The space complexity is O( f ), where f represents the size

of the feature maps and intermediate tensors during detection.

Process

The CARLA simulator generates real-time image frames, which are

passed to the YOLOv8 model for analysis. The model detects regions of interest
(ROIs) containing vehicles and pedestrians based on pre-trained patterns.
After initial detection, Non-Maximum Suppression (NMS) is applied to refine
the results, merging or discarding overlapping detections. The final output
includes bounding boxes around each detected object with their respective
labels. Python, OpenCV, and YOLOv8 libraries were used to streamline the
detection process and visualize the results. This setup ensures accurate object
identification and real-time decision-making in the system by processing images
from the simulator efficiently and handling overlapping detections effectively.

Expected Output

• Labeled bounding boxes around detected vehicles and pedestrians,

enabling downstream applications like collision avoidance.

• Processed data stored for further analysis and use in reinforcement

learning-based task offloading.
 35

4.4 OFFLOADING RL

This module employs a Deep Q-Learning (DQL) algorithm to make

optimal decisions for offloading computationally intensive tasks. The agent
considers real-time state information, including vehicle positions, available
computing resources, and tolerance times, to decide whether tasks should be
processed locally, at RSUs, edge servers, or cloud servers. The objective is to
balance latency, resource utilization, and processing efficiency.

Input

• State information: Includes vehicle positions, tolerance times, and

available computing resources.

• Action space: Defines possible offloading options, such as local

processing, RSU processing, edge server processing, or cloud server
processing.

Assumptions

• The system has real-time access to accurate state information, including

vehicle positions and available computing resources.

• Network connectivity between vehicles, RSUs, edge servers, and the cloud
server is reliable and has minimal packet loss.

• Reward signals are appropriately designed to guide the agent toward

optimizing offloading efficiency and reducing latency.
 36

Algorithm

Algorithm 4.3 Deep Q-Learning for Task Offloading

1: Initialize Q-table Q(s, a) with random values
2: Set learning rate α, discount factor γ, exploration rate ε
3: for each episode do
4: Initialize state s
5: while task is not offloaded do
6: if random value ¡ ε then
7: Select random action a ▷ Exploration
8: else
9: a ← arg max Q(s, a) ▷ Exploitation: Choose action with max
Q-value
10: end if
11: Perform action a, observe next state s′ and reward r
12: Update Q-value:

Q(s, a) ← Q(s, a) + α · r + γ · max Q(s′ , a′ ) − Q(s, a)

 

13: Update state: s ← s′

14: if convergence criteria met then
15: Break
16: end if
17: end while
18: Decrease ε over time ▷ Encourage exploitation in later stages
19: end for
20: return optimal Q-table

Time Complexity:

• O(N · A), where N is the number of states and A is the number of actions.

Space Complexity:

• O(N · A), for storing the Q-table.

 37

Process

The offloading reinforcement learning (RL) agent is implemented

using Python and interacts with the CARLA simulator to make decisions about
task offloading. The simulator provides dynamic state information, such as
vehicle positions and available computing resources. The agent evaluates
potential offloading actions, deciding whether to offload tasks to the edge
or cloud server or keep them local, and updates the Q-values iteratively
based on the outcomes of these actions. Tools like NumPy are used for
efficient matrix operations, while TensorFlow or PyTorch can be integrated for
scalable deep Q-learning. The system achieves a balance between exploring
new strategies and exploiting previously learned ones, allowing the agent to
progressively optimize offloading decisions. Over time, the agent fine-tunes its
decisions, improving the overall system’s performance by efficiently managing
available resources. This dynamic learning process enables the RL agent to
make intelligent offloading choices in real-time, adapting to changes in the
environment and resource availability.

Expected Output

• Optimal task offloading decisions, specifying whether tasks should be

processed locally, at RSUs, edge servers, or the cloud server.

• Reduced latency and improved resource utilization across the system.

• Enhanced overall efficiency of the vehicular edge computing framework.

 38

4.5 DECISION AGENT

This module is responsible for integrating outputs from the detection

and offloading modules to determine the most appropriate action for the
vehicle. The decision-making process considers multiple factors, such as traffic
signals, nearby pedestrians, and vehicles, to ensure safe navigation through
the CARLA environment. A priority-based approach is adopted to resolve
conflicts, ensuring critical situations are addressed effectively. The agent sends
the computed actions to CARLA, enabling real-time vehicle control.

Input

• Detection results from YOLOv8, including identified traffic signals, signs,

pedestrians, and vehicles.

• Results of offloading decisions from the RL module, specifying whether a

task is handled locally or offloaded.

• Current vehicle state, such as speed and heading.

Assumptions

• Detection outputs from the YOLOv8 model are accurate and reliable for
decision-making.

• The vehicle’s control system responds promptly to computed actions.

• Real-time processing of inputs ensures no significant delays in decision

execution.
 39

Algorithm

Algorithm 4.4 Decision Agent for Vehicle Control

1: Initialize priority rules for signals, pedestrians, and vehicles
2: Set default action ← maintain speed
3: Step 1: Process Detection Outputs
4: if signal = "red" then
5: action ← stop
6: else if signal = "yellow" then
7: action ← slow down
8: else if signal = "green" then
9: Continue to next checks
10: end if
11: Step 2: Check Pedestrian Proximity
12: if pedestrian distance < safety threshold then
13: action ← stop
14: end if
15: Step 3: Check Vehicle Proximity
16: if vehicle distance < close threshold AND relative speed >
safe closing speed then
17: action ← slow down
18: end if
19: Step 4: Evaluate Turning Requirements
20: if turn required then
21: if turn direction = "left" then
22: action ← turn left
23: else if turn direction = "right" then
24: action ← turn right
25: end if
26: end if
27: Step 5: Execute Action
28: Send action to CARLA for execution

Time Complexity:

• Per Step Complexity: O(1) for each decision check.

• Overall Complexity: O(k), where k is the number of decision

factors (e.g., signals, pedestrians, vehicles).
 40

Space Complexity:

• O(1), as the module processes inputs in real time without storing

large datasets.

Process

The decision agent logic is implemented using Python and integrated

into the CARLA simulator. Detection outputs from YOLOv8 are passed to the
agent, along with task results from the RL module. Priority rules are applied to
resolve conflicts, ensuring safe navigation. The agent computes the appropriate
action, such as braking, slowing down, or turning, and sends these commands
to CARLA in real time.

Expected Output

• Safe and efficient navigation of the ego vehicle through the CARLA
environment.

• Real-time application of control actions (e.g., stopping, turning) to avoid

collisions and adhere to traffic rules.

• Consistent prioritization of critical scenarios (e.g., red signals, pedestrians)

for effective decision-making.
 41

CHAPTER 5

IMPLEMENTATION RESULTS

In this phase of the project, the system has implemented the detection
models using YOLOv8, a state-of-the-art object detection framework known for
its high efficiency and accuracy. The implementation was carried out in the
CARLA simulation environment, which emulates real-world driving scenarios.

5.1 SETTING UP THE ENVIRONMENT MODULE

Road-Side Units (RSUs) were deployed at strategic locations in the

CARLA environment to simulate dynamic vehicular communication, collecting
data such as speed, location, and images. A separate system was configured
to act as an edge server for processing the data received from the RSUs in
real time. Communication between RSUs and the edge server was established
using a Flask-based client-server architecture, enabling seamless data transfer
via HTTP. Sensor data, including vehicle metrics and images, was transmitted
from CARLA to the RSUs and then forwarded to the edge server for storage
and analysis. This setup ensured the efficient collection and transmission of
real-time vehicular data, enabling further processing in downstream modules.

Figure 5.1 illustrates the CARLA environment setup used for

simulating autonomous driving scenarios. The environment includes urban
infrastructure such as roads, intersections, traffic signals, and pedestrian
crossings. Various weather and lighting conditions can be configured to
test vehicle performance under dynamic real-world scenarios. The simulated
environment serves as a safe platform for evaluating navigation, object
detection, and task offloading in autonomous driving systems.
 42

Figure 5.1: CARLA Environment

Figure 5.2 showcases the cloud server configuration using an AWS

EC2 instance. The instance, named cloudserver, operates on a t3.micro
configuration with the status marked as running. This setup provides the
necessary computational resources to handle offloaded tasks from edge devices,
ensuring efficient processing and performance in the simulation.

Figure 5.2: Cloud Server Configuration

 43

Challenges Faced

• Simulator Installation: Encountered issues related to version mismatches

and unresolved dependencies during the installation of the CARLA
simulator, requiring multiple configuration adjustments and environment
setup fixes.

• RSU Placement Optimization: Faced challenges in optimizing the

placement of Road-Side Units (RSUs) to ensure maximum coverage and
efficient data transmission within the simulated environment.

5.2 SIGNAL AND SIGN DETECTION MODULE

A dataset of over 1,500 images was generated in the CARLA

environment, with traffic signals and signs manually annotated using LabelImg.
The dataset was enhanced with rotated, flipped, and scaled images to improve
generalization. A YOLOv8 model was trained on this dataset, with the data
split into 80% for training and 20% for validation. The trained YOLOv8 model
was deployed on the edge server, enabling real-time detection of signals and
signs. Images captured by vehicles were transmitted to RSUs and processed
by the edge server to identify traffic elements critical for safe navigation. This
integration ensured accurate and efficient detection in diverse traffic scenarios.
 44

Figure 5.3: Signal detection in the Environment

Figure 5.3 depicts the signal detection process using yolov8 model
in the simulation environment. The system uses real-time image processing
techniques to accurately identify and interpret traffic signals. This functionality
is crucial for enabling autonomous vehicles to make informed navigation
decisions and adhere to traffic rules within dynamic environments.
 45

Figure 5.4: Detection of speed limit 60

Figure 5.4 demonstrates the detection of a ”60 km/h” speed limit sign
as part of the signboard detection process using the YOLO model. This feature
allows the system to accurately identify regulatory speed limits in real time,
ensuring that autonomous vehicles adapt their speed accordingly to maintain
safe and lawful driving behavior.
 46

Figure 5.5: Detection of stop sign

Figure 5.6: Detection of speed limit 90

Figure 5.5 shows the detection of a ”Stop” sign using the YOLO
model. This capability enables autonomous vehicles to recognize and respond
 47

appropriately to stop signs, ensuring safe navigation at intersections and

pedestrian crossings.

Figure 5.6 illustrates the detection of a ”90 km/h” speed limit sign
using the YOLO model. This ensures that autonomous vehicles can identify
and adhere to the prescribed speed limits, promoting safe and efficient driving.

5.3 VEHICLE AND PEDESTRIAN DETECTION MODULE

To expand the dataset, additional images capturing vehicles and

pedestrians under varying weather conditions and times of day were collected
in the CARLA environment. The YOLOv8 model was updated to include new
labels for vehicles and pedestrians, with training focused on improving detection
accuracy for small objects and crowded scenes. Computationally intensive
detection tasks, such as identifying multiple pedestrians in busy intersections,
were offloaded to the cloud server, while simpler tasks were processed locally on
the edge server. This setup ensured the efficient handling of complex detection
scenarios and maintained real-time performance for critical tasks.

Figure 5.7 showcases vehicle detection, identifying cars and other

vehicles on the road using the YOLO model. This functionality is essential for
collision avoidance and user safety in autonomous driving systems.
 48

Figure 5.7: vehicle detection in the environment

Figure 5.8: Pedestrian detection in the environment

 49

Figure 5.8 illustrates pedestrian detection using the YOLO model.

This enables the system to recognize and respond to pedestrians in real time,
ensuring safety and adherence to traffic regulations in urban environments.

5.4 OFFLOADING RL MODULE

The state space for RL was designed to include parameters such

as edge server load, bandwidth, cloud latency, and task complexity, while the
action space defined options like local processing or offloading to RSUs or the
cloud server. A Deep Q-Learning (DQL) algorithm was implemented with a
reward function balancing latency, resource utilization, and task completion
efficiency. The RL model was trained in CARLA using dynamic traffic
scenarios, with the trained agent deployed on the edge server. The agent
analyzed real-time inputs to make optimal offloading decisions, ensuring
efficient resource allocation while maintaining low latency.

Figure 5.9: output of Offloading RL agent

 50

Figure 5.9 displays the output of the offloading process implemented

using Deep Q-Learning. The system intelligently decides whether tasks are
processed locally on the edge server or offloaded to the cloud server, optimizing
computational efficiency and reducing latency. The results are visualized
through a user-friendly UI, showcasing real-time decision-making and task
allocation, enhancing the interpretability and monitoring of the offloading
mechanism.

5.5 DECISION AGENT

The decision agent integrated outputs from the YOLOv8 detection

model (traffic signals, signs, vehicles, and pedestrians) and the RL offloading
decisions to determine the most appropriate action for the ego vehicle. Based
on the inputs, the agent evaluated possible actions such as stopping, slowing
down, steering left or right, or maintaining speed. A priority hierarchy was
established to resolve conflicts, giving precedence to critical elements such as
traffic signals and pedestrians over vehicles. The computed action was then
sent to the CARLA environment, ensuring real-time execution and effective
responses to dynamic road scenarios. This module played a pivotal role in
enabling the vehicle to make safe and accurate decisions during navigation.

Challenges Faced

• Conflicting Inputs: Simultaneous detections (e.g., a red signal and a

nearby pedestrian) created conflicts in prioritizing actions.

• Real-Time Constraints: Ensuring that decisions were computed and

executed within the time constraints of real-time navigation was
challenging in scenarios with multiple detections.

• Action Granularity: Mapping detection outputs to precise actions (e.g.,

 51

gradual braking versus immediate stops) required careful tuning and

extensive testing.

Figure 5.10: Decision agent controlling the autonomous car

 52

Figure 5.11: Decision agent controlling for left turn

Figures 5.10 and 5.11 illustrate the functioning of the decision agent
responsible for managing task execution and agent control. After completing
assigned tasks, the decision agent evaluates the outcomes and dynamically
adjusts the agent’s actions based on real-time conditions, ensuring optimal
performance and efficient resource utilization.
 53

CHAPTER 6

CONCLUSION AND FUTURE WORK

In this phase, we focused on real-time detection of traffic signals and

signs using the YOLOv8 model within the CARLA simulation environment,
while also incorporating Reinforcement Learning (RL) for offloading tasks.
The RL agent efficiently decided when to offload complex tasks to the
cloud, optimizing edge server resources and minimizing latency. The model’s
performance was evaluated using precision, recall, and F1-score metrics to
ensure accuracy and reliability in detecting traffic signals and signs. The
results showed that the model successfully detected and classified signals and
signs, and the RL-based offloading mechanism performed well in ensuring
tasks were handled effectively based on computational needs. While the
system performed well in controlled scenarios, there is potential for further
optimization, especially in processing speed for real-time applications in more
complex environments. This phase established a strong foundation for the
project, setting the stage for future stages incorporating Federated Learning and
further optimization.

6.1 FUTURE WORK

• Enhanced Federated Learning (FL): Implement advanced aggregation

techniques like FedProx and FedDyn for better convergence. Use adaptive
model updates based on vehicles’ resources and network conditions.

• Privacy and Edge Integration: Apply techniques like differential

privacy and secure aggregation for data security. Optimize edge server
coordination for efficient model updates.
 54
REFERENCES

[1] Hui Huang, Qiang Ye, and Yitong Zhou. 6g-empowered offloading for
real-time applications in multi-access edge computing. IEEE Transactions
on Network Science and Engineering, 10(3):1311–1323, May-June 2023.
[2] W. Sun, Y. Zhao, B. Guo, L. Xu, W. Ma, and T. Q. Duong. Accelerating
convergence of federated learning in mec with dynamic community. IEEE
Transactions on Mobile Computing, 23(2):1769–1785, Feb 2024.
[3] Jing He, Songtao Guo, Mingyan Li, and Yongdong Zhu. Acefl:
Federated learning accelerating in 6g-enabled mobile edge computing
networks. IEEE Transactions on Network Science and Engineering,
10(3):1200–1214, May-June 2023.
[4] X. Peng, Z. Han, W. Xie, C. Yu, P. Zhu, J. Xiao, and J. Yang. Deep
reinforcement learning for shared offloading strategy in vehicle edge
computing. IEEE Systems Journal, 17(2):2089–2101, June 2023.
[5] W. Wang, Y. Zhang, Q. Liu, T. Wang, and W. Jia. Edge-intelligence-based
computation offloading technology for distributed internet of unmanned
aerial vehicles. IEEE Internet of Things Journal, 11(12):20948–20961,
June 15 2024.
[6] S. Liu, J. Yu, X. Deng, and S. Wan. Fedcpf: An efficient-communication
federated learning approach for vehicular edge computing in 6g
communication networks. IEEE Internet of Things Journal,
11(12):20962–20975, June 15 2024.
[7] J. Liu, J. Ren, Y. Zhang, X. Peng, Y. Zhang, and Y. Yang. Efficient
dependent task offloading for multiple applications in mec-cloud system.
IEEE Transactions on Network Science and Engineering, 11(1):1–14, Jan
2024.
[8] M. K. Hasan, S. Islam, N. Jahan, M. Z. A. Nazri, M. A. Khan, A. I.
Alzahrani, N. Alalwan, and Y. Nam. Federated learning for computational
offloading and resource management of vehicular edge computing in
6g-v2x network. IEEE Transactions on Vehicular Technology, 73(6):1–14,
June 2024.
[9] A. F. Şahin, A. El Saleh, I. Shayea, İ. Yazıcı, and S. A. Saad. Offloading
techniques in mobile edge computing (mec) for future wireless networks.
In 2023 International Conference on Smart Computing and Application
(ICSCA), 2023.
 55

[10] T. H. Binh, D. B. Son, H. Vo, B. M. Nguyen, and H. T. T. Binh.

Reinforcement learning for optimizing delay-sensitive task offloading
in vehicular edge–cloud computing. IEEE Transactions on Vehicular
Technology, 73(8):7631–7643, Aug 2024.
[11] J. Liu, K. Xue, Q. Miao, S. Li, D. Wang, and K. Li. Mcvco: Multi-mec
cooperative vehicular computation offloading. IEEE Transactions on
Intelligent Vehicles, 9(1):813–825, Jan 2024.
[12] L. Zhao, E. Zhang, A. Y. Al-Dubai, S. Wan, G. Min, A. Hawbani, and
A. Y. Zomaya. Meson: A mobility-aware dependent task offloading
scheme for urban vehicular edge computing. IEEE Transactions on Mobile
Computing, 23(5):4259–4273, May 2024.
[13] S. Li, W. Sun, Q. Ni, and Y. Sun. Road side unit-assisted learning-based
partial task offloading for vehicular edge computing system. IEEE
Transactions on Vehicular Technology, 73(4):4931–4944, Apr 2024.
[14] X. Wang, J. Ye, and J. C. S. Lui. Online learning aided decentralized
multi-user task offloading for mobile edge computing. IEEE Transactions
on Mobile Computing, 23(4):1938–1953, Apr 2024.
[15] S. Shen, G. Shen, Z. Dai, K. Zhang, X. Kong, and J. Li. Asynchronous
federated deep-reinforcement learning-based dependency task offloading
for uav-assisted vehicular networks. IEEE Internet of Things Journal,
11(19):31561–31575, Oct 2024.
[16] D. Wu, Q. Liu, T. Chen, H. Huang, X. Jia, and K. Li.
Collaborative edge-cloud computing for delay-sensitive task offloading
in vehicular networks. IEEE Transactions on Vehicular Technology,
73(10):8217–8231, Oct 2024.
[17] X. Yang, J. Jiang, H. Guo, Y. Guo, Y. Zhou, and S. Wan.
Fl-etm: Federated learning-empowered task migration in vehicular edge
computing networks. IEEE Transactions on Network Science and
Engineering, 11(4):1398–1412, July-August 2024.
[18] M. Zhang, T. Wang, K. Wang, Y. Zheng, and Z. Chen. Learning-based
dynamic resource allocation for vehicular edge computing systems. IEEE
Transactions on Vehicular Technology, 73(2):1197–1210, Feb 2024.
[19] W. Li, Z. Liu, F. Fang, Y. Xie, S. Yan, and Y. Tang. Joint
optimization of computation offloading and resource allocation in
vehicular edge computing systems. IEEE Transactions on Mobile
Computing, 22(11):8551–8563, Nov 2023.
[20] H. Xu, J. Zhou, F. Wu, and L. Xu. Task offloading for energy-efficient
federated learning in vehicular edge computing networks. IEEE Internet
of Things Journal, 11(14):22488–22503, July 15 2024.
 56

[21] L. Gao, F. Li, Y. Hou, X. Xu, Y. Zhang, and C. Wang. Reinforcement

learning-based delay-aware computation offloading in mec-cloud systems
for vehicular networks. IEEE Transactions on Intelligent Transportation
Systems, 24(4):4316–4329, April 2023.
[22] H. Wang, Y. Zhao, X. Gao, Z. He, and J. Huang. Cooperative
learning-based distributed task offloading in vehicular edge computing.
IEEE Transactions on Vehicular Technology, 72(11):11501–11514, Nov
2023.
[23] S. Hou, L. Huang, W. Zhang, and H. Wu. Multi-objective optimization for
federated learning-based task offloading in vehicular edge networks. IEEE
Transactions on Intelligent Transportation Systems, 25(3):2039–2052,
March 2024.
[24] J. Lee, Y. Kim, and H. J. Park. Energy-efficient deep reinforcement
learning-based task offloading for autonomous driving in vehicular edge
computing networks. IEEE Transactions on Vehicular Technology,
72(10):10415–10428, Oct 2023.
[25] X. Yin, Y. Cheng, and M. Peng. Mobility-aware task offloading for
federated learning in vehicular edge computing: A deep reinforcement
learning approach. IEEE Internet of Things Journal, 10(23):23235–23249,
Dec 2023.
[26] Y. Zhao, L. Zhang, X. Tang, S. Yu, and F. Qian. Resource allocation
for federated learning-assisted task offloading in mec networks. IEEE
Transactions on Network Science and Engineering, 10(4):3445–3459, Dec
2023.
[27] C. Yu, F. Liu, G. Wang, and X. Liu. Privacy-preserving federated learning
for task offloading in vehicular edge computing. IEEE Transactions on
Vehicular Technology, 72(9):9409–9423, Sep 2023.
[28] Y. Zhang, X. Xu, J. Ren, H. Zhao, and Y. Guo. Energy-efficient
federated deep learning for task offloading in 6g edge computing networks.
IEEE Transactions on Network Science and Engineering, 11(2):567–581,
March-April 2024.
[29] L. Wang, X. Yang, X. Zhang, and J. Yan. Delay-aware task offloading
in dynamic mec networks using deep reinforcement learning. IEEE
Transactions on Vehicular Technology, 72(7):8475–8489, July 2023.
[30] Z. Pan, H. Wu, K. Xu, Y. Shen, and X. Song. Reinforcement
learning-based federated task offloading for smart vehicular
networks. IEEE Transactions on Intelligent Transportation Systems,
25(6):5503–5516, June 2024.