VISVESVARAYA TECHNOLOGICAL UNIVERSITY

Belagavi – 590018

A Project Report on

TRACK VISION-REAL TIME OBSTRUCTION

DETECTION AND PEDESTRIAN ALERT SYSTEM
Submitted by
DEEPAK KUMAR A 4SN20IS005
DHANUSH 4SN20IS006
T VAISHNAV 4SN20IS018
YAJNESH J KULAL 4SN20IS020

In partial fulfillment of the requirements for the degree of

BACHELOR OF ENGINEERING
IN
INFORMATION SCIENCE & ENGINEERING
Under the Guidance of
Mrs. SHREEKSHITHA
Assistant Professor

Department of Information Science & Engineering

SRINIVAS INSTITUTE OF TECHNOLOGY
(Accredited by NAAC)

MANGALURU - 574143, KARNATAKA.

2023 – 2024
 SRINIVAS INSTITUTE OF TECHNOLOGY
(Accredited by NAAC)
MANGALURU -574143, KARNATAKA
DEPARTMENT OF INFORMATION SCIENCE & ENGINEERING

CERTIFICATE

Certified that the project work entitled “TRACK VISION- REAL TIME OBSTRUCTION
TRACKING AND PEDESTRIAN ALERT SYSTEM” is a bonfide work carried out by

DEEPAK KUMAR A 4SN20IS005

DHANUSH 4SN20IS006
T VAISHNAV 4SN20IS018
YAJNESH J KULAL 4SN20IS020

in partial fulfillment for the award of BACHELOR OF ENGINEERING in INFORMATION

SCIENCE & ENGINEERING of the VISVESVARAYA TECHNOLOGICAL UNIVERSITY,
BELGAVI during the year 2023-2024. It is certified that all corrections/suggestions indicated
for Internal assessment have been incorporated in the report deposited in the departmental
library. The project report has been approved as it satisfies the academic requirements in
respect of Project work prescribed for the Bachelor of Engineering Degree.

Mrs. Shreekshitha Mr. Sudarshan K Dr. Shrinivasa Mayya D

Project guide Head of the Department Principal

Name of the Examiners Signature with Date

1.

2.
 ACKNOWLEDGEMENT
We are very grateful to our guide Mrs. Shreekshitha, Assistant Professor,
Department of Information Science and Engineering, who has been a great mentor
and guide at all times. The project wouldn’t have been a success without her help.

We would like to thank our project coordinator Mr. Athmaranjan K Associate

Professor, Department of Information Science and Engineering for his valuable
suggestions.

Our heartfelt thanks to Mr. Sudarshan, Head of the Department, Information

Science and Engineering, for his cordial support, valuable information and guidance,
which helped us in completing this project through various stages.

We also express our heartfelt gratitude to our Principal Dr. Shrinivasa Mayya D. for
his kind co-operation and encouragement which helped us in the completion of this
project.

We would like to place on record our gratitude to our Management for providing a
wonderful academic atmosphere.

We would like to thank all the Teaching and Non-Teaching staff for their help
during the course of the project.

Lastly, we would like to thank our Parents for their moral support and our Friends
with whom we shared our day-to-day experiences and received lots of suggestions that
improved our quality of work

- DEEPAK KUMAR A(4SN20IS005)

-DHANUSH(4SN20IS006)

-T VAISHNAV(4SN20IS018)

-YAJNESH J KULAL(4SN20IS020)
 ABSTRACT
The advancement of computer vision and deep learning technologies has ushered in a new era of
innovation by entering the railway industry. Traditional train control systems rely on fixed infrastructure
and human operators, which can be inflexible and prone to errors. A new age technological solution for
problems faced in efficient railway transportation is in need that also safeguards the passenger’s life.
One such approach is this project that helps ai assisted driving of passenger trains using computer vision
along with deep-learning for efficient decision makings and prevent negligence of loco-pilot resulting in
life risks. This project contains the design, implementation, and testing of a robust system that can detect
obstacles on the track, pedestrians crossing the track and effective decision-making system that can
decide any collision or pedestrian accidents enabling loco pilots to make informed decisions about when
and where to apply emergency breaks etc. The integration of computer vision technology enables real-
time data acquisition, and deep learning algorithms empower trains to adapt to dynamic and complex
environments. Improved safety, reduced human error, increased operational efficiency, and enhanced
scheduling flexibility are some of the key advantages. While exploring the applications of AI in the field
of railways this approach can subsequently help the railway authorities to enable the existing safety
system “Kavach” and implement it to safeguard citizens at risk.
 TABLE OF CONTENTS
CHAPTER TITLE PAGE
No No
1 INTRODUCTION 1-4
1.1 Problem Statement 2
1.2 Proposed Solution 2
1.3 Scope Of The Project 3
1.4 System Requirements 3
2 LITERATURE SURVEY 5-9
2.1 Computer Vision And Image Processing: A Paper 5
Review
2.2 Yolov4: Optimal Speed And Accuracy Of Object 5
Detection
2.3 A Computer Vision System For Detection And 6
Avoidance For Automotive Vehicles
2.4 Computer Vision Based Obstacle Identification 6
Using Real-Time Illumination Sensor Data
2.5 Development Of Computer Vision Based Obstacle 6
Detection And Human Tracking On Smart
Wheelchair For Disabled Patient
2.6 Application Of Computer Vision And Deep 7
Learning In The Railway Domain For
Autonomous Train Stop Operation
2.7 Iot Based Vehicle Tracking And Monitoring 7
System Using Gps And Gsm
2.8 Automatic Obstacle Detection Method For The 7
Train Based On Deep Learning
2.9 An Intelligent Depth-Based Obstacle Detection 8
System For Visually-Impaired Aid Applications
2.10 Object Detection Using Deep Learning, CNNs 8
And Vision Transformers: A Review
3 SYSTEM DESIGN 10-16
 3.1 Use case diagram 10
3.2 Sequence diagram 11
3.3 Data flow diagram 13
3.4 Activity diagram 14
4 IMPLEMENTATION 17-27
4.1 Visual Studio Code 17
4.2 Convolutional Neural Network 18
4.3 TensorFlow and Keras 19
4.4 Matplotlib 19
4.5 OpenCV 20
4.6 Pseudo codes 21
4.6.1 Pseudo code for Image Classification 21
1. Pseudo code for Data Preparation 21
2. Pseudo code for load and preprocess data 22
3. Pseudo code for Model Development 23
4. Pseudo Code for Model Evaluation 24
5. Pseudo code for Model Deployment 24
4.7 Algorithms 25
4.7.1 Algorithm for Data Preparation 25
4.7.2 Algorithm for Load and Preprocess Data 25
4.7.3 Algorithm for Model Development 26
4.7.4 Algorithm for Model Evaluation 26
4.7.5 Algorithm for Model Deployment 27
5 SYSTEM TESTING 28-33
5.1 Testing objective 28
5.2 Testing levels 29
5.2.1 Unit testing 29
5.2.2 Integration testing 29
5.2.3 System testing 30
5.2.4 Acceptance testing 30
5.3 Testing methods 30
5.3.1 Black box testing 30
5.3.2 White box testing 31
5.4 Test cases 32
6 SCREENSHOTS 34-38
6.1 Convolutional Neural Network Graph 34
6.2 Obstacle Detected 35
6.3 No Obstacle Detected 36
6.4 Training Data-Clear Track 37
6.5 Training Data-Obstacles in Tracks 37
CONCLUSION 39
REFERENCES
 LIST OF FIGURES
FIGURE NO FIGURE NAME PAGE NO
3.1 Use case diagram for system and user 10
3.2 Sequence diagram for user and System 12
3.3 Data flow diagram for Real time object and 13
pedestrian detection
3.4 Activity diagram for storing details 15
5.1 Object Detection Image 32
5.2 No Object Detection Image 32
5.3 Edge Case Detection Image 33
6.1 Convolutional Neural Network Graph 34
6.2 Obstacle Detection 35
6.3 No Obstacle Detected 36
6.4 Training Data-clear Tracks 37
6.5 Training Data-Obstacles in Track 37
 CHAPTER 1
INTRODUCTION
Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Introduction

Chapter 1
INTRODUCTION
The railway industry is undergoing a transformative shift propelled by the integration of
computer vision and deep learning technologies. Traditional train control systems reliant
on fixed infrastructure and human operators often encounter limitations, including
inflexibility and susceptibility to errors. To address these challenges and ensure both
efficient transportation and passenger safety, there's a pressing need for innovative
technological solutions. The advancement of computer vision and deep learning
technologies has ushered in a new era of innovation by entering the railway industry.
Traditional train control systems rely on fixed infrastructure and human operators, which
can be prone to errors.

A new age technological solution for problems faced in efficient railway transportation is
in need that also safeguards the passenger’s life. This project contains the design,
implementation, and testing of a robust system that can detect obstacles on the track,
pedestrians crossing the track and effective decision-making system that can decide any
collision or pedestrian accidents enabling loco pilots to make informed decisions about
when and where to apply emergency breaks. Leveraging computer vision and deep
learning technologies, this project endeavors to enhance decision-making capabilities,
mitigating the risk of accidents due to negligence or unforeseen obstacles on the tracks.

The project entails the design, implementation, and rigorous testing of a robust system
capable of detecting track obstacles and pedestrians, thereby enabling an AI-driven
decision-making process. By empowering train operators with real-time information and
insights, the system aims to prevent collisions and pedestrian accidents

Through the exploration of AI applications in railway transportation, this project aims to

shed light on the potential of this cutting-edge technology in improving safety protocols
and operational efficiency. The successful integration of computer vision and deep
learning holds the promise of transforming the railway industry, ensuring safer and more
reliable transportation systems for the future.

Department of ISE, SIT, Mangaluru Page 1

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Introduction

1.1 Problem statement

The conventional railway industry heavily relies on fixed infrastructure and manual
control systems for train operation and safety, leading to limitations in Adaptability,
Efficiency, and Human Error. With rising need for safe and reliable approaches to run
railways system in Bharat there is a pressing need to explore and implement advanced
technologies like computer vision and deep learning to develop an AI assisted train
driving system that can intelligently take decisions analysing real time data. This system
should be designed such that it can overcome all traditional challenges in the system.

1.2 Proposed solution

The solution is to develop an innovative train assistance system that leverages the power
of computer vision and deep learning technologies. This system will be designed to
enhance the safety, efficiency, and adaptability of railway operations. Implement
advanced computer vision algorithms to continuously monitor the railway tracks in real-
time for any potential obstructions. Detect and classify objects such as obstacles, tracks,
humans, or any miscellaneous object being placed on the tracks.
Employ deep learning models, such as convolutional neural networks (CNNs) and
recurrent neural networks (RNNs), to process the data collected by the computer vision
system. Train the models to make informed decisions about when and where to stop the
train based on the detected objects and dynamic environmental factors.
Establish a robust data integration pipeline to ensure seamless communication
between the computer vision system, deep learning models, and the train control system.
Enable the transfer of real-time data to inform braking and acceleration decisions.
Implement multiple redundancy layers in the system to guarantee passenger safety and
operational reliability. Develop fail-safe mechanisms to handle system failures or
unexpected situations.
Design the system with scalability in mind, allowing for easy integration into
existing railway infrastructure. Provide support for gradual deployment, enabling a
phased transition to autonomous train stop operations.
Conduct rigorous testing and validation procedures, including simulations and
real-world testing, to fine-tune the system's performance and safety. Continuously
monitor and improve the system's accuracy and reliability.

Department of ISE, SIT, Mangaluru Page 2

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Introduction

1.3 Scope of the project

Compared to the existing system the proposed system will increase the efficiency of the
flow of vehicles with a valid document’s and thereby saving a lot of time. Provides an
easy and faster way to verify each and every document of all the vehicles and can detect
the vehicle with expired or out of validity documents. To effectively implement and
develop a system in the railways includes the following key aspects:

1. System Development: Design and develop AI assisted obstruction tracking and

pedestrian alert

2. Data Acquisition and Processing: Implement mechanisms for real-time data acquisition
from sensors, cameras, and other relevant sources along the railway track. Develop data
processing pipelines to analyze and interpret the acquired data system, integrating
computer vision-based object detection and deep learning-based decision-making
components.

3. Object Detection and Classification: Implement computer vision algorithms to detect

and classify objects, including obstacles, and pedestrians. Ensure accurate and real-time
object recognition.

4. Deep Learning Models: Develop deep learning models, including convolutional neural
networks (CNNs) and recurrent neural networks (RNNs), for decision-making. Train and
fine-tune these models to take decisions based on detected objects analyzing the dynamic
environment.

5. Scalability and Integration: Design the system to be scalable and adaptable for
integration into existing railway infrastructure. Plan for a phased deployment approach to
accommodate different rail networks.

6. Testing and Validation: Conduct extensive testing, including simulations and real-
world testing, to validate the system's performance, accuracy, and safety. Continuously
monitor and improve the system's capabilities

1.4 System requirements

Software Requirements
• Operating System: Android.
• Front end: Android Studio with XML 1.0, Java JDK 8.0.

Department of ISE, SIT, Mangaluru Page 3

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Introduction
• Back end: MySQL 5.0., PHP 5.6.31.
• Emulator: Genymotion 2.12.0.
• Designed to work on android 4.2 or higher.
Hardware Requirements
• Processor: 1.5GHz Quad-Core processor.
• RAM: 1 GB or above.
• Internal Memory: 32 GB or above.
• Phone Camera minimum 12MP required.

Department of ISE, SIT, Mangaluru Page 4

 CHAPTER 2
LITERATURE SURVEY
Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Literature Survey

Chapter 2
LITERATURE SURVEY
Literature survey is the documentation of a comprehensive review of the published and unpublished
work from secondary sources data in the areas of specific interest to the researcher. It is important
for gathering the secondary data for the research which might be proved very helpful in the research.
The literature survey can be conducted for several reasons. The literature review can be in any area
of the business. Below are the few papers referred.

2.1 COMPUTER VISION AND IMAGE PROCESSING: A PAPER

REVIEW
Computer vision has been studied by many perspectives. It expands from raw data recording into
techniques and ideas combining digital image processing, pattern recognition, machine learning and
computer graphics. The extensive usage has attracted many scholars to integrate with many
disciplines and fields. This paper provides a survey of the recent technologies and theoretical
concept explaining the development of computer vision mainly related to image processing using
different areas of their field application. Computer vision helps scholars to analyze images and
video to obtain necessary information, understand information on events or descriptions, and
beautiful pattern. It used the method of multi-range application domain with massive data analysis.
This paper contributes to recent development on reviews related to computer vision, image
processing, and their related studies. We categorized the computer vision mainstream into four
group, e.g., image processing, object recognition, and machine learning. We also provide brief
explanation on the up-to-date information about the techniques and their performance.

2.2 YOLOv4: OPTIMAL SPEED AND ACCURACY OF OBJECT

DETECTION
There are a huge number of features which are said to improve Convolutional Neural Network
(CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical
justification of the result, is required. Some features operate on certain models exclusively and for
certain problems exclusively, or only for small-scale datasets; while some features, such as batch-
normalization and residual-connections, are applicable to the majority of models, tasks, and
datasets. We assume that such universal features include Weighted-Residual-Connections (WRC),
Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-
training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish
activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and

Department of ISE, SIT, Mangaluru Page 5

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Literature Survey

combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS
COCO dataset at a real time speed of ∼65 FPS on Tesla V100.

2.3 A COMPUTER VISION SYSTEM FOR DETECTION AND

AVOIDANCE FOR AUTOMOTIVE VEHICLES
This paper presents a vision based algorithm for obstacle detection and avoidance for autonomous
land vehicle. For an efficient algorithm it is necessary to know the information about surrounding
environment of vehicle. The whole environment is classified in two groups as obstacle and path.
This classification helps to find the space for avoiding the obstacle. Data is acquired by a high-
resolution digital camera. Relative distance of the obstacle, dimension of the obstacle, turning radius
of the vehicle and required turning angle of the steering are estimated.

2.4 COMPUTER VISION BASED OBSTACLE IDENTIFICATION

USING REAL-TIME ILLUMINATION SENSOR DATA
In this study, an IoT based system is developed to monitor obstacles at the indoor surface using
mobile sensors in a client-server wireless network. An IR transceiver system gives the positional
information and light intensity sensor measures the lux values. An embedded wifi enabled
microcontroller is interfaced with the sensors and performs as the client system. The client module
is placed over the roof of a car and when it moves through a particular indoor space, it collects the
positional illumination data and transmit them to the server unit. The captured sensor values are
stored in server laptop as MS-Excel file under the influence of a wifi router. By using offline
processing, the real-time sensor data is converted into an image and filtering methods are applied for
linear and nonlinear noise removal. Then, edge detection techniques like Canny, Prewitt, Sobel, and
Roberts methods are applied to detect the presence of obstacles. The study is repeated for another
room to find out the best possible obstacle identification method. Finally, it was concluded that the
Canny’s algorithm provides the most accurate identification of static obstacles for both the rooms.

2.5 DEVELOPMENT OF COMPUTER VISION BASED OBSTACLE

DETECTION AND HUMAN TRACKING ON SMART
WHEELCHAIR FOR DISABLED PATIENT
People with physical disability such as quadriplegics may need a device which assist their mobility.
Smart wheelchair is developed based on conventional wheelchair and is also generally equipped
with sensors, cameras and computer-based system as main processing unit to be able to perform
specific algorithm for the intelligent capabilities. We develop smart wheelchair system that
facilitates obstacle detection and human tracking based on computer vision. The experiment result
of obstacle distance estimation using RANSAC showed lower average error, which is only 1.076 cm

Department of ISE, SIT, Mangaluru Page 6

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Literature Survey

compared to linear regression which is 2.508 cm. The average accuracy of human guide detecting
algorithm also showed acceptable result, which yield over 80% of accuracy.

2.6 APPLICATION OF COMPUTER VISION AND DEEP

LEARNING IN THE RAILWAY DOMAIN FOR AUTONOMOUS
TRAIN STOP OPERATION
The purpose of this paper is to present the results of the analysis of the application of Deep Learning
in the railway domain with a particular focus on a train stop operation. The paper proposes an
approach consisting of monocular vision-based and Deep Learning architectures. Even the
difficulties imposed by actual regulation, the findings show that Deep Learning architecture can
offer promising results in railway localization using techniques like visual odometry, SLAM or pose
estimation. Besides, in spite of the many datasets available in the literature needed to train the neural
network, none of them have been created for indoor railway environments. Therefore, a new dataset
should be created. Furthermore, the paper presents future research and development suggestions for
railway applications which contribute to guiding the mid-term research and development.

2.7 IOT BASED VEHICLE TRACKING AND MONITORING

SYSTEM USING GPS AND GSM
Generally the usage of vehicle tracking system has been increased rapidly. The major concern of the
proposed system is identifying the vehicle theft by implementing anti-system. Vehicle tracking
system is beneficial in many ways such as providing security to the personal vehicles, taxis, cabs,
school buses/cars and others. Vehicle tracking system is designed to know the position of the
vehicle. Tracking system is developed by using GPS and GSM modules to locate the user’s vehicle
easily. GPS module is used to track the location of the vehicle in the form of values such as latitude
and longitude. These values are transmitted to the user using GSM modem through mobile network.
Different sensors are used to detect alcohol consumption and to identify the accident. The Sensor
values can be monitored by anyone from anywhere in the world using thingspeak channel. RFID
technology is used to facilitate security to the vehicle tracking and anti-theft system. Ignition key is
used to detect the theft and when theft is detected, GSM control app is used to control the vehicle
remotely.

2.8 AUTOMATIC OBSTACLE DETECTION METHOD FOR THE

TRAIN BASED ON DEEP LEARNING
Automatic obstacle detection is of great significance for improving the safety of train operation.
However, the existing autonomous operation of trains mainly depends on the signaling control

Department of ISE, SIT, Mangaluru Page 7

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Literature Survey

system and lacks the extra equipment to perceive the environment. To further enhance the efficiency
and safety of the widely deployed fully automatic operation (FAO) systems of the train, this study
proposes an intelligent obstacle detection system based on deep learning. It collects perceptual
information from industrial cameras and light detection and ranging (LiDAR), and mainly
implements the functionality including rail region detection, obstacle detection, and visual–LiDAR
fusion. Specifically, the first two parts adopt deep convolutional neural network (CNN) algorithms
for semantic segmentation and object detection to pixel-wisely identify the rail track area ahead and
detect the potential obstacles on the rail track, respectively. The visual–LiDAR fusion part
integrates the visual data with the LiDAR data to achieve environmental perception for all weather
conditions. It can also determine the geometric relationship between the rail track and obstacles to
decide whether to trigger a warning alarm. Experimental results show that the system proposed in
this study has strong performance and robustness. The system perception rate (precision) is
99.994% and the recall rate reaches 100%. The system, applied to the metro Hong Kong Tsuen Wan
line, effectively improves the safety of urban rail train operation.

2.9 AN INTELLIGENT DEPTH-BASED OBSTACLE DETECTION

SYSTEM FOR VISUALLY-IMPAIRED AID APPLICATIONS
In this paper, we present a robust depth-based obstacle detection system in computer vision. The
system aims to assist the visually-impaired in detecting obstacles with distance information for
safety. With analysis of the depth map, segmentation and noise elimination are adopted to
distinguish different objects according to the related depth information. Obstacle extraction
mechanism is proposed to capture obstacles by various object proprieties revealing in the depth
map. The proposed system can also be applied to emerging vision-based mobile applications, such
as robots, intelligent vehicle navigation, and dynamic surveillance systems. Experimental results
demonstrate the proposed system achieves high accuracy. In the indoor environment, the average
detection rate is above 96.1 %. Even in the outdoor environment or in complete darkness, 93.7%
detection rate is achieved on average.

2.10 OBJECT DETECTION USING DEEP LEARNING, CNNS AND

VISION TRANSFORMERS: A REVIEW
Detecting objects remains one of computer vision and image understanding applications’ most
fundamental and challenging aspects. Significant advances in object detection have been achieved
through improved object representation and the use of deep neural network models. This paper
examines more closely how object detection has evolved in the era of deep learning over the past
years. We present a literature review on various state-of-the-art object detection algorithms and the

Department of ISE, SIT, Mangaluru Page 8

Track Vision- Real Time Obstruction Tracking And Pedestrian Alert System Literature Survey

underlying concepts behind these methods. We classify these methods into three main groups:
anchor-based, anchor-free, and transformer-based detectors. Those approaches are distinct in the
way they identify objects in the image. We discuss the insights behind these algorithms and
experimental analyses to compare quality metrics, speed/accuracy tradeoffs, and training
methodologies. The survey compares the major convolutional neural networks for object detection.
It also covers the strengths and limitations of each object detector model and draws significant
conclusions. We provide simple graphical illustrations summarising the development of object
detection methods under deep learning. Finally, we identify where future research will be
conducted.

Department of ISE, SIT, Mangaluru Page 9

 CHAPTER 3
SYSTEM DESIGN
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

Chapter 3
SYSTEM DESIGN
3.1 Use case diagram
A use case diagram in software development illustrates how users interact with a system to
accomplish tasks. It comprises actors (users or external systems), use cases (system
functionalities), and their connections. This visual representation clarifies the relationships
between actors and use cases, aiding in understanding system behavior. By providing a
structured overview, it facilitates collaboration and ensures the system meets user
requirements. In essence, use case diagrams are essential for capturing and conveying
system functionality, promoting collaboration, and user satisfaction.
• System: The system is depicted as a rectangle.
• Actor: Each actor is shown as a stick man.
• Use case: Each use case is shown as a solid bordered oval labeled with the
name of the use case.

Figure 3.1: Use case diagram for system and user

Figure 3.1 shows the interaction between use and system. In this collaborative
process between the user and the system, a machine-learning algorithm is applied to
image processing tasks. Initially, the user collects image data from diverse sources, after
which the system undertakes pre-processing tasks to prepare the images for algorithmic
application. Subsequently, the system selects an appropriate algorithm (e.g.,

Department of ISE, SIT, Mangaluru Page 10

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design
Convolutional Neural Network, Artificial Neural Network, Support Vector Machine, or
Random Forest) based on task requirements and data characteristics, followed by
training using the pre-processed images. Once training concludes, the user supplies a
separate set of test images for model evaluation. The system then utilizes the trained
model to generate predictions for these test images, presenting the outcomes to the user
for analysis. Based on the evaluation, the user may make necessary adjustments or
initiate further iterations. Throughout this iterative cycle, the user inputs image data and
selects algorithms, while receiving outputs such as the trained model, predictions, and
performance metrics. Meanwhile, the system manages computationally intensive tasks
like pre-processing, training, testing, and prediction, leveraging its capacity to handle
large datasets and complex calculations.

3.2 Sequence diagram

A sequence diagram in Unified Modeling Language (UML) is a kind of

interaction diagram that shows how processes operate with one another and in what order.
It is a construct of a Message Sequence Chart. A sequence diagram shows, as parallel
vertical lines (“lifelines”), different processes or objects that live simultaneously as
horizontal arrows, the messages exchanged between them, in the order in which they
occur. This allows the specification of simple runtime scenarios in graphical manner.
Figure 3.2 shows the sequence diagram for user and system.

1) Initialization
a) Set up system parameters, such as camera settings and detection thresholds.
b) Initialize necessary variables and data structures.
2) Live Video Capture
i) Continuously capture live video from a camera (e.g., mounted on a vehicle or
surveillance system).
ii) The video frames serve as input for subsequent processing.
3) Data Stream Task
i) Process the live video frames to create a continuous data stream.
ii) This involves tasks like frame extraction, noise reduction, and feature
extraction.
iii) The resulting data stream contains relevant information about the
environment.

Department of ISE, SIT, Mangaluru Page 11

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

4) Object Detection Workflow:

i) Pre-processing:
(a) Enhance image quality (e.g., adjust brightness, contrast).
(b) Remove noise and artifacts.
ii) Feature Selection:
(a) Identify relevant features (e.g., edges, corners) in each frame.
(b) Extract meaningful information for subsequent analysis.
iii) Convolutional Neural Network (CNN):
(a) Apply a trained CNN model to detect objects.
(b) The CNN learns to recognize patterns associated with pedestrians,
vehicles, and other obstacles.
iv) Training and Testing:
(a) Train the CNN using labeled data (positive and negative examples).
(b) Evaluate its performance on test data.
(c) Fine-tune parameters to improve accuracy.
v) Object Detection:
(a) Identify and localize obstacles (e.g., pedestrians, vehicles) in the video
frames.
(b) Trigger alerts (visual, auditory) when an obstruction is detected.

Figure 3.2: Sequence diagram for user and System

Department of ISE, SIT, Mangaluru Page 12

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

3.3 Data flow diagram

A Data Flow Diagram (DFD) is a graphical depiction showing how data moves through a
system, commonly used in software engineering and systems analysis. It simplifies
complex systems by focusing on data flow, illustrating how data enters, undergoes
processing, and exits the system. It abstracts system details and highlights data
movement. Key shapes include ovals for start/end points, lines for data flow direction,
parallelograms for input/output, rectangles for processes, and diamonds for decision
points. DFDs consist of several components, including processes, data stores, data flows,
and external entities. Processes represent the functions or activities that manipulate data
within the system, such as calculations or transformations. Data stores are repositories
where data is stored temporarily or permanently. Data flows are the paths along which
data travels between processes, data stores, and external entities. External entities are
sources or destinations of data outside the system boundary.
DFDs use standardized symbols to represent different components:
• Oval: Represents the start or end points of the process flow, indicating the
beginning or conclusion of a process.
• Line: Connects components and shows the direction of data flow.
• Parallelogram: Represents input or output of data, indicating data flow into or
out of the system.
• Rectangle: Represents processes or transformations that occur within the system,
such as calculations or data manipulation.
• Diamond: Represents decision points in the process flow, where the flow of data
may take different paths based on certain criteria or conditions being met.

Figure 3.3: Data flow diagram for Real time object and pedestrian detection
Department of ISE, SIT, Mangaluru Page 13
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

3.4 Activity diagram

An activity diagram, a type of UML (Unified Modeling Language) diagram, illustrates
workflows or processes by visualizing the sequence of actions within a system. It
showcases how various activities are coordinated and the control flow among them,
particularly beneficial for grasping the behavior of intricate systems or software
applications.

Key elements and ideas within activity diagrams include:

• Activity: Represents specific actions or operations within the system, which could
be tasks, processes, or operations occurring sequentially or concurrently.
• Transition: Signifies the flow of control from one activity to another, typically
depicted as an arrow indicating the direction of flow. Transitions may have
conditions or triggers determining their activation.
• Initial Node: Marks the starting point of the activity diagram, representing the
initial state or the process's beginning.
• Final Node: Marks the ending point of the activity diagram, indicating the final
state or completion of the process.
• Decision Node: Also known as a branching point, it represents a decision point
within the process where the flow of control can follow different paths based on
certain conditions.
• Merge Node: Merges multiple control paths into a single path, symbolizing the
convergence of different flows within the process.
• Fork Node: Splits the flow of control into multiple concurrent paths, representing
parallel or concurrent execution of activities.
• Join Node: Synchronizes multiple concurrent control paths into a single path,
representing the point where concurrent activities converge back into a single
flow.
• Action Flow: Illustrates transitions from one action state to another, usually
depicted with an arrowed line, also known as edges or paths.

Department of ISE, SIT, Mangaluru Page 14

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

Figure 3.4: Activity diagram for storing details

Figure 3.4 shows the activity diagram for storing details.
• The process starts with loading the dataset, which contains images related to
tracking and identifying pedestrians or obstacles.
• The next step involves pre-processing the images. This can include tasks like
resizing, normalization, augmentation, and noise reduction to prepare the images
for further analysis.
• After pre-processing, the relevant features from the images are selected. This step
involves identifying the attributes or parts of the images that are most useful for
the model to learn from, such as edges, textures, or shapes.
• Once the features are selected, an appropriate machine learning model is chosen.
This could be a type of neural network, decision tree, support vector machine, etc.,
depending on the nature of the problem and the data.
• The selected model is then trained using the processed images. During this step,
the model learns to recognize patterns and make predictions based on the training
data.

Department of ISE, SIT, Mangaluru Page 15

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Design

• After training, the model is evaluated and tested to see how well it performs. This
involves running the model on a separate set of test images that were not used
during training to assess its accuracy and ability to generalize.
• The model's performance is measured in terms of success or failure rate. This step
involves calculating metrics such as accuracy, precision, recall, and F1 score to
quantify the model's effectiveness.
• The overall accuracy of the model is calculated, combining the results from
various evaluations to give a comprehensive performance metric.
• The final step involves using the model's accuracy to trigger an LED blink. This
could be an indicator of the model's success, where the LED blinks based on the
accuracy threshold met by the model. For instance, a certain number of blinks
might indicate a high accuracy, while fewer blinks might indicate a lower
accuracy.

Department of ISE, SIT, Mangaluru Page 16

 CHAPTER 4
IMPLEMENTATION
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

Chapter 4
IMPLEMENTATION
During system implementation, the theoretical design is realized into a functional system
using Visual Studio Code as the development environment. This system incorporates
advanced machine learning models like CNN. Whether it replaces manual processes,
automates existing systems, or introduces modifications, this implementation maximizes
Visual Studio Code's flexibility alongside the powerful capabilities of these algorithms.

4.1 Visual Studio Code

Visual Studio Code (VSCode) is a free, cross-platform source code editor created by
Microsoft for Windows, Linux, and macOS. Launched in April 2015, it quickly gained
traction and is now one of the most popular code editors among developers. Its appeal lies
in its powerful features, user-friendly interface, and extensive customization options.
VSCode is known for its flexibility, supporting numerous programming languages and
frameworks, making it ideal for various development tasks. Moreover, its rich ecosystem
of extensions enhances its capabilities, allowing developers to customize the editor to their
specific needs. Key features of VSCode include:

• Cross-Platform Compatibility: Runs on Windows, macOS, and Linux, making

it accessible to many developers.

• Integrated Development Environment (IDE): Despite being lightweight, it

offers many features found in full IDEs, such as syntax highlighting, code
completion, debugging, and version control integration.

• Extensions: A vast ecosystem of extensions for various purposes, including

language support, debugging, themes, and productivity tools, available directly
from the editor.

• Customization: Users can personalize themes, key bindings, and settings to

match their preferences.

• Integrated Terminal: Includes a built-in terminal for running command-line

tasks without leaving the editor.

• IntelliSense: Provides code completion suggestions as you type, analyzing

Department of ISE, SIT, Mangaluru Page 17

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

your code and libraries for accurate recommendations.

• Language Support: Supports a wide range of programming languages,

including JavaScript, Python, Java, C++, and more, with additional
extensions for less common languages.

• Version Control Integration: Seamlessly integrates with Git, allowing you to

view changes, commit code, and manage branches within the editor.

• Debugging: Supports debugging for various languages and platforms,

enabling you to set breakpoints, inspect variables, and step through your code
to troubleshoot issues.

• Free and Open Source: Available for free and its source code is accessible on
GitHub under the MIT License.

4.2 Convolutional Neural Network

Convolutional Neural Networks (CNNs) are deep learning models tailored for visual tasks
such as image classification, object detection, and image segmentation. Inspired by the
structure of the animal visual cortex, CNNs are composed of layers including
convolutional layers for feature extraction, pooling layers for downsampling, and fully
connected layers for classification. They have transformed computer vision by
automatically learning hierarchical data representations, achieving state-of-the-art
performance in various image-related tasks. By capturing spatial feature hierarchies,
CNNs are adept at recognizing patterns within images and use techniques like pooling to
enhance computational efficiency. When trained on large datasets, CNNs can effectively
generalize to new data, making them crucial in fields such as medical imaging,
autonomous driving, and natural language processing.
This CNN architecture starts by extracting features like edges and textures with 32
filters in the convolutional layers. ReLU activation introduces non-linearity to learn
complex patterns, followed by max-pooling to reduce dimensions while preserving key
features. Subsequent layers with 64 and 128 filters further extract more abstract features.
Post-convolution, the feature maps are flattened and passed into two dense layers, each
containing 256 neurons with ReLU activation. A dropout layer with a 0.5 rate helps
prevent overfitting by randomly deactivating neurons. Finally, a single neuron output

Department of ISE, SIT, Mangaluru Page 18

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

layer with sigmoid activation provides a probability indicating the input image's
likelihood of belonging to one of two classes.

4.3 TensorFlow and Keras

TensorFlow and Keras are two formidable libraries for constructing and training deep
learning models in Python. TensorFlow, developed by Google, stands as an open-source
machine learning framework offering a flexible and extensive ecosystem for crafting
various machine learning and deep learning models. It boasts efficient computation
capabilities, supporting both CPU and GPU acceleration, making it apt for training models
on extensive datasets. TensorFlow provides both a low-level API for defining and
executing complex computational graphs and a high-level API that streamlines the process
of constructing and training neural networks.

On the contrary, Keras serves as a high-level neural networks API built on top of
TensorFlow (as well as other deep learning frameworks like Theano and Microsoft
Cognitive Toolkit). Keras presents users with a user-friendly and intuitive interface for
model construction, fostering rapid prototyping and experimentation. It furnishes a uniform
and straightforward API for defining neural networks, catering to both novices and
seasoned deep learning practitioners.

An evident advantage of employing Keras with TensorFlow is its user-friendliness

and adaptability. Keras empowers users to define neural network architectures using either
a sequential or functional API, while also offering an array of pre-built layers, activation
functions, optimizers, and loss functions that can be effortlessly combined to forge custom
models. Furthermore, Keras extends support for both CPU and GPU acceleration, enabling
efficient model training across various hardware setups.

TensorFlow and Keras complement each other seamlessly, with TensorFlow

delivering a robust computational backend and Keras furnishing a user-friendly interface
for constructing and training deep learning models. Together, they form a versatile and
comprehensive toolkit for crafting state-of-the-art machine learning applications.

4.4 Matplotlib
Matplotlib is a comprehensive data visualization library in Python that enables users to
create high-quality plots, charts, and graphs to visualize their data effectively. Developed
by John D. Hunter in 2003, Matplotlib has since become one of the most widely used

Department of ISE, SIT, Mangaluru Page 19

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

plotting libraries in the Python ecosystem. It provides a flexible and intuitive interface for
generating a wide variety of plots, ranging from simple line plots and scatter plots to
complex 3D plots and interactive visualizations. Matplotlib is designed to work seamlessly
with other Python libraries such as NumPy and pandas, making it a versatile tool for data
analysis and exploration.

One of the key features of Matplotlib is its extensive customization options, allowing
users to tailor every aspect of their plots to suit their specific needs and preferences. Users
can customize the appearance of plot elements such as colors, line styles, markers, fonts,
and annotations, as well as adjust the layout and formatting of the plot axes and labels.
Matplotlib also supports the creation of multi-panel plots, enabling users to visualize
multiple datasets or plots side by side for comparison or analysis.

Matplotlib offers a wide range of plot types and styles to accommodate various data
visualization requirements. In addition to basic 2D plots like line plots, scatter plots, and
bar charts, Matplotlib supports advanced plot types such as histograms, heatmaps, contour
plots, box plots, violin plots, and more. Furthermore, Matplotlib provides support for
creating interactive plots using tools like zooming, panning, and tooltips, as well as saving
plots in various file formats including PNG, PDF, SVG, and EPS. Overall, Matplotlib
provides a powerful and flexible platform for creating publication-quality visualizations for
both exploratory data analysis and presentation purposes, making it an essential tool for
data scientists, researchers, and analysts.

4.5 OpenCV

OpenCV (Open Source Computer Vision Library) stands as a cornerstone in the field of
computer vision, offering an extensive suite of tools and functionalities for image and
video analysis. Initially developed by Intel in 1999, OpenCV has evolved into a
collaborative effort driven by a global community of developers, researchers, and
enthusiasts. Its open-source nature fosters innovation and facilitates the development of
cutting-edge computer vision applications across a myriad of domains. With support for
multiple programming languages including C++, Python, Java, and MATLAB, OpenCV
caters to a diverse audience, enabling developers to harness its capabilities irrespective of
their preferred programming environment.

At its core, OpenCV empowers developers to tackle a wide range of image and video
processing tasks with ease and efficiency. From basic operations such as resizing and
Department of ISE, SIT, Mangaluru Page 20
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

cropping to more advanced functionalities like object detection, feature extraction, and
camera calibration, OpenCV provides a comprehensive toolkit for transforming raw
visual data into meaningful insights. Moreover, its integration with machine learning
frameworks such as TensorFlow and PyTorch opens up avenues for developing custom
models and algorithms, further expanding its utility and versatility.

OpenCV's impact extends far beyond academia and research, finding applications in
real-world scenarios across various industries. In automotive, OpenCV plays a pivotal
role in autonomous vehicles, enabling tasks such as lane detection, pedestrian detection,
and obstacle avoidance. In healthcare, it aids in medical image analysis for diagnosis and
treatment planning. In surveillance and security, OpenCV facilitates video analytics for
monitoring and threat detection. Its broad adoption and continuous development make
OpenCV a vital tool for driving innovation and advancing the frontier of computer vision
technology.

4.6 Pseudo Codes

4.6.1 Pseudo Code for Image Classification

1. Pseudo code for Data Preparation

BEGIN
1.1 Import Libraries:
IMPORT os
IMPORT tensorflow as tf
IMPORT cv2
IMPORT imghdr
IMPORT matplotlib.pyplot as plt
IMPORT numpy as np

1.2 Set Data Directory:

SET data_dir = os.path.join('data', 'traindata', 'datasetSeparation')
CALL os.listdir(data_dir)
1.3 Validate and Clean Data:
SET image_exts = ['jpeg', 'jpg', 'bmp', 'png']
FOR EACH image_class IN os.listdir(data_dir):
FOR EACH image IN os.listdir(os.path.join(data_dir, image_class)):

Department of ISE, SIT, Mangaluru Page 21

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

SET image_path = os.path.join(data_dir, image_class, image)

TRY:
SET img = cv2.imread(image_path)
SET tip = imghdr.what(image_path)
IF tip NOT IN image_exts:
PRINT "Image not in ext list", image_path
CALL os.remove(image_path)
EXCEPT Exception AS e:
PRINT "Exception:", e
CALL os.remove(image_path)

2. Pseudo Code For Load And Preprocess Data

2.1 Load Dataset:
SET data = tf.keras.utils.image_dataset_from_directory(data_dir)

2.2 Visualize Sample Images:

SET data_iterator = data.as_numpy_iterator()
SET batch = data_iterator.next()
CALL fig, ax = plt.subplots(ncols=4, figsize=(20, 20))
FOR idx, img IN ENUMERATE(batch[0][:4]):
CALL ax[idx].imshow(img.astype(int))
SET ax[idx].title.set_text(batch[1][idx])

2.3 Normalize Images:

SET scaled = batch[0] / 255
CALL scaled.min()

2.4 Split Dataset:

SET data = data.map(lambda x, y: (x / 255, y))
SET train_size = int(len(data) * 0.7)
SET val_size = int(len(data) * 0.15)
SET test_size = len(data) - train_size - val_size

Department of ISE, SIT, Mangaluru Page 22

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

SET train = data.take(train_size)

SET val = data.skip(train_size).take(val_size)
SET test = data.skip(train_size + val_size).take(test_size)

3. Pseudo code for Model Development

3.1 Build Model:
FROM tensorflow.keras.models IMPORT Sequential
FROM tensorflow.keras.layers IMPORT Conv2D, MaxPooling2D, Dense, Flatten

SET model = Sequential()

CALL model.add(Conv2D(16, (3, 3), 1, activation='relu', input_shape=(256, 256, 3)))
CALL model.add(MaxPooling2D())
CALL model.add(Conv2D(32, (3, 3), 1, activation='relu'))
CALL model.add(MaxPooling2D())
CALL model.add(Conv2D(16, (3, 3), 1, activation='relu'))
CALL model.add(MaxPooling2D())
CALL model.add(Flatten())
CALL model.add(Dense(256, activation='relu'))
CALL model.add(Dense(1, activation='sigmoid'))

CALL model.compile('adam', loss=tf.losses.BinaryCrossentropy(), metrics=['accuracy'])

CALL model.summary()

3.2 Compile Model:

SET logdir = 'logs'
SET tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=logdir)
CALL hist = model.fit(train, epochs=20, validation_data=val,
callbacks=[tensorboard_callback])
3.3 Plot Training Performance:
CALL fig = plt.figure()
CALL plt.plot(hist.history['loss'], color='teal', label='loss')
SET fig.suptitle('Loss', fontsize=20)
CALL plt.legend(loc='upper left')

Department of ISE, SIT, Mangaluru Page 23

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

CALL plt.show()
CALL fig = plt.figure()
CALL plt.plot(hist.history['accuracy'], color='teal', label='accuracy')
SET fig.suptitle('Accuracy', fontsize=20)
CALL plt.legend(loc='upper left')
CALL plt.show()

4. Pseudo Code For Model Evaluation

4.1 Evaluate on Test Data:
FROM tensorflow.keras.metrics IMPORT Precision, Recall, BinaryAccuracy

SET pre = Precision()

SET re = Recall()
SET acc = BinaryAccuracy()

FOR EACH batch IN test.as_numpy_iterator():

SET X, y = batch
SET yhat = model.predict(X)
CALL pre.update_state(y, yhat)
CALL re.update_state(y, yhat)
CALL acc.update_state(y, yhat)

PRINT "Precision:", pre.result().numpy(), "Recall:", re.result().numpy(), "Accuracy:",

acc.result().numpy()
5. Pseudo code for Model Deployment
5.1 Save Model:
FROM tensorflow.keras.models IMPORT load_model

CALL model.save(os.path.join('models', 'track.keras'))

5.2 Load and Predict:

SET new_model = load_model(os.path.join('models', 'track.keras'))

# Predict for first image

Department of ISE, SIT, Mangaluru Page 24

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

SET img = cv2.imread('data/traindata/datasetSeparation/Obstacles/track2_220.jpg')

SET resize = tf.image.resize(img, (256, 256))
SET yhat = new_model.predict(np.expand_dims(resize / 255, 0))
IF yhat[0][0] > 0.5:
PRINT "Given Image has obstacle"
ELSE:
PRINT "Given Image has no obstacle"
# Predict for second image
SET img = cv2.imread('data/traindata/datasetSeparation/ClearTrack/track1_0.jpg')
SET resize = tf.image.resize(img, (256, 256))
SET yhat = new_model.predict(np.expand_dims(resize / 255, 0))
IF yhat[0][0] > 0.5:
PRINT "Given Image has obstacle"
ELSE:
PRINT "Given Image has no obstacle"

4.7 Algorithms
4.7.1 Algorithm for Data Preparation
Step 1: Import Libraries

- Import necessary libraries for file handling, image processing, data visualization, and
deep learning.

Step 2: Set Data Directory

- Define the path to the dataset directory.

Step 3: Validate and Clean Data

- Check each image for valid file extensions (e.g., jpeg, jpg, bmp, png).

- Remove images with invalid extensions or errors.

4.7.2 Algorithm for Load and Preprocess Data

Step 1: Load Dataset
- Load images from the directory using a function that creates a dataset object.

Department of ISE, SIT, Mangaluru Page 25

 Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

Step 2: Visualize Sample Images

- Plot a few sample images and their labels for inspection.

Step 3: Normalize Images:

- Scale image pixel values to the range 0-1 for better model performance.

Step 4: Split Dataset

- Divide the dataset into training, validation, and testing sets based on specified ratios.

4.7.3 Algorithm for Model Development

Step 1: Build Model

- Define a Sequential model.

- Add convolutional layers for feature extraction.

- Add pooling layers to reduce dimensionality.

- Flatten the output of the convolutional layers.

- Add dense (fully connected) layers, with a final output layer suitable for classification
(e.g., sigmoid for binary classification).

Step 2: Compile Model

- Compile the model with an optimizer (e.g., adam), a loss function (e.g., binary
crossentropy for binary classification), and evaluation metrics (e.g., accuracy).

Step 3: Train Model

- Train the model using the training dataset.

- Use validation data to monitor the training process and prevent overfitting (e.g., through
early stopping or callbacks).

Step 4: Plot Training Performance

- Plot training and validation loss and accuracy over epochs to visualize model
performance.

4.7.4 Algorithm for Model Evaluation

Department of ISE, SIT, Mangaluru Page 26
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Implementation

Step 1: Evaluate on Test Data

- Compute evaluation metrics such as precision, recall, and accuracy on the test dataset.

- Print or log these metrics for performance assessment.

4.7.5 Algorithm for Model Deployment

Step 1: Save Model

- Save the trained model to a file for future use

Step 2: Load and Predict

- Load the saved model when needed.

- Preprocess input images (e.g., resize and normalize).

- Use the model to make predictions on new images.

- Interpret and display the predictions.

Department of ISE, SIT, Mangaluru Page 27

 CHAPTER 5
SYSTEM TESTING
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

Chapter 5
SYSTEM TESTING
Software testing is an essential procedure that evaluates the accuracy, comprehensiveness,
security, and overall excellence of computer software. It entails methodically running the
program or application to detect errors or flaws, ensuring it adheres to predetermined
specifications. Because quality is subjective and varies among stakeholders, testing
cannot guarantee flawless software but offers a crucial comparison against expectations.

Testing serves as the primary stage in detecting errors, relying heavily on the
quality of test cases. Since code is the sole executable product, testing serves as the
frontline for identifying remaining errors from earlier phases. Testers execute the program
with a set of test cases and assess its output against anticipated behavior to ascertain if the
software operates as intended.

Various criteria, including specifications, agreements, past iterations, and user

expectations, are employed to evaluate correctness during testing. Testers utilize these
principles to identify any issues, ensuring the software meets desired standards.
Ultimately, testing provides valuable insights into software quality and its susceptibility to
failure, informing enhancements for improved performance.

5.1 Testing objective

The main goal of software testing is to guarantee that the software aligns with the
specifiedrequirements and operates correctly as intended. This involves:
• Confirming Functionality: Testing ensures that the software executes its designated
functions accurately and effectively.
• Detecting Defects: Testing aids in the detection of defects, glitches, or mistakes
within the software, enabling them to be rectified before deployment to users.
• Ensuring Reliability: Testing strives to ensure that the software functions reliably
under diverse conditions and situations, without unexpected crashes or
malfunctions.
• Evaluating Performance: Testing assesses the software's performance, encompassing
its speed, responsiveness, and resource utilization, to verify it meets performance
criteria.
• Compliance: Testing validates that the software adheres to relevant standards,

Department of ISE, SIT, Mangaluru Page 28

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

Regulations and industry norms.

• Ensuring Security: Testing identifies vulnerabilities and weaknesses in the

software that could be exploited by attackers, safeguarding user data and system
integrity.

• Confirming Usability: Testing evaluates the usability of the software, ensuring it

isuser-friendly and intuitive for its intended users.

5.2 Testing levels

5.2.1 Unit testing

Unit testing marks the outset of the testing process in software development, during
which developers scrutinize individual components or units of the software separately.
The aim is to validate that each unit operates accurately in line with its planned design.
Usually, developers conduct unit testing, utilizing dedicated tools and frameworks to
automate procedures. Through examining each unit autonomously, developers can detect
any potential issues at an early stage, thus bolstering the dependability and effectiveness
of the overall software system.

5.2.2 Integration testing

Integration testing follows unit testing and focuses on assessing the interactions among
different software units or modules. The primary objective is to ensure that when these
units are combined, they function smoothly within the overall system. Integration testing
checks that interfaces between units are seamless and that data flows correctly among
them. This testing can be done incrementally as new units are developed and integrated,
allowing for early detection of compatibility issues or component discrepancies. Various
approaches to integration testing include big bang, incremental (top-down, bottom-up),
and sandwich (hybrid) methodologies. Tools such as stubs and drivers are often used to
simulate interactions with higher-level or lower-level modules during incremental testing.
By using integration testing techniques like black box, white box, and gray box testing,
developers can thoroughly examine both the functionality and internal workings of
module interactions. This ensures that software components work together effectively,
laying the foundation for a robust and unified system. Furthermore, continuous
integration practices can enhance this process by automating the integration and testing
phases, ensuring frequent testing and prompt identification of integration issues.

Department of ISE, SIT, Mangaluru Page 29

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

5.2.3 System testing

System testing involves a thorough evaluation of the entire software system to
ensure it complies with specified requirements and functions correctly within its intended
environment. This phase assesses the system's behavior and functionality across diverse
scenarios, including normal operation, stress conditions, and boundary cases. Testers
replicate real-world usage to confirm that the system behaves as anticipated and can
manage various inputs and conditions proficiently. Typically, system testing is performed
by independent testers or quality assurance teams who assess the system's performance
against predetermined criteria. Through comprehensive system testing, organizations can
uncover any defects or discrepancies prior to deployment, guaranteeing that the software
meets quality standards and user expectations.

5.2.4 Acceptance testing

Acceptance testing represents the final validation step before software deployment,
focusing on verifying that the software meets user requirements and supports business
objectives. Its primary aim is to determine whether the software meets the predefined
acceptance criteria set by stakeholders. This testing phase typically involves end-users,
clients, or other key stakeholders who evaluate the software's functionality, usability, and
overall suitability for their needs. By actively engaging these stakeholders, acceptance
testing ensures that the software aligns with their expectations and business requirements.
Any discrepancies or issues identified during acceptance testing are resolved before the
final deployment, guaranteeing that the software meets quality standards and provides
value to its users.

5.3 Testing methods

5.3.1 Black box testing
Black Box Testing is an evaluation approach for software functionality that relies solely
on observing its external behavior, avoiding exploration of its internal mechanisms or
code structure. Testers treat the system as an opaque entity, concentrating solely on
understanding its inputs, outputs, and responses. Interacting with interfaces like user
interfaces, APIs, or command-line interfaces, testers simulate various scenarios to verify
whether the system behaves as expected per its specifications.

This method is particularly valuable for assessing the system from an end-user

Department of ISE, SIT, Mangaluru Page 30

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

standpoint, focusing on how the software functions and behaves without needing insight
into its internal workings. Black box testing empowers testers to uncover defects, errors,
or inconsistencies in the system's behavior without being influenced by its underlying
codebase. It fosters thorough testing of all potential inputs and pathways through the
system, ensuring the software meets its functional requirements and delivers a satisfactory
user experience.

5.3.2 White box testing

White Box Testing, also referred to as structural or glass-box testing, entails examining
the internal structure and operations of the software system. Testers have access to the
source code, architectural design, and detailed specifications, empowering them to devise
tests that scrutinize specific code paths, conditions, and branches within the software. By
analyzing the internal logic, data flows, and control structures of the code, white box
testing aims to ensure comprehensive coverage and validation of the system's
functionality, while also identifying potential defects or vulnerabilities that may not be
evident externally.

This approach provides insights into the system's internal behavior, enabling testers
to formulate tests that target critical areas of the codebase and verify its accuracy and
resilience. Through techniques such as statement coverage, branch coverage, and path
coverage, white box testing offers a deeper understanding of the software's execution flow
and assists in pinpointing potential areas for optimization or enhancement. By integrating
white box testing with other testing methodologies, organizations can attain extensive test
coverage and guarantee the reliability and integrity of their software systems.

Department of ISE, SIT, Mangaluru Page 31

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

5.4 Test Case

Test case 1:
Given input: Image containing obstacles placed on the track

Fig 5.1: Object Detection Image

Expected output: Change detect label to obstacles detected

Output: Success

Test case 2:
Given input: Image containing no obstacles being placed on the track

Fig 5.2: No Object Detection Image

Department of ISE, SIT, Mangaluru Page 32

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System System Testing

Expected output: Change detect label to no obstacles detected

Output: Success

Test case 3:
Given input: Image containing obstacles placed on the edge of the track and edge of the

video frame

Fig 5.3: Edge Case Detection Image

Expected output: Change detect label to obstacles detected

Output: Failure, fluctuating output between obstacles detected and no obstacles detected

Department of ISE, SIT, Mangaluru Page 33

 CHAPTER 6
SCREENSHOTS
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Screen Shots

Chapter 6
SCREENSHOTS
6.1 Convolutional Neural Network Graph

Figure 6.1: Convolutional Neural Network Graph

The diagrams show the training accuracy and loss of a Convolutional Neural Network
(CNN) model over 35 epochs, revealing a rapid improvement in the initial phase (epochs
0-5) where accuracy jumps from around 0.75 to nearly 1.0 and loss plummets from 0.6 to
almost 0. This is followed by a middle phase (epochs 5-10) where the model continues to
improve, though at a slower rate, with accuracy nearing perfect levels and loss reaching
minimal values. Finally, in the last phase (epochs 10-35), the model stabilizes,
maintaining high accuracy around 1.0 and a very low loss, indicating convergence. The
training process involves data preparation, model design, training, evaluation, adjustment,
and final model saving, ensuring the CNN effectively learns and performs the task.

Department of ISE, SIT, Mangaluru Page 34

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Screen Shots

6.2 Obstacle Detected

Figure 6.2: Obstacle Detection

Figure 6.2 presents the output from the "obstacleDetection" program, which has
successfully identified an obstacle on a set of toy train tracks. This output illustrates the
system's proficiency in detecting obstructions in real-time, a crucial feature for
applications where safety and collision prevention are paramount. The image highlights
the system's capability to recognize and signal the presence of obstacles in the trajectory
of a moving object, such as a train. This functionality demonstrates the potential of the
system to significantly enhance safety operations across various sectors, including
transportation and robotics. In automated systems, where precise and dependable
detection mechanisms are necessary for safe navigation, this technology proves especially
beneficial. By ensuring accurate and timely identification of obstructions, the system
supports the seamless operation of autonomous vehicles and robotic systems, mitigating
risks associated with collisions and improving overall safety.

Department of ISE, SIT, Mangaluru Page 35

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Screen Shots

6.3 No Obstacle Detected

Figure 6.3: No Obstacle Detected

Figure 6.3 The image provided displays a clear toy train track accompanied by the
message “No Obstacle Detected.” This indicates the system’s capability to monitor track
conditions in real-time. Such functionality is vital for the safety of automated systems, as
it verifies that the path is free from any obstructions that could potentially cause accidents
or delays.Ensuring that the track is clear is just as crucial as detecting obstacles. The
system's effectiveness in confirming the absence of hazards supports the continuous and
smooth operation of trains or other automated vehicles. This ability to provide real-time
status updates enhances operational efficiency and safety, allowing for uninterrupted and
safe navigation.This technology is versatile and can be applied in numerous settings
where real-time monitoring and prompt response are essential. Its applications extend
beyond transportation to any industry that relies on the safe and efficient movement of
automated systems. By maintaining clear pathways and promptly identifying any
potential hazards, this system plays a critical role in operational safety and reliability.

Department of ISE, SIT, Mangaluru Page 36

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Screen Shots

6.4 Training Data-Clear Track

Figure 6.4: Training Data-clear Tracks

Figure 6.4 shows that these images likely serve as training data for a machine learning
model, specifically designed for the “Track Vision” system you mentioned earlier. The
training data is crucial for the model to learn and distinguish between scenarios where the
track is clear and when there are obstacles present. By analyzing these images, the model
can improve its accuracy in real-time obstruction detection and pedestrian alerting,
ensuring the safety and efficiency of railway operations.

6.5 Training Data-Obstacles in Tracks

Figure 6.4: Training Data-Obstacles in Track

Department of ISE, SIT, Mangaluru Page 37

Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Screen Shots

Figure 6.5 training data for obstacle detection on railway tracks, we create synthetic
images representing various scenarios. These images include railway tracks, backgrounds
and strategically placed obstacles (debris, animals, structures, people, and vehicles).
Annotations indicate the obstacle type and location. Data augmentation techniques
enhance diversity by varying orientation, scale, and introducing realistic imperfections.
This comprehensive dataset enables machine learning models to learn and accurately
detect obstacles, ensuring safety and efficiency in rail transport systems.

Department of ISE, SIT, Mangaluru Page 38

 CHAPTER 7
CONCLUSION
Track Vision- Real Time Obstruction Tracking and Pedestrian Alert System Conclusion

CONCLUSION
The development of an AI-assisted obstruction tracking and pedestrian alert system for
the Indian railway system represents a significant advancement in railway safety and
efficiency. By leveraging computer vision and deep learning technologies, this project
addresses critical challenges faced by traditional train control systems, such as
inflexibility and human error. The integration of real-time data acquisition, accurate
object detection, and robust decision-making models ensures that train operators are
equipped with timely and precise information to prevent collisions and accidents. The
scalability and adaptability of the system further enhance its potential for widespread
implementation across various railway networks. Through rigorous testing and
continuous improvement, this innovative solution promises to transform the railway
industry, providing safer and more reliable transportation for passengers and goods.

Department of ISE, SIT, Mangaluru Page 39

REFERENCES
 REFERENCES
1. Wiley, Victor Lucas, Thomas. (2018). Computer Vision and Image Pro- cessing: A Paper
Review. International Journal of Artificial Intelligence Research. 2. 22. 10.29099/ijair.v2i1.42.

2. Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy
of object detection. arXiv preprint arXiv:2004.10934.

3. A. Alam and Z. A. Jaffery,” A computer vision system for detection and avoidance for
automotive vehicles,” 2015 Annual IEEE India Conference(INDICON), New Delhi, India, 2015,
pp. 1-6, doi: 10.1109/INDI- CON.2015.7443366.

4. Ghosh, P. K. Kundu and G. Sarkar,” Computer Vision based Obstacle Identification using Real-
Time Illumination Sensor Data,” 2021 IEEE Second International Conference on Control,
Measurement and Instrumentation (CMI), Kolkata, India, 2021, pp. 190-195,
doi:10.1109/CMI50323.2021.9362734.

5. F. Utaminingrum et al.,” Development of computer vision based obstacle detection and human
tracking on smart wheelchair for disabled patient,” 2017 5th International Symposium on
Computational and Business Intelligence (ISCBI), Dubai, United Arab Emirates, 2017, pp. 1-5,
doi: 10.1109/ISCBI.2017.8053533.

6. M. Etxeberria-Garcia, M. Labayen, M. Zamalloa and N. Arana- Arexolaleiba,” Application of

Computer Vision and Deep Learn- ing in the railway domain for autonomous train stop opera-
tion,” 2020 IEEE/SICE International Symposium on System In-tegration (SII), Honolulu, HI,
USA, 2020, pp. 943-948, doi: 10.1109/SII46433.2020.9026246.

7. A. H. Alquhali, M. Roslee, M. Y. Alias and K. S. Mohamed,” IOT Based Real-Time Vehicle

Tracking System,” 2019 IEEE Confer- ence on Sustainable Utilization and Development in
Engineering and Technologies (CSUDET), Penang, Malaysia, 2019, pp. 265-270, doi:
10.1109/CSUDET47057.2019.9214633.

8. Zhang, Qiang Yan, Fei Song, Weina Wang, Rui Li, Gen. (2023). Automatic Obstacle Detection
Method for the Train Based on Deep Learning. Sustainability. 15. 1184. 10.3390/su15021184.

9. Chia-Hsiang Lee, Yu-Chi Su and Liang-Gee Chen,” An intelligent depth-based obstacle

detection system for visually-impaired aid ap- plications,” 2012 13th International Workshop on
Image Analysis for Multimedia Interactive Services, Dublin, Ireland, 2012, pp. 1-4, doi:
10.1109/WIAMIS.2012.6226753.

10. A. B. Amjoud and M. Amrouch,” Object Detection Using Deep Learn- ing, CNNs and Vision
Transformers: A Review,” in IEEE Access, vol. 11, pp. 35479-35516, 2023, doi:
10.1109/ACCESS.2023.3266093.