.

Indian Institute of Technology Kharagpur

Embedded Machine Learning
Class Test 1

Full Marks: 40
Time allowed: 1.5 hour
1. (a) What are the scenarios where you should not opt for machine learning? (2 marks)
(b) Comment on the deciding parameters for selecting filters. Justify your choices. (4 marks)
2. Select the correct answer with the explanation. Partial answer without explanation will not be
considered for evaluation.

[Each question carries 3 marks (8 ∗ 3 = 24). ]

(a) Which of the following statements correctly describes the role of L2 regularization in preventing
overfitting in a machine learning model?
I. L2 regularization adds a penalty to the loss function based on the absolute values of the weights,
which encourages the model to ignore less important features.
II. L2 regularization adds a penalty to the loss function based on the square of the weights, which
helps to prevent the model from assigning too much importance to any single feature, thereby
reducing overfitting.
III. L2 regularization adds noise to the input data during training, which helps to prevent the model
from memorizing the training data and overfitting.
IV. L2 regularization reduces the number of parameters in the model by setting some of the weights
to zero, which directly decreases the complexity of the model and prevents overfitting.
V. None of the above
(b) Which of the following best describes the concept of the Inception module in deep learning?
I. The Inception module is designed to replace traditional convolutional layers with fully connected
layers to increase the depth of a neural network without increasing the computational complexity.
II. The Inception module allows a neural network to process an input image at multiple scales
by using multiple convolution filters with different sizes in parallel, followed by concatenating the
results, which helps in capturing various levels of detail.
III. The Inception module reduces the dimensionality of the input data before passing it through a
series of pooling layers, thereby minimizing the risk of overfitting and reducing the computational
cost.
IV. The Inception module is used to increase the model’s ability to recognize objects by applying
dropout at multiple stages within a deep neural network, improving generalization performance.
V. None of the above
(c) When mapping a Deep Neural Network (DNN) to a GPU, which of the following approaches best
optimizes the use of GPU resources to achieve high computational efficiency and minimize training
time ?
I. Assigning each layer of the DNN to a different GPU ensures that each GPU processes one layer
at a time, balancing the computational load across multiple GPUs and maximizes parallelism.
II. Breaking down the DNN into smaller sub-networks and assigning each sub-network to a different
GPU, allowing for distributed processing, but potentially increasing the communication overhead
between GPUs.
III. Exploiting data parallelism by dividing the input data into batches and processing each batch
 simultaneously across multiple GPU cores, while ensuring that operations such as matrix multipli-
cations and convolutions are parallelized at the level of individual neurons and synaptic weights.
IV. Mapping each neuron of the DNN to a separate GPU core, ensuring that each core is respon-
sible for the computation of a single neuron’s output, thereby fully utilizing the GPU’s parallel
processing capabilities.
(d) In an embedded system, a microcontroller executes the following functions in sequence:

func1() {
int sensorData = readSensor(); // Read sensor data
storeInMemory(sensorData); // Store data in memory
}

func2() {
int data = retrieveFromMemory(); // Retrieve data from memory
int result = processData(data); // Process the data
}

Assume the memory layout of the microcontroller is such that the functions are located at different
memory addresses, but the cache is small and handles only a few functions at a time.

In the given embedded system with functions func1() and func2(), how does spatial locality affect
the execution of these functions?

I. Spatial locality is beneficial because both functions func1() and func2() are likely to be stored
close to each other in memory, improving cache efficiency.
II. Spatial locality is detrimental because func1() and func2() are stored far apart in memory,
leading to frequent cache misses
III. Spatial locality has no effect because each function operates on separate memory addresses
that do not affect the cache.
IV. Spatial locality is irrelevant because the functions are executed sequentially, and cache
performance is determined only by the execution order.

(e) In previous question, suppose func1(), and func2() are optimized to be stored consecutively in
memory (e.g., starting at 0x1000). What is the most likely impact on cache performance?

I. Cache performance will improve due to increased spatial locality, as the functions are now loaded
into the cache in a single operation.
II. Cache performance will degrade because storing functions consecutively does not affect the spa-
tial locality of memory access.
III. Cache performance will remain unchanged, as the cache access pattern is not affected by func-
tion placement.
IV. Cache performance will degrade because consecutive function storage increases cache con-
tention.
(f) Consider an embedded system that processes a stream of sensor data in real-time using a small,
fixed-size cache memory. The system calculates a moving average over the last 10 sensor readings.
The sensor data is stored in an array of size 100, and the moving average is updated for every new
data point that arrives.
[Consider, simple moving average (SMA), which is the unweighted mean of the previous k data-
points ]
 Given: Sensor data array: data[100] Moving average window size (k): 10 Cache size: 20 data points
Assume the system uses direct-mapped caching and the sensor data is accessed sequentially. The
system needs to update the moving average every time a new sensor reading is added to the array.
1) How does the locality of reference impact the cache performance in this scenario?

I. Temporal locality improves cache performance because the same data points are frequently
reused within the moving average calculation.
II. Spatial locality degrades cache performance because the data points accessed sequentially are
spread across the entire array.
III. Temporal locality has no impact on cache performance because each data point is accessed
only once.
IV. Spatial locality improves cache performance because the data points accessed sequentially are
located close to each other in memory.

(g) In the above problem, if the array size is increased to 200 while keeping the cache size fixed at 20,
what is the most likely effect on the cache hit rate and locality of reference?

I. The cache hit rate will increase due to improved temporal locality, as more data points are
reused within the moving average calculation.
II. The cache hit rate will decrease due to reduced temporal locality, as the increased array size
means older data points are less likely to be in the cache.
III. The cache hit rate will remain the same, but spatial locality will improve, as the larger array
allows for better memory organization.
IV. The cache hit rate will decrease due to reduced spatial locality, as the data points in the larger
array are more spread out, leading to fewer consecutive cache hits.

(h) In an embedded system with a custom-designed instruction set, which of the following instruction
set design choices is most likely to optimize performance and reduce power consumption in a
real-time signal processing application?

I. Implementing a large set of complex instructions that can perform multiple operations in a
single instruction, reducing the total number of instructions executed.
II. Using a simple, reduced instruction set with a limited number of basic operations, which
allows for faster instruction execution and easier pipelining, even if it requires more instructions to
complete a task.
III. Including instructions that support dynamic reconfiguration of the hardware, allowing the
processor to adapt its instruction set on-the-fly based on the application requirements.
IV. Designing the instruction set with a focus on maximizing the number of registers available
for computation, thereby reducing the need for memory access and improving overall system
performance.

3. Answer each of the below questions:

(a) You are solving the binary classification task of classifying images as dogs vs. non-dogs. You
design a neural network with a single output neuron. Let the output of this neuron be z. The
final output of your network, ŷ, is given by ŷ = σ(ReLU(z)). You classify all inputs with a final
value ŷ ≥ 0.5 as dog images. What problem are you going to encounter? Justify with a proper
explanation. (4 marks)
(b) Consider a neural network trained on a dataset of 50,000 images. The initial training configuration
uses a batch size of 100 and trains for 50 epochs. The total training time for this configuration is
10 hours.
 If the batch size is increased to 200 while keeping the number of epochs constant at 50, determine
the new total training time, assuming the time per iteration remains constant and scales linearly
with the number of iterations. (2 marks)
(c) Justify the statement from the basic definition of RNN: ”RNN works on variable-length in-
put/output and suffers from high latency overhead. ” (4 marks)