Gpu acceleration for simulating massively parallel many core platforms

Editor's Notes

• #2: Good Morning Everyone, I am today’s first speaker Caroline. The paper I want to present today is GPU Acceleration for Simulating Massively Parallel Many-Core Platforms. And it comes from IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, MAY 2015
• #3: This is the outline: First is introduction, and the related work. After this section I will go through the Simulation Approach The Functional Simulation, Performance Prediction Model The way how to do synchronization in many core And finally is the Experimental Results and conclusion.
• #4: Ok Let’s begin at the introduction and related works
• #5: In this paper, the main question the author try to deal with is how to speed up the simulation of Heterogeneous System Architecture
• #6: With increasing complexity and performance demands of emerging applications, heterogeneous platforms are becoming a popular trend in computer design.
• #7: These Heterogeneous platforms consist of traditional multi-core CPUs in combination with a many-core coprocessor which we usually call GPU composed of thousands or hundreds of embedded cores. Increased use of parallel algorithms creates a market for HSA to manipulate large amounts of data in parallel with high energy efficiency.
• #8: So before the real hardware is available, we need a simulation platforms to help us make meaningful predictions of design alternatives and early software development, as well as to be able to assess the performance of a system. Current state-of-the-art sequential simulators leverage SystemC [11], binary translation [18], smart sampling techniques [12] or tunable abstraction levels [13] for hardware description. However, one of the major limiting factors in utilizing current simulation methodologies is simulation speed, these simulation technologies are not sufficiently well to handle the incoming complexity.
• #9: Most of the existing simulation techniques are slow and have poor scalability, which leads to an unacceptable performance when simulating a large number of cores. Since next generation many-core coprocessors are expected to have thousands of cores, there is a great need to have simulation frameworks that can handle target workloads with large data sets. In addition, these new and scalable simulation solutions must be inexpensive, easily available, with a fast development cycle, and able to provide good tradeoffs between speed and accuracy.
• #10: It is easy to notice that simulating a parallel system is an inherently parallel task. This is because individual processor simulation can be independently proceed until the point where communication or synchronization with other processors is required. Parallel simulators have been proposed in the past [14], [15], [16], which leverage the availability of multiple physical processing nodes to increase the simulation rate. However, this requirement may turn out to be too costly if server clusters or computing farms are adopted as a target to run the many-core coprocessor simulations.
• #11: The development of computer technology has recently led to an great performance increase of General- Purpose Graphical Processing Units (GPGPU). Modern GPGPUs integrate hundreds of processors on the same device, communicating through low-latency and high bandwidth on-chip networks. This allows us to reduce inter-processor communication costs by orders of magnitude with respect to server clusters.
• #12: Moreover, scalable computation power and flexibility is delivered at a rather low cost by commodity GPU hardware. Besides hardware performance improvement, the programmability of GPUs also has been significantly increased in the last 5 years [17], [19]. So this scenario motivated author’s idea of developing a novel parallel simulation technology that leverages the computational power of widely-available and low-cost GPUs.
• #13: The main novelty of this simulation methodology is to use heterogeneous system as a host platform to tackle the challenge of simulating the future heterogeneous architectures
• #14: Specifically, this paper make the following contributions: Present an approach to build the full system simulation frameworks for HSA, and present the idea to partition simulation workloads between CPU and GPU. Discuss the code optimization techniques to maximize concurrency and gain maximum benefit from parallel GPU hardware. Provide a cycle-approximate model for fast performance prediction of the overall simulated platform. Uses a relaxed-synchronization technique to minimizing synchronization overhead and achieving speedup.
• #15: In next section, we provide architectural details of the heterogeneous platform targeted by our simulation technique. And give an overview of the simulation flow for the full system simulator.
• #16: Our target architecture is representative of future heterogeneous platforms. It consists of a general purpose processor connected with a many-core coprocessor. While currently available many-core coprocessors only integrate up to hundreds of cores interconnected via a network-on-chip (NoC) [5], [6], [8], in the near future the number of cores is likely to increase to thousands [22].
• #17: The target coprocessor features thousands of simple ARM cores. As shown in this figure, each core is equipped with data and instructions scratchpad memories (SPM), private caches, private and distributed shared memory of target (TDSM). Caches are private to each core, means they only deal with data or instructions allocated in the private memory.
• #18: The cores are interconnected via an on-chip network organized as a rectangular mesh. The architecture of a core in the coprocessor is based on a simple single issue, in-order pipeline.
• #19: The applications and program binaries targeted for this system are launched from an Linux OS running on the general-purpose CPU. The execution of a parallel application is divided between the two entities, the CPU host and the GPU coprocessor. CPU runs the sequential part of target application up to a point where a computation intensive and highly-parallel program region is encountered. When a parallel program region is encountered, this particular part of the program is offloaded to the coprocessor to gain benefit from its high performance.
• #20: The considered memory model of coprocessor applies the Partitioned Global Address Space (PGAS) paradigm [20]. Each thread mapped to a single node of target coprocessor has private memory for local data items, and shared memory for globally shared data values. Both private and shared memory is mapped in a common single global address space, which is logically partitioned among a number of threads. In this distributed-memory programming model, each thread maintains its own local memory and has no direct knowledge about another’s. Greater performance is achieved when a thread accesses data locally. Non-local access will generate communication traffic across on-chip interconnect, therefore incur performance overhead.
• #21: To simulate general-purpose CPU, we selected QEMU [18] which is an ARM Versatile Platform Baseboard featuring an ARM v7- based processor and input-output devices. QEMU is a popular, open-source, fast emulation platform based on dynamic binary translation (DBT and provides a set of device models, enabling it to run a variety of unmodified guest operating systems. #在Linux上使用Qemu執行ReactOS作業系統
• #22: Fig. 2 depicts the full-system simulation methodology this paper propose. The simulator consists of two main blocks. First, QEMU emulates the target general purpose processor, capable of executing a Linux OS and file system. Next, our coprocessor simulator uses GPUs for accelerating simulation of its thousands of cores.
• #23: Coprocessor simulator is entirely written using C for CUDA [17] and we map each instance of a simulated node to a single CUDA thread. These CUDA threads are then mapped to streaming processors of GPU by its hardware scheduler and run concurrently in parallel. Each target core model is written using an interpretation-based method to simulate the ARM pipeline. Each simulated core is assigned its own context structure, which represents register file, status flags, program counter, etc.
• #24: The necessary support for data structures are initially allocated from the main (CPU) memory for all the simulated cores. The host program (running on the CPU) initializes these structures, and then copies them to the GPU global device memory. Once the main simulation kernel is offloaded to the GPU, each simulated core repeatedly fetches, decodes and executes instructions from the program binary. Similar to the operation on the hardware. Each core updates its simulated registers file and program counter until program completion.
• #25: As mentioned last section, full system simulation is partitioned between CPU and GPU therefore it is essential to find an efficient way to offload the data parallel part from QEMU to coprocessor simulator.
• #26: As shown in Fig. 3, the target application running on guest kernel space of the target platform (simulated by QEMU) is named QEMU target process. And QEMU process running on host CPU is named QEMU-Hprocess.
• #27: Our coprocessor simulator program written in C and CUDA is designed to execute partly on the host CPU and host GPU. The part of coprocessor simulator program executed on host CPU is named CPHprocess. The other part that runs on host GPU platform is named CP-Gprocess.
• #28: Target application needs to forward requests (data structure and parameters) between QEMU and the coprocessor simulator. Therefore an interface is needed so that the QEMU target process can communicate to the QEMU-Hprocess and finally to the CP-Gprocess. To implement this communication between the processes, we use semihosting [25].
• #29: Semihosting is a technique developed for ARM targets allowing the communication between an application running on the target and a host computer running a debugger. It enables the redirection of all the application’s IO system calls to a debugger interface. We leverage this method to provide a direct communication channel between QEMU target process (QEMU-Tprocess) and QEMU-Hprocess. Next, we used Unix Domain Sockets to transfer data between the QEMU-Hprocess and the CP-Gprocess.
• #30: The CP-Hprocess initially boots as a server and waits for an offloading request from the client. When the QEMU-Hprocess encounters a data parallel function from the QEMU-Tprocess, it transfers the structure of parameters pointed by semihosting call to the server socket. When the CP-Hprocess receives all the necessary data structures (data and code segment), it launches the CUDA kernel as a CP-Gprocess for simulation on GPU platform. The QEMU-Hprocess waits for computation to end from GPU side and releases it.
• #32: The coprocessor simulator comprises many modules that simulate the various components of the target architecture. In particular, the core model is responsible for modeling the computational pipeline; the memory model includes scratchpad memory models, cache models for private memory and distributed shared memory of the target (TDSM) models. The network model handles distributed shared memory operations and allows routing of network packets over the onchip network.
• #33: The entire simulation flow is structured as a single CUDA kernel, the simplified structure is explained in previous Fig. One physical GPU thread is used to simulate one single node, the program is run inside a main loop until all simulated nodes in turn finish their simulation. The core model is executed first, during the instruction fetch phase and while executing memory instructions, the core issues memory requests to the cache model. The cache model is in charge of managing data/ instructions stored in the private memory of each core.
• #34: Although CUDA provides a powerful API with a swift learning curve, designing a parallel simulator for many core running on GPUs is not a straightforward task and implementing a simulator for such a platform imposes several challenges: One of the main challenge is to identify the parallelization in various stages of target node pipeline and map the target nodes on CUDA. The layout of data structure that represent the context of target node has to be carefully organized in GPU memory to utilize high bandwidth GPU global memory and low-latency shared memory (GSM). Then we have to optimize the simulation code to ensure GPU device shared memory (GSM) is free of conflicts. Finally the interaction between CPU and GPU is costly, so we have to minimize the amount of data transfer required between host CPU and GPU platform. The methods implemented to overcome the first three limitation are provided with other supplemental material, so this paper will discuss how to use the relaxed-synchronization method to deal with the fourth challenge.
• #36: A performance model gives an insight about the application primitives that are particularly costly in a certain heterogeneous platform and allows us to predict the runtime behavior of a program in that target platform This paper present a cycle-approximate method to predict the performance of many core coprocessor. Allows designers to make decisions about configurations of architectural parameters and to predict the scalability of their applications.
• #37: In order to quantitatively estimate performance of an application running on target many-core coprocessor, we apply simple fixed, approximated delays to represent the timing behaviors of simulation models. Every core has a local clock variable and this variable is updated after completion of each instruction. Considering a unit-delay model all computational instructions have fixed latency of a single clock cycle.
• #38: Caches have one cycle hit latency and 10-cycle cache miss penalty. Each simulated node thread simulates a local clock which provides a timestamp to every locally generated event, and clock cycles are updated for each event with round trip latency As the simulator executes an application code on a core, it increases its local clock in accordance with the event executed on the code block.
• #39: Each memory access or remote request is initially stamped with the initiator core’s local clock time and is increased by a specific delay as it traverses the architecture’s communication model. When the initiator core finally starts processing the reply, its own local clock is updated to that of the reply. To summarize, the sum of all delays induced by all the device models traversed is added to a core’s local time in case of interaction.
• #40: Communication packets through on-chip network update their latency as they traverse through the system and collect the delay due to congestion and network contention. Single-hop traversal cost on on-chip network is assumed to be one. Finally, the simulator output consists of global clock cycles of many-core processor as well as total instructions per cycles (IPC) calculated for running the parallel application allowing the designers to forecast scalability and performance.
• #41: When simulating a many core coprocessor, synchronization requirements can add significant overhead to the performance efficiency. This section focuses on the efforts to increase the efficiency of required synchronization operations.
• #42: Application programs offloaded to the coprocessor, may contain inter-thread interactions using various synchronization primitives such as barrier and locks. In this case, application threads running on many cores will generate accesses to the shared memory distributed across the simulated many nodes (TDSM), which in turn will result in traffic (remote packets) over the NoC. We call these remote packets s-events. To simulate these s-events, it is important to have a synchronization mechanism to ensure the timing and functional fidelity of the many core simulation maintaining both simulation speed and accuracy.
• #43: Cycle approximate timing simulation assesses the target performance by modeling the node latencies using local clock cycles. Since this is simulated using parallel host GPU hardware threads, node clocks are non-synchronized and run independent of each other at different simulation speeds. To accurately model the timing behavior of the entire system, these simulated local clocks should be synchronized at some point in time to keep the accounting of global target time of coprocessor.
• #44: This is essential to determine the correct round trip latency of NoC packet communication between otherwise unsynchronized nodes. For example, when application threads are mapped on different cores of simulated coprocessor and during an s-event such as spin lock for a shared memory (TDSM) variable. The local clocks of each node needs to know the correct number of iterations that each thread should wait before acquiring the lock.
• #45: From the functional accuracy point of view, synchronization between many nodes is essential to maintain functional simulation accuracy particularly during simulation of s-events. Since only s-events modify the state of the shared memories of the target system, they needs to be simulated in non-decreasing timestamp order so that shared memory (TDSM) accesses with data dependencies are executed in order. S-events with smaller timestamp have potential to modify the state of the system and thereby affect events that happen later.
• #46: Implementing synchronization behavior on GPU requires periodically synchronizing hardware threads on a barrier. Barrier functionality can only be imposed on threads within the same block synchronization among thread blocks is not natively supported by CUDA and GPU hardware. This is achieved by terminating the GPU kernel function call and using costly communication between CPU and GPU (Host-CPU-Barrier). Host-CPU-Barrier requires suspension of execution on GPU, saving the states of all simulated cores in GPU’s global memory and transferring control to host CPU where barrier synchronization is performed. This cause a serious performance, bottleneck and slows down the entire simulation.
• #47: A traditional cycle accurate simulation would force synchronization of each node simulation at every clock cycle, so that the simulated program can execute in synchronized target time. The major drawback of this approach is immense synchronization overhead, as the simulation would stop and synchronize at each clock-tick. To address the problem of high overhead due to Host-CPU-Barrier synchronization and gain significant speedup, we use less frequent synchronization, introducing a technique which we call relaxed-synchronization. In relaxed synchronization, instead of synchronizing at every few clock cycles, we choose to synchronize only upon s-events to minimize the number of synchronization points. The steps taken to achieve fast synchronization on GPU as described below:
• #48: Step 1—Simulation of all nodes are allowed to run freely and cores simulate with high speed until an s-event is detected in coprocessor system. This is to ensure that we achieve fast simulation by allowing the hardware context thread to execute independently without the need to synchronize frequently. All nodes which are simulated independently, continuously checks for the presence of any s-event in the system after execution of each local instruction by polling an s-event flag in GPU device memory. When any of the nodes in the system detects the first occurrence of an s-event, it sets the s-event flag and all other nodes are notified of the presence of an s-event in the system. In this figure node 0 is the first to detect the presence of an s-event at its local clock cycle T0 reads 4. As explained above, at this point it is necessary to synchronize all nodes to maintain the functional and timing fidelity of the simulation.
• #49: Step 2—As explained in previous Section, to avoid data dependency violation we need to make sure that all the nodes should have local clock cycles at least equal to the timestamp of the node generating the s-event. Local clocks Tn of all nodes are collected and Lower Bound on Timestamps (LBTS) is calculated, where LBTS is equal to the clock cycle of the node that notifies the presence of an s-event.
• #50: Step 3—Calculating LBTS requires inter-thread synchronization of GPU CUDA threads. Therefore all nodes suspend their independent simulation and control returns to the host CPU platform to perform Host-CPU-Barrier operation. The LBTS is calculated as 4 in this case.
• #51: Simulation of nodes with local clock lower than LBTS proceed while nodes with timestamp greater than or equal to the lower bound wait until all local clocks are at least equal to LBTS = 4 as shown in figure. If there are any previous s-events present in the system, Steps 2, 3 and 4 are repeated until all of them are detected and LBTS is set to the minimum value of their timestamp.
• #52: Step 5—The simulation proceeds in lock-step fashion until all detected s-events in the system are retired. Lock-step simulation also helps to ensure that timing fidelity is maintained and we get the expected round trip latency of the packet traveling across NoC. Which will be introduced later
• #53: Step 6—Once the simulation of s-events in the system is complete, normal simulation resumes without barriers until the simulation ends or next s-event is encountered at which point Step 2 is invoked again.
• #54: Next, we explain lock step simulation implemented in NoC simulation. We recall that a single CUDA thread simulates a single node of the NoC, which has a network switch connected to local and neighboring queues as shown in Fig. 1. Local processor and memory queues are bidirectional and switch receives requests and inserts the forwarding packets from/to them independently. However neighboring nodes are connected using two queues—incoming and outgoing. The incoming queue for one switch of the neighbor acts as an outgoing queue for the second one and vice versa.
• #55: Within one single kernel step of lock-step synchronization, every switch queries its incoming packet queues, then selects a single packet from one of its queues and forwards it to the destination outgoing queue. This implies that in a single step, two neighboring switches may be reading and writing from the same packet queue. Therefore additional synchronization is needed where one of the neighbors is trying to insert the packet in the outgoing queue while another one is trying to read the packet from the same queue location (which may result in write-after-read hazard).
• #56: This synchronization is done using a combination of CPU barrier synchronization and a lock-free fast barrier synchronization [37] (see Fig. 5). A single step between two CPU Barrier Synchronization points is further divided into a read and writes cycles. First, in a read cycle all switches poll their incoming queue for requests and read a packet to be serviced. This is followed by a lock-free fast barrier synchronization [37] to ensure that all threads have finished reading their incoming queues, before writing the selected packet into their outgoing queues. Finally, the following CPU barrier synchronization ensures that packets written to all queues in the write cycle.
• #57: In this experiment section we first present our experimental set up and benchmarks. And show performance results for benchmarks running on top of our simulator. Finally, we show the evaluation of scalability of our simulator and a detailed comparison of its performance with respect to other state-of-the-art commercial platform simulation approaches.
• #58: For evaluating the simulation methodology, they carefully selected our target architecture and benchmarks for simulation. As a target architecture for heterogeneous platforms, they decided to simulate an ARM versatile baseboard with dual core ARM1136 CPU connected with a many-core programmable coprocessor. The architecture they are targeting for target coprocessor has thousands of very simple in-order cores. And they use simple RISC32 cores (based on ARM instruction set architecture [25]).
• #59: Since they model various different system components in the coprocessor simulator, it is important to understand the cost of modeling each component on performance of simulator. Therefore they conducted the experiments with two different architectures.
• #60: The complete list of architectural features is provided in this Table.
• #61: We characterize our simulator’s performance using a metric called S-MIPS. S-MIPS presents the rate at which the host platform simulates target instructions. We define S-MIPS as follows:
• #62: For all experiments, we used a NVIDIA C2070 Tesla graphic card (the Device), equipped with 6 GB memory and 448 CUDA cores. The QEMU ARM emulator runs a Linux kernel image compiled for the Versatile platform with EABI support. As a host CPU platform, we used an Intel Xeon 2.67 GHz multi-core system running Linux 2.6.32. We vary the number of processors in the simulated many-core from 128 to 4,096. This allows us to explore the performance of our simulator when modeling both current and future many-core designs.
• #63: The benchmarks we have selected aim at evaluating the scalability of target workload on many-cores, the design alternatives for target architecture with CPU coprocessor scheme the efficacy of simulator implementation on GPU. We measure the impact on four most important factors:
• #64: In Table 2, we list the benchmarks adopted for our experiments with their data set size which are scaled up so that they can be parallelized for as many as four thousands cores. Each of these benchmarks is either representative of a category of applications widely used in the many-core domain Or contains specific computation or memory patterns frequently found in highly parallel applications The first four benchmarks are extracted from a JPEG decoder and from the OpenMP Source Code Repository [30] benchmark suite and exhibits high degree of data parallelism. The EPCC benchmark is taken from the well-known OpenMP Microbenchmarks Suite [31], which contains large data parallel phases interspersed with several implicit synchronizations.
• #65: This figure presents the instruction profile showing percentage of different mix of instructions in each of these benchmarks. The percentage of instructions is referred to the application portion that is running on the target coprocessor. We can see that for EPCC benchmark, the fraction of synchronization instructions represents a small percentage of the total. The dataset size changes between benchmarks. In particular, MM and NCC have larger datasets, which implies a longer duration of their overall execution time compared to the other benchmarks. FFT was chosen as a representative of task (MIMD) parallelism. Indeed, in this benchmark, threads with an odd ID perform different computation than threads with an even ID. These threads, when mapped to the simulation cores, create control flow divergence.
• #66: Finally, synchronization is an important feature for any shared memory programming model and it is important to measure the overhead of using synchronization primitives in a given workload. Therefore, we selected a worst case scenario and used a barrier synchronization benchmark, described in [33] and [34]. This benchmark consists of a sequence of data-parallel tasks (or algorithmic phases) and a final barrier synchronization, which makes threads wait for each other at the end of parallel region. We consider three different implementations of barrier algorithm to show that our simulator can precisely capture the performance impact of software and architectural design implementations.
• #67: In the first implementation of the barrier benchmark (BC), a centralized shared barrier is used. It uses shared entry and exit counters atomically updated through lock-protected write operations. Implementation of synchronization primitives is done using spinlocks and polling a shared variable which is stored in one single segment of the distributed shared memory of the target (TDSM). Threads busy-waiting for barrier to complete are constantly sending memory packets with high latency towards a single node. Therefore the number of synchronization instructions increases with the increasing number of simulated cores, creating increasingly high traffic towards a single tile in the network.
• #68: In this barrier algorithm BMSD, we work around the contention problem by designating a master core, responsible for collecting notifications from other cores (the slaves). Each slave notifies its presence on the barrier to the master on a separate location of an array stored in DSM portion local to the master. The master core polls on this array until it receives the notification from all the slaves. Distributing the polling location for slaves local to their DSM segment, greatly reduces the traffic on network due to busy-waiting.
• #69: This algorithm is similar to B-MSD, but further improves performance by using a tree based multi-stage synchronization mechanism (BMSD-T) where cores are organized in clusters. In this case, the first core of each sub-cluster is master to all slave cores in that sub-cluster. When all slaves in a sub-cluster reach the barrier, they trigger top level synchronization between local (cluster) masters [34]. The tree-based implementation (BMSD-T) is better suited for a large number of processors, And it is expected to further mitigate the effect of barrier synchronization.
• #70: In this section, we present the results related to the performance prediction capabilities of our many-core coprocessor simulator. Application performance depends upon a large number of parameters. such as parallel programming model, number of loop iterations per thread, chunk size and software implementation of synchronization directives In addition to this, it also depends upon architectural features such as cache design, memory locality and communication cost due to network delay. Therefore, with this set of experiments we assess how each of these characteristics affects application performance.
• #71: We report average instructions per cycles for each benchmark in Fig. 7. The first five benchmarks have high data parallelism and therefore show increasing IPC of up to 2,000 with increasing number of cores. EP and EPCC benchmark have a small percentage of synchronization instructions, so they still benefit from the rest of the available data parallel section. BC is our worst case scenario and maximum IPC it achieves is around 350, for the case when coprocessor is simulated for 1,024 cores. This is due to heavy traffic coming from synchronization directives, which creates a bottleneck in the NoC and imposes sequential execution. B-MSD and BMSD-T are more suitable implementations for the assumed memory model and hence show better results compared to BC. As expected, due to multi-stage implementation of Master-Slave algorithm, the simulation cycles are further reduced in BMSD-T as compared to BMSD, and therefore show higher instructions per cycles.
• #72: In Figs. 9a and 9b, we show scalability results for up to 4,096 cores. The application speedup is calculated by dividing the cycle counts of a parallelized benchmark running on N cores by the cycle count of the same benchmark running on a single core of the coprocessor. Results for data parallel kernels are shown in Fig. 9a and barrier algorithms are shown in Fig. 9b. All the benchmarks achieve good scalability due to high parallelism. FFT features data dependent conditional execution and EPCC includes synchronization primitives. Due to this, the inherent parallelism of EPCC and FFT is lower than that of the other benchmarks and consequently we see a slight decrease in application scalability when parallelizing for 4,096 cores. In Fig. 9b, as we expect, BC does not scale, showing the effect of heavy network congestion due to contention for centralized barrier counters. B-MSD mitigates the effect of synchronization and shows significantly better results where the application speed up increases to 1,000. Employing a tree-based algorithm in BMSD-T further removes the traffic due to busy-waiting, and therefore shows better speedup (3,300) than BMSD with increasing number of cores.
• #73: Fig. 10a shows S-MIPS for increasing core count of the simulated accelerator for Architecture I. We can notice that the performance is as high as 600 S-MIPS when simulated for 4K cores for the MM benchmark with biggest dataset sizes. The same experiment for Architecture II is shown in in Fig. 10b. The performance is close to 50 SMIPS. The performance of our simulator is directly related to the level of parallelism available in the coprocessor workload. High level of data parallelism in application implies longer simulation of workload in parallel, thus benefiting from GPU hardware parallelism and therefore results in better scalability and performance. Therefore, MM and NCC benchmarks with the largest parallel datasets benefits the most and show highest performance compared to rest other benchmarks. For other benchmarks, due to the small workload size on coprocessor, the overhead associated with semihosting and parallelization directives wins over the benefits of parallelism on GPU. In addition to this, as explained in Section 4, task parallel workload has a performance impact on architecture simulation due to the serialization of execution when control flow divergence occurs (during the execution phase of core pipeline simulation), which is visible in the simulation performance of the FFT benchmark in both Fig. 10a and Fig. 10b.
• #74: In Fig. 10b, we see that the increase in S-MIPS stagnate between 1,024 and 2,048 cores. This happens because when simulating Architecture 2, due to a very large data structure of NoC, Caches etc, we exhaust the available shared memory resource on GPU device (GSM) and GPU scheduler can only launch a limited number of total CUDA thread blocks per multiprocessor, which can simulate up to 1792 cores concurrently. Simulation of rest of the cores wait until the first batch of simulation finishes before launching the next batch of core simulation. Due to this serialization, we see no gain in performance between 1,024 and 2,048 cores, however when simulating 4,096 cores, we again see increase in performance due to the fact now a very high number of cores are simulating in parallel, although in three batches of 1,792 cores simultaneously.
• #75: Fig. 10c shows our implementation of barrier benchmarks. BC being a worst-case scenario incurs a huge performance overhead due to slow CPU-GPU communication. The impact of synchronization overhead is visible in Architecture II, where each application synchronization requires frequent CPU-GPU barrier synchronization to faithfully simulate on-chip network communication and synchronize the cycles counts of the performance model. B-MSD and BMSD-T show significant improvement in performance. The cost of synchronization is visible beyond the simulation of 1,024 cores, but as we expect, BMSD-T shows the best performance among all three implementations.
• #76: Fig. 8 gives further insight into our heterogeneous simulator scalability. It shows the breakdown of total simulation time for different benchmarks. We evaluate the amount of time spent on the QEMU, the time spent on many-core simulation, overhead of semihosting required for the communication between the two and time spent in CPU-GPU communication. These results refer to a 1,024-core instance of many-core coprocessor. Since the MM benchmark has the largest dataset, the time spent on the GPU is highest, whereas for BC, B-MSD, BMSD-T, most of the time is spent in the CPU-GPU communication (due to synchronization). For most cases, we can see that semihosting time is a small fraction of total execution time.
• #77: In this section we compare our simulation methodology with dynamic binary translation. Single core simulation on a powerful CPU using DBT is likely to outperform our interpretation-based simulation approach on the GPU. However, we can expect that the high number of streaming processors available on a single graphics card would allow our simulation method to deliver significant speedup benefits when simulating thousands of cores on coprocessor.
• #78: To the best of our knowledge, none of the currently available simulators can simulate thousands of ARM cores along with caches and interconnect similar to the full system architecture of our target coprocessor. As a term of comparison, we selected OVPSim [35], which is a famous commercial state-of-the-art simulation platform able to model architectures composed of thousands of ARM-based cores. OVPSim is a sequential simulator where each core takes turn after certain number of instructions, however it exploits the benefits of Just in Time Code Morphing and translation caching system to accelerate the simulation. The host platform used for running OVPSim is the same we use for our QEMU-based target CPU simulator. We compare the performance of OVPSim against Architecture I, which most closely matches what OVPSim is capable of modeling.
• #79: We conducted two different experiments. First we consider two benchmarks from the OVPSim test suite, Dhrystone and Fibonacci. Unlike our other benchmarks, these two benchmarks are not parallelized and every core on the coprocessor simulator executes the benchmarks entirely. In both benchmarks, OVPSim shows a constant performance with increasing number of simulated core because of its code morphing technology. With these benchmarks, OVPSim needs to invoke its morphing phase just once and exploits the translation caching system to speed up the simulation. Our GPU based simulator, on the other hand, scales well up to 2,048 simulated. Beyond 2,048 cores the achievable throughput only increases very slightly. Due to per-block GPU device shared memory (GSM) requirements on the GPU, we are only able to run at most three blocks per multi-processor at a time. When simulating 4,096 cores we exceed this limit and extra blocks are dynamically scheduled thus impacting the final scalability. The breakeven performance point between our coprocessor simulator and OVPSim is 1,024 cores.
• #80: In the second experiment, we consider our data-parallel benchmarks MM and NCC. We recall here that, with this parallelization approach, smaller chunks of data are processed by each core when the core count increases. Results for this test are shown in Figs. 11c and 11d. These graphs show that performance of OVPSim decreases significantly, showing less than 50 S-MIPS when simulating 4,096 cores. On the contrary, our simulator performs equally well in this context (600 MIPS), even for very small workloads. In these scenarios our simulation approach performs better than OVP. The breakeven performance point between coprocessor simulator and OVPSim for data-parallel kernels is close to 512 cores
• #81: Finally is the conclusion
• #82: In this paper, they have presented a novel methodology to use GPU acceleration for architectural simulation of heterogeneous. They have shown in this work how to effectively partition simulation workload between the host machine’s CPU and GPU. Thousands of hardware thread contexts available in GPU hardware make a perfect match for simulation of the simple, single issue, in-order cores of our target many-core coprocessor. Compared to performance results from the OVPSim simulator, this solution demonstrates better scalability when simulating a target platform with increasing number of cores.