Concurrency and Computation

Marowka, Ami

doi: 10.1002/cpe.7868pmid: N/A

The rise in the demand for new performance portability frameworks for heterogeneous computing systems has brought with it a number of proposals of workable metrics for evaluating the performance portability of applications. The aim of this article is twofold. First, we analyze the underlying principles of the criteria and definition of the revised Ⴔ metric and show that these principles are partially correct and suffer from a lack of a solid and clear performance portability model. We prove mathematically and demonstrate it practically that the principles are only correct for the architectural efficiency approach based on throughputs but are incorrect for the popular application efficiency approach based on run‐times. Second, we are examining whether the Ⴔ and Ⴔ‾ metrics meet the requirements of consistency, proportionality, and lossless information. We use examples from the scientific literature to show the reader that the Ⴔ metric loses information while the Ⴔ‾ metric and other metrics do not lose information.

Wilhelm, Martin; Geppert, Hanna; Drewes, Anna; Pionteck, Thilo

doi: 10.1002/cpe.7909pmid: N/A

We introduce a new model for the task mapping problem to aid in the systematic design of algorithms for heterogeneous systems including, but not limited to, CPUs, GPUs, and FPGAs. A special focus is set on the communication between the devices, its influence on parallel execution, as well as on device‐specific differences regarding parallelizability and streamability. We give a comprehensive description on how a given task mapping can be abstractly evaluated including mappings to dataflow‐based hardware accelerators. We show how this model can be utilized in different system design phases and present two novel mixed‐integer linear programs to demonstrate the usage of the model, showing significant improvements compared to pure CPU mapping for randomly generated task graphs. To the best of our knowledge, we present the first ILP for task mapping that considers pipelining effects when streaming tasks on an FPGA.

Zhao, Jisheng; Bertoni, Colleen; Young, Jeffrey; Harms, Kevin; Sarkar, Vivek; Videau, Brice

doi: 10.1002/cpe.7866pmid: N/A

While heterogeneous computing has emerged as a dominant trend in current and future High‐Performance Computing (HPC) systems, it is also widely recognized that this shift has led to increased software complexity due to a proliferation of programming systems for different heterogeneous processors. One such example is the Heterogeneous‐Compute Interface for Portability from AMD (HIP ), which is composed of a C Runtime API and C++ Kernel Language. Many HPC applications will likely use HIP on future exascale systems (e.g., Frontier and El Capitan), but HIP currently only targets AMD and NVIDIA processors. This limitation creates challenges for users who would also like to run their applications on exascale systems based on other architectures (e.g., Aurora, which is based on Intel hardware) that are currently not targeted by HIP . In this paper, we introduce the design and implementation of HIPLZ , a compiler and runtime system that uses the Intel Level Zero API to support HIP on Intel GPU architectures. We discuss the design of HIPLZ , derived from HIPCL (an implementation of HIP on top of OpenCL ), and portability issues that occur from using the Level Zero runtime as a backend. We evaluate our implementation by running several performance benchmarks and mini‐apps written in HIP on Intel architectures using HIPLZ . Our results show that this approach provides competitive performance relative to Intel's OpenCL implementations on Intel Gen9 and UHD Graphics 770 GPUs, while providing good coverage of features needed by HPC applications. Overall, this approach is a promising demonstration of enabling performance portability for exascale systems.

Faverge, Mathieu; Furmento, Nathalie; Guermouche, Abdou; Lucas, Gwenolé; Namyst, Raymond; Thibault, Samuel; Wacrenier, Pierre‐André

doi: 10.1002/cpe.7811pmid: N/A

Task‐based systems have become popular due to their ability to utilize the computational power of complex heterogeneous systems. A typical programming model used is the Sequential Task Flow (STF) model, which unfortunately only supports static task graphs. This can result in submission overhead and a static task graph that is not well‐suited for execution on heterogeneous systems. A common approach is to find a balance between the granularity needed for accelerator devices and the granularity required by CPU cores to achieve optimal performance. To address these issues, we have extended the STF model in the StarPU runtime system by introducing the concept of hierarchical tasks. This allows for a more dynamic task graph and, when combined with an automatic data manager, it is possible to adjust granularity at runtime to best match the targeted computing resource. That data manager makes it possible to switch between various data layout without programmer input and allows us to enforce the correctness of the DAG as hierarchical tasks alter it during runtime. Additionally, submission overhead is reduced by using large‐grain hierarchical tasks, as the submission process can now be done in parallel. We have shown that the hierarchical task model is correct and have conducted an early evaluation on shared memory heterogeneous systems using the Chameleon dense linear algebra library.

Sadiq, Muhammad Umair; Yousaf, Muhammad Murtaza

doi: 10.1002/cpe.7888pmid: N/A

Sequence alignment is a critical computational problem in various domains, including genomics, proteomics, and natural language processing. The Needleman‐Wunsch (NW) algorithm is a classical dynamic programming approach for finding the optimal global alignment between two sequences. However, its quadratic time and space complexity make it impractical for aligning large‐scale sequences, which are increasingly common in modern applications. In this article, we propose a parallel variation of the NW algorithm that enables scalable global sequence alignment with customizable scoring schemes. Our approach re‐formulates the dependencies in the NW algorithm to enable parallel execution, thereby leveraging the computational power of modern parallel architectures, such as graphics processing unit (GPU). Furthermore, our algorithm supports arbitrary linear scoring schemes, which allows us to use domain‐specific knowledge to improve alignment accuracy. We establish the correctness of our algorithm and evaluate its performance using real DNA and user trajectory sequences on GPUs. Our parallel algorithm has shown impressive results in our experiments, with a peak performance of 27.99 GCUPS (giga cell updates per second) and a maximum speedup of 48.18 times compared to the traditional sequential implementation. Additionally, our algorithm demonstrates remarkable scalability, enabling the alignment of sequences of any length while ensuring balanced work distribution and optimal utilization of resources. Our primary objective is to harness the computational capabilities of a single GPU and fully utilize the processing power of multi‐core CPUs.

Incardona, Pietro; Gupta, Aryaman; Yaskovets, Serhii; Sbalzarini, Ivo F.

doi: 10.1002/cpe.7870pmid: N/A

We present a C++ library for transparent memory and compute abstraction across CPU and GPU architectures. Our library combines generic data structures like vectors, multi‐dimensional arrays, maps, graphs, and sparse grids with basic generic algorithms like arbitrary‐dimensional convolutions, copying, merging, sorting, prefix sum, reductions, neighbor search, and filtering. The memory layout of the data structures is adapted at compile time using C++ tuples with optional memory double‐mapping between host and device and the capability of using memory managed by external libraries with no data copying. We combine this transparent memory layout with generic thread‐parallel algorithms under two alternative common interfaces: a CUDA‐like kernel interface and a lambda‐function interface. We quantify the memory and compute performance and portability of our implementation using micro‐benchmarks, showing that the abstractions introduce negligible performance overhead, and we compare performance against the current state of the art in a real‐world scientific application from computational fluid mechanics.

Kulagina, Svetlana; Meyerhenke, Henning; Benoit, Anne

doi: 10.1002/cpe.7842pmid: N/A

Directed acyclic graphs are commonly used to model scientific workflows, by expressing dependencies between tasks, as well as the resource requirements of the workflow. As a special case, rooted directed trees occur in several applications, for instance in sparse matrix computations. Since typical workflows are modeled by large trees, it is crucial to schedule them efficiently, so that their execution time (or makespan) is minimized. Furthermore, it is usually beneficial to distribute the execution on several compute nodes, hence increasing the available memory, and allowing us to parallelize parts of the execution. To exploit the heterogeneity of modern clusters in this context, we investigate the partitioning and mapping of tree‐shaped workflows on two types of target architecture models: in AM1, each processor can have a different memory size, and in AM2, each processor can also have a different speed (in addition to a different memory size). We design a three‐step heuristic for AM1, which adapts and extends previous work for homogeneous clusters [Gou C, Benoit A, Marchal L. Partitioning tree‐shaped task graphs for distributed platforms with limited memory. IEEE Trans Parallel Dist Syst 2020; 31(7): 1533–1544]. The changes we propose concern the assignment to processors (accounting for the different memory sizes) and the availability of suitable processors when splitting or merging subtrees. For AM2, we extend the heuristic for AM1 with a two‐phase local search approach. Phase A is a swap‐based hill climber, while (the optional) Phase B is inspired by iterated local search. We evaluate our heuristics for AM1 and AM2 with extensive simulations, and we demonstrate that exploiting the heterogeneity in the cluster significantly reduces the makespan, compared to the state of the art for homogeneous processors.

Bilbao, Carlos; Saez, Juan Carlos; Prieto‐Matias, Manuel

doi: 10.1002/cpe.7814pmid: N/A

Asymmetric multicore processors (AMPs) couple high‐performance big cores and power‐efficient small ones, all exposing a shared instruction set architecture to software, but with different microarchitectural features. The energy efficiency benefits of AMPs, together with the general‐purpose nature of the various cores, have led hardware manufacturers to build commercial AMP‐based products, first for the mobile and embedded domains, and more recently, for the desktop market segment, as with the Intel Alder Lake processor family. This trend indicates that AMPs may become a solid and more energy efficient replacement for symmetric multicores in a wide range of application domains. Previous research has demonstrated that the system software can substantially improve scheduling—critical to get the most out of heterogeneous cores—by leveraging hardware facilities that are directly managed by the OS, such as performance monitoring counters, or the recently introduced Intel Thread Director technology. Unfortunately, the OS‐level support enabling access to these hardware facilities may often take a long time to be adopted in operating systems, or may come in forms that make its utilization challenging from specific levels of the system software stack, especially in production systems. To fill this gap, we propose PMCSched, an open‐source framework enabling rapid development and evaluation of custom scheduling‐related support in the Linux kernel. PMCSched greatly simplifies the design and implementation of a wide range of scheduling policies for multicore systems that operate at different system software layers without requiring to patch the kernel. To demonstrate the potential of our framework, we conduct a set of experimental case studies on asymmetry‐aware scheduling for Intel Alder Lake processors.

doi: 10.1002/cpe.7094pmid: N/A

Ilic, Aleksandar; Sousa, Leonel

doi: 10.1002/cpe.7915pmid: N/A