hrvatski jezikClear Cookie - decide language by browser settings

Batched matrix operations on distributed GPUs with application in theoretical physics

Mijić, Nenad; Davidović, Davor (2022) Batched matrix operations on distributed GPUs with application in theoretical physics. In: 2022 45th Jubiilee International Convention on Information, Communication and Electronic Technology (MIPRO). pp. 293-299 .

PDF - Accepted Version - article
Download (318kB) | Preview


One of the most important and commonly used operations in many linear algebra functions is matrix-matrix multiplication (GEMM), which is also a key component in obtaining high performance of many scientific codes. It is a computationally intensive function requiring O(n3) operations, and its high computational intensity makes it well-suited to be significantly accelerated with GPUs. Today, many research problems require solving a very large number of relatively small GEMM operations that cannot utilise the entire GPU. To overcome this bottleneck, special functions have been developed that pack several GEMM operations into one and then compute them simultaneously on a GPU, which is called a batch operation. In this research work, we have proposed a different approach based on linking multiple GEMM operations to Message Passing Interface (MPI) processes and then binding multiple MPI processes to a single GPU. To increase GPU utilisation, more MPI processes (i.e. GEMM operations) are added. We implement and test this approach in the field of theoretical physics to compute entanglement properties through simulated annealing Monte Carlo simulation of quantum spin chains. For the specific use case, we were able to simulate a much larger spin system and achieve a speedup of up to 35× compared to the parallel CPU-only version.

Item Type: Conference or workshop item published in conference proceedings (UNSPECIFIED)
Uncontrolled Keywords: matrix multiplication ; batched operations ; GPU ; MPI ; HPC
Subjects: NATURAL SCIENCES > Mathematics
NATURAL SCIENCES > Mathematics > Algebra
TECHNICAL SCIENCES > Computing > Process Computing
Divisions: Center for Informatics and Computing
Project titleProject leaderProject codeProject type
Skalabilni algoritmi visokih performansi za buduće heterogene distribuirane računalne sustaveDavidović, DavorUIP-2020-02-4559HRZZ
Napredne metode i tehnologije u znanosti o podatcima i kooperativnim sustavima - IJ za znanost o podatcimaLončarić, SvenKK.
Depositing User: Davor Davidović
Date Deposited: 18 Aug 2022 09:23
DOI: 10.23919/mipro55190.2022.9803591

Actions (login required)

View Item View Item


Downloads per month over past year

Increase Font
Decrease Font
Dyslexic Font