Enrique S. Quintana-Orti
Recipient of the Euro-Par Achievement Award 2023
A Continuum of Matrix Multiplications: From Scientific Computing to Deep Learning
Matrix multiplication (GEMM) is a key, pervasive computational kernel that spans across multiple domains. On the one hand, many applications arising in scientific computing require the solution of linear systems of equations, least-square problems, and eigenvalue problems. For portability, these applications often rely on linear algebra routines from LAPACK (linear algebra package). In turn, in order to deliver high performance, LAPACK heavily relies on GEMM and other Basic Linear algebra subroutines (BLAS). On the other hand, to a large extent, the computational cost for the convolutional neural networks (CNNs) that dominate machine learning algorithms for signal processing and computer vision tasks, as well as the transformers behind recent deep learning (DL) applications, such as ChatGPT, is largely determined by the performance of GEMM.
In this talk we will first expose caveats of current instances of GEMM in linear algebra libraries for conventional multicore architectures: suboptimal performance and missing support for DL-oriented data types. Starting from that point, we will then demonstrate how these problems can be overcome via tools for the (semi-)automatic generation of the only architecture-specific piece of GEMM, known as micro-kernel, together with an analytical-based model to capture the cache hierarchy configuration. In addition, we will show that this approach carries over to more "exotic" architectures, from high-end vector accelerators and the Xilinx artificial intelligence engine (AIE) to low-power designs such as RISC-V processors and ARM-based (Arduino) micro-controllers.
Enrique S. Quintana-Orti received his bachelor and Ph.D. degrees in computer sciences from the Universitat Politecnica de Valencia (UPV), Spain, in 1992 and 1996, respectively. After 20+ years at the Universitat Jaume I of Castellon, Spain, he came back to UPV in 2019, where he is now Professor in Computer Architecture. For his research, he received the NVIDIA 2008 Professor Partnership Award and two awards from the USA National Space Agency (NASA). He has published 400+ articles in journals and international conferences. Currently he participates in the EU projects APROPOS (approximate computing), RED-SEA (exascale computer networks), eFLOWS4HPC (workflows for HPC and AI) and Nimble AI (neuromorphic chip for sensing & processing). His research interests include parallel programming, linear algebra, energy consumption, transprecision computing and deep learning as well as advanced architectures and hardware accelerators.
Distributed Intelligence in the Computing Continuum
Modern distributed systems also deal with uncertain scenarios, where environments, infrastructures, and applications are widely diverse. In the scope of IoT-Edge-Fog-Cloud computing, leveraging these neuroscience-inspired principles and mechanisms could aid in building more flexible solutions able to generalize over different environments.A captivating set of hypotheses from the field of neuroscience suggests that human and animal brain mechanisms result from few powerful principles. If proved to be accurate, these assumptions could open a deep understanding of the way humans and animals manage to cope with the unpredictability of events and imagination
Schahram Dustdar is a Full Professor of Computer Science at the TU Wien, heading the Research Division of Distributed Systems, Austria. He holds several honorary positions: Univers ity of California (USC) Los Angeles; Monash University in Melbourne, Shanghai University, Macquarie University in Sydney, University Pompeu Fabra, Barcelona, Spain. From Dec 2016 until Jan 2017 he was a Visiting Professor at the University of Sevilla, Spain and from January until June 2017 he was a Visiting Professor at UC Berkeley, USA.
From 1999 – 2007 he worked as the co founder and chief scientist of Caramba Labs Software AG in Vienna (acquired by ProjectNetWorld AG), a venture capital co-funded software company focused on software for collaborative processes in teams. He is co-founder of edorer.com (an EdTech company based in the US) and co founder and chief scientist of Sinoaus.net, a Nanjing, China based R&D organization focusing on IoT and Edge Intelligence.
He serves as Editor-in-Chief of Computing (Springer).Dustdar is recipient of multiple awards: IEEE TCSVC Outstanding Leadership Award (2018), IEEE TCSC Award for Excellence in Scalable Computing (2019), ACM Distinguished Scientist (2009), ACM Distinguished Speaker (2021), IBM Faculty Award (2012). He is an elected member of the Academia Europaea: The Academy of Europe, as well as an IEEE Fellow(2016) and an Asia-Pacific Artificial Intelligence Association (AAIA) Fellow (2021) and the AAIA president (since 2021).