Publications

2025

1. FC-GPU: Feedback Control GPU Scheduling for Real-Time Embedded Systems

   Srinivasan Subramaniyan and Xiaorui Wang

   ACM Transactions on Embedded Computing Systems, Sep 2025

   Abs DOI PDF Code Slides

   

   GPUs have recently been adopted in many real-time embedded systems. However, existing GPU scheduling solutions are mostly open-loop and rely on the estimation of worst-case execution time (WCET). Although adaptive solutions, such as feedback control scheduling, have been previously proposed to handle this challenge for CPU-based real-time tasks, they cannot be directly applied to GPUs, because GPUs have different and more complex architectures and so schedulable utilization bounds cannot apply to GPUs yet. In this article, we propose FC-GPU, the first feedback control GPU scheduling framework for real-time embedded systems. To model the GPU resource contention among tasks, we analytically derive a multi-input–multi-output (MIMO) system model that captures the impacts of task rate adaptation on the response times of different tasks. Building on this model, we design a MIMO controller that dynamically adjusts task rates based on measured response times. Our extensive hardware testbed results on an NVIDIA RTX 3090 GPU and an AMD MI-100 GPU demonstrate that FC-GPU can provide better real-time performance even when the task execution times significantly increase at runtime.

2024

1. Latency-Guaranteed Co-Location of Inference and Training for Reducing Data Center Expenses

   Guoyu Chen, Srinivasan Subramaniyan, and Xiaorui Wang

   In 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS), Sep 2024

   Abs DOI Slides

   

   Today’s data centers often need to run various machine learning (ML) applications with stringent SLO (Service-Level Objective) requirements, such as inference latency. To that end, data centers prefer to 1) over-provision the number of servers used for inference processing and 2) isolate them from other servers that run ML training, despite both use GPUs extensively, to minimize possible competition of computing resources. Those practices result in a low GPU utilization and thus a high capital expense. Hence, if training and inference jobs can be safely co-located on the same GPUs with explicit SLO guarantees, data centers could flexibly run fewer training jobs when an inference burst arrives and run more afterwards to increase GPU utilization, reducing their capital expenses. In this paper, we propose GPUColo, a two-tier co-location solution that provides explicit ML inference SLO guarantees for co-located GPUs. In the outer tier, we exploit GPU spatial sharing to dynamically adjust the percentage of active GPU threads allocated to spatially co-located inference and training processes, so that the inference latency can be guaranteed. Because spatial sharing can introduce considerable overheads and thus cannot be conducted at a fine time granularity, we design an inner tier that puts training jobs into periodic sleep, so that the inference jobs can quickly get more GPU resources for more prompt latency control. Our hardware testbed results show that GPUColo can precisely control the inference latency to the desired SLO, while maximizing the throughput of the training jobs co-located on the same GPUs. Our large-scale simulation with a 57-day real-world data center trace (6500 GPUs) also demonstrates that GPU Colo enables latency-guaranteed inference and training co-location. Consequently, it allows 74.9 % of GPUs to be saved for a much lower capital expense.

2023

1. OptiCPD: Optimization For The Canonical Polyadic Decomposition Algorithm on GPUs

   Srinivasan Subramaniyan and Xiaorui Wang

   In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Sep 2023

   DOI

   
2. Enabling High-Level Design Strategies for High-Throughput and Low-Power NB-LDPC Decoders

   Srinivasan Subramaniyan, Oscar Ferraz, M. R. Ashuthosh, and 6 more authors

   IEEE Design & Test, Sep 2023

   DOI PDF

   

2022

1. MAPPARAT: A Resource Constrained FPGA-Based Accelerator for Sparse-Dense Matrix Multiplication

   M. R. Ashuthosh, Santosh Krishna, Vishvas Sudarshan, and 2 more authors

   In 2022 35th International Conference on VLSI Design and 2022 21st International Conference on Embedded Systems (VLSID), Sep 2022

   Abs DOI PDF

   

   Matrix Multiplication has gained importance due to its wide usage in Deep Neural Networks. The presence of sparsity in matrices needs special considerations to avoid redundancy in computations and memory accesses. Sparsity becomes relevant in the choice of compression format for storage and memory access. In addition to compression format, the choice of the algorithm also influences the performance of the matrix multiplier. The interplay of algorithm and compression formats results in significant variations in several performance parameters such as execution time, memory, and total energy consumed. This paper presents MAPPARAT, a custom FPGA-based hardware accelerator for sparse×dense matrix multiplication. Our analysis show that the choice of the compression format is heavily dependent on sparsity of the input matrices. We present two variants of MAPPARAT based on the algorithm used for sparse×dense matrix multiplication, viz., row-wise and column-wise product. A difference in speedup of 2.5× and a difference in energy consumption by about 2.6× is seen between the two variants. We show that an intelligent choice of algorithm and compression format based on the variations in sparsity, matrix dimensions, and device specifications is necessary for performance acceleration. For identical sparse matrices, a speedup of up to 3.6× is observed, when the dense format is chosen for one of the matrices for sparsity in the range of 30% to 90%. MAPPARAT on a resource-constrained device shows performance efficiency of up to 7 GOPs-per-second-per-watt.

2021

1. A survey on high-throughput non-binary LDPC decoders: ASIC, FPGA, and GPU architectures

   Oscar Ferraz, Srinivasan Subramaniyan, Ramesh Chinthala, and 7 more authors

   IEEE Communications Surveys & Tutorials, Sep 2021

   Abs DOI PDF

   

   Non-binary low-density parity-check (NB-LDPC) codes show higher error-correcting performance than binary low-density parity-check (LDPC) codes when the codeword length is moderate and/or the channel has bursts of errors. The need for high-speed decoders for future digital communications led to the investigation of optimized NB-LDPC decoding algorithms and efficient implementations that target high throughput and low energy consumption levels. We carried out a comprehensive survey of existing NB-LDPC decoding hardware that targets the optimization of these parameters. Even though existing NB-LDPC decoders are optimized with respect to computational complexity and memory requirements, they still lag behind their binary counterparts in terms of throughput, power and area optimization. This study contributes to an overall understanding of the state-of-the-art on application-specific integrated-circuit (ASIC), field-programmable gate array (FPGA) and graphics processing units (GPU) based systems, and highlights the current challenges that still have to be overcome on the path to more efficient NB-LDPC decoder architectures.

2020

1. Pushing the Limits of Energy Efficiency for Non-Binary LDPC Decoders on GPUs and FPGAs

   Srinivasan Subramaniyan, Oscar Ferraz, M. R. Ashuthosh, and 6 more authors

   In 2020 IEEE Workshop on Signal Processing Systems (SiPS), Sep 2020

   Abs DOI PDF

   

   Signal processing hardware designers of Low-Density Parity-Check (LDPC) decoders used in modern optical communications are confronted with the need to perform multi-parametric design space exploration, targeting very high throughput (hundreds of Mbit/s) and low-power systems. This work addresses the needs of current designers of dedicated GF(2m) NB-LDPC decoders that necessitate robust approaches for dealing with the ever-increasing demand for higher BER performance. The constraints pose tremendous pressure on the on-chip design of irregular data structures and micro-circuit implementation for supporting the complex Galois field mathematics and communications of hundreds of check nodes with hundreds of variable node processors. We have developed kernels targeting GPU and FPGA (HLS and its equivalent RTL) descriptions of this class of complex circuits for comparing area, frequency of operation, latency, parallelism and throughput. Exploiting techniques such as using custom bit-widths, pipelining, loop-unrolling, array-partitioning and the replication of compute units, results in considerably faster design cycles and demands less non-recurring engineering effort. We report a throughput performance of 800 Mbps for the FPGA case.

2. Gbit/s Non-Binary LDPC Decoders: High-Throughput using High-Level Specifications

   Oscar Ferraz, Srinivasan Subramaniyan, Guohui Wang, and 3 more authors

   In 2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), Sep 2020

   Abs DOI PDF

   

   It is commonly perceived that an HLS specification targeted for FPGAs cannot provide throughput performance in par with equivalent RTL descriptions. In this work we developed a complex design of a non-binary LDPC decoder, that although hard to generalise, shows that HLS provides sufficient architectural refinement options. They allow attaining performance above CPU- and GPU-based ones and excel at providing a faster design cycle when compared to RTL development.

3. CoIn: Accelerated CNN Co-Inference through data partitioning on heterogeneous devices

   K. Vanishree, Anu George, Srivatsav Gunisetty, and 3 more authors

   In 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Sep 2020

   Abs DOI PDF

   

   In Convolutional Neural Networks (CNN), the need for low inference time per batch is crucial for real-time applications. To improve the inference time, we present a method (CoIn) that benefits from the use of multiple devices that execute simultaneously. Our method achieves the goal of low inference time by partitioning images of a batch on diverse micro-architectures. The strategy for partitioning is based on offline profiling on the target devices. We have validated our partitioning technique on CPUs, GPUs and FPGAs that include memory-constrained devices in which case, a re-partitioning technique is applied. An average speedup of 1.39x and 1.5x is seen with CPU-GPU and CPU-GPU-FPGA co-execution respectively. In comparison with the approach of the state-of-the-art, CoIn has an average speedup of 1.62x across all networks.