hiw3c 日报精选
arXiv cs.AI/CL/LG 网页快照

用于GPU发起通信的快速、可编程GPU至CPU调用层

原文标题 · A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication

作者: Jiakun Yan, Shiksha Patel, David Sidler | 现代GPU工作负载越来越多地依赖设备发起的通信,使GPU内核能够直接通信,而无需在内核边界处与主机同步。现有库通常倾向于将传输控制置于GPU线程上,这种方式称为GPUDirect异步(GDA),并将反向卸载至主机代理线程视为一种较慢的备用方案。我们认为,反向卸载可以超出

来源:arXiv cs.AI/CL/LG · arxiv.org

正文为英文,可一键机器翻译(仅首次需要等待)

Computer Science > Distributed, Parallel, and Cluster Computing

Title: A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication

Abstract: Modern GPU workloads increasingly rely on device-initiated communication to let GPU kernels communicate directly without synchronizing with the host at kernel boundaries. Existing libraries commonly favor placing transport control on GPU threads, an approach known as GPUDirect Async (GDA), and treat reverse offload to host proxy threads as a slower fallback. We argue that reverse offload can outperform GDA and should be a first-class communication path. We further propose exposing reverse offload as a programming interface that lets users define workload-specific communication operations. We present KIWI, a programmable reverse-offload framework that achieves sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. We address practical challenges of scaling reverse offload, including contention on shared queues, polling overhead, and load imbalance across proxies. KIWI-P2P, a SHMEM-like library built on KIWI, outperforms state-of-the-art GDA implementations in both latency and bandwidth. More importantly, KIWI-EP redesigns low-latency mixture-of-experts dispatch and combine kernels around workload-defined operations, lowering expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and time per output token in vLLM by up to 29%. The source code of KIWI is available at this https URL .

Submission history

Access Paper:

Additional Features

Current browse context:

References & Citations

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .