#vLLM

GigaTap articles tagged vLLM.

• DFlash on TPUs targets LLM inference’s sequential bottleneck - UCSD researchers report 3.13x average LLM inference speedups on Google TPUs by using block-diffusion speculative decoding instead of one-token-at-a-time dr