We present performance results for dense linear algebra using
recent NVIDIA GPUs. Our matrix-matrix multiply routine
(GEMM) runs up to 60% faster than the vendor’s implementation and approaches the peak of hardware capabilities. Our LU,
QR and Cholesky factorizations achieve up to 80–90% of the
peak GEMM rate. Our parallel LU running on two GPUs
achieves up to ~540 Gflop/s. These results are accomplished by
challenging the accepted view of the GPU architecture and programming guidelines. We argue that modern GPUs should be
viewed as multithreaded multicore vector units. We exploit
blocking similarly to vector computers and heterogeneity of the
system by computing both on GPU and CPU. This study includes detailed benchmarking of the GPU memory system that
reveals sizes and latencies of caches and TLB. We present a
couple of algorithmic optimizations aimed at increasing parallelism and regularity in the problem that provide us with slightly
higher performance.
Make this part of your paper trail
Save this paper to a shelf, write a review, and keep your own notes.