Runtime measurement for deep learning/PyTorch: CPU vs. GPU (torch.cuda.Event)
#ai #machinelearning #pytorch
In this video, we discuss how to accurately measure runtime on CPU vs. GPU. Timing your code is essential for performance tuning. But did you know that GPU operations are asynchronous? That means your timing might be wrong if you're not careful. Let’s break it down. On CPU, you can simply use time.time(). But on GPU, you need to use torch.cuda.Event and synchronize the device.
--------------- GPU Timing ---------------
start_gpu = torch.cuda.Event(enable_timing=True)
end_gpu = torch.cuda.Event(enable_timing=True)
start_gpu.record()
result_gpu = torch.mm(a, b) # Matrix multiplication on GPU
end_gpu.record()
Wait for GPU to finish
torch.cuda.synchronize()
gpu_time = start_gpu.elapsed_time(end_gpu) / 1000 # Convert ms to seconds
print(f"GPU runtime: {gpu_time:.4f} seconds")