Autonomous driving systems require performant and reliable perception, though they only possess limited computational resources, which places a high priority on the efficiency of the underlying algorithms. Radar sensors play an important role in this context, because they provide data in the form of sparse point clouds, which can be stored and processed in a condensed and efficient manner. However, this sparsity is often overlooked in the design of perception algorithms, such as convolutional object detection networks. In this work we investigate how sparse submanifold convolutions can be used to exploit this sparsity to drastically reduce the computational complexity of a CNN-based radar object detector. To this end, we propose an efficient implementation of submanifold convolutions on a vertical vector processor architecture called V \(^2\) PRO, which is emulated on an FPGA board. Benchmarks on the public nuScenes dataset and an internal dataset show, that the sparse models provide competitive detection performance, while achieving average speedups of up to 27x over their dense counterparts on the considered vector processor. Finally, the sparse model deployed on the FPGA is integrated into a measurement vehicle with three front-facing high-resolution radars, to demonstrate real-time online radar object detection running at 15 Hz.