Real-time Traffic Sign Recognition (TSR) on the Raspberry Pi 5 is constrained by limited computational resources. We introduce YOLO-LitePi, an optimized detector that incorporates hardware-aware architectural scaling and is deployed using the NCNN inference engine to reduce end-to-end latency. A two-stage pipeline–class-agnostic detection followed by a lightweight ShuffleNetV2 classifier–is evaluated on the TT100K and VN-Signs datasets. Compared with a YOLOv8n baseline under identical deployment settings, YOLO-LitePi achieves a 24.5%–74.4% improvement in throughput while preserving recognition accuracy suitable for real-time TSR. On the Raspberry Pi 5, the complete pipeline sustains 13.22–16.83 FPS with stable responsiveness across diverse scenes, fully satisfying the latency and reliability requirements of edge driver-assistance applications. We further analyze key engineering choices, including ONNX export, post-processing optimization, and CPU threading, which account for most of the observed latency reduction. These results highlight that backend and runtime optimization can outweigh incremental architectural modifications on resource-constrained devices, offering a practical recipe for deploying TSR on low-cost edge platforms without specialized accelerators.