Pavement condition monitoring is important for road safety, vehicle durability, and passenger comfort. Inspection methods are often labor-intensive and impractical for large-scale deployment. In this study, we propose a lightweight, edge-friendly model for road surface classification using smartphone accelerometer data. Our model is a dual-branch quantized neural network (QNN) that combines raw vibration signals with engineered statistical features to capture both temporal and structural characteristics of the data. The model is optimized with Bayesian hyperparameter tuning and validated using 5-fold cross-validation. We achieve on AsphaltPavementType dataset a macro-averaged F1-score of 93.9% on three different classes flexible pavement, cobblestone, and dirt, while keeping the model size of 354 KB, making it well-suited for real-time inference on low-power embedded devices. When compared to state-of-the-art models such as Complexity Invariant Distance-Longest Common Subsequence Similarity, our architecture achieves superior accuracy and efficiency. Additional analysis on class-wise performance confirms strong generalization, especially for the class flexible pavement. Our solution finds a good trade-off between accuracy and memory-constrained deployment, supporting broader adoption of smart infrastructure tools in resource-limited settings.