As a sensor providing long-range perception and precise measurements, Light Detection and Ranging (LiDAR) can assist the development of smart cities in various aspects, which raises the rising interest in the topic of LiDAR Odometry (LO) research. Despite the popular application of LiDAR Odometry for the localization purpose, existing LiDAR Odometry methods are still limited by the estimation error in the elevation direction due to the low vertical resolution of LiDAR and the increasing storage cost with time. To address these issues, we propose Ground and Memory Optimized LiDAR Odometry (GMLO) for autonomous vehicles by integrating a joint scan-matching optimization between consecutive LiDAR frames. In addition, an efficient and robust ground extraction method, Polar Region Ground Plane Fitting (PR-GPF), is integrated into GMLO. Finally, we propose a novel scan-to-mapping scheme to eliminate accumulated errors with low storage cost and good efficiency in updating historical mapping data. To evaluate the performance of GMLO in various aspects, we have conducted extensive experiments based on the KITTI Odometry and Semantic-KITTI datasets, and the test results show that GMLO is effective with 1.03% average drifting error and over 25 Hz tracking frequency.