Representation learning is central to structural bioinformatics, enabling shared embeddings for protein fold recognition, structural similarity search, and function prediction. However, the organization and interpretability of latent representations learned from protein distance or contact maps remain poorly understood. We present a representation-focused comparative analysis of two of our previously developed geometric autoencoders, SuperFoldAE and its contractive variant ConSOLAE, trained on \(C_{\alpha }\) distance maps. To characterize latent representations, we examine latent organization using visualizations, clustering alignment, perturbation sensitivity, resolution studies, and transfer tasks. While both models capture coarse fold structure, their latent spaces differ markedly in geometry and robustness. Representations learned with contractive regularization exhibit smoother, more compact, and more stable latent geometry, with stronger unsupervised alignment to fold structure (ARI = 0.87, NMI = 0.91). These geometric advantages are reflected in downstream behavior, including consistent Top-1 and Top-5 accuracy and support for unsupervised protein length prediction ( \(R^2 = 0.64\) ). Overall, this study provides a concise comparative characterization of latent spaces learned by distance-map autoencoders, offering a geometric perspective that links latent geometry and stability to robust protein fold representations.