We investigate the space of Hermitian metrics on a fixed complex vector bundle. This infinite-dimensional space has appeared in the study of Hermitian-Einstein structures, where a special \(L^2\) -type Riemannian metric is introduced. We compute the metric spray, geodesics and curvature associated to this metric, and show that the exponential map is a diffeomorphism. Though being geodesically complete, the space of Hermitian metrics is metrically incomplete, and its metric completion is proved to be the space of “ \(L^2\) integrable” singular Hermitian metrics. In addition, both the original space and its completion are CAT(0). In the holomorphic case, it turns out that Griffiths seminegative/semipositive singular Hermitian metric is always \(L^2\) integrable in our sense. Also, in the Appendix, the Nash-Moser inverse function theorem is utilized to prove that, for any \(L^2\) metric on the space of smooth sections of a given fiber bundle, the exponential map is always a local diffeomorphism, provided that each fiber is nonpositively curved.