With low memory cost and high query speed, hashing has been widely used in unsupervised cross-modal retrieval. Recently, to build efficient hash codes, neural networks are adopted. However, existing deep methods relax the properties of hash codes, which limits the semantic representation ability of hash codes and thus affects the retrieval accuracy. To circumvent this drawback, based on the information theory, this paper points out the properties of ideal hash codes (i.e., uniformity, binarization, balance and uncorrelation). Correspondingly, Information-theoretic Deep Quantification (ITDQ) is proposed, which simultaneously equips the outputs of networks with the semantics of instances and the properties of hash codes. Specifically, a cross-quantification (CQ) head is designed, which adaptively fuses contributions of different modalities to build unified hash codes for multimodal instances. During code construction, CQ head also utilizes the Optimal Transport to directly generate binary codes with balanced bits. To further decouple the bit-wise correlation, the total correlation among each bit is minimized. For semantics, ITDQ aligns the semantic similarity and the hash codes. Extensive experiments demonstrate that ITDQ achieves superior performance.