Residue contact maps are a useful compressed representation that can be used as constraints for structural modeling, but can also help identify inter-helical binding sites and are hence effective on their own. In this work, we hypothesize that crowdedness around a target residue pair influences whether it is a contact point. We developed two measures of crowdedness in a residue’s 3D neighborhood: bin counts - defined in terms of relative residue distance; and residue contact number for inter-helical TM proteins - the number of residues in a specified relative distance. Since unsupervised language models such as MSA transformer, trained on millions of sequences, are very accurate but also complementary to our approach, we combined MSA transformer score with our proposed features to assess the impact of crowdedness on residue contact prediction. We found that crowdedness measures can in fact increase the upper bound performance by at least 7.65% average precision in cross validation experiments and by at least 11.59% average precision in held out experiments. Further, we developed a method to “transfer” this information when ground truth crowdedness measures are unavailable. Our approach outperformed MSA transformer by at least 1.15% average precision in cross validation experiments and 1.85% average precision in held-out experiments.