Computer vision is a driver of many applications, as it provides helpful tools to analyze images. However, there are challenges with respect to privacy, since faces or features of people can be recorded. While face obfuscation helps remove personal identifiers, it induces modifications that can affect the extraction of semantic information with implications for downstream computer vision tasks. In this paper, we aim to quantify the impact of face obfuscation on the extraction of semantic information concerning instances of faces, persons, and other objects. To gain deeper insights, we analyze the effects across different categories and highlight how the spatial arrangement of faces and objects impacts semantic interpretation. Traditional techniques provide high de-identification accuracy, but they reduce face-detection accuracy to below 24%. In contrast, face-preserving approaches improve detection of faces (>90% at IoU thresholds 25–75%), persons, and other objects. Furthermore, we observe that all techniques lead to more accurate object detection when faces do not overlap with objects. Depending on the data, overlaps can lead to a \(\sim \) 40% drop in accuracy for traditional approaches, while there is a significantly smaller gap ( \(\sim \) 15%) for face-preserving techniques. These effects are amplified when objects are semantically or spatially related to persons, such as accessories or cell phones.