Transfer learning leverages knowledge gained from previous tasks to accelerate learning in related target tasks. In robotics, and especially in Human-Robot Interaction, this capability is crucial due to the scarcity and high cost of collecting social interaction data. Using transfer learning, a robot can learn new tasks faster and with less data. Successor Representations (SR) have traditionally been used to transfer knowledge between tasks with shared environment dynamics but differing reward functions. In this work, we propose a novel decomposition of SR, Modular Successor Representations (MSR) that facilitates transfer between tasks where only a subset of the environment dynamics changes, while others remain invariant. We evaluate MSR in a multi-agent Social Navigation scenario in simulation and show that it reduces the amount of social data required for training. Finally, we discuss remaining challenges, including scaling to high-dimensional continuous state spaces and handling dynamic social behaviors.