<p>Crowd authoring has mainly focused on generalised agent interactions such as collision avoidance and grouping. However, in society, people interact more intentionally with specific “sources” such as exhibits, or inspectors. Uncovering these interactions is essential for understanding and characterising social behaviours. We propose a model that learns from trajectories, the localised agent interactions imposed by the context of the object or agent source. Our model decomposes agent paths into sequential combinations of simple and understandable “core” behaviours, like approach, stop, and circle around, temporally dissecting source-centric trajectories into standardised movements. We train on pairs of trajectory-encoded images and their associated core behaviour combination. Given a set of trajectories around a specific source, our framework can be applied to build a behaviour distribution, summarising how people interact with the source type. The inferred distribution can then be sampled to generate diverse crowds of context-aware agents. We evaluate our model using collected ground-truth data and perform a case study that showcases the utility of this decomposition of context-aware interactions in other tasks, such as measuring behaviour similarity.</p>