Event recognition, a process facilitated by the use of photos or videos, can pertain to events of either a temporal or spatial nature. This field has gained popularity with the rise of deep learning technologies. However, learning from the vast quantities of video data, especially accident-related content from various demographics, poses challenges due to the distinct spatial characteristics inherent to different demographic groups. Additionally, the need for extensive data annotation complicates the process of fully supervised event detection from videos. To overcome these hurdles, we introduce a framework for developing an initial foundation model, referred to as LERM (large event recognition model), a concept not thoroughly explored in existing federated learning literature. This framework addresses the challenges of data and model heterogeneity, which complicates the implementation and maintenance of federated learning systems. In our research, we focus on the recognition of road accidents, situations where video frames are typically sparse. Marking a novel approach in the field, we for the first time integrate federated deep learning, multi-instance learning, knowledge distillation, and event recognition to tackle the aforementioned challenges comprehensively. Due to the absence of a dataset specifically designed for federated learning applications, we adapt our methodology to a composite of multiple road accident datasets. Furthermore, our method can be applied to train any extensive event recognition foundation models, as we have adapted EfficientNet by integrating a transformer block, making it more appropriate for future research.