In this paper, a novel universal learning framework, named Fireworks Swarm Learning (FSL), is built to address some critical challenges in the optimization in machine learning. The framework of FSL primarily comprises two core structures that are the FSL Algorithm and FSL Adapter. The FSL Algorithm simulates the swarm behavior to efficiently explore the solution space, while the FSL Adapter primarily interacts with specific learning tasks, acting as an intermediary layer. Concisely, FSL leverages explosive learning and cooperative mechanism to effectively tackle those difficult optimization issues, occurred in machine learning, such as local optima, saddle points as well as complex optimization objectives. The framework of FSL is characterized by high parallelism, low human intervention, strong exploratory capabilities, and adaptability to a diverse range of complex learning tasks. A case study in a reinforcement learning environment demonstrates that the proposed FSL outperforms SOTA algorithms, highlighting its effectiveness and versatility. FSL represents a significant advancement in machine learning in terms of offering a powerful and flexible framework in both solving optimization challenges and pushing the boundaries of the modern machine learning.