Accurate orbit prediction is crucial for space situational awareness. However, Physics-based approaches can fail to achieve the required accuracy for collision avoidance of Resident Space Objects (RSOs). This paper presents a Machine Learning-based approach for RSOs orbit prediction leveraging Two-Line Element (TLE). Taking the dynamic nature of orbital deviations into consideration, we integrate a dynamic loss function into the orbit prediction framework, allowing for a more adaptive and accurate prediction model. The experiments demonstrate the superior performance of our proposed method in predicting RSOs orbits over extended periods.