This paper presents a novel artificial intelligence-based system for real-time identification and tracking of railcar movements within a yard, enabling precise asset location and inventory management. The proposed solution addresses the inefficiencies and limitations of current yard management practices, which rely heavily on manual updates provided by the crew or reading RFID tags attached to rail cars. By leveraging low-cost camera hardware and advanced AI techniques, including object detection, segmentation, and optical character recognition, the system accurately detects railcar IDs, identifies car types, and records movement directions at yard entries and exits. Additionally, the system employs a counting model to process videos overlooking critical switch zones, determining the classification tracks and the number of railcars moving through each track. The metadata generated by the edge devices is transmitted via IoT connectivity to the cloud, where a data synchronization algorithm processes the information to produce comprehensive yard maps and summarize complete movements. This concept has been tested and proven as a pilot in one of the railroads in North America, demonstrating its effectiveness in enhancing visibility, improving data accuracy and timeliness, increasing crew and management productivity, boosting asset velocity, and automating administrative tasks. Furthermore, this system can be scaled across multiple yards with limited hardware (cameras, and edge devices) installation, without making infrastructure changes to either rail cars or tracks, making it a viable solution for widespread implementation. The proposed solution offers a modular and scalable approach, making it accessible and adaptable to various yard configurations and requirements.