In this work, we explore the application of machine learning techniques in archaeology with the goal of automating the identification and classification of images of clans and family symbols on household items found during archaeological excavations. These images are known as “tags”. We propose a novel approach to object recognition and classification based on a data-driven method using two models. The study investigates the possibility of enhancing prediction accuracy by merging data from two algorithms to improve object classification. We conducted tests and experiments to validate the effectiveness of our chosen models, and the results show an accuracy rate of 0.762 for known categories and 0.507 for newly discovered categories. Additionally, we propose a method for using the created dataset to train algorithms for detecting and categorizing objects.