The exponential growth of data has intensified the challenge of meeting labeling demands, driving the need for innovative solutions like domain adaptation and transfer learning. This paper explores these approaches by focusing on the alignment of labeled data from a source domain with unlabeled or semi-labeled data from a target domain. The goal is to develop a model that leverages source domain labels to predict target domain labels effectively. A key aspect of this alignment process is the generation of pseudo-labels for the target domain. Our contribution enhances the accuracy of domain adaptation models through an improved pseudo-labeling procedure. We introduce a novel, low-computation solution for generating pseudo-labels using techniques such as Transfer Component Analysis (TCA), Geodesic Flow Kernel (GFK), and Correlation Alignment (CORAL). Our experimental results demonstrate that this approach yields superior accuracy on datasets like PIE and Office-Decaf compared to traditional pseudo-label generation methods, such as K-Nearest Neighbors (KNN). This advancement represents a significant step forward in effectively utilizing domain adaptation techniques amidst the growing volume of data.