Recent advancements in technology have shown significant improvement in the area of sign language recognition. Though there are sign language recognition applications, there are seldom tutoring systems for sign language. In this paper, we introduce “Sign Buddy” an innovative, user-friendly sign language android app with various features that bridge a significant gap in sign language education to educate and facilitate communication for both the deaf community and those with various linguistic backgrounds. The app provides interactive lessons, tutorials, vocabulary building, quizzes, and assessments to improve sign language skills. For effective learning, it features an inclusive design, a user-friendly interface, real-time detection, and a large video collection. One valuable feature of the app is its bidirectional conversion between sign language and text by utilizing image processing and machine learning algorithms. Sign Buddy integrates technologies like OpenCV, YOLO, and Kotlin, ensuring a user-friendly experience supporting sign language learning and communication. The app’s accessibility is a significant advantage, as it is compatible with most Android devices, making it available to a wide range of users. Additionally, we introduce creating a comprehensive dataset covering all sign language alphabets to enhance the app’s training data. The validation of the YOLOv8 model demonstrates impressive results, with high precision and recall, making Sign Buddy a powerful tool for sign language recognition. A pilot study with 10 users confirms the app’s intuitive interface and real-time learning capability. The System Usability Survey (SUS) reports an average score of 80.5, reflecting high usability and effectiveness.