Online social networks have become a breeding ground for rumors due to faster dissemination of information to a large scale of users. Safeguarding them from rumors is critical due to potential harmful societal impacts. Using the rumor control strategies for their mitigation involves significant efforts, which can be reduced by knowing the veracity of rumors, as we need not control true rumors. Detecting the rumor veracity involves challenges like complexity in natural language communication, varying forms of rumor content and its sources, and a massive amount of content to be processed for detection. Many machines and deep learning-based models are proposed to address rumor veracity detection problems. However, these models deal with language subtleties, dependency on the context of rumor, and user purpose, further complicating veracity detection. Also, rumors keep evolving with new versions requiring continuous updates to veracity detection models. Lastly, the limited availability of labeled datasets for training models further increases the complication in veracity detection. This paper explores various machine and deep learning-based solutions proposed for rumor veracity detection tasks. We provide a brief review of proposed approaches and potential issues and challenges that can be addressed using multi-disciplinary approaches capable of integrating linguistic, social, and technical insights to improve the reliability and scalability of rumor veracity detection models.