Tamil, a Dravidian language, is renowned for its complex word formations through prefixes and suffixes, intricate script, and detailed grammar. These features pose challenges for computational understanding. However, advancements in Natural Language Processing (NLP) are enhancing this comprehension. Literature reviews indicate that adhering to a universal standard for annotating sentences simplifies the development of NLP models and facilitates comparative linguistic analyses. When these standardized annotations are used in machine learning models, performance is greatly improved. This paper discusses the use of Recurrent Neural Networks (RNN) and Long Short Term Memory (LSTM) for Part-of-Speech (POS) tagging and dependency parsing on the annotated Tamil Treebank. The models achieved an accuracy of 73% for Universal POS (UPOS) tagging, 94% for eXtended POS (XPOS) tagging, and 94% for Dependency Parsing. The results indicate promising improvements for the development of future Tamil-based NLP applications.