Pre-trained Language Model facilitates contextual relation classification by capturing contextual information, addressing word ambiguity, encoding global sentence context, enabling transfer learning, handling out-of-vocabulary words, and improving performance with limited labelled data. Existing pre-training approaches suffer in size, bias, interpretability, generalization, and the lack of domain specificity. To address this, HyPRETo, the hybrid model that combines the strength of token replacement and dynamic masking is proposed to achieve upgraded performance to increase classification accuracy. The Mosquito Vector Biocontrol Agents data is used for implementing the model for a contextual relation classification task. HyPRETo uses ontology to provide structured knowledge. HyPRETo is pre-trained by ELECTRA and fine-tuned by RoBERTa models. Feedforward and softmax activation function is used for classification. The Natural Language Processing technique and SQL database are used to develop an automated question-answering system. The HyPRETo was evaluated with state-of-art models and achieved 98.42% accuracy. As a contribution, the manually annotated input dataset on the mosquito vector control agent is prepared for the classification task. Subsequently, the enhanced model is developed. The interface for an automated question-answering system for mosquito vector biocontrol agents is developed to assist public health applications such as mosquito vector control, disease control, ecosystem management, environmental conservation, and so on.