Exemplar-Free Class Incremental Learning (EFCIL) is a specialized form of Class Incremental Learning (CIL), where a model sequentially learns new classes without storing past data. As a subset of Continual Learning (CL), EFCIL presents greater challenges due to its heightened susceptibility to Catastrophic Forgetting (CF), stability-plasticity tradeoff, and feature drift. A recent trend in CIL for pre-trained models involves freezing the feature extractor after the first task and incrementally learning the classifier, attracting considerable attention. To address the research gap of the prototypical network for CIL leveraging new molecular class prototypes that can be generated using a frozen feature extractor, we propose a novel Multi-Task Learner (MTL) using Elastic Weight Consolidation (EWC) and Feature Covariance Aware Metric (FECAM) in ChemBERTa model named MTL-FECAM. The proposed framework uses the Mahalanobis metric for CIL setting for three molecular datasets BBBP, bitter, and sweet. The empirical analysis showcases that modeling feature covariance relationships outperform previous methods that sample features from normal distributions train the MTL and balance stability-plasticity tradeoff and minimum CF (Accuracy - 90.89%, 92.62%, 91.95% and Forgetting Measure (FM) - 0.27, 0.0, 0.0 on BBBP, Bitter, and Sweet datasets respectively). The proposed framework is adaptable to various molecular datasets for both Few-Shot CIL (FSCIL) and Many-Shot CIL (MSCIL), setting it apart from existing approaches. Remarkably, MTL-FECAM achieves state-of-the-art results on multiple standard CL benchmarks. Code link : https://github.com/sran07/ MTLFECAM.