Parameter-efficient transfer learning (PETL) has emerged as a powerful approach for adapting pre-trained models to downstream tasks while minimizing trainable parameters and computational costs. However, existing PETL methods often alter the internal structures of pre-trained models, leading to high memory usage due to intermediate feature storage during backpropagation. In this paper, we introduce a novel PETL framework that generates task-specific learning queries for pre-trained knowledge utilization, enabling efficient adaptation without extensive resource demands. The TSQG module synthesizes these queries independently of the backbone, selectively extracting informative features from intermediate representations with minimal memory overhead. To further boost performance, the extracted features are refined through a Multi-Scale Feature Enhancement module. Unlike conventional PETL approaches that rely on extensive feature propagation, our architecture achieves substantial memory savings without compromising accuracy. We validate our approach in medical image classification tasks, and extensive experiments across multiple datasets demonstrate that our method achieves state-of-the-art results with memory efficiency, making it well-suited for real-world scenarios. Our findings highlight the practicality of lightweight PETL solutions, offering a scalable and resource-efficient alternative for deploying pre-trained models in medical imaging domain.