Financial sentiment analysis encounters significant challenges, including domain-specific vocabulary, class imbalance, and the computational demands of advanced models, which create deployment barriers in resource-constrained environments. Parameter-efficient fine-tuning methods, such as LoRA, achieve computational savings but often overlook the integration of domain-specific financial knowledge, limiting their effectiveness in specialized contexts. We propose LAPEFT (Lexicon-Augmented Parameter-Efficient Fine-Tuning), which unifies financial domain lexicons with LoRA adapters via learnable gating mechanisms. This approach enables domain knowledge to directly inform neural parameter updates during fine-tuning, effectively combining lexicon-based and neural methodologies. Evaluation on financial sentiment datasets demonstrates that LAPEFT achieves 88.56% accuracy, outperforming both full fine-tuning (88.18%) and standard LoRA (87.21%), while requiring 95.23% fewer trainable parameters and 35% less GPU memory than conventional approaches. These findings question the perceived trade-off between lexicon-based and neural techniques, demonstrating that domain knowledge integration enhances rather than limits neural architectures. This work enables broader adoption of advanced NLP capabilities across the financial sector.