Adapting general-purpose large language models (LLMs) to domain-specific tasks remains challenging, especially under limited computational resources and strict privacy constraints. In-context learning (ICL) offers a lightweight adaptation method but suffers from sensitivity to example selection and limited generalization. A promising strategy is to introduce small auxiliary models that assess task relevance and enhance ICL inputs. However, privacy concerns and non-IID data distributions hinder centralized training of such models. To address these challenges, this paper presents FedBridgeICL, a framework that leverages federated learning (FL) and in-context learning (ICL) to bridge small language models (SLMs) and LLMs, thereby enhancing domain adaptation under privacy constraints. The framework introduces three mechanisms: (1) federated SLM fine-tuning for global feature extraction; (2) embedding SLM predictions with confidence scores into LLM prompts to enable privacy-aware knowledge transfer; (3) dynamic confidence-based reasoning for optimized LLM decisions. Evaluations on the GLUE benchmark demonstrate FedBridgeICL’s consistent outperformance over conventional ICL and centralized baselines in heterogeneous federated settings, confirming its viability for privacy-sensitive deployments.