Among other drawbacks, models trained via federated learning (FL) often struggle to generalize to new, unseen clients – a challenge that becomes more severe in low-data regimes. Most existing solutions address this issue using hypernetworks, which do not scale well with the size of the underlying model. To overcome these limitations, we introduce Federated Personalized Client Embeddings (FedPCE) – a method that uses embeddings to distill personalized knowledge from existing FL approaches. Our results show that FedPCE performs comparably to popular FL algorithms during both training and personalization. Notably, it outperforms competing methods when only limited data is available for personalization—even with as few as 25 labeled samples (Code available at https://github.com/somcogo/fedpce ).