In few-shot learning for named-entity recognition (NER), each labeled example critically contributes to model performance, making the selection of examples crucial. However, most existing works overlook the importance of exemplar selection, often relying on random sampling to choose examples. This work explores the problem of selecting exemplars for few-shot NER by training numerous models on a randomly sampled set of exemplars and examining how various properties correlate with the resulting F1 scores in six NER datasets. Our findings reveal that 1) the number of sampled sentences can be equally or even more important than the number of entities, and 2) it is critical to sample diverse examples. We propose an analysis-synthesis-based approach to pinpoint the characteristics of an effective sampling strategy and construct it accordingly. This new method effectively reduces variance while maintaining or improving F1 scores. Our experiments demonstrate that strategic exemplar selection in few-shot NER significantly improves F1 scores and reduces variability. The complete code and data are available at https://github.com/qtelnoff/kmle-sampling .