Extractive Question Answering (EQA) in the few-shot learning scenario is one of the most challenging tasks of Machine Reading Comprehension (MRC). Some previous works employ external knowledge for data augmentation to improve the performance of few-shot extractive question answering. However, there are not always available external knowledge or language- and domain-specific NLP tools to deal with external knowledge such as part-of-speech taggers, syntactic parsers, and named-entity recognizers. In this paper, we present a novel Plug-and-Play Data Augmentation Component (PPDAC) for the few-shot extractive question answering, which includes a paraphrase generator and a paraphrase selector. Specifically, we generate multiple paraphrases of the question in the (question, passage, answer) triples using the paraphrase generator and then obtain highly similar statements via paraphrase selector to form more training data for fine-tuning. Extensive experiments on multiple EQA datasets show that our proposed plug-and-play data augmentation component significantly improves question-answering performance, and consistently outperforms state-of-the-art approaches in few-shot settings by a large margin.