Entity matching (EM) is a critical task for data integration. Existing methods based on Pre-trained Language Models (PLMs) have achieved promising performance with sufficient training samples, but exhibit significant performance drops in few-shot scenarios. To address the issue of poor performance in few-shot scenarios, we propose a novel RoBERTa-based framework called FSEM, which uses multi-loss adversarial training with multi-attention masking. The multi-attention masking module adopts four complementary masking mechanisms to capture multi-level semantic interactions, while the multi-loss adversarial training module, leveraging these multi-attention features, generates diverse adversarial samples by the Fast Gradient Method (FGM) to supplement scarce training data in few-shot scenarios and enriches gradient signals through multi-loss fusion to alleviate gradient sparsity caused by limited annotations. Experiments on nine benchmark datasets demonstrate that FSEM not only significantly outperforms baselines by an average of 14.94% in few-shot scenarios, but also remains competitive when data is sufficient. Furthermore, ablation studies validate the necessity of both modules.