Recent studies have highlighted factual inconsistency as a critical challenge in abstractive summarization, with knowledge injection emerging as a key solution. However, how and where to inject knowledge into the model remain open research questions. Inspired by existing work on knowledge storage in Transformer, we propose a factual factors attribution algorithm for the Feed-Forward Networks (FFNs) in the decoder and analyze the distribution of factual neurons in BART, PEGASUS, and LLaMA. We also introduce FactoScalpel, a knowledge injection module that integrates a Knowledge Bank and a router-controlled mechanism into the FFNs. Based on attribution analysis, we inject FactoScalpel into the layers with the densest distribution of factual neurons. We use five factual consistency metrics to compare the performance of BART and PEGASUS with FactoScalpel against four fact-aware summarization models on the XSum dataset. The results demonstrate that FactoScalpel significantly improves factual consistency, achieving state-of-the-art performance in most experiments. Furthermore, we find that knowledge injection method substantially reduces extrinsic errors in the summaries, offering valuable insights for future research. The code of all experiments is available at https://github.com/haohiehie/FactoScalpel