The properties of silver nanoparticles (Ag NPs) are often affected by particle size, especially when the particle size is less than 10 nm. However, the controllable preparation methods of such small-size Ag NPs reported in the literature, e.g., microemulsion or laser ablation, are often either costly or complicated. In this paper, a chemical reduction method was used to controllably prepare small-size Ag NPs by a simple chemical reaction of AgNO3 and NaBH4 with the protection of polyvinylpyrrolidone (PVP) under mild conditions. Four samples of Ag NPs with target particle diameters of 5.0 nm, 15.0 nm, 25.0 nm, and 35.0 nm were prepared. The actual average particle diameter of obtained Ag NPs was 5.1 nm, 15.7 nm, 24.1 nm, and 36.8 nm respectively, which is a good match to the target diameters. The findings in this research are believed to promote the development of controllable preparation of small-size Ag NPs.