Purpose <p>Xiaoyao San (XYS) is an ancient clinical dosage form in China. Our previous research discovered that the compound decoction of traditional Chinese medicine presents a mixed-phase system, encompassing nanophase, precipitated phase, suspension phase, and true solution phase. Hence, this study is intended to screen out the effective phases among the four phases of XYS and explore its anti-hyperplasia effect on mammary glands.</p> Methods <p>Four phases of XYS were prepared by centrifugation-dialysis. The preparation process, characterization, determination of chemical composition, and anti-breast hyperplasia effects were investigated.</p> Results <p>The four phases of XYS were successfully separated. The characterization results indicated that the nanophase of XYS had the smallest particle size, polydispersity index (PDI) value, and the largest potential value, with an average particle size of approximately 100&#xa0;nm. In the content determination, the component content ratios of the four phases were consistent, and the content of the nanophase was much higher than that of the other phases. The pharmacodynamic results demonstrated that all indicators of rats in the XYS whole liquid and nanophase groups were significantly improved compared with those in the model group (<i>p</i> &lt; 0.05). At the same time, there were no significant differences in the indicators of rats in the other phase groups. The grey correlation degree analysis revealed that the common peaks of both XYS whole liquid and nanophase were correlated with the pharmacological effect, and the correlation degrees were all greater than 0.7.</p> Conclusion <p>This research successfully proved from physicochemical characterization and pharmacodynamics perspectives that the nanophase is the effective phase and that multiple components jointly exert the anti-hyperplasia effect of mammary glands.</p>