This study aimed to quantitatively evaluate the risk of soil toxic metals in the Youjiang River Basin to human health. Shallow soil samples (n = 71) were collected and their concentrations of Cd and Pb were measured. The United States Environmental Protection Agency (EPA) risk to health evaluation model was used to quantitatively evaluate the risk of toxic metals to human health in this basin. The results showed that: (1) All non-cancer-causing risk to health indices were below 1, indicating a small non-cancer-causing risk to health in the study area, mainly due to Pb. Among three exposure pathways investigated, ingestion through contamination of the hands was shown present the highest risk. Exposure to non-cancer-causing risk to health in children was shown to exceed that of adults by a factor of 10. (2) The maximum, minimum, and average total cancer-causing risk to health index values for children were  > 10−4, 10−6, and 10−4, respectively. This result indicated the presence of a certain cancer-causing risk to health by soil toxic metals to children in most regions, with significant localized risks. The total cancer-causing risk to health index for adults was between 10−6 and 10−4. Most cancer-causing risk to health occurred though ingestion from contaminated hands. (3) There were higher non-cancer-causing and cancer-causing risk to health indices in the eastern part of the region, whereas those in other regions remained relatively stable. The higher risks to health in the east could be attributed to the production, development, and utilization of bauxite resources.