Acoustic decoy is one kind of essential equipment in Underwater Acoustic Countermeasures (UACM), primarily serve ship or submarine defense against torpedo threats. Modern torpedoes exhibit trends toward extended range, higher velocity, and enhanced intelligence, significantly improving their counter-countermeasure capabilities. Single acoustic decoy always suffers from short deception durations, failing to achieve sustained decoying against torpedoes, and resulting in reduced countermeasure effectiveness during UACM, when using multiple acoustic decoys as instead, challenges such as communication condition limitations and low coordination efficiency are faced. We propose a multiple acoustic decoys UACM technique based on dual time scale coordinated multi-agent proximal policy optimization (DTSC-MAPPO) algorithm, enabling cooperative decoy operations to enhance defensive efficacy. Firstly, a simulated UACM environment incorporating eight mobile acoustic decoys was established, modeling the behavior logic and maneuverability of decoys, torpedoes, and submarines. Secondly, a multi-agent dual time scale coordinated decision model with acoustic decoys as agents was constructed, namely a centralized decision model for a long time scale and an independent decision model for each single acoustic decoy with continuous time. In communication limited underwater acoustic environment, continuous decisions are made for the eight acoustic decoys respectively. The independent model of the single acoustic decoy makes continuous decisions based on the local observations of themselves and then coordinates with the centralized decision model. Simulation results indicate that the proposed intelligent multiple acoustic decoys decision algorithm facilitates coordinated operations under communication constraints, achieving superior countermeasure effectiveness compared to the independent decisions using local observations.