Open-plan offices have gained popularity in recent years; however, a lack of shielding can lead to the leakage of conversation and noise, potentially reducing work efficiency. Improving the acoustic environment is important for the comfort of workers because it enhances the indoor environmental quality of the workplace. Although previous studies have provided insight on the acoustic environment for each type of work, there have been few studies on acoustic environments that assume spaces where multiple types of work are carried out simultaneously. Therefore, this study conducted two experiments to clarify the effects on workers’ intellectual productivity and psychology of sound masking as a method of improving the acoustic environment. In the first experiment, various sound sources were created and evaluated by participants to identify effective masking sounds. It was found that the combination of pink noise, water sound, and classical music resulted in a higher level of satisfaction with the acoustic environment than the HVAC noise. In the second experiment, the selected masking sound was played in a room in which different groups of participants simultaneously engaged in group discussions and individual tasks requiring working memory. The results showed that the number of conversations between participants in the discussion task increased in the masking sound condition and that participants who were sensitive to environmental stimulation tended to feel more relaxed in the masking sound condition. Additionally, individual task performance was retained in the masking sound condition. The findings showed that improving the acoustic environment using sound masking in the workplace may contribute to workers’ comfort.