This study investigated why social media users are hesitant to use the metaverse, along with their perceived risks and proposed mitigations. We conducted semi-structured interviews with 16 adult social media users who had not used and were not interested in the metaverse. Guided by the Technology Acceptance Model (TAM) and its extended frameworks, a theory-driven thematic analysis was conducted. We constructed five dimensions for social media users’ metaverse acceptance, including perceived usefulness, perceived ease of use, perceived enjoyment, perceived connectedness, and perceived risks. The key influential factors included in each dimension were also revealed, which helped in understanding why people were hesitant to use the metaverse. In addition, we further identified four major perceived risks, encompassing data security, abuse or misuse, overuse, and health issues. Given these perceived risks, a set of corresponding mitigations were uncovered: data security protection, governance and management, access control and restriction, and user education. Based on the findings, we discussed theoretical contributions and practical implications.