Federated learning (FL) is a collaborative technique for training large-scale models while protecting user data privacy, but its applications and further development are hindered by the free-riding behavior of participating firms, especially in competitive markets. In particular, to obtain a competitive advantage, a participating firm under FL has an incentive to take advantage of the global model without willingly contributing its information, discouraging information contribution by other firms and making FL formation collapse. We build a parsimonious game theoretical model and explore how free-riding behavior in a competitive market affects FL formation and welfare. Our analyses show several new findings with FL formation as follows: (1) the free-riding behavior does not necessarily discourage competing firms from contributing all its available information; (2) an intensified competition among firms does not necessarily benefit consumers or the social planner; (3) as long as FL is formed, there exists an “All-Win” situation in which all stakeholders (participating firms, consumers, and social planners) benefit; (4) subsidizing by the free-riding firm can align its rival’s incentive to form FL only when the level of competition is intermediate.