This paper surveys recent developments at the intersection of formal language theory and logical theories of arithmetic, focusing particularly on Büchi and Semënov arithmetic. We explore the connections between sets of integers defined by logical formulas and their representation as formal languages, and present some key results concerning the expressive power and algorithmic properties of these arithmetic theories. We also outline several open problems that arise from the current body of literature.