Analyzing Text the Tidy Way
摘要
The rapidly expanding field of cultural analytics has necessitated the development of robust methods for analyzing large volumes of textual data. Central to this endeavor is transforming text, a fundamentally human medium, into quantitative data that machines can process. This chapter details this transformation by applying the tidy text format, simplifying complex text into manageable tokens. Techniques such as sentiment analysis and term frequency-inverse document frequency (TF-IDF) probe deeper into the language, revealing patterns that indicate overall sentiment and word significance respective to distinct contexts. The chapter culminates in exploring topic modeling, specifically latent Dirichlet allocation (LDA), to discern thematic structures within a corpus of text by Mary Shelley, F. Scott Fitzgerald, and Charlotte Perkins Gilman. This analytical process highlights the power of tidy text in unveiling cultural and linguistic patterns and the potential of computational methods in enriching our understanding of cultural phenomena through text.