This research aims to carve a path in understanding oral history and biographical interviews by utilizing a robust Natural Language Processing (NLP) model. This model is designed to discern and extract topics from the vast array of interview transcripts housed in the Institut für Geschichte und Biographie’s (IGB) archive “Deutsches Gedächtnis” (ADG). The research methodology has been structured meticulously into six distinct stages, underpinning the end-to-end process flow. It begins with extensive data preprocessing to ensure data quality and consistency, followed by the creation of embeddings to convert the data into a numeric form for additional processing. To optimize computational efficiency, it includes the crucial step of dimension reduction. Subsequently, clustering techniques are employed, serving the dual purpose of grouping similar data and introducing structure into the large volume of data. A Large Language Model (LLM) was deployed for labeling the data, enhancing its comprehensibility and ease of retrieval. Lastly, the methodology employs classification to impose a more refined structure on the data. By diving into the depths of the interviewees’ perspectives, beliefs, and experiences, this research draws out the invaluable nuances hidden within the interviews and presents insightful perspectives on various themes or topics discussed therein. As a result, the ADG is transformed into a well-structured, intuitive, and easily navigable resource, thus significantly enriching its value to researchers and making intricate historical narratives easily accessible for exploration and study.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Topic Extraction from Biographical Interviews

  • Shahriyar Babaki,
  • Shital Bagankar,
  • Tanya Goyal,
  • Fatemeh Shahriarizadeh,
  • Kamellia Reshadi,
  • Sina Mehraeen

摘要

This research aims to carve a path in understanding oral history and biographical interviews by utilizing a robust Natural Language Processing (NLP) model. This model is designed to discern and extract topics from the vast array of interview transcripts housed in the Institut für Geschichte und Biographie’s (IGB) archive “Deutsches Gedächtnis” (ADG). The research methodology has been structured meticulously into six distinct stages, underpinning the end-to-end process flow. It begins with extensive data preprocessing to ensure data quality and consistency, followed by the creation of embeddings to convert the data into a numeric form for additional processing. To optimize computational efficiency, it includes the crucial step of dimension reduction. Subsequently, clustering techniques are employed, serving the dual purpose of grouping similar data and introducing structure into the large volume of data. A Large Language Model (LLM) was deployed for labeling the data, enhancing its comprehensibility and ease of retrieval. Lastly, the methodology employs classification to impose a more refined structure on the data. By diving into the depths of the interviewees’ perspectives, beliefs, and experiences, this research draws out the invaluable nuances hidden within the interviews and presents insightful perspectives on various themes or topics discussed therein. As a result, the ADG is transformed into a well-structured, intuitive, and easily navigable resource, thus significantly enriching its value to researchers and making intricate historical narratives easily accessible for exploration and study.