This paper presents an automated methodology for clustering text entities within salary datasets, in particular employer names. This is a crucial step for data standardization and subsequent robust regression analysis, particularly when investigating employer- and job-specific effects. The approach leverages the power of transformer models (MPNer-base, MiniLM-L6 and RoBERTa-large) to generate semantically rich vectorial representations (embeddings) from diverse text descriptions of employer names in the data. These high-dimensional embeddings then serve as input for subsequent clustering algorithms, which group together similar descriptions (e.g., “University of Guelph” and “Guelph University” for employer). The identified clusters enable the standardization and cleaning of employer. We illustrate the effect of clustering employer data in a regression model framework applied to a public salary disclosure data for Ontario, Canada.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text Standardization in Public Salary Disclosure Data

  • Luiza Antonie,
  • Mohammad Haveledar,
  • Miana Plesca

摘要

This paper presents an automated methodology for clustering text entities within salary datasets, in particular employer names. This is a crucial step for data standardization and subsequent robust regression analysis, particularly when investigating employer- and job-specific effects. The approach leverages the power of transformer models (MPNer-base, MiniLM-L6 and RoBERTa-large) to generate semantically rich vectorial representations (embeddings) from diverse text descriptions of employer names in the data. These high-dimensional embeddings then serve as input for subsequent clustering algorithms, which group together similar descriptions (e.g., “University of Guelph” and “Guelph University” for employer). The identified clusters enable the standardization and cleaning of employer. We illustrate the effect of clustering employer data in a regression model framework applied to a public salary disclosure data for Ontario, Canada.