Scholarlink

Context: <p>Java source code may inadvertently embed personally identifiable information (PII), creating privacy, security, and regulatory compliance risks in contemporary software engineering pipelines.</p> Objective: <p>We empirically compare three architectural strategies for entity-level PII detection in Java source code: a classifier-only pipeline based on transformer models, a hybrid classifier plus large language model (LLM) pipeline in which an open-weight LLM judges classifier-generated candidates, and an LLM-centered structured-extraction pipeline with deterministic validation and sanitization.</p> Method: <p>We evaluate the three pipelines on a synthetic Java dataset using shared preprocessing, value-level matching, and micro-averaged precision, recall, and F<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(_1\)</EquationSource> </InlineEquation>-score. The experiments use controlled configurations, fixed prompt templates, deterministic post-processing, and archived artifacts to support reproducibility and auditability.</p> Results: <p>Individual classifier baselines showed limited recall in isolation, while union-based ensemble aggregation increased coverage at the cost of additional false positives. The hybrid architecture reduced some false positives, but also systematically removed baseline true positives, producing precision-oriented filtering behavior that prioritizes selectivity over coverage. In contrast, the LLM-centered structured-extraction pipeline achieved the most favorable precision–recall balance among the evaluated strategies when structured-output compliance was stable and deterministic sanitization was enforced.</p> Conclusions: <p>The results indicate that architectural integration strategies and pipeline constraints substantially shape PII detection behavior, often beyond the isolated choice of individual models. Under controlled conditions, open-weight LLMs can support privacy-oriented source code analysis when embedded in reproducible, auditable, and validation-aware pipelines that explicitly monitor recall loss and structured-output robustness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating foundation model integration strategies for detecting PII in java software engineering pipelines

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月8日

查看全文
  • Fabiano Damasceno Sousa Falcão,
  • Edna Dias Canedo

摘要

Context:

Java source code may inadvertently embed personally identifiable information (PII), creating privacy, security, and regulatory compliance risks in contemporary software engineering pipelines.

Objective:

We empirically compare three architectural strategies for entity-level PII detection in Java source code: a classifier-only pipeline based on transformer models, a hybrid classifier plus large language model (LLM) pipeline in which an open-weight LLM judges classifier-generated candidates, and an LLM-centered structured-extraction pipeline with deterministic validation and sanitization.

Method:

We evaluate the three pipelines on a synthetic Java dataset using shared preprocessing, value-level matching, and micro-averaged precision, recall, and F \(_1\) -score. The experiments use controlled configurations, fixed prompt templates, deterministic post-processing, and archived artifacts to support reproducibility and auditability.

Results:

Individual classifier baselines showed limited recall in isolation, while union-based ensemble aggregation increased coverage at the cost of additional false positives. The hybrid architecture reduced some false positives, but also systematically removed baseline true positives, producing precision-oriented filtering behavior that prioritizes selectivity over coverage. In contrast, the LLM-centered structured-extraction pipeline achieved the most favorable precision–recall balance among the evaluated strategies when structured-output compliance was stable and deterministic sanitization was enforced.

Conclusions:

The results indicate that architectural integration strategies and pipeline constraints substantially shape PII detection behavior, often beyond the isolated choice of individual models. Under controlled conditions, open-weight LLMs can support privacy-oriented source code analysis when embedded in reproducible, auditable, and validation-aware pipelines that explicitly monitor recall loss and structured-output robustness.