Deploying large language models (LLMs) on resource-constrained edge device presents significant challenges due to their high computational and memory demands. This paper introduces DLUSEdge, an efficient algorithmic framework designed to dynamically manage the loading and unloading of quantized LLMs on edge devices. The framework employs time-bound scheduling to optimize task execution while minimizing resource overhead. Four quantized LLMs, including qwen2.5:0.5b-instruct and granite3-moe:1b-instruct-q4_K_M, were evaluated in real-world scenarios, demonstrating task latency as low as <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$1.97 \times 10^9$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.97</mn> <mo>×</mo> <msup> <mn>10</mn> <mn>9</mn> </msup> </mrow> </math></EquationSource> </InlineEquation> ns and switching latency as low as <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$2.25 \times 10^9$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>2.25</mn> <mo>×</mo> <msup> <mn>10</mn> <mn>9</mn> </msup> </mrow> </math></EquationSource> </InlineEquation> ns. Correlation analysis revealed that prompt evaluation metrics strongly influence task latency (<InlineEquation ID="IEq3"> <EquationSource Format="TEX">$r > 0.8$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>r</mi> <mo>></mo> <mn>0.8</mn> </mrow> </math></EquationSource> </InlineEquation>), highlighting key optimization areas. Statistical analysis confirmed significant differences in task performance across models (<InlineEquation ID="IEq4"> <EquationSource Format="TEX">$p < 0.001$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>p</mi> <mo><</mo> <mn>0.001</mn> </mrow> </math></EquationSource> </InlineEquation>). The results validate the effectiveness of DLUSEdge in optimizing resource utilization and task performance, providing a robust solution for localized LLM inferencing. The code is hosted on <a href="https://github.com/ParthaPRay/llm_dynamic_load_unload">https://github.com/ParthaPRay/llm_dynamic_load_unload</a>.

DLUSEdge: Dynamic Load–Unload Scheduling for Localized LLMs on Resource-Constrained Edge

期刊论文
发表日期：

2025年8月20日

查看全文

Partha Pratim Ray,
Mohan Pratap Pradhan

摘要

Deploying large language models (LLMs) on resource-constrained edge device presents significant challenges due to their high computational and memory demands. This paper introduces DLUSEdge, an efficient algorithmic framework designed to dynamically manage the loading and unloading of quantized LLMs on edge devices. The framework employs time-bound scheduling to optimize task execution while minimizing resource overhead. Four quantized LLMs, including qwen2.5:0.5b-instruct and granite3-moe:1b-instruct-q4_K_M, were evaluated in real-world scenarios, demonstrating task latency as low as $1.97 \times 10^9$ $1.97 \times 10^{9}$ ns and switching latency as low as $2.25 \times 10^9$ $2.25 \times 10^{9}$ ns. Correlation analysis revealed that prompt evaluation metrics strongly influence task latency ( $r > 0.8$ $r > 0.8$ ), highlighting key optimization areas. Statistical analysis confirmed significant differences in task performance across models ( $p < 0.001$ $p < 0.001$ ). The results validate the effectiveness of DLUSEdge in optimizing resource utilization and task performance, providing a robust solution for localized LLM inferencing. The code is hosted on https://github.com/ParthaPRay/llm_dynamic_load_unload.