The way in which data-driven services are rendered and operated at industrial scale often assumes the connotation of LLMOps, or large-language model (LLM) operations, specifically, service operations stemming from data-driven, LLM-instrumented business processes. At the same time, a general understanding over the principles of DevOps services computing is still lacking. To gain such a generalisable understanding, In this study we perform mixed-method research using in-depth interviews in the largest service provider—a logistics service provider industry also active around the EU—of the Netherlands combined with an online survey to assess the relevance, challenges, and best practices of LLMOps services computing in industry. Our findings show that, on the one hand the literature has provided so far valuable insights into many practical lessons already applied in action, but, on the other hand, not all such practicalities have equal application to various services computing scenarios in the same way. In conclusion the study shows that the assessment of LLMOps in industry is considerable but its level of maturity—intended as the established way to employ its benefits while minimising the risks connected to its use—is still very low. We conclude recommending a careful evaluation of the principles and goals emerging from our study.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

On the Maturity of LLMOps Services Computing: An Industrial Study

  • Nemania Borovits,
  • Damian A. Tamburri,
  • Willem-Jan van den Heuvel

摘要

The way in which data-driven services are rendered and operated at industrial scale often assumes the connotation of LLMOps, or large-language model (LLM) operations, specifically, service operations stemming from data-driven, LLM-instrumented business processes. At the same time, a general understanding over the principles of DevOps services computing is still lacking. To gain such a generalisable understanding, In this study we perform mixed-method research using in-depth interviews in the largest service provider—a logistics service provider industry also active around the EU—of the Netherlands combined with an online survey to assess the relevance, challenges, and best practices of LLMOps services computing in industry. Our findings show that, on the one hand the literature has provided so far valuable insights into many practical lessons already applied in action, but, on the other hand, not all such practicalities have equal application to various services computing scenarios in the same way. In conclusion the study shows that the assessment of LLMOps in industry is considerable but its level of maturity—intended as the established way to employ its benefits while minimising the risks connected to its use—is still very low. We conclude recommending a careful evaluation of the principles and goals emerging from our study.