Applying LLMOps to Support the Automatic Analysis of CAs
摘要
Context. Many applications following a service-based architecture are regulated by Customer Agreements (CA) that define the conditions of use, responsibilities and limitations between suppliers and customers. There are tools for the automatic analysis of CAs that use Large Language Models (LLM) for some analysis operations. However, these tools do not take advantage of the Large Language Model Operations (LLMOps) cycle and best practices. Objective. Focusing on the need to automate the process of developing and operationalizing automatic analysis of CAs tools, this paper aims to explore the possibilities and limitations offered by LLMOps for this purpose. Method. To meet the proposed objective, we define an ideal LLMOps application for CAs analysers. In addition, a reference implementation has been designed that includes each phase of the LLMOps cycle. Results. After reviewing the automatic analysis of CAs tools and investigating the phases of the LLMOps cycle, an analysis and discussion of the alternatives for applying LLMOps for these tools has been conducted. Conclusions. From the eight stages of the LLMOps cycle, it can be concluded that some stages are more relevant than others. Ideally, the entire LLMOps process should be automated so that a new LLM can be incorporated in the future or an advanced feature of an LLM already in use can be activated.