EdLLM: A Novel Entity Detection Method for Test Data Integrating YOLO-World and LLM
摘要
The test data often exhibit the characteristics of multimodality and environmental complexity, and these characteristics have brought significant challenges to the detection and analysis work. The existing methods have deficiencies in interacting with the system through natural language processing, resulting in additional human resource consumption. The frontier topic of the integrated application of the You Only Look Once World (YOLO-World) and the large language model (LLM) is focused on by this paper. Firstly, the pre-trained LLM is used to tokenize the natural language and understand the user task, and then the tokenized data is passed into the agent designed based on YOLO-World to detect the entities of the test task. Then, based on prompt engineering, YOLO-World and LLM are innovatively integrated and the EdLLM is proposed, which creatively integrates the multimodal data, aiming to break the barrier between vision and language and realize the collaborative interaction between visual information and semantic information. Finally, it is applied in the field of test data to provide a more efficient, accurate, and intelligent solution. Compared with the existing open source model, this model can obtain entity information more effectively and quickly, and can detect entities based on prior knowledge combined with LLM under the condition of limited computing resources, which opens up a new way for the further development of related fields.