Argument Mining (AM) is a significant task in the field of Natural Language Processing (NLP). However, effectively utilizing the structural information of these components remains a challenge for traditional approaches. Transformer architectures, such as RoBERTa and DeBERTa, have emerged as the unified paradigm for NLP tasks. Integrating structural information effectively using transformer architectures remains a challenge. To tackle this challenge, we employed advanced modeling techniques by utilizing RoBERTa for sentence-level modeling, integrating attention mechanisms to capture comprehensive textual information. Concurrently, we utilized DeBERTa, leveraging prompts to introduce structural information. We observed significant differences in predictions between models incorporating full-text information through attention mechanisms and those using prompt-based approaches, and simultaneously among models with varying parameter sizes. Therefore, we applied model ensemble techniques to enhance predictive performance. To ensure the reliability of the ensemble results, we supplemented our predictions with inference from large language models. By integrating these approaches, we established a system for argument components identification. In the NLPCC-2024 shared task 5, our system achieved fifth place with a score of 74.54. We released our code at https://github.com/captainpickstar/New Folder-on-NLPCC2024-task5 .