Sentiment analysis allows researchers to analyze word-of-mouth without the help of human raters automatically. However, traditional machine learning or Bidirectional Encoder Representations from Transformer (BERT) models usually ask researchers to build their model or fine-tune the model, which creates a barrier to sentiment analysis in business and social science research. The Large Language Model (LLM), such as ChatGPT, provides researchers with a feasible approach to conducting sentiment analysis by developing or fine-tuning the model. However, it is essential to know how precise the LLM-based word-of-mouth sentimental analysis is and if they can directly use LLM-based tools to analyze word-of-mouth. The study compares the LLM-based semantic analysis results and the analysis results evaluated by human raters. The conclusions obtained from the study will help academic and industrial researchers understand whether the LLM-based sentiment analysis can replace human raters.