This paper presents a comprehensive study on the sentiment analysis of hotel reviews in Vietnam, utilizing various deep learning models. Among the conducted models, the Transformer model on the pre-trained ‘all-mpnet-base-v2’ embedding model achieved the highest accuracy of 85.73% and averaged F1-score of 0.8580, outperforming CNN, LSTM, BiLSTM, and BiLSTM-CNN on word2vec embeddings. Furthermore, we propose a cold-start recommendation system designed to assist new customers in finding suitable hotels based on their keywords. By employing a cosine similarity matrix over the embedding matrices obtained from sentence transformer model, the system calculates similarity scores between the input sentence and previous customer reviews. The quality of the recommendations is improved by integrating additional weights, including hotel ratings, total number of reviews, and sentiment derived from our sentiment analysis model. Experimental results demonstrate the effectiveness of our approach in providing accurate and relevant hotel recommendations.