<p>Emotional support conversation aims to convey understanding, sympathy, care, and support through conversation, to help others cope with emotional distress, pressure, or challenges. In this study, we conduct a holistic comparative study to investigate how well the most recent large language models (LLMs), which have recently proved to have empathy, and act as emotional supporters. To this end, we make use of the emotional support conversation (ESC) framework and assess multiple maintain LLMs accordingly. We then have an in-depth comparison between these LLM-based emotional supporters to humans in terms of the use of emotional support strategies as well as the use of language. Surprisingly, we find that there is still a huge gap until these LLMs become effective emotional supporters. This is because, on the one hand, they have strong preference biases on using a limited set of strategies, making them always show empathy but rarely take real actions (such as providing suggestions), which is key in ESC. On the other hand, they often over-generate responses, making what they utter a departure from those of human experts.</p>