Radiology report generation (RRG) aims to generate medically valuable textual reports from radiological images. Recent advances in large language models (LLMs) and vision-language models (VLMs) have highlighted their remarkable capabilities in RRG. Compared with X-ray imaging, chest CT offers higher density resolution and improved convenience for detecting lung lesions. However, research specifically focused on CT remains limited. In this paper, we introduce a vision-language model CTGLM for Chinese chest CT report generation, which connects a image encoder and an LLM using a lightweight Querying Transformer (Q-Former). In particular, Q-Former utilizes a self-attention mechanism to capture global image representations and focuses on specific image regions through a set of learnable queries. Moreover, to facilitate effective cross-modal alignment in LLM, we explore three alignment strategies by varying the trainable parameters. Experimental results on two public datasets COVID-19 CT and COV-CTR show that our proposed model CTGLM improves the BLUE-4 metrics by 16.2% and 12.5%, respectively, compared to the baseline model.