Story visualization intends to generate a coherent sequence of images based on the textual description of a story. However, the sparse description of characters provided by story sentences results in models’ inability to maintain the consistency of characters in dynamic story scenes. To address the issue, this paper presents a story visualization model with attribute encoding and adaptive attention, named AEAA-SV. In particular, this model utilizes graph convolutional networks to encode attributes of characters and uses the encoded attributes as supplementary information for the characters in the story. Simultaneously, a character attribute classification loss is introduced during training to promote semantic alignment between attributes and images. Furthermore, we propose an adaptive attention module with a gating mechanism to enhance attention between text and image features in the generator. Experiments on Pororo-SV and Abstract Scenes datasets show AEAA-SV outperforms baselines in character consistency and image quality.