Conditional image generation aims to generate consistent images with user’s input. For handling complex situations with several objects and their relations, scene graphs have been proposed as the input of conditional image generation. Existing scene-graph-to-image models struggle to generate three or more objects in proper positional relations because an edge of the scene-graph represents a binomial relation of two objects. To overcome this difficulty, we proposed a hyper scene-graph-to-image model hsg2im. In a hyper scene graph, a hyperedge represents a trinomial relation of three objects, which can be reflected by one graph convolution. Nonetheless, hsg2im still struggles to capture relations of distant objects in a hyper scene graph and often fails to generate consistent images with the user’s input in the presence of many objects. In this paper, we evaluate our new model OA-hsg2im which reflects the relations of distant objects by introducing object attention layers on COCO-Stuff and Visual Genome datasets. Ablation studies and human evaluations show that OA-hsg2im improves the consistency of images to the user’s inputs compared to hsg2im as well as the naturalness of the generated images.