Graph convolutional networks (GCNs) have found wide applications through effectively learning node, edge or graph embedding. While many existing works focus on accelerating GCN inference on a single large graph, in this work, we propose an approach to accelerate GCN inference on a large number of small graphs. The main idea is to implement GCN inference fully relying on dense operators, which enables us to rearrange the order of basic operators and to leverage deep learning compilers like TVM to lift the performance for both single graph and batched graphs. Experimentation on typical small graph datasets shows that our approach achieves significant speedup over DGL. It also outperforms TVM on GCN inference for batched graphs.