Identifying genes that play a causal role in carcinogenesis remains one of the major challenges in cancer biology. With the accumulation of high-throughput multi-omics data over decades, it has become a great challenge to effectively integrate these data into the identification of cancer driver genes. Here, we propose MODIG, a graph attention network (GAT)-based framework, to identify cancer driver genes by combining multi-omics pan-cancer data (mutations, copy number variants, gene expression, and methylation levels) with multidimensional gene networks. Among them, the multidimensional gene network is constructed by using genes as nodes and five types of gene associations (protein-protein interaction, gene sequence similarity, KEGG pathway co-occurrence, gene co-expression patterns, and gene ontology terms) as multiplex edges. We apply a GAT encoder to model within-dimension interactions to generate a gene representation for each dimension based on this graph, introduce a joint learning module to fuse multiple dimension-specific representations to generate general gene representations, and use the obtained gene representation to perform a semi-supervised driver gene identification task. The MODIG program is available at https://github.com/zjupgx/modig . The code and data are also available on Zenodo, at https://doi.org/10.5281/zenodo.7057241 .