High-throughput sequencing technology provides the opportunity to conduct cancer analysis at the gene level. Despite the emergence of survival prediction methods for transcriptome RNA-seq data, discovering valuable information from numerous genes remains a challenging due to the high dimension of RNA-seq data. An ideal strategy to solve this challenge is to adopt prior knowledge and eliminate the influence of noise. Here, we propose the Pathway Variational Auto Encoder (PathVAE) framework, which leverages prior knowledge of biological pathways to encode gene expression profile into pathway activities. To highlight pathways that associated with survival prediction and limit noise interference, we further introduce the Gated Attention Pooling (GAP) layer to enhance and recalibrate the fused pathway representation. To evaluate the effectiveness of our proposed PathVAE, we use nearly 2500 cancer samples across four different cancer types sourced from TCGA cancer datasets to conduct experiments. Our experimental results demonstrate that our proposed method outperforms current methods.