Multimodal medical imaging is of great importance in contemporary healthcare to integrate complementary information from various imaging modalities, for example, MRI, CT, PET, and X-ray. These modalities offer complementary information covering anatomic and functional information of the human body, which results in enhanced ability to reach more accurate diagnoses and to plan appropriate treatments. Nevertheless, combining and comparing information between these modalities is still challenging because of inherent discrepancies in resolution, contrast, and anatomical correspondence. This paper will describe some of the recent progress and challenges associated with this problem, and proposes a new computational framework using generative adversarial networks (GAN) for multimodal medical image translation problem, and will demonstrate the validity for two different kinds of downstream clinical tasks.