Cancer tissue or cancer cells do not exist in isolation, but start life within the microenvironment called tumor microenvironment (TME). The surrounding TME consists of an extracellular matrix (ECM), endothelial cells, cancer-associated fibroblasts (CAFs), and other components and is characterized by hypoxia, low pH, pathological pressure gradients, dysregulated extracellular enzymes, and more. The existence of the TME is a major reason underlying cancer occurrence, development, invasion, metastasis, drug resistance, and recurrence after treatment. Therefore, regulating the TME might be regarded as a rational choice beyond only killing cancer cells. However, there is difficulty in targeting cancers using free TME-modulating and/or anticancer drugs as a result of low bioavailability and high toxicity. Accordingly, it is necessary to develop delivery systems that can carry the corresponding therapeutic drugs to the therapeutic site. In recent years, a series of drug delivery systems (DDSs) integrating TME modulation and cancer cell targeting have been developed. Here, we summarize recent advances in their development, some considerations about the composition of the TME, and the factors necessary for TME- and tumor-targeted therapy, including the selection of homing functionalities, therapeutic targets, and typical DDSs.