Learning a deep neural network requires massive amount of data and intensive computation capacity. For various reasons data may be distributed across multiple places. Even if we have all the data, training requires massive computations, which may not be handled by a single machine. In this section we will introduce distributed learning, which is a practical technique that can partition the learning task to multiple machines. Distributed learning can reduce the communication and storage costs to gather all the data and reduce the computation costs on each machine.