Phenotypic characterization allows the study of cellular effects caused by substances or genetic modifications. Here, we demonstrate how deep learning tools can help advance phenotypic characterization using high-throughput microscopy imaging. We focus on a particular example, segmenting and classifying different cellular phenotypes, to demonstrate how an automated image analysis can be performed using open-source software that can be used without advanced deep learning knowledge. In particular, we give in-depth descriptions of how to train and deploy a deep learning pipeline for phenotypic characterization, including troubleshooting when things go wrong in different scenarios.