Deep Neural Networks (DNNs) are successful but work as black-boxes. Elucidating their inner workings is crucial but a difficult task. In this work, we investigate how activity and confidence of a DNN relate in a simple Multi-Layer Perceptron. Further, we observe how activity, confidence and their relation develop during model training. For ease of visual comparison, we use a technique to display DNN activity as topographic maps, similar to common visualization of brain activity. Our results indicate that activity becomes stronger and distinguished both with training time and confidence.