Micro-expression is an involuntary facial expression that involves subtle facial muscle movements and only lasts for a very short duration. Given the potential of micro-expressions to reveal the true emotion of a person, research on automated micro-expression recognition has been gradually getting more attention over the last few years. However, many works tend to focus either on recognition based on three (3) general emotions (positive, negative, surprise) or based on the only available emotions in the dataset, which may not be sufficient when used for real-world applications. Meanwhile, other studies focus on building deep neural networks to extract relevant features and improve the performance of the recognition system. Hence, we present a shallow three-stream CNN (S3S-CNN) architecture that is capable to distinguish the six (6) basic emotions, namely: anger, sadness, surprise, fear, happiness, and disgust. We used estimated optical flow and optical strain as the set of features and followed the Leave One Subject Out Cross Validation (LOSOCV) protocol to train and evaluate our model. Our model reached an accuracy of 72.45%, with a score of 0.6182 for the unweighted F1-score (UF1) and 0.6089 for the unweighted average recall (UAR).