Musical onset detection, a cornerstone in automatic music transcription, involves identifying the precise moments when notes or sounds begin. This paper proposes a neural network architecture for this task that takes advantage of the temporal order of features extracted from the context windows of the spectrogram. The architecture integrates convolutional layers for the extraction of spectral features with recurrent layers to capture sequential temporal patterns. Evaluated on the Böck dataset, the proposed method achieves results comparable to the state-of-the-art convolutional neural network models and surpasses other approaches leveraging temporal information, such as convolutional recurrent neural networks and temporal convolutional networks. These results emphasize the efficacy of the proposed architecture in capturing the complex temporal dependencies inherent in musical onsets.