In the ever-changing and dynamic realm of high-end fashion marketplaces, providing accurate and personalized size recommendations has become a critical aspect. Meeting customer expectations in this regard is not only crucial for ensuring their satisfaction but also plays a pivotal role in driving customer retention, which is a key metric for the success of any fashion retailer. We propose a novel sequence classification approach to address this problem, integrating implicit (Add2Bag) and explicit (ReturnReason) user signals. Our approach comprises two distinct models: one employs LSTMs to encode the user signals, while the other leverages an Attention mechanism. Our best model outperforms SFNet, improving accuracy by 45.7%. By using Add2Bag interactions we increase the user coverage by 24.5% when compared with only using Orders. Moreover, we evaluate the models’ usability in real-time recommendation scenarios by conducting experiments to measure their latency performance.