For quantitative portfolio optimization problems, predicting the various asset returns accurately is essential. Transformer architectures, which have achieved state-of-the-art performance on a broad variety of sequence modeling tasks, offer significant potential for capturing the complex temporal dynamics inherent in financial markets, going beyond traditional limitations. However, financial markets present unique challenges due to non-stationarity and complex relationships between stocks. Also, while numerous variants of the Transformer have been proposed for general time series forecasting (TSF) with a variety of mechanisms, their relative performance and best suitability for ranking-based portfolio selection are barely examined. We close this gap by adapting and evaluating several Transformer architectures (Vanilla Transformer, CrossFormer, adapted MASTER and iTransformer) for daily return forecasting to enable ranking-based portfolio selection on S&P500 using stock market data. Focusing on their ability to model both temporal patterns in individual stocks in the market and valuable relationships between them. Our contribution is a benchmark comparing these adapted architectures in a realistic ranking task. The analysis shows how the different mechanisms capture temporal and cross-sectional information, identifying the strengths and weaknesses of each model in generating profitable rankings and offering practical insight into the choice of suitable transformers for quantitative trading.