The ideas surrounding transformers, self-attention, and other attention mechanisms have reshaped the direction of computer vision, originating in NLP feature learning. Transformers can be parallelized very well compared to CNN-pipelines and LSTM-recursive architectures (see Chaps. 9 and 10 ). The power of the transformer architecture is being applied to other data modalities as well and is perhaps the most popular architecture used in AI today with gradient descent training for machine learning. In this chapter, we survey transformer architectures beginning with NLP to find the roots and then proceed to explore how transformers are applied to computer vision. A range of visual attention methods are discussed, differing from text-based attention methods, starting from key historical milestones in natural language processing using the Vaswani method through BERT, since these methods inspire and inform visual attention and visual transformers for computer vision, first by the ViT method, followed by several other methods. In addition, we touch on third-generation AI directions in multimodal classification and AML, moving away from hand-crafted softmax-style single-class classifiers toward multivariate and multimodal systems with agent classifiers and continuously learning classifiers, using a set of multi-size multi-class models from small to huge—these topics are also discussed with more depth and examples in Chap. 12 . We review the applications of attention in deep learning to retrofit CNNs and explore in more detail the transformer models using encoder/decoder architectures for visual attention. We survey hybrid feature learning backbone architectures using a variety of feature learning methods and network innovations such as self-attention, Fourier features, NL-Means features, and binary features which may be combined with convolutional CNN-style n × n feature weight learning. We touch on third-generation AI directions emerging with multivariate and multimodal feature models and classifiers which implement continuous learning. Background details are provided on NLP topics related to transformers and attention, such as encodings, embeddings and attributes, positional encodings, and multi-head attention, with historical milestones in NLP attention and transformers which have informed computer vision methods.