This chapter, based on the detection of spatiotemporal point and the analysis of spatiotemporal trajectory, gets into the classification and recognition of various actions that occur in the scene. Visual based human action classification and recognition is the process of labeling images (also videos, image sequences) with action (class) labels. Various techniques for action classification and recognition have been proposed, but work in this area is still ongoing. Multisensory data is used to enhance the accuracy of action classification in image. Attention mechanisms is used to focus more on frame images with more important content in video. This chapter will first introduce basic action classification techniques, including direct classification, time state models, and direct detection techniques. Then, the action recognition methods for various activities as well as various networks used for action recognition are discussed. In practice, an action classification method that combines pose and contextual information is introduced, and specifically classifiers based on pose and context are used. This chapter also analyzes two methods using attention mechanisms for classifying and recognizing actions in videos. One method utilizes attention mechanism to generate candidate actions, while the other embeds attention mechanism into the temporal action detection process.