How to use lightweight algorithms to accurately identify multimodal fruits during fruit recognition is one of the key technical issues for agricultural robots. One of the important approaches in biomimetic optimization is to optimize robot recognition strategies using the inspiration of human attention distribution characteristics. This study conducted attention human factors experiments on apples, potatoes, and cotton in unnatural and storage scenarios, analyzed the attention distribution characteristics of different modal fruits in different scenarios, explained the attention distribution phenomenon using ergonomics principles, and proposed inspirations for improving existing algorithm frameworks. The results show that in terms of attention distribution for fruit localization, the attention is not only determined by the shape and color of the current image, but also by the image of the fruit in memory. Therefore, when designing machine recognition algorithms, it is necessary to add a memory module to the recognition algorithm framework. In terms of attention distribution for fruit lesion recognition, attention does not cover the entire fruit, but only focuses on specific areas. This can reduce the resource consumption of recognition and shorten the recognition time. When optimizing the design of fruit recognition algorithms, it is necessary to consider the most characteristic parts of the fruit as the recognition focus, which can reduce the computational resources of the algorithm.