Eye tracking and egocentric vision technologies have emerged as powerful tools for analysing human behaviour, offering deep insights into attention patterns, decision-making processes, and mental workload. These tools have been successfully applied in diverse fields such as medicine, marketing, and human-computer interaction, yet their potential remains largely unexplored in the agricultural domain. Thus, this study investigates for the first time the feasibility and effectiveness of leveraging eye tracking and egocentric video analysis to assess human performance in soft fruit harvesting tasks, specifically focusing on strawberry as a use case. This work employs mobile eye-tracking glasses worn by workers as they navigate dynamic farm settings. The unstructured and fast-paced nature of harvesting, along with the small size of fruits and their frequent overlap with other objects, makes the annotation of the resultant eye-tracking data complex and time-consuming. To address this problem, this work introduces an egocentric vision system designed to automate the spatial and temporal annotation of key events and interactions relevant to fruit harvesting. Validated with field recordings of professional pickers, the system achieved substantial inter-rater agreement with human coders (mean Fleiss’ kappa score of 0.692 with a standard deviation of 0.06). Moreover, when fine-tuning its behaviour to match human manual annotations, the automatic reached an average accuracy of 0.845 with an average F1 score of 0.735. These results show that the system generates annotations with accuracy comparable to that of human coders while reducing processing time from several hours to just minutes, demonstrating its potential for scalable use in agricultural labour analysis and training.