Automating the assessment of software delivered as part of student coursework is an important tool to make better use of time for academics with teaching requirements. Here, eye tracking software has been tested with the goal of automatically capturing how the user interacts with software, in this case, simulation software. Useful usage metrics can be easily captured, that along with manual assessment metrics, provide a detailed and insightful visualisation of how the end user experiences the delivered software. Quantification of the eye tracking detail can be systematically achieved and can act as a grading guideline. The results indicate additional eye movement data collection and evaluations of different eye tracking algorithms are needed for better automatic scoring.