Industrial Control Systems (ICS) are paramount to the efficient operation of Critical National Infrastructure (CNI) ranging from electricity generation and distribution to manufacturing. However, the growing convergence of ICS with Information Technology (IT) systems renders CNI vulnerable to a range of cyber threats. Graph neural networks are being increasingly used for anomaly detection by adding granularity to the detection process. In this paper, we present a comparative study of graph-based deep learning models for ICS anomaly detection. Through the evaluation of four models using three multivariate industrial datasets, we aim to discern the effectiveness of prediction and reconstruction-based graph models in the ICS domain. We investigate data reduction techniques to minimise features needed to represent the window size and examine the representation of sliding window in terms of feature size for time-series analysis. Additionally, we assess the impact of the length of a context window on anomaly detection performance. Our results show that using feature reduction techniques on a longer context window produces better results while having the computational advantages of a shorter window size. Graph autoencoder is the most resilient to feature size reduction by maintaining similar F1 and AUC-PR score regardless of the number of features used to represent a context window. The results also provide insight to the suitability of graph-based models in this domain and offer recommendations for their optimal usage, paving the way for enhanced security and resilience in ICS.