Reasoning about temporal commonsense knowledge in action sequences is important for applications involving task planning, automation, anomaly detection, etc. This paper explores the fine-tuning of various language models including RoBERTa, SmolLM2, DistilBERT, and the edge-optimized smaller TinyBERT for effectively capturing and predicting temporal commonsense knowledge and action relationships while maintaining energy efficiency and suitability for deployment on low-resource devices. Leveraging an existing commonsense temporal action knowledge dataset, we modified and extended the data to evaluate the ability of several language models to classify action-describing natural language sentences into various temporal categories in terms of the time they take to perform and make accurate predictions about the relationships between action sequence goal/steps. Our results demonstrate that even relatively compact models such as TinyBERT can achieve competitive performance in predicting commonsense knowledge about action-describing sentences and also perform anomaly detection tasks. The findings highlight the trade-offs between model size and prediction accuracy, providing insights into the viability of using lightweight models for real-world applications.