Attitude toward quality of life in the current century is unhealthy, resulting in several diseases. Thermal imaging technology accompanied with robust and automated computational-aided diagnostic systems can be deployed at public gathering locations (like malls, hospitals, etc.) and on smartphones to timely detect and warn about potential health issues of their body caused by inflammation. In this paper, we aim to develop a lightweight model that can be deployed on mobile devices. For the proposed statement, we extracted and compared the performance of various Manually-Designed and lightweight Deep Neural Networks-based feature-sets using four widely used classifiers along with Cross-Validation (tenfold) sampling strategy. We performed the empirical study on two datasets, TH-CANCER-DB and TH-PLANTAR-DB. We obtained the best performance with accuracy values as 99.3 and 93.46% with SVC and Nasnet-Mobile-based features for TH-CANCER-DB and with k-NN and ShuffleNet-based features for TH-PLANTAR-DB. Also, on an average, the performance of features extracted from ShuffleNet and Nasnet-Mobile architectures is better in overall for both the datasets.