In recent years, the increase in Emergency Department (ED) visits worldwide, exacerbated by the COVID-19 pandemic, has posed considerable challenges to healthcare systems. Electronic health records (EHRs), which serve as comprehensive digital repositories of patient data, offer a valuable resource for developing predictive models to mitigate these issues. However, the effectiveness of such models is often compounded by the inherent heterogeneity of EHRs. A particularly significant obstacle is the prevalence of high-cardinality nominal features (NFs) within EHRs, which complicates accurate predictions and effective model performance. Their extensive distinct values often lead to their exclusion from analysis, which not only diminishes model performance and interpretability but also risks the loss of valuable information. Additionally, selecting a manageable subset of these values relies heavily on domain knowledge. This study introduces TE-PrepNet, a method that leverages MIMIC-IV-ED data to efficiently handle NFs in machine learning (ML) models by combining highly correlated features and incorporating target encoding. We applied TE-PrepNet on two ED prediction tasks: hospital admissions based on triage and re-admissions to the ED within 72 h after discharge. Our approach beats the baseline and earlier models that removed NFs by combining highly correlated characteristics and integrating three important NFs. The XGBoost model with TE-PrepNet outperformed the baseline with an AUROC of 0.8451 in hospital admission prediction. Logistic Regression model using TE-PrepNet increased the AUROC to 0.7004 for predicting ED reattendance within 72 h, above the baseline of 0.6267.