The commonality of high-dimensional data encountered in numerous application areas poses significant challenges and impacts the performance and accuracy of machine learning models, including fuzzy models. The effect known as a concentration phenomenon has been discussed intensively in the literature. However, this detrimental aspect has not been thoroughly studied and quantified in the area of fuzzy models. To narrow down the existing gap and bring some original design insights, in this study, we investigate the impact of data dimensionality on the performance of fuzzy rule-based models, in particular, Takagi-Sugeno models. The effect of increased dimensionality becomes clearly manifested when building fuzzy condition parts (fuzzy sets) realized with the use of Fuzzy C-Means clustering. The Fuzzy C-Means algorithm dwells on the distances between data and prototypes, and these distances are directly impacted by the concentration phenomenon. Furthermore, the study leverages partition matrix-based concentration indices and histograms of membership grades to evaluate clustering results that have a direct impact on the performance of ensuing fuzzy models. Based on the results of the Fuzzy C-Means clustering, we explore the effect of the increasing dimensionality of the feature space on the performance of K-Nearest Neighbor classifiers. By working with high-dimensional data, we experimentally explore the effect of successive random reduction of the number of features (input variables). The experimental results reveal that the performance of the model (both predictors and classifiers) can often be enhanced by reducing the dimensionality of the feature space, leading to models of lower complexity and improved generalization abilities. In the study, we also offer some design guidelines.