Tabular data synthesis provides promising solutions addressing the limited availability of machine learning training data and privacy concerns by generating synthetic datasets that mirror real data characteristics. Despite the field’s rapid evolution, systematic benchmarking approaches remain underdeveloped, with evaluations typically limited to hand-picked datasets and tasks. We introduce ReL8r, a benchmarking framework that evaluates tabular data generators using predefined inter-column relationships (e.g., 1:n, XOR) commonly found in real datasets. ReL8r systematically assesses generators’ ability to learn these constructed relations and analyzes how the quality of learned dependencies influences the performance of supervised machine learning models. Even with small datasets, ReL8r provides deep insights into different generative architectures’ strengths and weaknesses. Our benchmark of state-of-the-art methods (GANs, transformers, autoencoders) reveals that transformer models achieve near-optimal performance across many dependency domains. While GANs mostly exceed autoencoder performance, autoencoders show comparable results to GANs in scenarios characterized by less complex, linear relationships.