Privacy-aware data synthesis is a field aiming to liberate data access through the generation of synthetic data which mirrors the original without resulting in privacy exposure. State-of-the-art algorithms for structured data perform well in datasets with tables of a few million rows but result in prohibitive runtimes when scaling to hundreds of millions of rows. In addition, due to the sensitive nature of data, practitioners are often limited to a single server environment. In this paper, we present the framework Pasteur, which aims to scale privacy-aware data synthesis linearly under a single server environment. Pasteur achieves this through a parallelization approach tailored for synthesis, optimized memory representations, and an accelerated marginal calculation algorithm (bottleneck in a class of privacy-aware algorithms). We show Pasteur performing pre-processing, synthesis, and evaluation of a tabular dataset with 1 billion rows (200 GB) in 1 h on a 16 core CPU server.