The binder material in conventional concrete is cement. In addition to its binding properties, it also provides the finest particle size range in the mix. The cement content in normal concrete fundamentally influences strength and durability. Higher strength and better durability require a higher amount of binder. Cement is the most polluting and energy-intensive component of concrete. The most effective way to reduce CO2 emissions is to lower the cement content in concrete. Green cement with low clinker content can be used but directly utilizing cement replacement materials (CRM) is even more effective. Well-established CRMs include blast furnace slag cement and pozzolan cement, but using local materials is particularly beneficial. In many cases, this will depend on the geographical location or the state of the industry. It is even better if waste materials or industrial by-products are used. Our experiments used low and high water-cement ratios (0.45–0.65) in mortars at low and high cement contents (360 and 500 kg/m3) as reference series and then pure Portland cement was partially replaced with CRM. Both are wastes from the production of construction elements that should be recycled in other sectors of the construction industry. The use of marble powder is still less widespread, limestone flour being more common. Their optimal and maximum potential applications were investigated, and they were compared with other cement replacement materials using both our own results and results from the literature.