The construction sector consumes a massive amount of natural resources and energy and is a major emitter of greenhouse gases resulting in global temperature increase. Thus, the employment of recycled and green materials is essential for sustainable advancement. Hemp-lime composites, commonly known as hempcrete, are a promising low-carbon alternative to conventional building materials; however, further research is needed to optimize hempcrete mix designs and binder composition. This study investigated the impact of using different pozzolanic materials including metakaolin, silica fume, GGBFS, and fly ash on the physical and mechanical properties of the lime binder used to produce hempcrete. The experimental procedure was executed by testing 27 different mix designs generated by varying three parameters, namely the lime replacement ratio of the pozzolanic materials (10% and 20%), type of pozzolan, and the water/binder ratio (0.8, 1, and 1.2) in the mix. The physical and mechanical properties of the mortar including compressive strength, drying shrinkage, and flowability were carried out. The results showed that batches containing GGBFS and silica fume performed the best compared with the other mixes.