The urgent need to reduce the environmental impact of the construction sector has spurred growing interest in sustainable and circular approaches to building materials. This work investigates the mechanical behavior and long-term durability of bio-based cementitious composites incorporating agricultural waste, specifically olive kernel aggregates (OKA). These composites, designed as environmentally friendly alternatives to traditional mortars, aim to contribute to sustainable masonry practices and future applications in historical masonry repair. An experimental campaign was carried out to characterize the swelling behavior of OKA under varying moisture content, simulating conditions relevant to in-service masonry elements. The results were integrated into a finite element modeling framework using representative volume elements (RVEs) with degradable matrices and moisture-sensitive inclusions. By adopting a Concrete Damaged Plasticity (CDP) model, the study evaluates the internal damage mechanisms driven by shrinkage and swelling of bio-aggregates, with particular focus on their influence on matrix degradation and stiffness reduction. This study contributes to the ongoing discourse on masonry/composite interaction, with emphasis on bond behavior, long-term durability, and the life-cycle implications of substituting traditional aggregates with biodegradable alternatives. The findings provide insight into the performance of composites designed for sustainable construction and open perspectives for their responsible use in strengthening strategies and compatible repairs of historical masonry structures.