Masonry walls, arches, and other structural elements have been primary components of buildings from ancient to modern times; residential and public masonry buildings are widespread in North America and Europe. These buildings often have distinctive architectural and cultural character which needs to be preserved. To assess these buildings’ response to hazards, such as seismic action and settlements, high-fidelity computational models are needed. Discontinuum-based structural analysis tools, where the contact of each individual unit (e.g., brick or stone blocks) to other units is modeled, can be used for this purpose. However, manually generating the computational models with thousands of units per building is arduous and time-consuming. To this end, this chapter introduces a new data-driven framework where AI-assisted object detection and instance segmentation algorithms are used to generate discontinuum-based models from images of the building. The modeling pipeline is illustrated with an application to a stone masonry building with a nonperiodic masonry wall morphology. It is envisioned that the developed tools will enable reliable structural analysis of masonry buildings and support future activities related to their sustainable rehabilitation and/or reuse.