<p>Hierarchical Deep-learning Neural Networks (HiDeNN) have recently been introduced to solve PDEs. Thanks to their mesh-based architecture, they have significantly fewer trainable parameters than fully connected neural networks. Moreover, these parameters are entirely interpretable. In this paper, we investigate the properties of HiDeNN and introduce several modifications to extend their capabilities. We describe in detail two approaches to shape function construction: the interpolation-layer-based approach and the reference-element-based approach. We investigate the performance of the framework, focusing on the impact of different loss functions on training efficiency and solution accuracy, as well as on the effect of numerical integration method used for loss evaluation. Additionally, we introduce the following two extensions to the existing HiDeNN framework: rh-adaptivity in higher dimension, which incorporates a Jacobian-based criterion to guide adaptive mesh refinement, and a multi-level training strategy, which leverages the network interpretability to efficiently transfer the learned weights across refinement levels. The capabilities of the proposed framework are demonstrated on linear elasticity problems in 1D, 2D, and 3D, and we compare the results against analytical solutions and a classical FEM solver. We show that variational loss performs comparably to energy-based loss and outperforms residual-based loss. Furthermore, the multi-level strategy significantly improves training robustness and efficiency, while the combined rh-adaptivity further enhances solution accuracy compared to fixed and purely r-adaptive meshes.</p>