A methodology is proposed to estimate the dilation parameter, \({m}_{dil}\) for intact sedimentary rocks in the plastic region based on the Hoek–Brown yield criterion. Monotonic triaxial compression tests are performed on shaly sandstone and fine-grained sandstone under confining stresses ranging between 4 and 30 MPa. The results show that the peak \({m}_{dil}\) is related to \({\sigma}_{3}/{\sigma}_{c}\) , where \({\sigma}_{c}\) represents the uniaxial compressive strength of the rock. A three-parameter non-linear \({m}_{dil}\) model describing the development of dilation as a function of cumulative plastic shear strain is developed based on the observed experimental results, and the model parameters are determined by regression analysis. These parameters are found to be closely related to the confining stress. A dilation index is also estimated as a function of stress ratio between deviatoric principal stress and normal stress to highlight the trends of dilation/contraction in the elastic region, yield point, peak stress point, and post-peak region. The proposed model is then implemented in an in-house developed finite element code to compare displacements and plastic volumetric strains around a circular tunnel with those of constant \({m}_{dil}\) models. The simulation results show that a constant \({m}_{dil}\) does not produce realistic plastic deformation around the tunnel boundary in comparison to a mobilized \({m}_{dil}\) . The study concludes that the mobilized \({m}_{dil}\) model may be combined with the Hoek–Brown yield criterion for sedimentary rocks to analyze non-associative plastic behavior and displacement distribution in engineering practice.
