Machining high-aspect ratio holes using a conventional solid boring bar (BB) is a challenging task; hence, the tuned mass damper (TMD) is generally preferred to enable a higher machining stability limit (blim). In TMD-based passive damped boring bars (PDBB), cavity geometry and its location dictate their performance besides other parameters; however, limited studies are available and to date, an approach for optimizing in combination has not been reported. For this purpose, the present work initially presents a comprehensive study for understanding the effect of the cavity geometry and its location on the BB’s (length to diameter (L/D) ratios 7–12) effective modal mass (meff) of the fundamental governing mode and the TMD’s mass ratio (µ- the ratio of absorber mass (mabs) to the meff). Furthermore, to obtain the optimum values of the cavity geometry and its location, numerical optimization under conditions without and with constraints (i.e., allowable mabs) is performed to maximize the µ of the TMD. The results show that increasing mabs or the µ does not increase the blim indefinitely since higher mabs require more space within the BB, resulting in a loss of rigidity when the cavity is sub-optimally placed. To assess the optimal cavity geometry quickly, a simplified approach is proposed using the insights gained from a comprehensive study and then numerically optimized for its location. The blim in the proposed approach was closer to the other optimization approaches performed and further improved for 8 – 10 L/D with a maximum of 29.6% at 8 L/D.