Deep Neural Network (DNN) training is both compute- and memory-intensive. In this work, we propose a hardware-software co-design approach that leverages ReRAM-based process-in-memory (PIM) technology and second-order training to enhance DNN training efficiency. Second-order training reduces the number of iterations. Importantly, the key operation in second-order training, matrix inversion (INV), can be performed in ReRAM crossbars with \(O\left( 1\right) \) time complexity, minimizing the overhead. However, current ReRAM-based INV circuits face insufficient precision. To overcome this limitation, we propose a high-precision matrix inversion method with 8-bit INV circuit. Building on this foundation, we introduce STAMP, a ReRAM-based PIM accelerator specifically designed for second-order training. Experimental results demonstrate that STAMP achieves an average speedup of 114.8 \(\times \) and an energy saving of 41.9 \(\times \) compared to a GPU counterpart on large-scale DNNs.