Rich reserves of shale oil are considered a viable alternative to traditional energy sources. However, the dynamics of pore pressure, pore width, fluid distribution, and flow mechanisms induced by shale oil flow in deformable shale nanopores remain unclear. In this study, molecular dynamics simulations are employed to investigate the occurrence and flow of n-octane in rough kerogen nanopores under three cases: fixed pore width, fixed pore pressure, and coupled rock compressibility coefficient. Simulation results show that both pore pressure and pore width increase gradually with increasing pressure gradient. Under the same pressure gradient, the pore pressure is: fixed pore width > coupled compressibility coefficient > fixed pore pressure, while the pore width is: fixed pore width < coupled compressibility coefficient < fixed pore pressure. As the pressure gradient increases, the adsorption layer density decreases; the pore center density increases for fixed pore width case, remains nearly constant for coupled compressibility coefficient case, and decreases for fixed pore pressure case. Under the same pressure gradient, the adsorption layer density and pore center density are: fixed pore width > coupled compressibility coefficient > fixed pore pressure, while the fluid flow velocity is: fixed pore width < coupled compressibility coefficient < fixed pore pressure. Therefore, the actual flow of shale oil in nanopores will lead to increases in pore pressure and pore width, a decrease in the adsorption layer density, and little change in the pore center density. This research can provide a theoretical foundation for the efficient extraction of shale oil.