A generalization of the quantum Rabi model is obtained by replacing the linear (dipole) coupling between the two-level system and the radiation mode by a non-linear expression in the creation and annihilation operators, corresponding to multi-photon excitations. If each spin flip involves k photons, it is called the “k-photon” quantum Rabi model. While the formally symmetric Hamilton operator is self-adjoint in the case \(k=2\) , it is demonstrated here that the Hamiltonian is not self-adjoint for \(k\ge 3\) . Therefore it does not generate a unitary time evolution and is unphysical. This result cannot be obtained by numerical calculations in finite-dimensional spaces which attempt to approximate an unbounded operator by a finite-rank operator.