Arc-Kayles is a game where two players alternate removing two adjacent vertices until no move is left, the winner being the player who played the last move. Introduced in 1978, its computational complexity is still open. More recently, subtraction games, where the players cannot disconnect the graph while removing vertices, were introduced. In particular, Arc-Kayles admits a non-disconnecting variant that is a subtraction game. We study the computational complexity of subtraction games on graphs, proving that they are PSPACE-complete even on very structured graph classes (split, bipartite of any even girth). We give a quadratic kernel for Non-Disconnecting Arc-Kayles when parameterized by the feedback edge number, as well as polynomial-time algorithms for clique trees and a subclass of threshold graphs. We also show that a sufficient condition for a second player-win on Arc-Kayles is equivalent to the graph isomorphism problem.