Deep Reinforcement Learning has become an effective method for solving the Routing, Modulation, Spectrum and Core Allocation problem in Space-Division Multiplexing Elastic Optical Networks. The design of the reward function plays a crucial role in guiding the learning process. A significant number of studies have optimized the reward function by introducing path and intra-core spectral distributions. However, there has been little discussion of inter-core resource distribution. This paper proposes an Inter-Core Resource-aware DRL algorithm as a potential solution to this issue. The IR-DRL algorithm introduces the inter-core spectral adjacency to design the reward function, thereby encouraging the agent to allocate spectrum in a manner that mitigates inter-core crosstalk. The simulation results demonstrate that the IR-DRL algorithm exhibits superior performance in terms of blocking and spectrum resource utilization when compared to reference algorithms. IR-DRL algorithm achieved blocking rate reductions of 19.01% and 21.05%, and spectrum resource utilization improvements of 15.91% and 17.11%, in the NSFNET and Cost-239 networks, respectively.