Deep reinforcement learning is a widely established technique used for the navigation of autonomous mobile robots. Existing literature on deep reinforcement learning-based methods (DRL) for mobile robot navigation demonstrates the effectiveness of these techniques in understanding indoor environments. However, when deployed in complex, cluttered and unknown surroundings with moving obstacles, the requirement for exploration increases rapidly, leading to performance deterioration. This paper proposes a study on heuristic-based DRL technique for autonomous mobile robot navigation to address this performance degradation. The Soft Actor-Critic (SAC) method is considered for implementation due to its ability to balance exploration and exploitation, efficiently reusing past experiences to accelerate learning and reduce sampling complexity. The proposed heuristic SAC (h-SAC) system utilizes inputs from the neural network, including previous linear and angular velocities, ten 2D LiDAR data points, and the angle and relative position of the mobile robot with respect to the goal. A goal-based heuristic reward function is proposed to accelerate exploration by providing a positive reward when the goal is reached and a negative reward when obstacles are encountered. The proposed h-SAC method is implemented in the Robot Operating System (ROS) using Python code. A complex virtual environment is created for training and evaluation in the Gazebo Simulator, and the TurtleBot3 robot is used as an agent. The proposed system shows improved training efficiency, completing twice as many iterations compared to SAC in the same computational time, and achieves a maximum of up to 20% improvement in the success ratio over traditional SAC algorithms.