Hierarchical reinforcement learning is a type of reinforcement learning that decomposes the reinforcement learning problem into multiple less difficult subproblems. Hierarchical reinforcement learning has potential advantages over non-hierarchical reinforcement learning algorithms. This includes the potential to improve sample efficiency. This paper presents a new hierarchical reinforcement learning algorithm used in playing the game of Othello. An experiment has been done to compare the sample efficiency of the new hierarchical reinforcement learning algorithm to the Deep Q-Learning algorithm. The new hierarchical reinforcement learning algorithm demonstrated significant improvements in sample efficiency.