Distributed computing systems such as Spark have become the essential infrastructure for processing large-scale data. In heterogeneous cloud environments, the cost of resource usage is a key factor in system design and scheduling. Inefficient resource allocation can significantly increase the cost of resource usage. Therefore, it becomes crucial to minimize monetary expenses while ensuring performance requirements. However, most existing strategies fail to adapt to varying workload intensities or address multiple objectives simultaneously. In this paper, we formulate the Spark resource allocation process as a Multi-Objective Markov Decision Process and propose a learning-based resource allocation agent, ThetaMCTS, which integrates neural networks with Monte Carlo Tree Search to achieve forward-looking executor placement decisions. Extensive experiments under diverse workload patterns show that ThetaMCTS effectively adapts to different load conditions and heterogeneous resource conditions, reducing resource usage cost by up to 34.3% and shortening the average job turnaround time by up to 31%.