This article investigates the application of the MADDPG algorithm within deep reinforcement learning to analyze and optimize bidding strategies across various auction scenarios. Initially focusing on classical auctions, we verify convergence in linear bidding strategies, establishing a fundamental understanding of agent behavior. Extending our analysis, our primary goal lies in examining the convergence behavior of nonlinear bidding strategies, particularly in the context of all-pay auctions, where bidding policies take on polynomial functions. Through this investigation, we aim to understand and evaluate the effectiveness of this algorithm in learning optimal bidding behaviors, shedding light on various convergence equilibrium. This study contributes valuable insights into the adaptability of deep reinforcement learning within auction settings, advancing our understanding of decision-making dynamics in multi-agent environments.