Aerodynamic shape optimization is the process of designing and improving an object's external geometry to enhance its aerodynamic efficiency. Traditional approaches rely on viscous or inviscid flow simulations coupled with mathematical optimization algorithms to generate efficient designs in terms of aerodynamics, based on the sampling methods. This study focuses on developing a reinforcement learning based optimization algorithm for aerodynamic shape optimization, leveraging reinforcement learning (RL) and supervised machine learning techniques. Specifically, Proximal Policy Optimization (PPO), and XGBoost are employed to optimize the airfoil geometry. The primary objective is to maximize the aerodynamic efficiency, specifically the lift-to-drag ratio, of a two-dimensional airfoil by modifying its upper and lower surface geometries. The optimization process is driven by an RL agent and a machine learning model, which learn the relationship between geometric modifications and aerodynamic efficiency through iterative interactions with the flow solver. A key novelty of this work lies in the direct comparison of PPO, and XGBoost for aerodynamic shape optimization, as no prior studies have systematically evaluated these algorithms in this context. By exploring and exploiting optimal shape variations, the proposed machine learning based framework aims to achieve superior aerodynamic performance while minimizing computational cost.