This study proposes an adaptive system to regulate humidity and temperature in oyster mushroom cultivation using a Markov Decision Process (MDP) framework. The system predicts future environmental conditions and determines optimal water spraying actions to maintain humidity within the desired range. It considers the nonlinear relationship between temperature and humidity, integrating this into the decision-making process to enhance environmental stability and resource efficiency. The results show that the MDP-based approach outperforms traditional threshold methods in terms of water usage and humidity control, especially under varying temperature conditions. The proposed model is well suited for implementation in an Arduino-based integrated platform at low cost for Oyster mushroom cultivation.