Parameters play a key role in ensuring the expected behaviors of systems or achieving certain objectives, which gives rise to countless parameter optimization (PO) frameworks. In view of their shortcomings of weak adaptability to different scenarios caused by internal predefined configurations, this paper proposes a general controller named POAgent based on an efficient learning paradigm and multi-agent reinforcement learning, which can adaptively adjust the configurations and guide the PO process towards better outcomes according to the on-site situations. Experimental results show that significant improvements can be achieved when incorporating it into an existing SOTA PO framework.