Enhancing Proximal Policy Optimization in the Pendulum-v1 Environment Through Clustering-Based State Space Simplification

This study aims to enhance the performance of the Proximal Policy Optimization (PPO) algorithm in continuous control tasks, particularly in the Pendulum-v1 environment. We introduce a clustering-based approach to simplify the state-action space, improving learning efficiency and maximizing cumulative rewards. The state-action pairs are clustered using k-means, emphasizing regions with higher rewards to guide the agent's policy optimization. Experimental results demonstrate that clustering significantly improves PPO's performance, with the 10 -cluster configuration yielding the best outcomes in terms of reward consistency and efficiency. After tuning, the clustering-enhanced PPO achieves near-optimal performance, outperforming the baseline PPO by a wide margin.

• proximal policy optimization
• K-means clustering
• continuous control
• reinforcement learning

• Reinforcement Learning in Robotics
• Adaptive Dynamic Programming Control
• Simulation Techniques and Applications
• proximal policy optimization
• K-means clustering
• continuous control
• reinforcement learning
• Artificial Intelligence
• Computational Theory and Mathematics
• Management Science and Operations Research
• Computer Science
• Decision Sciences