This project explores reinforcement learning in Rocket League, training an AI agent to perform complex in-game maneuvers such as ball control, shooting, and dribbling. Using deep reinforcement learning techniques, the agent improves over time, competing against human players and demonstrating advanced gameplay strategies.

Reinforcement Learning (RL) plays a crucial role in Artificial Intelligence (AI), particularly in complex decision-making environments. Rocket League provides an ideal platform for testing and refining RL algorithms due to its dynamic, physics-based gameplay. Despite advancements in RL, awareness in both academic and professional circles remains limited. Showcasing successful RL applications in popular games like Rocket League serves as an engaging method to educate and inform the public about RL’s potential. By highlighting practical RL implementations, this project contributes to inspiring further research and development, bridging the gap between theoretical AI advancements and real-world applications.

Overview: Our project focuses on developing a reinforcement learning (RL) agent for Rocket League that can compete at a high level against human players and other bots. By leveraging state-of-the-art RL techniques, we aim to push the boundaries of AI-driven gameplay in complex, real-time physics environments. Our work is built upon the RLGym and RLBot frameworks, which provide the necessary tools to train and evaluate our agents in the Rocket League environment. Technical Approach: We utilize Proximal Policy Optimization (PPO), a popular on-policy RL algorithm known for its stability and efficiency in continuous action spaces. Our implementation refines existing RL techniques by incorporating: An improved reward structure: Our custom reward function balances offensive and defensive play, rewarding actions that contribute to sustained possession, efficient positioning, and successful goal-scoring opportunities. A discrete action space: Instead of using continuous control, we employ a carefully designed discrete action space that enables better learning and execution of complex maneuvers, such as dribbling, flicks, and aerial hits. Enhanced observation processing: The agent receives a structured set of inputs, including ball position, velocity, car states, and opponent positioning. We preprocess these observations to improve the agent's spatial awareness and decision-making. Performance and Capabilities: Our agent demonstrates significantly improved gameplay compared to earlier RL-trained bots like Nexto and Necto. Key abilities include: Ball control: Unlike traditional scripted bots, our agent can effectively control the ball, dribble it across the field, and execute flicks for scoring opportunities. Strategic positioning: The agent learns to maintain optimal field coverage, ensuring defensive stability while seeking offensive openings. Adaptive decision-making: Through extensive training, the agent adapts to various in-game situations, reacting appropriately to opponent movements and game state changes. Challenges and Future Work: While our agent has made notable progress, challenges remain: Opponent diversity: Training against a limited set of opponents can lead to overfitting. We aim to introduce more varied playstyles and adversarial training techniques to improve generalization. Aerial play: Although the agent can handle ground-based maneuvers well, aerial control remains a challenge. Future work will explore hierarchical learning approaches and imitation learning from high-level human replays to improve aerial play. Real-time adaptation: We are investigating ways to make the agent dynamically adjust its strategy mid-game based on opponent behavior and team dynamics. Conclusion: Our project demonstrates the potential of reinforcement learning in competitive Rocket League gameplay. By leveraging PPO, structured observations, and optimized reward functions, we have created an agent that surpasses previous RL models in skill and strategy. Continued development will focus on improving adaptability, aerial proficiency, and robustness against diverse opponents, with the ultimate goal of achieving human-level performance in Rocket League.