NUST Institutions Library Catalogue

Liquid sloshing presents a critical challenge for mobile robots tasked with transporting
partially filled containers, as it destabilizes motion, reduces accuracy, and increases the
risk of spillage. Traditional passive and active control methods—such as baffles, PID, and
LQR—are limited in adaptability and computational efficiency when faced with nonlinear,
time-varying fluid–structure interactions. This thesis addresses these challenges by
formulating slosh suppression as a reinforcement learning problem, leveraging the Twin
Delayed Deep Deterministic Policy Gradient (TD3) algorithm integrated with a surrogate
ball-in-container analogy in the Webots simulator. The surrogate transforms complex
liquid dynamics into a tractable cart-pole inspired system, enabling efficient training of
robust control policies without reliance on high-fidelity but computationally expensive
CFD models.
A custom simulation–learning pipeline was developed, coupling Webots with a PyTorchbased TD3 agent via JSON-based communication. State and action spaces were defined
using the ball’s displacement ratio and robot velocity, while a multi-component reward
function balanced slosh minimization, forward progress, and smooth energy-efficient
motion. Training results demonstrated that the TD3 agent learned to achieve stable, sloshfree navigation over a 10 m trajectory, outperforming traditional controllers and earlier RL
variants in stability and adaptability.
Results shows that the Model Learns well after 600 Episodes attaining optimal velocity of
1.25 – 0.5 m/s keeping ball within 2 mm limit, maximizing the reward to around 1180 at
1700 Episodes, eventually reducing sloshing by 65 % during Training phase. While, during
Test phase robot is tested for 0 to 4 m/s velocity with control and no control scenarios,
showing the results RL algorithm suppress the ball displacement (liquid slosh) by
approximately 45 %. The study validates reinforcement learning as a viable paradigm for
real-time liquid slosh suppression in robotics, offering superior robustness and scalability.
Contributions include an AI-driven control framework for slosh suppression, integrationxvii
of surrogate modeling with DRL for real-time feasibility, and a reward design framework
encoding domain-specific stability objectives. For Further Future working it is
recommended to include extending to real liquid models and hardware validation, adopting
symmetric action spaces and advanced RL methods, and integrating slosh control with
autonomous navigation in dynamic environments.