Reward-Shaped Proximal Policy Optimization for Emergency Department Scheduling

Abstract

The scheduling of emergency departments when stochastic patient streams are involved habitually fails with the fixed heuristic, resulting in huge wait times, SLA breach, and disastrous patient abandonment. This paper presents a fairness-conscious Proximal Policy Optimization (PPO) model to achieve dynamically responding emergency responsiveness, regular patient equity, and queue stability. We compare three variants of rewards that are progressively shaped (PPO-A, PPO-B, PPO-C) with FCFS and Priority baselines in a high-load stochastic simulation. Queues collapse with traditional rules, but none of the PPO agents causes the latter as they shorten the average wait times over 98 percent. Nonetheless, this security comes at the cost of a calculated downgrading of the raw throughput by about 45 percent, as the agents will give preference to the critical cases and preemptive clearance rather than volume. Ablation experiment results show that PPO-C offers the most consistent convergence to training, the highest fairness ratios and the best balance of operations. These findings indicate that explicit reward structuring, and not algorithmic complexity, is the most significant aspect in extrapolating projecting reinforcement learning to clinically applicable scheduling systems. The framework offers a blueprint of how to do resource allocation in high-acuity healthcare settings in an adaptive, abandonment-proof way. These dynamics will be proved in future work with real-world multi-resource departmental data and incorporated with short-horizon demand forecasting.