This thesis consists of two parts. In the first part, we study an advance scheduling problem with waiting-time-dependent patient cancellations. This work is motivated by a government-funded infertility clinic that suffers from high cancellation rates. Unused cancelled slots lead to low system utilization and raise patients' waiting time, which further aggregates patient cancellations. However, overbooking is not a good solution since it may increase the provider's overtime, which is undesirable, especially for a public clinic. The objective of the clinic is to dynamically schedule patients to improve system utilization while avoiding excessive overtime so that the total expected discounted profit is maximized. We formulate the problem as a Markov decision process. The objective is to maximize the expected total discounted profit over an infinite time horizon. We show that the optimal policy in a special case is a switching curve policy. However, the optimal policy in a general setting is difficult to derive due to the curse of dimensionality. Hence, we resort to heuristic policies. We propose an upper bound problem for policy evaluation. Through an extensive simulation study, we find that the proposed heuristic policy performs nearly optimally when demand variability is low, and it significantly outperforms other simple heuristics in most of the experiments. Moreover, the heuristic policy is consistent with the derived structure of the special case.

In the second part, we study the optimal patient care handoff decision in an emergency department (ED). Approaching the end of a physician's shift, the physician will need to sign out the patient and transfer her care to another physician on duty if her diagnosis and treatment are not complete. This practice is referred to as patient handoff, which results in discontinuity in patient care. Moreover, it creates the possibility of medical errors when the patient's information is not accurately transferred. Some EDs in North America advise a physician to stop accepting new patients near the end of her shift to prevent handoffs. The physician can instead focus on her existing patients and ''slack off'' at times. However, this strategy may lead to the underutilization of physicians' service capacity, which decreases ED throughput and aggravates ED congestion. Hence, the problem is how to optimally implement a slack-off strategy to strike a balance between patient safety and ED productivity. We formulate the problem as a finite-horizon discrete-time Markov decision process (DTMDP). The decision in each period is whether to serve a new patient when the physician becomes available. We derive structural properties of the optimal value functions and show that the optimal policy is a threshold-type policy. Furthermore, we propose an easy-to-implement heuristic policy, which performs near optimally in an extensive numerical study.