Optimal Allocation Strategies in a Discrete-Time Exponential Bandit Problem

This study addresses a theoretic-bandit problem involving a "safe" and a "risky" arm across countable periods. Departing from the "either-or" binary choices in previous studies, we explore smooth allocation strategies using the first-order approach. Modelling both the action and the posterior as state variables, we obtain clear characterizations of the optimal allocation strategies and comparative statics. The optimal plan significantly enhances the binary strategies, yielding a higher probability of breakthrough and a higher expected payoff. The Goldilocks principle emerges in that the incentives for exploring the risky arm peak at a level that is neither too difficult nor too easy.