Dynamic budget adjustment in search auctions

With serious advertising budget constraints, advertisers have to adjust their daily budget according to the performance of advertisements in real time. Thus we can leave precious budgets to better opportunities in the future, and avoid the surge of ineffective clicks for unnecessary costs. However, advertisers usually have no sufficient knowledge and time for real-time advertising operations in search auctions. We formulate the budget adjustment problem as a state-action decision process in the reinforcement learning (RL) framework. Considering dynamics of marketing environments and some distinctive features of search auctions, we extend continuous reinforcement learning to fit the budget decision scenarios. The market utility is defined as discounted total clicks to get during the remaining period of an advertising schedule. We conduct experiments to validate and evaluate our strategy of budget adjustment with real world data from search advertising campaigns. Experimental results showed that our strategy outperforms the two other baseline strategies.