What you look matters? Offline evaluation of advertising creatives for cold-start problem

Zhichen Zhao; Lei Li; Bowen Zhang; Meng Wang; Yuning Jiang; Li Xu; Fengkun Wang; Wei-Ying Ma

doi:10.1145/3357384.3357813

What you look matters? Offline evaluation of advertising creatives for cold-start problem

Zhichen Zhao^*
, Lei Li^*
, Bowen Zhang
, Meng Wang
, Yuning Jiang
, Li Xu
, Fengkun Wang
, Wei-Ying Ma

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

18 Citations (Scopus)

Abstract

Modern online-auction-based advertising systems utilize user and item features to automatically place ads. In order to train a model to rank the most profitable ads, new ad creatives have to be placed online for hours to receive sufficient user-click data. This corresponds to the cold-start stage. Random strategy lead to inefficiency and inferior selections of potential ads. In this paper, we analyze the effectiveness of content-based selection during the cold-start stage. Specifically, we propose Pre Evaluation of Ad Creative Model (PEAC), a novel method to evaluate and select ad creatives offline before being placed online. Our proposed PEAC utilizes the automatically extracted deep feature from ad content to predict and rank their potential online placement performance. It does not rely on any user-click data, which is scarce during the cold-starting phase. A large-scale system based on our method has been deployed in a real online advertising platform. The online A/B testing shows the ads system with PEAC pre-ranking obtains significant improvement in revenue gain compared to the prior system. Furthermore, we provide detailed analyses on what the model learned, which gives further suggestions to improve ad creative design. © 2019 Association for Computing Machinery.

Original language	English
Title of host publication	CIKM '19
Subtitle of host publication	Proceedings of the 28th ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	2605-2613
Number of pages	9
ISBN (Print)	978-1-4503-6976-3
DOIs	https://doi.org/10.1145/3357384.3357813
Publication status	Published - Nov 2019
Externally published	Yes
Event	28th ACM International Conference on Information and Knowledge Management (CIKM 2019): AI for Future Life - Beijing, China Duration: 3 Nov 2019 → 7 Nov 2019 https://www.openresearch.org/wiki/CIKM_2019

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	28th ACM International Conference on Information and Knowledge Management (CIKM 2019)
Abbreviated title	CIKM ’19
Place	China
City	Beijing
Period	3/11/19 → 7/11/19
Internet address	https://www.openresearch.org/wiki/CIKM_2019

Research Keywords

Advertisement ranking
Cold start
Deep Neural Networks

Access Output

10.1145/3357384.3357813

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Zhao, Z., Li, L., Zhang, B., Wang, M., Jiang, Y., Xu, L., Wang, F., & Ma, W.-Y. (2019). What you look matters? Offline evaluation of advertising creatives for cold-start problem. In CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 2605-2613). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3357384.3357813

Zhao, Zhichen ; Li, Lei ; Zhang, Bowen et al. / What you look matters? Offline evaluation of advertising creatives for cold-start problem. CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. pp. 2605-2613 (International Conference on Information and Knowledge Management, Proceedings).

@inproceedings{bbb60701df0847bfbc57f9c58191b296,

title = "What you look matters? Offline evaluation of advertising creatives for cold-start problem",

abstract = "Modern online-auction-based advertising systems utilize user and item features to automatically place ads. In order to train a model to rank the most profitable ads, new ad creatives have to be placed online for hours to receive sufficient user-click data. This corresponds to the cold-start stage. Random strategy lead to inefficiency and inferior selections of potential ads. In this paper, we analyze the effectiveness of content-based selection during the cold-start stage. Specifically, we propose Pre Evaluation of Ad Creative Model (PEAC), a novel method to evaluate and select ad creatives offline before being placed online. Our proposed PEAC utilizes the automatically extracted deep feature from ad content to predict and rank their potential online placement performance. It does not rely on any user-click data, which is scarce during the cold-starting phase. A large-scale system based on our method has been deployed in a real online advertising platform. The online A/B testing shows the ads system with PEAC pre-ranking obtains significant improvement in revenue gain compared to the prior system. Furthermore, we provide detailed analyses on what the model learned, which gives further suggestions to improve ad creative design. {\textcopyright} 2019 Association for Computing Machinery.",

keywords = "Advertisement ranking, Cold start, Deep Neural Networks",

author = "Zhichen Zhao and Lei Li and Bowen Zhang and Meng Wang and Yuning Jiang and Li Xu and Fengkun Wang and Wei-Ying Ma",

year = "2019",

month = nov,

doi = "10.1145/3357384.3357813",

language = "English",

isbn = "978-1-4503-6976-3",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "2605--2613",

booktitle = "CIKM '19",

address = "United States",

note = "28th ACM International Conference on Information and Knowledge Management (CIKM 2019) : AI for Future Life, CIKM {\textquoteright}19 ; Conference date: 03-11-2019 Through 07-11-2019",

url = "https://www.openresearch.org/wiki/CIKM\_2019",

}

Zhao, Z, Li, L, Zhang, B, Wang, M, Jiang, Y, Xu, L, Wang, F & Ma, W-Y 2019, What you look matters? Offline evaluation of advertising creatives for cold-start problem. in CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 2605-2613, 28th ACM International Conference on Information and Knowledge Management (CIKM 2019), Beijing, China, 3/11/19. https://doi.org/10.1145/3357384.3357813

What you look matters? Offline evaluation of advertising creatives for cold-start problem. / Zhao, Zhichen; Li, Lei; Zhang, Bowen et al.
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. p. 2605-2613 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - What you look matters? Offline evaluation of advertising creatives for cold-start problem

AU - Zhao, Zhichen

AU - Li, Lei

AU - Zhang, Bowen

AU - Wang, Meng

AU - Jiang, Yuning

AU - Xu, Li

AU - Wang, Fengkun

AU - Ma, Wei-Ying

PY - 2019/11

Y1 - 2019/11

N2 - Modern online-auction-based advertising systems utilize user and item features to automatically place ads. In order to train a model to rank the most profitable ads, new ad creatives have to be placed online for hours to receive sufficient user-click data. This corresponds to the cold-start stage. Random strategy lead to inefficiency and inferior selections of potential ads. In this paper, we analyze the effectiveness of content-based selection during the cold-start stage. Specifically, we propose Pre Evaluation of Ad Creative Model (PEAC), a novel method to evaluate and select ad creatives offline before being placed online. Our proposed PEAC utilizes the automatically extracted deep feature from ad content to predict and rank their potential online placement performance. It does not rely on any user-click data, which is scarce during the cold-starting phase. A large-scale system based on our method has been deployed in a real online advertising platform. The online A/B testing shows the ads system with PEAC pre-ranking obtains significant improvement in revenue gain compared to the prior system. Furthermore, we provide detailed analyses on what the model learned, which gives further suggestions to improve ad creative design. © 2019 Association for Computing Machinery.

AB - Modern online-auction-based advertising systems utilize user and item features to automatically place ads. In order to train a model to rank the most profitable ads, new ad creatives have to be placed online for hours to receive sufficient user-click data. This corresponds to the cold-start stage. Random strategy lead to inefficiency and inferior selections of potential ads. In this paper, we analyze the effectiveness of content-based selection during the cold-start stage. Specifically, we propose Pre Evaluation of Ad Creative Model (PEAC), a novel method to evaluate and select ad creatives offline before being placed online. Our proposed PEAC utilizes the automatically extracted deep feature from ad content to predict and rank their potential online placement performance. It does not rely on any user-click data, which is scarce during the cold-starting phase. A large-scale system based on our method has been deployed in a real online advertising platform. The online A/B testing shows the ads system with PEAC pre-ranking obtains significant improvement in revenue gain compared to the prior system. Furthermore, we provide detailed analyses on what the model learned, which gives further suggestions to improve ad creative design. © 2019 Association for Computing Machinery.

KW - Advertisement ranking

KW - Cold start

KW - Deep Neural Networks

UR - https://www.scopus.com/pages/publications/85075461594

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85075461594&origin=recordpage

U2 - 10.1145/3357384.3357813

DO - 10.1145/3357384.3357813

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-1-4503-6976-3

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 2605

EP - 2613

BT - CIKM '19

PB - Association for Computing Machinery

T2 - 28th ACM International Conference on Information and Knowledge Management (CIKM 2019)

Y2 - 3 November 2019 through 7 November 2019

ER -

Zhao Z, Li L, Zhang B, Wang M, Jiang Y, Xu L et al. What you look matters? Offline evaluation of advertising creatives for cold-start problem. In CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2019. p. 2605-2613. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3357384.3357813

What you look matters? Offline evaluation of advertising creatives for cold-start problem

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this