基於數據驅動的新零售模式下社區生鮮消費關聯推薦機制研究

Research on the Recommendation Mechanism of Community Fresh Food Consumption in Data-Driven New Retail Mode

Student thesis: Doctoral Thesis

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Awarding Institution
Supervisors/Advisors
Award date15 Aug 2019

Abstract

近年來處於爆發式發展、並被認為代表未來商業發展方向的“新零售”商業模式,著重於在一定經營場景內,以強關聯的多種商業形態及商品組合、線上線下全渠道協同經營、多維度經營數據深度挖掘利用,來精准匹配用戶的個性化消費需求,從而實現商品銷售與企業經營的最高效率。

萊布尼茨(1804)的經典哲學“單子論”中,認為世界上所有的信息都包含在物體的屬性及物體間的關係中,只要知道這些屬性和關係,就能從現在推理到未來,從本地推理到任何地方,也就是未來在任何地方發生的事理論上都是可以通過推理預測出來。

無論從哲學還是物理學的辯證角度,世間萬物之間都存在或強或弱、顯性或隱性的“關聯規則”;而在社會科學研究和商業實踐中,如何通過有效的理論與技術工具,基於諸多商業要素與變量間的關聯規則來獲得更好的商業“推薦”與“預測”效果,從而提升商業經營的效率與效果,決定了商業發展迭代的方向與進程。

因為“新零售”這種深度結合數據技術的線上線下全渠道商業模式,僅依據傳統社會學領域的管理學、經濟學、統計學理論工具,難以從 “人、貨、場” 的龐大、複雜、動態、彼此相互影響的變量因素中,迅速找到穩定與收斂的邏輯規律,因此本研究增加了計算機科學和人工智能領域的數學模型、深度神經網絡工具,來使得傳統商業理論與人力原本無法分析的複雜變量間的關聯機制研究,轉化成為可結構化、向量化分析的數學計算問題。

所以本文將商業運營行為、以及消費者消費過程,視作為一個“人、貨、場”業務中的特徵數據集合的持續擬合過程;所以本文研究問題的核心,希望從“人、貨、場”這個三個包含大量“顯性與隱性主題”、“結構化與非結構化”數據結構、具有時間序列關係的特徵向量集合間計算相似度的角度,來尋找三者間可量化分析的關聯邏輯證據,從而發現在特定場景下,消費者行為與商品品類的時序性消費組合的關聯機制。

以上這種關聯邏輯關係可以簡化為一個數學模型r=f(u,i,c) 其中u代表“人”,即消費者特徵,c代表“場景”,即場景特徵,i代表“商品”,即商品特徵,f代表匹配度計算模型,r代表匹配度。

即商業運營和消費者消費行為,可描述為一個(組)消費者特徵集合,在一個(組)消費場景特徵集合下,所對應一種(組)商品特徵集合的持續匹配、決策、優化的數學擬合過程。

本文選擇“新零售社區生鮮商業”來作為具體的研究樣本。社區生鮮作為最具典型性的新零售商業形態之一,具有完整的經營發展歷史、歷史數據獲取相對便捷完整、線上線下全渠道營銷模式、消費者購買頻次高、重複購買率高、消費粘性大等特點。

本研究的數據主要採集自MONE生鮮平臺,收集方式兼顧線上和線下,即通過線上生鮮平臺收集歷史交易數據和商品數據,通過生鮮平臺線下發放的調查問卷收集用戶數據部分。其中,線上歷史交易數據集匯總描述如下表所示,含用戶8812名,商品143種,共93183個交易樣本。該交易數據集所含的有效字段為訂單號、子訂單號、下單時間、提貨方式、實付價格、數量、單價和總價等交易信息,用戶編號、用戶昵稱、部分用戶手機號和地址等用戶信息以及商品編號和商品名稱等商品信息。線下的調查問卷補充了用戶基本人口屬性以及商品品類和品質偏好等特徵,共回收問卷7850份。

本文將收集加工後的數據劃分為三部分:80%為訓練集,10%為驗證集和10%為測試集。其中訓練集用於對深度神經網絡推薦模型xDeepFM-TH進行訓練,驗證集用於對模型超參的選擇,測試集用於對模型的最終效果進行評測。

本文選取的指標包含了召回率、召回的平均排名百分比和覆蓋度指標。我們選取了多個常見的深度神經網絡推薦模型作為對比模型,如FNN, PNN, DeepFM,xDeepFM等多個模型,結果發現我們選擇的xDeepFM-TH在召回平均排名百分比、Top-5召回率和覆蓋率等所有關鍵指標上都有著最佳表現。

此外,我們將深度神經網絡系列的推薦模型在抽樣產生的負樣本和正樣本之上進一步進行實驗,同時對AUC和LogLoss兩個指標進行分析。結果表明我們構造的xDeepFM-TH模型在AUC和LogLoss兩個指標上均表現最優。此外,我們設計了三個消融實驗,來衡量用戶和商品文本向量化,場景特徵和前置關聯商品這三個設計的影響,結果表明混合模型xDeepFM-TH在除去了doc2vec,context和pre-item之後,在AvgRank,Recall@5上都出現了不同程度的下降。這進一步說明,在模型中增加屬性畫像文本化、場景信息和歷史交易信息,的確可以提升模型性能。

本文的研究貢獻在於給出了社區生鮮消費的場景下,基於“人、貨、場”三個維度影響下尋找關聯機制的完整解決方案。

首先,過往的關聯與推薦機制研究主要探討“人、貨、場”中的同一類型間,或者兩兩之間的特徵分析,較少進行完整的“消費者、場景、商品”三者之間的相互影響研究,而本文綜合考慮了這三者之間的相互影響,顯著提升了試驗結果。

其次,本文創新性的進行了多維度特徵向量數據采集與輸入方法,有效提升了模型的推薦能力;包括從建立向量化的社區消費者行為特徵詞典,社區生鮮場景特徵向量詞典,社區生鮮商品特徵向量詞典,從而使得採集獲得的大量結構化與非結構化的原始數據材料,經過過濾清洗的數據加工過程,可以形成結構化標準化的特徵數據集合,更完整的定義研究目標特徵屬性的邊界與深度,輸入模型進行訓練。

第三,本文在模型構建上也有一定創新;過往研究的數據輸入多為歷史交易記錄、忽略了近期購物所觸發的時序性影響,短期偏好影響,即時性消費行為影響。本研究的混合模型引入了深度神經網絡,使用了doc2vec、場景數據、個人歷史交易數據,並且考慮了顧客的長短期偏好,利用非結構化的信息進行輸入。

因為關於“人、貨、場”的研究無論在學術理論還是商業實踐領域屬新興領域,又涉及大量跨學科的課題研究,所以在過去的幾年論文撰寫研究過程中,本文研究課題組同時完成了一些相應的前置基礎研究(詳見本文4.12章節:已完成的關聯研究驗證),揭示了在推薦場景下,用戶和商品信息所對應的文本所具有的語義信息是非常有價值的,長期以來由於它們是非結構化信息難以被利用,但是借助NLP自然語言技術可以將其轉化為語義特徵向量。
In recent years, the “new retail” business model, which has developed rapidly and has been considered to represent the future direction of business development, focuses on the multi-format and commodity combination of strong correlation, online and offline omni-channel cooperation in a certain business context, and more dimensional business data are deeply exploited to accurately match the user's personalized consumption needs in order to achieve the highest efficiency of commodity sales and business operations.

According to Leibniz' "the Monadology", all the information is contained in the attributes of objects and relationships of objects. Thus as long as we know the attributes and relationships, we can predict the future from the present, predict any place from the local. That is to say, theoretically we can make inferences about anything.

No matter from the perspective of philosophy or physics, there are strong or weak, explicit or implicit "association rules" in all things. In the research on social scientific research and business practices, it determines the direction of business development to use theories and technological tools to have more targeted recommendations based on the association rules between business elements.

Since the "new retail" is an online and offline omni-channel business model and it is deeply integrated with data technology, it is difficult to use traditional tools in the management, economics, and statistical field to find the stable and convergent logics from complex, dynamic, and mutually influential variables. Therefore, this study adds mathematical models and deep neural network tools in the fields of computer science and artificial intelligence to make it.

Therefore, this paper regards the commercial operation behavior and the consumer consumption process as a continuous fitting process of the feature data set in the “people, goods, contexts” business; therefore, the core of the research question in this paper is from the perspective of “user(people) -- item(goods) -- contexts(environment)” -- three levels containing "explicit and implicit topics", "structured and unstructured" data, and the similarity between the sets of feature vectors with time series relationships, to find the logical evidence for quantifiable analysis and further to discover the association mechanisms between consumer behavior and the sequential consumption mix of commodity categories in a particular scenario.

The above related logical relationship can be reduced to a mathematical model r=f(u,i,c) where u stands for “person”, i.e., consumer feature; c stands for “context”, i.e., scene feature; i stands for “goods”, i.e., the commodity feature; f represents the matching degree calculation model, and r represents the matching degree.

That is, commercial operations and consumer consumption behaviors can be described as a (group) set of consumer features. Under one (group) set of consumption scene features, the corresponding (group) set of commodity features are continuously matched, determined, and optimized mathematical fitting process.

This paper selects “community fresh food consumption in new retail mode” as a specific research sample. As one of the most typical new retail business forms, community fresh food has a complete history of business development, relatively convenient and complete historical data collection, omni-channel online marketing model, high frequency of consumer purchases, high repeat purchase rate, and great consumption viscosity and other characteristics.

The data of this study is mainly collected from the MONE fresh platform. The collection method is both online and offline. The historical transaction data and commodity data are collected through the online fresh platform, and the user data is collected through the questionnaire distributed under the fresh platform. Among them, the online historical transaction data set summary description is as shown in the following table, including 8812 users, 143 kinds of goods, a total of 93183 transaction samples. The valid fields contained in the transaction data set are transaction information such as order number, sub-order number, order time, delivery method, paid price, quantity, unit price and total price, user number, user nickname, part of user mobile phone numbers and user information such as address, and product information such as product number and product name. The offline questionnaire supplemented the characteristics of the user's basic demographics, as well as the categories of goods and quality preferences. A total of 7,850 questionnaires have been collected.

In this paper, the collected data is divided into three parts: 80% for the training set, 10% for the verification set and 10% for the test set. The training set is used to train the deep neural network recommendation model xDeepFM-TH. The verification set is used to select the model super-parameters, and the test set is used to evaluate the final effect of the model.

The metrics selected in this article include the recall rate, the average percentage of the recall, and the coverage metric. We select several common deep neural network recommendation models as comparison models, such as FNN, PNN, DeepFM, xDeepFM and other models. We find that the xDeepFM-TH model has the best performance in terms of the average recall percentage, Top-5 recall rate and coverage rate and other key indicators.

In addition, we further conduct the experiment with the recommended models of the deep neural network series using the negative and positive samples generated by sampling, and we analyze the two indicators of AUC and LogLoss. The results show that the xDeepFM-TH model performs best on both AUC and LogLoss. In addition, we design three ablation experiments to measure the effects of user and product text vectorization, scene features and pre-associated goods. The results show that when the hybrid model xDeepFM-TH removes doc2vec, context and pre-item, the performances of AvgRank and Recall@5 decrease. This further illustrates that adding attribute textualization, scene information, and historical transaction information to the model can indeed improve model performance.

This paper contributes to giving a complete solution to finding the correlation mechanism under the influence of the three dimensions of “people, goods and contexts” under the scenario of fresh consumption in the community.

First of all, the past research on association and recommendation mechanism has mainly discussed the only one element, or two of three elements among "people, goods, and contexts", and less attention has been paid to three elements at the same time. This paper comprehensively considers the interaction among these three elements with significantly improvement of key indicators.

Secondly, this paper innovatively carries out the multi-dimensional feature vector data acquisition and input method, which effectively improves the recommendation accuracy of the model. It includes establishing vectorized dictionary of community consumer behavior features, vectorized dictionary of community fresh scene features, and vectorized dictionary of community fresh commodity feature vectors, so that a large number of structured and unstructured raw data materials, through the filtered and cleaned data processing process, can form a structured and standardized feature data set. We can more completely define the boundary of the research target feature attributes and train with models.

Thirdly, this paper also has some innovations in model construction. The data inputs of past research are mostly historical transaction histories without considering the sequential impact of recent shopping, short-term preference impact, and real-time consumption behavior. The hybrid model of this study introduces a deep neural network that uses doc2vec, context data, and personal historical transaction data. This model takes the customer's long- and short-term preferences into account and uses unstructured information as input.

The research on "people, goods, and contexts" is an emerging field in academic theory and business practice, and it involves a large number of interdisciplinary research projects. Thus in the past few years, the research group of this paper also completes some corresponding pre-basic studies (see Section 4.12 of this article: completed correlation research verification) at the same time. These studies reveal that the semantic information underlying the text of user and product information is very valuable in the recommendation scenarios. Since such user and product information is hard to get when it is unstructured, it can be transformed into semantic feature vectors by means of NLP natural language technology.

    Research areas

  • people, goods, and contexts, consumption behavior portrait, commodity portrait, consumption combinations, mixed model, neural networks, association analysis, collaborative filtering, recommendation system