A Trinity Framework for Mobile Outsourced Human-Ambiance Interactions

[Motivation] Ambient intelligence senses and responses to human’s presence, intentionsand activities in the physical world. It is indispensable in the creation of smart spaces, to assistpeople with essential helps for a more intellectual life quality. We envision the following sceneof the future. Bob is a veteran, whose legs were wounded. One day after dinner, he leans onsofa and wants some entertainments. He glances over the desktop. His smart glass shows thathis close comrades-in-arms are off-line right now and the therapy tutorial downloading is notfinished yet. He then runs his eyes over the television, and a program list seamlessly pops out.He is excited about his favorite movie currently playing and the television is then automaticallyturned on to this channel. Later on a call is coming but the phone is placed on the table. Insteadof moving his body to pick up the phone, Bob simply shifts his eyes along the sliding directionof the “answer” button, stares at the “speaker” option, and then talks through the speaker. Atnight, Bob enters the bedroom and lays on bed for sleep, yet he notices that the living roomlamps are still on. He swipes one hand towards the living room. Lamps are then switched off.The key in above intriguing vision is the intelligent sensing and prompt reaction. Traditionally, people deploy dedicated devices for sophisticated sensing and use clouds for responsivereaction, but deploying such systems usually incurs substantial deployment cost, limited servicecoverage, and high-speed Internet dependence. Recently, mobile and wearable devices,e.g., smart phones, tablets, laptops, glasses, become increasingly popular with capable CPUsand sensors. In fact, they already form a side sensor network embedded in the environment,with powerful sensing, computing and communication capabilities. High penetration and richavailability of such pervasive devices inspire us to outsource ambient sensing tasks to them,and further leverage their available computing resources to form a computing edge to locallydigest the sensing data for promptly reacting to users, since they perfectly reside on the human-ambianceboundary, and leave remote clouds as the last resort for backup merely.[Challenges] However, non-trivial research questions have to be addressed in developingsuch a system: 1) Contact-less sensing. Interactions usually take contexts as input from dualsides: not merely the user’s surrounding circumstances, but also user’s subtle attentional, emotionalor behavioral intentions as in the above example. However, mobile sensors are capablefor environment sensing, instead of subtle intentions, as these sensors have very loosely or evencompletely no contacts with the user. Such a contact-less nature makes subtle sensing vulnerableto inevitable mobility and dynamics during interactions, contradicting with desired sensingsettings. 2) Scattered resources. Sensing data need to be processed in time, but recent measurementsfind it is difficult to ensure the timeliness using remote clouds due to Internet delayuncertainties. Computing edge could bring the cloud closer, but these participatory devices areonly volunteers. They have their own resource budgets and mobility patterns. Hence, it is unclearhow to efficiently coordinate them to ensure the performance and also encourage them toactively contribute. In addition, the information flow, sensingRcomputingRreacting, should be accommodated at each intermediate step. Due to the limited and scattered resources, sensing needs to be carefully determined and adapted along the flow to ensure the user’s experience.[Uniqueness] In this project, we will address above questions: from novel human intention-profiledsensing designs, to resource optimizations for on-site edge computing, and finallysensing adaptations. The idea of re-factoring human-ambiance interactions into an outsourcedsensing, computing and orchestration trinity framework is of essential novelty. It instrumentsthe necessity to bring computing to surround sensing, and further assemble them as an organicentirety to comfortably accommodate the whole interaction process in a self-sufficiencymanner. Moreover, the research perspective that focuses on intelligent sensing, resource optimization,and their adaptation is also innovative. It will not only attract wide audiences frommobile computing, but may also appeal to data analytics, systems, and many other domains.