Generic Method for Crafting Deformable Interfaces to Physically Augment Smartphones

Though we live in the era of the touchscreen (tablet PCs and smart phones providing a rigid and flat interface) people and the industry are getting excited about the world of tangible 3D interfaces. This may be explained for two reasons: first, the emergence of cheap vision-based gestural interfaces conquering the space above and below the screen (but without haptic feedback), and second -and perhaps more important for the present discussion - the explosion of the 3D printing industry and the possibility for the end user to not only customise the layout of icons on a screen, but also of designing their own physical, deformable interface from scratch. Mass-produced smartphones could then be seen as bare-bone electronics devices whose shape can be physically augmented, personalised and crafted. Now, in order to introduce DIY techniques in the world of deformable input-output interfaces, it is necessary to provide a generic manufacturing/sensing method for such arbitrarily designed shapes. The goal of this paper is to demonstrate a minimally invasive method (i.e. no wiring) to physically augment rigid tablet PCs or smartphones. By putting a deformable object over the front or rear camera - this 'object' can be part of the smartphone case itself - and by making the inside of the object partially transparent, the complex light reflections can be used to recognise patterns of deformation/grasping and map them to different UI actions. A machine learning algorithm allows object shape and deformation to be designed arbitrarily, bringing the device physical personalisation at a level never reached before, with minimal interference with its original hardware.