Research On Natural Interaction Of Cross-device 3D Objects Based On Augmented Reality

3D sketch and 3D object selection technology are two classic research tasks in natural interaction of 3D objects.Sketching is an essential part of content creation on the go in Augmented Reality(AR).However,at present,the methods of 3D sketch are single,the limitation of interactive ranges and condition of use,which restrict the natural interaction of 3D sketch task.To support this,we present Magic Pen,a portable input device that offers pen-based interaction in AR scenario.The interaction was achieved by a set of ultrasonic receivers that can be worn by the user and an ultrasonic transmitter embedded in a pen-shape device.The system tracks the spatial position in real time of the pen with a series of ultrasonic signal processing.An evaluation was carried out and showed that Magic Pen was highly praised by users.We also explore the design space of using Magic Pen for sketching,model construction and manipulating 3D object.It offers insights and implications for the future design of 3D interaction.Meanwhile,this study also proposes a novel 3D selection technology to solve selection problems including far and small object selection,3D occlusion,dense object selection,and batch selection problem,which is incorporated onto head-mounted displays and called PhoneCursor.PhoneCursor is designed by combining the mapping method of 3D objects to 2D image planes and the superiority of progressive refinement.An experiment based on the ISO 9241-9 standard is conducted,wherein the PhoneCursor technique is compared with the widely used and well-performing headmovement-based selection technique.Results show that PhoneCursor performs better in terms of movement time and throughput than the conventional technique.Some applications are further proposed to showcase the potential of using the PhoneCursor technique in AR scenarious.