LI Xiaoling,ZHANG Lixia,LI Jiawei,et al.A Tangible Interaction Method for Recognizing Local Natural Frequency in Natural Human-Computer Interaction[J].Journal of Xi'an Jiaotong University,2020,54(06):010-16+35.[doi:10.7652/xjtuxb202006002]





A Tangible Interaction Method for Recognizing Local Natural Frequency in Natural Human-Computer Interaction
李晓玲12 张丽霞12 李佳伟12 高远1 朱慧进12 郑子明12
1.西安交通大学机械工程学院, 710049, 西安; 2.西安交通大学数字医疗器械与仪器国际联合研究中心, 710049, 西安
LI Xiaoling12 ZHANG Lixia12 LI Jiawei12 GAO Yuan1 ZHU Huijin12 ZHENG Ziming12
1. School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China; 2. International Joint Research Center for Digital Medical Devices and Instruments, Xi’an Jiaotong University, Xi’an 710049, China
自然人机交互 局域固有频率 实体交互 主动声传感 手势
natural human-computer interaction local natural frequency tangible interaction active acoustic sensing gesture
TB5; TP181
为了实现人-机-环的自然交互,解决按键以及图形界面等交互方法存在的用户学习成本高、操作不便等问题,提出了一种采用主动声传感系统的局域固有频率识别的实体交互方法。该方法首先运用主动声传感技术构造多频率恒频声源信号,以激励交互实体系统按固有频率产生振动响应; 然后识别并提取局域固有频率特征,对不同时长和幅值进行参数组合,选择稳定性最佳的组合模式定义成不同交互手势; 最后选择KNN算法进行特征分类和识别。设计了以木质台面控制雾化玻璃状态的交互实验,实验结果表明:所提局域固有频率识别方法能有效地过滤噪声,对交互过程中边界条件的改变响应迅速且结果准确; 相较于传统操控方法,所提方法的最大优势在于运用了自然人机交互方式,能够有效降低用户的学习成本,使“物物皆界面,处处可交互”成为可能。
To realize the multi-channel natural interaction and solve users’ problems of high learning cost and inconvenient operation in the key interaction and graphical interface interaction methods, this paper proposes a tangible interaction method based on recognizing the local natural frequency of active acoustic sensing system. Firstly, the active acoustic sensing technology is used to construct multi-constant-frequency sound signals, and to excite interactive entity system generating vibration resonance according to the natural frequency. Then, the local natural frequency characteristics are identified and extracted, and the parameters with different durations and amplitudes are combined. The combination of parameters with the highest stability is selected and defined as different interactive gestures. Finally, the KNN classification algorithm is selected for feature learning and recognition. An application experiment using wooden countertops to control fog glass is designed. Experimental results show that the system can effectively filter noise and respond the change of boundary conditions quickly with accurate results in the interaction process. Compared with traditional interaction methods, the method has the advantage of using a natural and intuitive way to effectively reduce users’ learning cost, making the concept of “things are all interface, everywhere can be interactive” possible.


[1] 嵇柔, 巩淼森. 自然交互设计在智慧家庭中的应用研究 [J]. 包装工程, 2019, 40(22): 208-213.
JI Rou, GONG Miaomiao. Research on the application of natural interaction design in smart home [J]. Packaging Engineering, 2019, 40(22): 208-213.
[2] 史小龙, 李晓玲, 高虹霓, 等. 全息显控界面中多通道人机交互技术研究 [J]. 包装工程, 2016, 37(4): 120-124, 167.
SHI Xiaolong, LI Xiaoling, GAO Hongni, et al. Research on multi-channel human-computer interaction technology in holographic display and control interface [J]. Packaging Engineering, 2016, 37(4): 120-124, 167.
[3] 郝凝辉, 鲁晓波. 实体交互界面设计的方法思辨 [J]. 装饰, 2014(2): 34-37.
HAO Ninghui, LU Xiaobo. Speculation on the method of designing tangible interaction interface [J]. Decoration, 2014(2): 34-37.
[4] 吴旭敏, 罗颖. 物联网时代的自然交互设计 [J]. 设计艺术研究, 2013, 3(5): 5-8.
WU Xumin, LUO Ying. Natural interaction design in the age of IoT [J]. Design Art Research, 2013, 3(5): 5-8.
[5] MAHER M L, LEE L, CARROLL J M. Designing for gesture and tangible interaction [M]. San Rafael, CA, USA: Morgan & Claypool Publishers, 2017: 1-11.
[6] LI Xiaoling, HAN Feng, SUN Xiuwen, et al. Bracelet: arms-down selection for Kinect mid-air gesture [J]. Behaviour & Information Technology, 2019, 38(4): 401-409.
[7] ISHII H, ULLMER B. Tangible bits: towards seamless interfaces between people, bits and atoms [C]∥Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 1997: 234-241.
[8] PARK M, EUNE J, KIM S. Tangible user interface design for home automation energy management appliances [C]∥ACM SIGGRAPH Asia 2010 Posters. New York, USA: ACM, 2010: 12.
[9] MAYANK G, JACOB W, SHWETAK P. GripSense: using built-in sensors to detect hand posture and pressure on commodity mobile phones [C]∥Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. New York, USA: ACM, 2012: 545-554.
[10] MAKOTO O, BUNTAROU S, JIRO T. Touch & activate: adding interactivity to existing objects using active acoustic sensing [C]∥Proceedings of the 26th annual ACM symposium on User Interface Software and Technology. New York, USA: ACM, 2013: 31-40.
[11] TADA K, TAKAHASHI S, SHIZUKI B. Smart home cards: tangible programming with paper cards [C]∥Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. New York, USA: ACM, 2016: 381-384.
[12] KATAKURA S, WATANABE K. ProtoHole: prototyping interactive 3D printed objects using holes and acoustic sensing [C]∥Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 2018: LBW112.
[13] VAN ROOIJ T, BAKKER S. Plux: exploring light settings through hybrid control [C]∥Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. New York, USA: ACM, 2018: 205-210.
[14] 张汉丹. 基于实体用户界面的音乐情感可视化系统及用户体验研究 [D]. 哈尔滨: 哈尔滨工业大学, 2016: 7-8.
[15] 陈怡, 刘振生. 基于实体交互的学龄前儿童早教产品设计机会 [J]. 设计, 2018(19): 14-16.
CHEN Yi, LIU Zhengsheng. Opportunities for designing early childhood education products based on physical interaction [J]. Design, 2018(19): 14-16.
[16] GONG C S A, SU C H S, CHUANG Y C, et al. Feature extraction of rotating apparatus using acoustic sensing technology [C]∥2019 11th International Conference on Ubiquitous and Future Networks. Piscataway, NJ, USA: IEEE, 2019: 254-256.
[17] GIERAD L, XIAO R, HARRISON C. ViBand: high-fidelity bio-acoustic sensing using commodity smartwatch accelerometers [C]∥Proceedings of the 29th Annual Symposium on User Interface Software and Technology. New York, USA: ACM, 2016: 321-333.
[18] COLLINS T. Active acoustic touch interface [J]. Electronics Letters, 2009, 45(20): 1055-1056.
[19] ZHANG Cheng, XUE Qiuyue, WAGHMARE A, et al. Fingerping: recognizing fine-grained hand poses using active acoustic on-body sensing [C]∥Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 2018: 20182105213008.
[20] LUO Gan, CHEN Mingshi, LI Ping, et al. Soundwrite ii: ambient acoustic sensing for noise tolerant device-free gesture recognition [C]∥2017 IEEE 23rd International Conference on Parallel and Distributed Systems. Piscataway, NJ, USA: IEEE, 2017: 121-126.
[21] 胡玉勇, 吕明, 王时英. 关于敲击法测量材料固有频率及弹性模量的研究 [J]. 机械工程与自动化, 2010, 5(10): 88-89.
HU Yongjun, Lü Ming, WANG Shiying. Research on measuring natural frequency and elastic modulus of materials by percussion [J]. Mechanical Engineering and Automation, 2010, 5(10): 88-89.
[22] 李锁斌, 窦益华, 陈天宁, 等. 局域共振型周期结构振动带隙形成机理 [J]. 西安交通大学学报, 2019, 53(6): 169-175, 182.
LI Suobin, DOU Yihua, CHEN Tianning, et al. Formation mechanisms of vibration band gaps in locally resonant periodic structures [J]. Journal of Xi’an Jiaotong University, 2019, 53(6): 169-175, 182.
[23] 张军锋. 手动变速器齿轮敲击噪声的实验分析 [D]. 上海: 上海交通大学, 2012: 14-16.
[24] 白杨. 基于数据挖掘的TDOA定位技术 [D]. 北京: 北京邮电大学, 2015: 17-25.
[25] MULLER A C, GUIDO S. Introduction to machine learning with python: a guide for data scientists [M]. Sebastopol, CA, USA: O’Reilly Media, 2016: 33-123.


收稿日期: 2019-10-04。 作者简介: 李晓玲(1971—),女,副教授。 基金项目: 陕西省重点研发计划资助项目(2018GY-142)。
更新日期/Last Update: 2020-06-10