一种采用机器阅读理解模型的中文分词方法

周裕林; 陈艳平; 黄瑞章; 秦永彬; 林川

doi:10.7652/xjtuxb202208010

您当前的位置：

首页 >

文章列表页 >

一种采用机器阅读理解模型的中文分词方法

—————————————————————— | 更新时间：2025-07-09

- 一种采用机器阅读理解模型的中文分词方法
- Machine Reading Comprehension Model for Chinese Word Segmentation
- 西安交通大学学报 2022年56卷第8期页码：95-103
- 作者机构：
  
  1. 公共大数据国家重点实验室,贵阳,550025
  2. 贵州大学计算机科学与技术学院,贵阳,550025
- 作者简介：
  
  : 2021-12-16。作者简介: 周裕林(1997—),男,硕士生
  陈艳平(通信作者),男,副教授。基金项目: 国家自然科学基金资助项目(62166007)
- 基金信息：
  
  : 2021-12-16。作者简介: 周裕林(1997—),男,硕士生;陈艳平(通信作者),男,副教授。基金项目: 国家自然科学基金资助项目(62166007)
- DOI：10.7652/xjtuxb202208010
  中图分类号： TP391
- 网络首发：2022-08-10，
  
  纸质出版：2022
- 稿件说明：
移动端阅览
周裕林, 陈艳平, 黄瑞章, 等. 一种采用机器阅读理解模型的中文分词方法[J]. 西安交通大学学报, 2022,56(8):95-103.

ZHOU Yulin, CHEN Yanping, HUANG Ruizhang, et al. Machine Reading Comprehension Model for Chinese Word Segmentation[J]. 2022, 56(8): 95-103.
周裕林, 陈艳平, 黄瑞章, 等. 一种采用机器阅读理解模型的中文分词方法[J]. 西安交通大学学报, 2022,56(8):95-103. DOI： 10.7652/xjtuxb202208010.

ZHOU Yulin, CHEN Yanping, HUANG Ruizhang, et al. Machine Reading Comprehension Model for Chinese Word Segmentation[J]. 2022, 56(8): 95-103. DOI： 10.7652/xjtuxb202208010.

摘要

针对中文分词序列标注模型很难获取句子的长距离语义依赖

导致输入特征使用不充分、边界样本少导致数据不平衡的问题

提出了一种基于机器阅读理解模型的中文分词方法。将序列标注任务转换成机器阅读理解任务

通过构建问题信息、文本内容和词组答案的三元组

以有效利用句子中的输入特征; 将三元组信息通过Transformer的双向编码器(BERT)进行预训练捕获上下文信息

结合二进制分类器预测词组答案; 通过改进原有的交叉熵损失函数缓解数据不平衡问题。在Bakeoff2005语料库的4个公共数据集PKU、MSRA、CITYU和AS上的实验结果表明:所提方法的F

分别为96.64%、97.8%、97.02%和96.02%

与其他主流的神经网络序列标注模型进行对比

分别提高了0.13%、0.37%、0.4%和0.08%。

Abstract

Conventional sequence models for Chinese word segmentation are difficult to encode long distance semantic dependencies of a sentence

cannot make full use of input features

and have few boundary samples for use

which leads to data imbalance. In view of this

this paper proposes a machine reading comprehension(MRC)model for Chinese word segmentation. First

the sequence labelling task for Chinese word segmentation is converted into a machine reading comprehension task. This model constructs a triple relationship among question information

text content and answers to enrich the input features. Then

the triple relationship information is pre-trained

by bidirectional encoder representation from transformers(BERT)to capture the contextual information

and a binary classifier is used to predict the word answers. Finally

the original cross-entropy loss function is improved to alleviate the data imbalance between examples. The experiment results show that machine reading comprehension model achieves F

value of 96.64%

97.8%

97.02% and 96.02% with four public datasets used: PKU

MSRA

CITYU and AS. Compared with other neural network sequence labeling models

this model improves the corresponding F

value by 0.13%

0.37%

0.4% and 0.08%.

关键词

Keywords

references

GONG Chen, LI Zhenghua, ZHANG Min, et al. Multi-grained Chinese word segmentation [C]∥Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2017: 692-703.

马海辉, 余小玲, 吕倩, 等. 一维卷积神经网络在往复式压缩机气阀故障诊断中的应用 [J]. 西安交通大学学报, 2022, 56(4): 101-108.

MA Haihui, YU Xiaoling, LÜ Qian, et al. Application of one-dimensional convolutional neural network in fault diagnosis of reciprocating compressor air valve [J]. Journal of Xi'an Jiaotong University, 2022, 56(4): 101-108.

方军, 管业鹏. 基于双编码器的会话型推荐模型 [J]. 西安交通大学学报, 2021, 55(8): 166-174.

FANG Jun, GUAN Yepeng. A session recommendation model based on dual encoders [J]. Journal of Xi'an Jiaotong University, 2021, 55(8): 166-174.

COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing(almost)from scratch [J]. The Journal of Machine Learning Research, 2011, 12: 2493-2537.

CHEN Xinchi, QIU Xipeng, ZHU Chenxi, et al. Long Short-term memory neural networks for Chinese word segmentation [C]∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2015: 1197-1206.

YAO Yushi, HUANG Zheng. Bi-directional LSTM recurrent neural network for Chinese word segmentation [C]∥ICONIP 2016: Neural Information Processing. Cham, Germany: Springer International Publishing, 2016: 345-353.

CHEN Xinchi, QIU Xipeng, ZHU Chenxi, et al. Gated recursive neural network for Chinese word segmentation [C]∥Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA, USA: ACL, 2015: 1744-1753.

CHEN Xinchi, SHI Zhan, QIU Xipeng, et al. Adversarial multi-criteria learning for Chinese word segmentation [C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2017: 1193-1203.

MA Ji, GANCHEV K, WEISS D. State-of-the-art Chinese word segmentation with Bi-LSTMs [C]∥Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2018: 4902-4908.

YANG Jie, ZHANG Yue, DONG Fei. Neural word segmentation with rich pretraining [C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2017: 839-849.

GONG Jingjing, CHEN Xinchi, GUI Tao, et al. Switch-LSTMs for multi-criteria Chinese word segmentation [C]∥Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2019: 6457-6464.

ZHOU Jianing, WANG Jianing, LIU Gongshen. Multiple character embeddings for Chinese word segmentation [C]∥Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. Stroudsburg, PA, USA: ACL, 2019: 210-216.

HE Han, WU Lei, YAN Hua, et al. Effective neural solution for multi-criteria word segmentation [M]∥Smart Intelligent Computing and Applications. Singapore: Springer Singapore, 2019: 133-142.

郭振鹏, 张起贵. 基于结合词典的CNN-BiGRU-CRF网络中文分词研究 [J]. 电子设计工程, 2021, 29(16): 64-69, 74.

GUO Zhenpeng, ZHANG Qigui. Research on Chinese segmentation of CNN-BiGRU-CRF network based on combined dictionary [J]. Electronic Design Engineering, 2021, 29(16): 64-69, 74.

DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]∥Proceedings of NAACL-HLT 2019. Stroudsburg, PA, USA: ACL, 2019: 4171-4186.

PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations [C]∥Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA, USA: ACL, 2018: 2227-2237.

DIAO Shizhe, BAI Jiaxin, SONG Yan, et al. ZEN: pre-training Chinese text encoder enhanced by N-gram representations [C]∥Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg, PA, USA: ACL, 2020: 4729-4740.

TIAN Yuanhe, SONG Yan, AO Xiang, et al. Joint Chinese word segmentation and part-of-speech tagging via two-way attentions of auto-analyzed knowledge [C]∥Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 8286-8296.

TIAN Yuanhe, SONG Yan, XIA Fei, et al. Improving Chinese word segmentation with wordhood memory networks [C]∥Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 8274-8285.

TIAN Yuanhe, CHEN Guimin, QIN Han, et al. Federated Chinese word segmentation with global character associations [C]∥Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Stroudsburg, PA, USA: ACL, 2021: 4306-4313.

LI Xiaoya, FENG Jingrong, MENG Yuxian, et al. A unified MRC framework for named entity recognition [C]∥Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 5849-5859.

LI Xiaoya, YIN Fan, SUN Zijun, et al. Entity-relation extraction as multi-turn question answering [C]∥Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2019: 1340-1350.

LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection [C]∥2017 IEEE International Conference on Computer Vision(ICCV). Piscataway, NJ, USA: IEEE, 2017: 2999-3007.

LI Buyu, LIU Yu, WANG Xiaogang. Gradient harmonized single-stage detector [C]∥Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2019: 8577-8584.

VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.

GARDNER M W, DORLING S R. Artificial neural networks(the multilayer perceptron): a review of applications in the atmospheric sciences [J]. Atmospheric Environment, 1998, 32(14/15): 2627-2636.

XIE Saining, TU Zhuowen. Holistically-nested edge detection [C]∥2015 IEEE International Conference on Computer Vision(ICCV). Piscataway, NJ, USA: IEEE, 2015: 1395-1403.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

不同板叠材料热声热机能量转换过程数值模拟研究

面向高海拔空冷发电机防晕性能演化规律的电热耦合研究

张角凹坑气膜孔对双层壁气膜冷却特性的影响

电子级超纯氮深冷空分流程节能优化研究

扇贝仿生密封腔室内超临界二氧化碳泄漏流湍动能耗散机理研究