ChatGPT的工作原理、关键技术及未来发展趋势

秦涛; 杜尚恒; 常元元; 王晨旭

doi:10.7652/xjtuxb202401001

您当前的位置：

首页 >

文章列表页 >

ChatGPT的工作原理、关键技术及未来发展趋势

更新时间：2025-05-13

- ChatGPT的工作原理、关键技术及未来发展趋势
  增强出版
- Principles, Key Technologies and Emerging Trends of ChatGPT
- 西安交通大学学报 2024年58卷第1期页码：1-12
- 作者机构：
  
  西安交通大学智能网络与网络安全教育部重点实验室,西安,710049
- 作者简介：
  
  : 2023-06-25。作者简介: 秦涛(1982—),男,教授,博士生导师。基金项目: 国家自然科学基金资助项目(62172324)
- 基金信息：
  
  : 2023-06-25。作者简介: 秦涛(1982—),男,教授,博士生导师。基金项目: 国家自然科学基金资助项目(62172324);陕西省重点研发计划资助项目(2023-YBGY-269,2022QCY-LL-33HZ)。
- DOI：10.7652/xjtuxb202401001
  中图分类号： TP391.9
- 网络首发：2024-01-10，
  
  纸质出版：2024
- 稿件说明：
移动端阅览
秦涛, 杜尚恒, 常元元, 等. ChatGPT的工作原理、关键技术及未来发展趋势[J]. 西安交通大学学报, 2024,58(1):1-12.

QIN Tao, DU Shangheng, CHANG Yuanyuan, et al. Principles, Key Technologies and Emerging Trends of ChatGPT[J]. 2024, 58(1): 1-12.
秦涛, 杜尚恒, 常元元, 等. ChatGPT的工作原理、关键技术及未来发展趋势[J]. 西安交通大学学报, 2024,58(1):1-12. DOI： 10.7652/xjtuxb202401001.

QIN Tao, DU Shangheng, CHANG Yuanyuan, et al. Principles, Key Technologies and Emerging Trends of ChatGPT[J]. 2024, 58(1): 1-12. DOI： 10.7652/xjtuxb202401001.

摘要

ChatGPT是自然语言处理领域的一项重要技术突破

专注于对话生成任务

在多种任务中表现出卓越的性能。主要探讨ChatGPT的演变历程、关键技术

并分析了其未来可能的发展方向。首先

介绍了ChatGPT的模型架构和技术演进过程。随后

重点讨论了ChatGPT的关键技术

包括提示学习与指令微调、思维链、人类反馈强化学习。然后

分析了由于基于概率生成原理所造成的固有局限

包括事实性错误、垂直领域深度性弱、潜在的恶意应用风险、可解释性及模型实时性差等。最后

探讨了其在典型应用中存在的问题和相应的解决途径

包括在训练评估过程中考虑道德和安全性因素

以降低潜在风险; 结合外部专家知识和迁移学习

以提高模型对特定领域的理解能力

更好地适应特定任务场景; 引入多模态数据

以提高模型信息理解能力

增强模型通用性和泛化性。通过对ChatGPT模型框架、技术演变与关键技术的分析

为深入理解ChatGPT提供帮助; 结合原理分析其固有缺陷

并结合实际应用中存在的问题

挖掘未来可能的研究方向

为自然语言处理领域的深入研究提供有益参考。

Abstract

ChatGPT has emerged as a significant advancement in natural language processing

specifically in the domain of dialogue generation

and has achieved excellent performance in many areas. This paper aims to explore its architecture

underlying technologies

and potential areas for further investigation. The paper begins by discussing the architecture and technology evolution process. Next

the focus shifts to a comprehensive analysis of the key technologies

including the prompt learning and instruction fine-tuning

chain of thought and reinforcement learning through human feedback. Furthermore

the paper addresses the limitations of ChatGPT stemming from its probabilistic generation principles

including factual errors

poor performance in specific domain

potential malicious risk

poor interpretability and real-time. Finally

the paper outlines possible research directions based on the practical challenges observed in real-world applications

including the ethical and safety factors in the training process to reduce potential risks. Additionally

integrating external expert knowledge and employing transfer learning methods are proposed to enhance ChatGPT's performance in domain-specific tasks. Moreover

improving its information understanding capabilities based on the multimodal data is also considered as a notable avenue for development. By providing an in-depth analysis of ChatGPT's framework and key technologies

this paper aims to foster a deeper understanding of the system and also presents potential research directions to inspire further investigation in the field.

关键词

Keywords

references

郝亚洲, 郑庆华, 陈艳平, 等. 面向网络舆情数据的异常行为识别 [J]. 计算机研究与发展, 2016, 53(3): 611-620.

HAO Yazhou, ZHENG Qinghua, CHEN Yanping, et al. Recognition of abnormal behavior based on data of public opinion on the web [J]. Journal of Computer Research and Development, 2016, 53(3): 611-620.

何炎祥, 孙松涛, 牛菲菲, 等. 用于微博情感分析的一种情感语义增强的深度学习模型 [J]. 计算机学报, 2017, 40(4): 773-790.

HE Yanxiang, SUN Songtao, NIU Feifei, et al. A deep learning model enhanced with emotion semantics for microblog sentiment analysis [J]. Chinese Journal of Computers, 2017, 40(4): 773-790.

RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors [J]. Nature, 1986, 323(6088): 533-536.

奚雪峰, 周国栋. 面向自然语言处理的深度学习研究 [J].自动化学报, 2016, 42(10): 1445-1465.

XI Xuefeng, ZHOU Guodong. A survey on deep learning for natural language processing [J]. Acta Automatica Sinica, 2016, 42(10): 1445-1465.

QIU Xipeng, SUN Tianxiang, XU Yige, et al. Pre-trained models for natural language processing: a survey [J]. Science China Technological Sciences, 2020, 63(10): 1872-1897.

WEISS K, KHOSHGOFTAAR T M, WANG Dingding. A survey of transfer learning [J]. Journal of Big Data, 2016, 3(1): 9.

MENON T, PFEFFER J. Valuing internal vs. external knowledge: explaining the preference for outsiders [J]. Management Science, 2003, 49(4): 497-513.

BALTRUAITIS T, AHUJA C, MORENCY L P. Multimodal machine learning: a survey and taxonomy [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(2): 423-443.

RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training [EB/OL].[2023-03-14]. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.

THOMPSON A D. What's in my AI? [EB/OL]. [2023-03-12]. https://lifearchitect.ai/whats-in-my-ai/.

DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA, USA: ACL, 2019: 4171-4186.

LEWIS M, LIUYinhan, GOYAL N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 7871-7880.

RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners [EB/OL]. [2023-03-06]. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.

LIU Zhuang, LIN W, SHI Ya, et al. A robustly optimized BERT pre-training approach with post-training [C]//Proceedings of the 20th Chinese National Conference on Computational Linguistics. Stroudsburg, PA, USA: ACL, 2021: 1218-1227.

SANH V, WEBSON A, RAFFEL C, et al. Multitask prompted training enables zero-shot task generalization [C]//ICLR 2022. New York, USA: ICLR, 2022: 1-216.

WANG T, ROBERTS A, HESSLOW D, et al. What language model architecture and pretraining objective works best for zero-shot generalization? [C]//Proceedings of the 39th International Conference on Machine Learning. Chia Laguna Resort, Sardinia, Italy: PMLR, 2022: 22964-22984.

LI X L, LIANG P. Prefix-tuning: optimizing continuous prompts for generation [C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg, PA, USA: ACL, 2021: 4582-4597.

RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners [EB/OL]. [2023-04-18]. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.

ZHANG Yizhe, SUN Siqi, GALLEY M, et al. DIALOGPT: large-scale generative pre-training for conversational response generation [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Stroudsburg, PA, USA: ACL, 2020: 270-278.

GU X, YOO K M, HA J W. Dialogbert: discourse-aware response generation via learning to recover and rank utterances [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto,CA,USA:AAAI, 2021, 35(14): 12911-12919.

BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners [C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2020: 1877-1901.

KAPLAN J, MCCANDLISH S, HENIGHAN T, et al. Scaling laws for neural language models [EB/OL].(2020-01-23)[2023-04-10]. https://arxiv.org/abs/2001.08361.

HOFFMANN J, BORGEAUD S, MENSCH A, et al. Training compute-optimal large language models [EB/OL].(2022-03-29)[2023-04-18]. https://arxiv.org/abs/2203.15556.

CHRISTIANO P F, LEIKE J, BROWN T B, et al. Deep reinforcement learning from human preferences [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 4302-4310.

SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms [EB/OL].(2017-08-28)[2023-04-30]. https://arxiv.org/abs/1707.06347.

STIENNON N, OUYANG Long, WU J, et al. Learning to summarize from human feedback [C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2020: 3008-3021.

OUYANG Long, WU J, JIANG Xu, et al. Training language models to follow instructions with human feedback [C]//Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2022: 27730-27744.

SCHICK T, SCHÜTZE H.Exploiting cloze-questions for few-shot text classification and natural language inference [C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Stroudsburg, PA, USA: ACL, 2021: 255-269.

WEI J, BOSMA M, ZHAO V, et al.Finetuned language models are zero-shot learners [C]//ICLR 2022. New York, USA: ICLR, 2021: 6255.

CHUNG H W, HOU Le, LONGPRE S, et al. Scaling instruction-finetuned language models [EB/OL].(2022-12-06)[2023-05-06]. https://arxiv.org/abs/2210.11416.

MUENNIGHOFF N, WANG T, SUTAWIKA L, et al. Crosslingual generalization through multitask finetuning [C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2023: 15991-16111.

WEI J, TAY Y, BOMMASANI R, et al. Emergent abilities of large language models [EB/OL].(2022-10-26)[2023-05-28]. https://arxiv.org/abs/2206.07682.

WANG Y, YAO Q, KWOK J T, et al. Generalizing from a few examples: a survey on few-shot learning [J]. ACM computing surveys(csur), 2020, 53(3): 1-34.

WEI J, WANG Xuezhi, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models [C]//Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2022: 24824-24837.

FU Yao, PENG Hao, KHOT T. How does GPT obtain its ability? Tracing emergent abilities of language models to their sources [EB/OL].(2022-12-01)[2023-05-21]. https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30alab9e3e36fa1dc1.

ZHENG Rui, DOU Shihan, GAO Songyang, et al. Secrets of RLHF in large language models: part Ⅰ PPO [EB/OL].(2023-06-18)[2023-06-19]. https: //arxiv.org/abs/2307.04964.

ZHANG Susan, ROLLER S, GOYAL N, et al. OPT: open pre-trained transformer language models [EB/OL].(2022-06-21)[2023-04-17]. https://arxiv.org/abs/2205.01068.

ASKELL A, BAI Yuntao, CHEN Anna, et al. A general language assistant as a laboratory for alignment [EB/OL].(2021-12-09)[2023-06-18]. https://arxiv.org/abs/2112.00861.

BAI Yuntao, JONES A, NDOUSSE K, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback [EB/OL].(2022-04-12)[2023-06-02]. https://arxiv.org/abs/2204.05862.

GLAESE A, MCALEESE N, TREBACZ M, et al. Improving alignment of dialogue agents via targeted human judgements [EB/OL].(2022-09-28)[2023-06-09]. https://arxiv.org/abs/2209.14375.

刘全, 翟建伟, 章宗长, 等. 深度强化学习综述 [J]. 计算机学报, 2018, 41(1): 1-27.

LIU Quan, ZHAI Jianwei, ZHANG Zongchang, et al. A survey on deep reinforcement learning [J]. Chinese Journal of Computers, 2018, 41(1): 1-27.

RENS G, RASKIN J F. Learning non-Markovian reward models in MDPs [J]. arXiv preprint arXiv:2001.09293, 2020.

YE Junjie, CHEN Xuanting, XU Nuo, et al. A comprehensive capability analysis of GPT-3 and GPT-3.5 series models [EB/OL].(2023-03-18)[2023-06-11]. https://arxiv.org/abs/2303.10420.

KENTON Z, EVERITT T, WEIDINGER L, et al. Alignment of language agents [EB/OL].(2021-03-26)[2023-05-18]. https://arxiv.org/abs/2103.14659.

ZHENG Rui, XI Zhiheng, LIU Qin, et al. Characterizing the impacts of instances on robustness [C]//Findings of the Association for Computational Linguistics: ACL 2023. Stroudsburg, PA, USA: ACL, 2023: 2314-2332.

BOLOOR A, GARIMELLA K, HE Xin, et al. Attacking vision-based perception in end-to-end autonomous driving models [J]. Journal of Systems Architecture, 2020, 110: 101766.

GU Tianyu, DOLAN-GAVITT B, GARG S. BadNets: identifying vulnerabilities in the machine learning model supply chain [EB/OL].(2019-03-11)[2023-06-20]. https://arxiv.org/abs/1708.06733.

CHEN Xiaoyi, SALEM A, CHEN Dingfan, et al. BadNL: backdoor attacks against NLP models with semantic-preserving improvements [C]//Annual Computer Security Applications Conference. New York, NY, USA: Association for Computing Machinery, 2021: 554-569.

PEREZ F, RIBEIRO I. Ignore previous prompt: attack techniques for language models [EB/OL].(2022-11-17)[2023-05-22]. https://arxiv.org/abs/2211.09527.

BORJI A. A categorical archive of ChatGPT failures [EB/OL].(2023-04-03)[2023-05-28]. https://arxiv.org/abs/2302.03494.

CALISKAN A, BRYSON J J, NARAYANAN A. Semantics derived automatically from language corpora contain human-like biases [J]. Science, 2017, 356(6334): 183-186.

GUOYue, YANG Yi, ABBASI A. Auto-debias: debiasing masked language models with automated biased prompts [C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2022: 1012-1023.

HUANG Xiaowei, RUAN Wenjie, HUANG Wei, et al. A survey of safety and trustworthiness of large language models through the lens of verification and validation [EB/OL].(2023-08-27)[2023-06-30]. https://arxiv.org/abs/2305.11391.

QIN Yujia, HU Shengding, LIN Yankai, et al. Tool learning with foundation models [EB/OL].(2021-06-15)[2023-06-30]. https://arxiv.org/abs/2304.08354.

ZHUO T Y, HUANG Yujin, CHEN Chunyang, et al. Red teaming ChatGPT via jailbreaking: bias, robustness, reliability and toxicity [EB/OL].(2021-05-29)[2023-06-19]. https://arxiv.org/abs/2301.12867.

桑基韬, 于剑. 从ChatGPT看AI未来趋势和挑战 [J]. 计算机研究与发展, 2023, 60(6): 1191-1201.

SANG Jitao, YU Jian. ChatGPT: a glimpse into AI's future [J]. Journal of Computer Research and Development, 2023, 60(6): 1191-1201.

LAZARIDOU A, GRIBOVSKAYA E, STOKOWIEC W, et al. Internet-augmented language models through few-shot prompting for open-domain question answering [EB/OL].(2022-05-23)[2023-07-01]. https://arxiv.org/abs/2203.05115.

MADAAN A, TANDON N, CLARK P, et al.Memory-assisted prompt editing to improve GPT-3 after deployment [C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2022: 2833-2861.

MENG K, BAU D, ANDONIAN A, et al. Locating and editing factual associations in GPT [C]//Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2022: 17359-17372.

BANG Yejin, CAHYAWIJAYA S, LEE N, et al. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity [EB/OL].(2021-02-28)[2023-06-05]. https://arxiv.org/abs/2302.04023.

FELTEN E, RAJ M, SEAMANS R. How will language modelers like ChatGPT affect occupations and industries? [EB/OL].(2021-03-18)[2023-05-31]. https://arxiv.org/abs/2303.01157.

孙颖, 丁卫平, 黄嘉爽, 等. RCAR-UNet:基于粗糙通道注意力机制的视网膜血管分割网络 [J]. 计算机研究与发展, 2023, 60(4): 947-961.

SUN Ying, DING Weiping, HUANG Jiashuang, et al. RCAR-UNet: retinal vessels segmentation network based on rough channel attention mechanism [J]. Journal of Computer Research and Development, 2023, 60(4): 947-961.

LI Junyi, TANG Tianyi, ZHAO W X, et al. Pretrained language models for text generation: a survey [EB/OL].(2021-05-25)[2023-06-22]. https://arxiv.org/abs/2105.10311.

俞凯, 陈露, 陈博, 等. 任务型人机对话系统中的认知技术: 概念、进展及其未来 [J]. 计算机学报, 2015, 38(12): 2333-2348.

YU Kai, CHEN Lu, CHEN Bo, et al. Cognitive technology in task-oriented dialogue systems: concepts, advances and future [J]. Chinese Journal of Computers, 2015, 38(12): 2333-2348.

曾毅, 刘成林, 谭铁牛. 类脑智能研究的回顾与展望 [J]. 计算机学报, 2016, 39(1): 212-223.

ZENG Yi, LIU Chenglin, TAN Tieniu. Retrospect and outlook of brain-inspired intelligence research [J]. Chinese Journal of Computers, 2016, 39(1): 212-223.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

应用深度强化学习的电池热管理系统控制策略

面向飞行器智能协同控制的分层双时延策略梯度强化学习方法

应用深度强化学习的电池热管理系统控制策略

联合遗传算法和强化学习的虚拟网络功能映射与调度方法

考虑驾驶风格的混合动力汽车强化学习能量管理策略