基础模型
外观
基础模型(英語:foundation model或base model)指一类大型机器学习模型[1],它们经大规模数据训练而成(通常以自监督学习或半监督学习方式进行)[2],以适应各种下游任务[3][4]。基础模型帮助实现了人工智能系统构建方式的重大革新,例如为聊天机器人和其他面向用户的人工智能提供支持。斯坦福人类中心人工智能研究所(Stanford Institute for Human-Centered Artificial Intelligence)旗下的基础模型研究中心(Center for Research on Foundation Models,简称CRFM)推广了“基础模型”这一术语的使用。[3]
早期的基础模型包括一些预训练语言模型,比如Google开发的BERT和各种早期的GPT基础模型,特别是OpenAI的“GPT-n”系列模型。这类用途广泛的模型可以通过进一步开发以适用于特定的任务或领域。[5]
除文本模型外,还先后诞生了各种视觉或多模式的基础模型,如DALL-E、Flamingo[6]、Florence和NOOR[7]等。视觉基础模型(visual foundation model,简称VFM)已与基于文本的大型语言模型相结合以构建适应特定任务的复杂模型。[8]此外,还有Meta AI开发的用于通用图像分割的Segment Anything[9]以及Google DeepMind开发的强化学习智能体Gato等。[10] [11]
参考文献
- ^ Perrigo, Billy. The A to Z of Artificial Intelligence. Time. 13 April 2023 [22 May 2023]. (原始内容存档于2023-06-16).
- ^ Goled, Shraddha. Self-Supervised Learning Vs Semi-Supervised Learning: How They Differ. Analytics India Magazine. 7 May 2021 [22 May 2023]. (原始内容存档于2023-06-18).
- ^ 3.0 3.1 Introducing the Center for Research on Foundation Models (CRFM). Stanford HAI. [11 June 2022]. (原始内容存档于2023-06-04).
- ^ Goldman, Sharon. Foundation models: 2022's AI paradigm shift. VentureBeat. 2022-09-13 [2022-10-24]. (原始内容存档于2023-11-28).
- ^ Steinberg, Ethan; Jung, Ken; Fries, Jason A.; Corbin, Conor K.; Pfohl, Stephen R.; Shah, Nigam H. Language models are an effective representation learning technique for electronic health record data. Journal of Biomedical Informatics. January 2021, 113: 103637. ISSN 1532-0480. PMC 7863633 . PMID 33290879. doi:10.1016/j.jbi.2020.103637.
- ^ Tackling multiple tasks with a single visual language model, 28 April 2022 [13 June 2022], (原始内容存档于2022-04-28)
- ^ Technology Innovation Institute Announces Launch of NOOR, the World's Largest Arabic NLP Model. [2023-12-07]. (原始内容存档于2023-01-15).
- ^ Chenfei Wu; et al. Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models. Cornell University. 2023. arXiv:2303.04671 .
- ^ Segment Anything | Meta AI. segment-anything.com. [2023-06-21]. (原始内容存档于2023-12-11) (英语).
- ^ A Generalist Agent. www.deepmind.com. [2023-06-21]. (原始内容存档于2022-08-02) (英语).
- ^ RoboCat: A self-improving robotic agent. www.deepmind.com. [2023-06-21]. (原始内容存档于2023-10-06) (英语).