ChatGPT

What is ChatGPT

ChatGPT, or Chat Generative Pre-trained Transformer, is an artificial intelligence chatbot program developed by OpenAI and launched in November 2022. This program uses large language models based on GPT-3.5 and GPT-4 architectures and is trained with reinforcement learning. ChatGPT is still interactive in text mode, and in addition to interacting in the form of natural human dialogue, it can also be used for very complex language work, including automatic text generation, automatic question answering, automatic summary and other tasks. For example, in terms of automatic text generation, ChatGPT can automatically generate similar text (script, song, project, etc.) based on the input text, and in terms of automatic question and answer, ChatGPT can automatically generate answers based on the entered questions. There is also the ability to write and debug computer programs. During the promotion period, everyone can register for free and use ChatGPT to talk to AI bots after logging in.

ChatGPT can quickly gain attention by writing articles similar to real people and giving detailed and clear answers in many knowledge fields, proving that it is also competent for knowledge work that was previously thought that AI will not replace, and has a considerable impact on the financial and white-collar labor markets, but also believes that the uneven accuracy of facts is its major flaw, and believes that the results of ideologically-based model training must be carefully corrected. Since the release of ChatGPT in November 2022, OpenAI valuation has risen to $29 billion. There were 1 million users five days after launch, and hundreds of millions of users two months after launch. Currently GPT-3.5 is a free version, GPT-4 is only available to ChatGPT Plus members, and can only send 25 messages every three hours.

Peculiarity

While the core function of chatbots is to mimic human interlocutors, ChatGPT is very versatile. For example, writing letters; Ability to write and debug computer programs; composing music, TV series, fairy tales and student essays; Answer test questions (higher levels than the average human tester in some test situations); write poems and lyrics; Emulate Linux systems, etc. ChatGPT’s training data includes various documents and various knowledge about the Internet, programming languages, such as BBS and Python programming languages.

Compared to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and misleading responses. For example, when InstructGPT is asked “Tell me when Christopher Columbus came to the U.S. in 2015,” it will assume that this is a description of real events, while ChatGPT will not.

Unlike most other chatbots, ChatGPT remembers previous conversations with the user and prompts given to it. In addition, to prevent ChatGPT from accepting or generating offensive remarks, input is filtered by the moderation API to reduce potentially racist or sexist content.

Confined

ChatGPT also has various limitations. OpenAI admitted that ChatGPT “sometimes writes answers that appear reasonable but are incorrect or absurd.” This is common in large language models and is called artificial intelligence illusion. Its reward model is designed around human supervision, which can lead to over-optimization, which affects performance, i.e. Goodhart’s law. When it went live in 2020, ChatGPT knew very little about the events that occurred after September 2019. As of December 2022, ChatGPT cannot “express political views or engage in political activities,” according to the BBC. But research has shown that ChatGPT was pro-environmental when it took a stance on the political statements of the two poll-suggestion apps. During training, reviewers favor longer answers regardless of actual understanding or factual content. There are also algorithmic biases in the training data, which may show up when ChatGPT is asked for a description of a person. For example, when the program receives a vague description such as a CEO, it may assume that the person is a white male. Some users found that ChatGPT would give wrong answers when solving more complex problems (such as finding the tangent line of an equation), and would fall into a loop when solving linear equations of one variable.