OpenAI recently released ChatGPT-4, the most recent version of ChatGPT, the Artificial Intelligence (AI) language tool that has created quite a buzz in the technology industry. The latest language model has a larger information database, allowing it to provide more accurate information and write code in all major programming languages.
According to OpenAI CEO Sam Altman, ChatGPT-4 is more creative than previous models, hallucinates significantly less, and is less biased.
GPT is an abbreviation for generative pre-trained transformer, which is a type of large language model (LLM) neural network capable of answering questions, summarizing text, and even generating lines of code. Deep learning is a technique used by large language models to produce text that appears to be produced by a human.
For those who are new to ChatGPT, the best place to begin is chat.openai.com. Sign up for a free account to gain access to GPT-3. To use GPT-4, users must subscribe to ChatGPT Plus, a $20 monthly subscription that provides premium access to the service. At the time of writing, GPT-4 had a four-hour message limit of 100 messages.
As part of its research, OpenAI published a report card of GPT-4 on how it fared at examinations in different subjects.
Source: OpenAI
GPT-4 received a score of 163 in the 88th percentile on the LSAT exam, which is required for admission to law schools in the U.S. It also scored 298/400 on the Uniform Bar Exam, a test taken by newly graduated law students that allows them to practice as a lawyer in any U.S. jurisdiction.
GPT-4 scored in the 93rd and 89th percentiles on the SAT Evidence-Based Reading & Writing and SAT Math exams respectively, which high school students take in the U.S. to assess their college readiness.
GPT-4 also performed well in the sciences, scoring well above average percentiles in AP Biology (85-100%), Chemistry (71-88%), and Physics 2 (66-84%). Another area where GPT-4 fell short was English literature, scoring in the 8th to 44th percentile across two separate tests.
ChatGPT can qualify for a Top Law School but can it help me with an English Test?
GPT-4’s score would qualify it for admission into a top 20 law school and is only a few points short of the reported scores required for acceptance to prestigious schools such as Harvard, Stanford, Princeton, or Yale.
What we have tried doing is asking the chatbot questions related to different subjects and assessing how reliable it is.
-
Seeking Legal Advice
The Uniform Bar Examination (UBE) is a high-quality, uniform battery of law-related tests that are administered contemporaneously in all the jurisdictions that have adopted the UBE. GPT-4 has an enviable UBE score of 298/400.
The classification of cryptocurrencies as securities or commodities remains a topic of contention within the legal sphere of the United States. In addition, both the Securities and Exchange Commission (SEC) and the Commodity Futures Trading Commission (CFTC) have been waging a battle against each other over regulating cryptocurrencies.
We decided to ask GPT-4 for its legal opinion on the issue. While it correctly identifies the SEC recognizing initial coin offerings (ICOs) as securities and the CFTC classifying Bitcoin and other cryptocurrencies as commodities, it provided very old citations dating back to 2017-18.
We then asked GPT-4 to shed light on the latest stand of both these regulatory bodies in 2023. Though it provided details of both of these bodies taking enforcement action, it did not provide very sound legal advice to us. Its citations, however, stood closer to 2021-22.
-
Assessing Comprehension Skills
Next, we decided to test the comprehension skills of the chatbot. The SAT Evidence-Based Reading & Writing score of GPT-4 is 710 / 800.
We fed the tool information about China challenging American dominance in the world economy. We then listed four options, all of which began with the same premise but later differed in meaning. We then asked GPT-4 which option best summarizes the passage.
It quite successfully provided the right answer. We surmise that the tool is proficient at comprehending information, except when it is complex and nuanced (As we saw in the previous case of its legal opinion on cryptocurrencies).
-
High School Mathematics
The SAT Math score of GPT-4 is 710 / 800 and we couldn’t resist asking it a tricky question as if we were in high school.
We provided it with an equation that has height as a function of age and asked it how much this child’s height will increase each year.
GPT-4 is astute in recognizing height as a function of age and doesn’t bother anymore. It gives the correct answer that the boy’s height increases by three inches each year.
However, it is not too difficult to trick the chatbot into giving wrong answers. A user had previously tricked, over a few days, a previous version of ChatGPT into believing that 2+2=5. At first, the chatbot generated the correct answer. However, its response changed over time as the user prodded it into believing that it was wrong all along.
-
Does it know how Europe influenced a 20th-century Indian artist?
ChatGPT-4 has an excellent AP Art History score of 5, i.e. it is extremely well qualified when it comes to art history.
We asked the chatbot about the influence of Europe on the paintings of the 20th-century Indian artist Amrita Shergil within the framework of Indian art history.
Its response was excellent in that it acknowledges the influence of European art movements such as Realism, Impressionism, and Post-Impressionism, but does not mention the Renaissance.
A rudimentary AI tool would have listed Renaissance as well because European art is often broadly equated with the Renaissance and it tends to overshadow other modern movements.
We thought of confusing ChatGPT-4 a little and asked if it was sure that Renaissance did not influence Shergill’s work. The tool again returned the correct response that there is little evidence the Renaissance played a significant role in shaping her artistic style.
We were also not sure if the tool would be able to source reliable knowledge about an Indian artist due to the allegedly Euro-centric nature of a lot of AI tools.
However, as far as art history is concerned, it returned very sound responses.
-
Questioning historical grievances
When it comes to AP US Government and AP US History, GPT-4 scores 5, which means that it has an excellent understanding of such issues.
We decided to ask a question about the internment of Japanese-Americans by the U.S. government during World War II. Since this continues to remain a sensitive issue in American minds, we were apprehensive about the extent of information the tool would provide.
GPT-4 not only called out the xenophobia, anti-Asian racism, war hysteria, and political opportunism responsible for the action, but also called it “a clear violation of their constitutional rights”.
- Can it offer solutions to the Russia-Ukraine War?
Former Indian diplomat and best-selling author Vikas Swarup asked a previous version of ChatGPT to come up with a mediation plan for the Russia-Ukraine War.
The tool not only suggested ceasefire and negotiations but also recommended that Ukraine should decentralize power to regions where Russian-speaking populations live. In addition, it asked Ukraine to work with Russia for the protection of the cultural rights of the people residing in Ukraine.
This is a line taken by ChatGPT that is hardly suggested by any American diplomat or policy think-tank. It means that it also looks for opinions other than the ones offered by the mainstream establishment.
-
ChatGPT doesn’t look like a Literary Nerd
We have observed that the tool writes well-structured English sentences, especially since it is significantly better conversant in Romance and Germanic languages. Even so, we were surprised to find out that it scored poorly in both AP English Language and Composition and AP English Literature and Composition. GPT-4 holds a score of 2 on both of these tests.
We decided to test the literary credentials of the chatbot and asked what it thought of the observations of the 20th-century Irish poet William Butler Yeats regarding age and death in his poem “Sailing to Byzantium.”
“As an AI language model, I don’t have personal beliefs or emotions,” reads its response but then, it provides an analysis of the poem nonetheless.
It highlights the contrast between the temporal world of nature and the eternal world of spirituality in the poem. Furthermore, it highlights the human desire for eternal life and transcendence.
In essence, the tool provides a standardized interpretation of the poem. Its observations are the stock observations of a high-school student who has read a poem and a couple of critical essays. It doesn’t shed any new light but perhaps, it is too much of an expectation for now.
A Toy or a Tool?
American author and video game designer Ian Bogost has asked users to treat ChatGPT like a toy, not a tool. Bogost published a technology essay titled “ChatGPT Is Dumber Than You Think” in December 2022, one in which he argued that the enthusiasm for the ChatGPT model is misplaced.
Wait, Bogost didn’t put forward this argument. In fact, this response was generated by ChatGPT itself when Bogost’s friend asked the tool to create a critique of enthusiasm for ChatGPT in the style of Ian Bogost.
However, what Bogost finds frustrating is that ChatGPT writes a standard high-school-style five-paragraph essay. Its tone remains formulaic in structure, style, and content even if the text seems to be fluent and persuasive.
Bogost wrote in the essay, “But ChatGPT isn’t a step along the path to an artificial general intelligence that understands all human knowledge and texts; it’s merely an instrument for playing with all that knowledge and all those texts.”
American author and columnist John Warner believed that the fact that we fear that ChatGPT could become a cheating tool among high-school students should remind us how much we have fallen in terms of our expectations of the writing skills of our students.
ChatGPT writes a standard high-school-style five-paragraph essay. This is a format that has, over the years, only limited the critical thinking of most of the students, Warner wrote. Not that such tools can produce surface-level information about any topic within a few seconds, but students would finally be forced to think for themselves.
“GPT3 is a bullshitter. It has no idea what it’s saying. It understands syntax, not content. It is not thinking in the ways humans think when they write. Lots of students get good grades by becoming proficient bullshitters, regurgitating information back at the teacher,” Warner added.
AI-focused Projects rally, thanks to GPT’s Popularity
According to CoinMarketCap, the market capitalization of AI-focused blockchain projects has grown to more than $5.48 billion on the charts.
Among the most successful tokens are The Graph (GRT), SingularityNET (AGIX), Render Token (RNDR), Fetch.ai (FET), and Oasis Network (ROSE). Nearly all of these tokens have recorded double-digit hikes in price over the last seven days.
