Google recently unveiled its latest AI model Gemini which comes in three tiers: Nano, Pro, and Ultra. Gemini Ultra, the most powerful among the three, outperforms OpenAI’s GPT in almost all industry-standard benchmarks, as per the company’s internal sources.
The company insiders report that Gemini Ultra beats ChatGPT in 30 out of the 32 widely used academic benchmarks for testing Large Language Models (LLM). This may allow Google to turn things around as the search engine giant has been accused of lagging behind its AI competitor OpenAI, whose GPT model is known as the most powerful and popular in the industry.
According to a report from Insider, Gemini Ultra scored 90% in the MMLU (massive multitask language understanding) benchmark, becoming the first model to outperform human experts. MMLU comprises 57 subjects such as math, physics, history, law, medicine, and ethics for testing both world knowledge and problem-solving abilities.
Google’s Gemini Nano and Pro AI models have already been deployed in the company’s product while Ultra is set to release in early 2024. Gemini Nano powers AI features in Google’s Pixel 8 Pro smartphone and Gemini Pro can be accessed through the company’s Bard chatbot. Gemini Pro will also be available to enterprise customers and AI studio developers through Google’s Vertex AI program.
According to Eli Collins, the Vice President of Product at DeepMind, the Google division responsible for developing the AI platform, Gemini Ultra possesses the capability to understand “nuanced” information across various formats such as text, images, audio, and code.
Collins revealed that some of the data utilized in creating the application was sourced from publicly available web content. Nevertheless, the company did not explicitly disclose the specific origins used to train the AI.
