New AI models that seek to challenge the dominance of OpenAI’s renowned GPT 4 large language model (LLM) seem to spring up every now and then and the latest one comes from a Chinese firm called SenseTime. The company has unveiled its latest AI model called SenseNova 5.0 which claims to beat GPT 4’s performance in benchmarks.
Of course, real-life performance and benchmark results are two different things, but for what it is worth, SenseNova 5.0’s test results show that it can surpass OpenAI’s flagship AI in logical reasoning, creative writing, and more. It is also able to understand and generate human-like text better than GPT 4, giving it the ability to apply effective solutions to real-world applications.
On April 8, 2024, SenseTime introduced its most expansive model to date, SenseNova 5.0, at a Tech Day celebration in Shanghai.
This model merges the functionalities of transformer and recurrent neural network architectures. It is also trained on a rich and varied dataset comprising over 10 billion tokens from several languages and diverse sources.
PR Newswire detailed that SenseNova 5.0 was trained with over 10TB of tokens, which included a substantial quantity of synthetic data.
The model uses a ‘Mixture of Experts’ approach to manage a context window of approximately 200,000 during inference, improving its performance. This context window is significantly higher than GPT 4 Turbo’s 128,000 input. However, let’s not forget that being able to handle data effectively is a preferred performance metric over handling vast amounts of data, which remains to be seen so far.
Dr. Xu Li, Chairman and CEO of SenseTime said:
In the era of AGI, the three elements of data, algorithms, and computing power are undergoing a new evolution. The number of model parameters will increase exponentially, and the volume of data will grow massively with the introduction of multi modalities, leading to a continuous surge in demand for computing power.
