📊 Full opportunity report: Minerva. The opposite path. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Italy’s Minerva project trained a large-scale Italian-language LLM from scratch, achieving impressive technical results but performing poorly on academic benchmarks. This challenges assumptions about investment levels needed for country-specific language models.
Italy’s Minerva-3B, a large-scale sovereign language model trained entirely from scratch on 2.5 trillion tokens with approximately 50% Italian data, scored just 4.9% on the Italian INVALSI school-exam benchmark, revealing significant challenges in achieving country-specific language understanding despite substantial investment.
The Minerva project, led by Sapienza University of Rome and supported by Italy’s national research infrastructure, built a 7-billion-parameter model trained on a massive dataset of 2.5 trillion tokens, half of which was Italian content. Despite this, Minerva-3B’s performance on the INVALSI exam was near chance levels, a result confirmed by official evaluation reports. The project aimed to demonstrate that large-scale, from-scratch training could produce a robust Italian language model, but the low exam score suggests that scale alone may not suffice for complex language understanding and knowledge depth. The empirical results contrast with the model’s impressive technical performance on benchmarks, prompting questions about the actual investment needed for effective country-specific AI models.Minerva.
The opposite
path.
Italy spent years building a European sovereign LLM from scratch. Then Minerva-3B scored 4.9% on the INVALSI Italian school exam.
Where AMÁLIA layered Portuguese specialization onto a multilingual foundation, Minerva trained from scratch on 2.5 trillion tokens with approximately 50% Italian content. Where AMÁLIA’s weights are not yet public, Minerva published weights, training data, and code as truly-open from day one. By every institutional measure, the Italian approach worked. But the empirical results contain a finding the press coverage has been quiet about — and it has implications that extend well beyond Italy.
Same problem. Opposite path.
European sovereign-LLM development has two primary architectural approaches. Italy chose from scratch with substantial native-language foundation. Portugal chose continuation pre-training of a multilingual model. The structural comparison surfaces what each commitment actually requires operationally.
The comparison is not “Italy did it better than Portugal.” Both projects respond to the same structural problem with different architectural strategies under different institutional and economic constraints. Italy’s national-AI investment is structurally larger by an order of magnitude — and Minerva is the visible artifact of that scale.
large language model training hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
4.9% on INVALSI. The bitter lesson surfaces.
In June 2024, researchers evaluated Minerva-3B on the Italian school-exam benchmark. The result was unambiguous. This is not a critique of Minerva — it is a critique of the public discourse around what Minerva’s empirical results actually demonstrate.
AI model training dataset management
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
350M to 7B. Four parameter scales, one architecture.
The Minerva model family covers four parameter tiers, each with specific training corpora. Each scale level reveals what the from-scratch path actually requires at different operating points.
Italian + English
100B English
~50% English
+ 200B code
AI model evaluation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three answers. Same question.
Minerva, AMÁLIA, and OpenEuroLLM represent the three operational answers to the European sovereign-LLM question. Each makes different architectural and institutional bets. The strategic discourse benefits from treating all three as data points in the same empirical experiment.
country-specific language AI models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three standards the movement should adopt.
The structural critique generalizes beyond Minerva. The European sovereign-LLM movement benefits from internalizing these lessons across every subsequent national project. Italy modeled the openness standard; the movement should adopt it as norm.
Minerva is one valid answer to the European sovereign-LLM question. AMÁLIA is another. OpenEuroLLM is potentially a third. The strategic discourse benefits from treating all three as data points in the same empirical experiment rather than as competing national-prestige projects. More analysis like this is needed. Not less.
Implications for European Sovereign-Language AI Strategies
This development indicates that even substantial investments in training large models from scratch may not guarantee deep language and knowledge capabilities in specific national contexts. It challenges the assumption that scale and data volume alone are sufficient, emphasizing the need for targeted investment and possibly different architectural approaches. The findings could influence future European AI policies, highlighting the importance of realistic expectations and strategic resource allocation in building effective country-specific language models.Italy’s Large-Scale Investment in Sovereign LLMs
Italy’s Minerva project represents a significant effort to develop a European sovereign language model, involving 15 researchers and access to Italy’s top supercomputing resources. Launched in 2024, it followed a strategy of training from scratch on a massive dataset, contrasting with approaches like Portugal’s AMÁLIA, which relied on continuation training of multilingual models. Despite the large-scale effort and open release of weights and data, Minerva’s performance on academic benchmarks has raised questions about the effectiveness of such investments for achieving country-specific knowledge depth.Unresolved Questions About Scale and Effectiveness
It remains unclear whether further scaling, different training methodologies, or additional data curation could improve Minerva’s performance on complex language and knowledge tasks. The long-term trajectory of the project and whether subsequent iterations will overcome current limitations are still uncertain.
Next Steps for Minerva and European Sovereign LLM Development
The Minerva team plans ongoing iterations, including continual training experiments, to enhance the model’s capabilities. Additionally, European policymakers and researchers will likely reassess investment strategies, considering the findings that scale alone may not suffice. Future benchmarks and real-world applications will determine whether the current approach can deliver practical, country-specific AI solutions.
Key Questions
Why did Minerva perform poorly on the Italian school exams?
Despite large-scale training on a substantial dataset, Minerva-3B’s low score suggests that simply increasing data volume and model size may not be enough to develop deep language understanding and knowledge in specific national contexts.
What does this mean for other European countries developing sovereign LLMs?
It indicates that countries may need to consider more targeted data, different architectures, or increased investment to achieve meaningful language and knowledge depth, rather than relying solely on scale and raw data.
Is the Minerva project still ongoing?
Yes, the team continues to iterate on the model, with plans for further training and evaluation to improve performance and address current limitations.
How does Minerva compare to other multilingual or European models?
Minerva’s approach of training from scratch on a large, language-specific dataset contrasts with models like Portugal’s AMÁLIA, which relied on continuation training of multilingual models. Its results highlight the trade-offs between scale, specialization, and performance.
What are the broader implications for AI policy in Europe?
The findings suggest that European AI strategies should incorporate realistic expectations about scale and data investment, emphasizing the importance of targeted, efficient approaches for building effective country-specific models.
Source: ThorstenMeyerAI.com
