Facilitating access to Retrievel Augmented Generation (RAG) in the Netherlands

The NCC Netherlands aims at facilitating knowledge sharing in the topics of HPC, HPDA, and AI among all interested national parties, including research institutes, SMEs, large industries, public administration, and society in general. It also provides access to these competencies for other stakeholders at the international level and serves as a driver for innovation. The NCC Netherlands acts as a starting point for accessing HPC, HPDA, and AI resources within the country, supporting projects involving fine-tuning a large language model, semantic search, and retrieval-augmented generation.

Collaboration with Zeta Alpha

Zeta Alpha, an advanced AI research and product lab located in Amsterdam Science Park, is building an Enterprise Retrieval-Augmented Generation (RAG) and AI Search platform for knowledge-intensive organizations. This platform enables researchers, analysts, and decision-makers to efficiently reuse both internal and external knowledge using Generative AI, securely deployed in enterprise environments.

To power its flagship SaaS product, the “Neural Discovery Platform,” Zeta Alpha combines deep-learning-based natural language processing with semantic search, question answering, summarization, and recommendation systems. The result is a powerful AI knowledge management solution that synthesizes scattered information from scientific papers, internal documentation, source code, and practical use cases into actionable insights.

Zeta Alpha focuses heavily on developing and improving domain-specific models for large language model (LLM AI) applications. Their goal is to boost the accuracy and reliability of RAG agents in highly specialized areas such as legal search, chemistry, high-tech manufacturing, life sciences, and R&D. Fine-tuning large language models for these use cases is a complex task that demands high-performance infrastructure.

The role of EuroCC Netherlands

With support from EuroCC Netherlands, Zeta Alpha was able to access expert knowledge and national high-performance computing (HPC) resources critical to the project. Rather than relying solely on commercial cloud providers, the collaboration gave Zeta Alpha access to HPC expertise, consultancy, and high-end GPU infrastructure. This enabled more efficient exploration of performance, scalability, and parallel I/O of LLM-based systems in an HPC setting.

This partnership was instrumental in tackling key technical challenges, such as generating high-quality embeddings for domain-specific semantic search. Fine-tuned embedding models are central to improving the performance of Retrieval-Augmented Generation pipelines, as they determine how accurately relevant content is retrieved, reduce hallucinations, and increase organizational trust in the AI output.

Achievements and technical success in rapidly experimenting with different Large Language Models

Through the resources of EuroCC Netherlands, Zeta Alpha successfully developed and open-sourced the “Zeta-Alpha-E5-Mistral” model, which quickly achieved a competitive score on the global Massive Text Embedding Benchmark (MTEB) Leaderboard. The accompanying documentation ensures reproducibility of the results, further contributing to the open AI research community.

This HPC-enabled fine-tuning workflow allowed Zeta Alpha to:

• Rapidly experiment with different foundation LLMs
• Train customized embedding models tailored to specific business domains and languages
• Achieve high performance across multilingual retrieval tasks relevant to European industry

The project is ongoing, with plans to continue improving multilingual performance and retrieval quality across more sectors.

Business impact through fine tuning a large language model (LLM) 

Using HPC to optimize LLM model development has significantly accelerated Zeta Alpha’s innovation roadmap. The ability to fine-tune models at scale ensures frequent updates, improved model accuracy, and enhanced user experience for enterprise-facing applications. It also enables informed investment decisions backed by solid proof-of-concept results.

By contributing to the European ecosystem of Neural Search and AI-powered embedding solutions, Zeta Alpha helps further the EU’s goals of digital sovereignty and technological independence. The success of this project demonstrates how HPC resources can be a strategic asset for startups and enterprises focused on cutting-edge AI applications.

Success story highlights

• Proof of concept conceptual exploration
• Exploration of performance, efficiency, and accuracy
• Reproducible workflow development

Technologies used

• High-Performance Computing (HPC)
• Artificial Intelligence (AI)
• High-end GPU infrastructure
• Semantic search and retrieval-augmented generation pipelines
• Large language model fine-tuning

Benefits realized

• Fine-tuning large language models on HPC accelerates development of high-performing AI agents
• Access to EuroCC Netherlands’ supercomputing infrastructure reduces cost and time-to-insight
• Semantic search quality and LLM AI accuracy are greatly enhanced for domain-specific applications
• RAG system performance improves through tailored vector embeddings, ensuring trust and reliability in enterprise AI tools

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 951732, supported by the European Union’s Horizon 2020 research and innovation program.