Spain’s Ministry of Science launches first national science map based on a Cloudera-powered Open Data Lakehouse

SICTILake project: a source of knowledge to drive innovation

The Ministry of Science, Innovation and Universities (MICIU) is the Spanish government department responsible for proposing and implementing government policies on science, technological development and innovation.

The institution covers a massive volume of data from organizations, professionals, research, patents, theses, R&D grants, etc. Through the SICTILake project, it has created a national science map, which provides access to all this information in a single repository through the application of advanced data management and processing capabilities. This project benefits not only the organisation, but also society, by creating a single source of shared knowledge.

A vast and heterogeneous volume of data

Before Cloudera, two previous attempts to unify Spain’s R&D data using other technologies had failed. The challenge wasn't just the 2.7 million records; it was the lack of a “common language” between universities, patent offices, and research centers. MICIU needed a platform that could handle this complexity without sacrificing data sovereignty or security. What further complicated the project was also the heterogeneity of the sources, the need to guarantee the quality and traceability of the data, and the possibility of reusing it in future analyses and public policies, which required a robust and scalable data platform.

Implementing this solution required the development of an Open Data Lakehouse architecture,capable of unifying structured, semi-structured, and unstructured data into a single open and governed format, thereby addressing the heterogeneity of data sources through a common language that also enables contextual interpretation of the information. This was particularly relevant in the research domain, where collaboration across multiple organizations and the diversity of data recording practices across projects demand a consistent understanding of the data, while ensuring data security and sovereignty and demonstrating that protection of information is fully compatible with controlled sharing and reuse.

Data sovereignty and privacy as key pillars 

To overcome these challenges, MICIU chose Cloudera on premises. This enabled the Ministry to maintain control and sovereignty over critical public sector information, ensuring security and regulatory compliance. 

The implementation of the architecture and platform enabled the elimination of silos and the creation of a secure and governed environment. From the start of the project, more than 3,8 million records have been processed. Specifically, more than 970,000 from researchers, 670,000 from universities and centers, and 2,200,000 from research projects. This shows Cloudera's platform's ability to operate at scale while maintaining data consistency, quality, and governance in highly complex environments.

By implementing Cloudera on premises, PUEDATA, a Spain-based systems integrator of Cloudera, technology in the project has enabled the Ministry to maintain total data sovereignty and security in a government environment while effectively eliminating legacy information silos. They developed a specialized interface that allows Ministry experts to manually unify or desynchronize entities, ensuring maximum data reliability.

By building an API REST and a web search engine powered by Apache Solr, they have enabled citizens to easily cross-reference researchers, patents, and funding projects, providing a clear view of the return on scientific investment.

Driving distributed architectures in the public sector

The ministry has become a pioneering public body in the application of modern distributed architectures, committing to cutting-edge mass data processing technologies to unify information sources. Thanks to this, scientific knowledge can be easily accessed through the search engine, and for the first time it is possible to directly cross-reference research projects, different authors and existing grants in a specific field.

In addition, a modification interface has been developed so that Ministry managers can respond to requests regarding researchers' data protection rights, thereby improving governance and data quality. 

The Open Data Lakehouse architecture implemented on the Cloudera platform has provided MICIU with a technological foundation based on reliable and governed data, ready to evolve towards new use cases based on advanced analytics and artificial intelligence without redesigning the architecture. At the same time, it will serve as a catalyst for a model based on collaboration and knowledge sharing in the scientific, academic and business spheres.  Starting with the creation of the search engine as a fundamental use case, the quality of the data will continue to be improved to achieve more advanced analysis and detect collaboration networks.  

For the first time in Spain’s history, scientific knowledge is searchable and cross-referenced in real-time. MICIU can now instantly map collaboration networks and track the ROI of public R&D investment. Most importantly, the architecture is “AI-Ready,” providing a governed foundation for future predictive analytics without requiring a system redesign.

Story developed in May 2026