Data governance practices for using RAG in generative AI-powered information systems supporting knowledge management (KM)

This article is part of an ongoing series looking at AI in KM, and KM in AI.

Retrieval-augmented generation (RAG)¹ improves large language model (LLM) outputs by incorporating external information retrieval before generating responses. Unlike LLMs alone that rely on static training data, RAG pulls relevant text from databases, uploaded documents, or web sources. Because of this, RAG is being increasingly used in the generative AI-powered information technology systems that are supporting knowledge management (KM).

A recent RealKM Magazine reported on research looking at how LLM outputs can be improved even further through GraphRAG² which also incorporates knowledge graphs.

A new study³ published in the Proceedings of the 59th Hawaii International Conference on System Sciences examines another important dimension of RAG use. This is that RAG introduces data-related risks including issues of quality, trust, and compliance. To address these risks, study authors Tilman Friedrich, Karl Akbari, and Daniel Fu¨rstenau set out to identify which data governance practices contribute to the success of generative AI-powered information systems using RAG. They broadly define data governance as the specification of rights, responsibilities, and controls over data assets.

Because of the newness of enterprise RAG applications and the limited availability of research, Friedrich and colleagues adopt a qualitative multiple case study design, following the eight-step method for building theories from case study research described in Kathleen M. Eisenhardt’s seminal 1989 article⁴ in The Academy of Management Review.

Findings

The study findings are summarised in Figure 1 below. Friedrich and colleagues advise that their findings suggest a strong link between data governance practices and information system success across the three quality dimensions of system, knowledge, and service quality, via six mechanisms:

1. Amplifying data discoverability during data curation.
2. Formalising quality assurance for knowledge content.
3. Defining metadata to enable additional source integration.
4. Enabling data enrichment for optimised retrieval.
5. Providing guidance for handling sensitive data.
6. Enforcing and monitoring data protection at the system level.

The contextual drivers shaping the adoption of these practices are the combination and interaction of dynamically evolving technology-induced requirements (particularly stemming from the reliance on semi-structured data) and contextual data-related challenges. While they enhance knowledge, system, and service quality, they also introduce strategic and operational trade-offs, notably due to their resource intensity and constraints on data integration for the sake of data protection.

Given the importance of high-quality proprietary unstructured data for generative AI value creation, governance becomes a strategic enabler, enhancing content richness, retrieval performance, and compliance.

Friedrich and colleagues also report that two findings diverge from expectations. Firstly, no practices were observed to align or structure data at source for improved pipeline integration, indicating that organizations prefer post-hoc transformation through technical architectures rather than proactive data design. Secondly, while procedural mechanisms (e.g., checks, classifications) dominate, relational mechanisms such as training or awareness-building are almost entirely absent. This under-representation suggests that relational mechanisms remain overlooked or under-reported in enterprise generative AI initiatives.

Header image source: Turing Commons, CC BY-SA 4.0.

References: