The knowledge systems nobody uses

Data isn't knowledge

When we looked inside the nonprofit's shared drive, we found 96 documents. Twenty named an owner. Seventy-six were flagged as stale. The sync had skipped the PDFs and kept only the first 2,000 characters of everything else. So the AI could answer "who does what." It couldn't answer "how does this actually get done."

That's the difference between data and knowledge, and it's the one we keep stepping over. Data is what a system records: the policy, the org chart, the job description, the donor record. Knowledge is know-how. It's how the work really runs. Russell Ackoff put it plainly: knowledge is "know-how, for example, how a system works" (Ackoff, 1989, p. 4, as cited in Frické, 2009).

I use an iceberg to explain it. The small white tip above the waterline is what gets written down: the rules, the policies, the org chart. That's what an AI system knows. Everything below the waterline is what people actually carry. The workarounds, the handoffs, the judgment calls, the guiding principles, the tacit knowledge.

Agents can only follow processes someone has written down clearly. And if a document was never written, no retriever can find it.

A ground-up knowledge audit

A knowledge audit asks what an organization knows, what it needs to know, where that knowledge lives, and where it breaks. It's an old idea. Twenty-five years ago it was already being called "one of the critical first steps" in knowledge management (Liebowitz et al., 2000). The usual starting point is the documents and the systems. That's the tip of the iceberg again.

A ground-up audit starts with the people doing the work. It goes like this:

1. Listen 1x1. A recorded conversation with each person about how their work really runs, not how the job description says it runs.
2. Write it down in their words. Each workflow becomes a short record, linked back to what was said.
3. Give it back to them. The person reads their own record and corrects it. Experts often can't recall their own steps out of context. Seeing a draft prompts corrections and additions they'd never have volunteered in the interview. Qualitative researchers call this member checking.
4. Give every piece an owner and a review date. Without one, accurate pages and outdated ones look exactly the same.

What comes out of the audit goes into a knowledge vault. A vault is a folder of plain text files (notes, decisions, transcripts, how-tos) with an index and a short page telling the AI what to read first. It's not a database. Anyone can open the files. If you want to think of it in the old terms, it's just a knowledge base. The difference is how it gets filled: from the ground up, and checked by the people who do the work.

We rebuilt the nonprofit's system that way. Every staff member had 1x1 listening sessions. The sessions became sixty-one workflow records in nine categories, each signed off by the person who does that work. And the audit found things no document held. The judgment step in the photo review lived in one person's head. "ACH" meant three different things. A workflow had stopped when one person left.

None of that was in the ninety-six documents.

Where qualitative AI analysis comes in

This isn't a new problem. The first expert systems hit the same wall. In 1977 Edward Feigenbaum wrote that "the acquisition of domain knowledge was the bottleneck problem" in building them (Feigenbaum, 1977, p. 1020). His explanation still holds. The expert "knows more than he is aware of knowing" (p. 1017).

When Patrick Lambe surveyed knowledge auditors, the methods considered most effective were interviews, "for their depth and richness," and workshops. Interviews also cost the most: they're "time consuming to conduct and analyse" (Lambe, 2017).

That's the part that's changed. Transcription and summarizing turn an hour of conversation into notes in minutes. Agents can help with the tedium of qualitative analysis: pulling quotes, proposing groupings, drafting the workflow record. The research on this is honest about the limits. When Stefano De Paoli had a language model do a thematic analysis of interviews, "the model never inferred as a theme the 'psychological perspective' or 'violence and aggression' which clearly had value for human analysts" (De Paoli, 2024, p. 19). The model also proposed themes the humans hadn't, and at least once it filed a code under the wrong theme. His conclusion was collaboration, not replacement. The likely future, he writes, is not human analysts replaced by AI but "a Human-AI collaboration" (De Paoli, 2024, p. 20).

Checking didn't get cheap. That's still human work, and it's where three decades of knowledge management cut corners. So I keep the agents behind us, not in front of us. They deal with the tedium. The person whose knowledge it is decides whether it's right.

My goal isn't automation. It's to raise conscious learning in the people doing the work. If we create automations without knowing how they work, we lose the depth.

Why operations know-how, why now

Getting operations know-how out of people's heads and into a vault isn't new work. Three things make this urgent now.

Retirement. Long-serving people are leaving, and most of what they know was never written down. Every retirement is a tacit knowledge transfer problem. In a 2025 APQC survey, "92% of organizations do not consistently capture knowledge from soon-to-be retirees" (APQC, 2025).

Onboarding and cross-training. A new hire learns the job as listed, then learns the real version by trial and error. A teammate covering for someone learns it the same way.

Personal AI. Everyone's getting faster on their own. But what one person learns stays in their own chat history, so the team's AI starts empty. Faster alone, no smarter together.

Writing in Harvard Business Review this fall, Kris Johnson Ferreira and Jordan Tong describe the same gap at company scale. Many companies now use AI to improve individual tasks, but "most still struggle to apply it to the cross-functional work that makes or breaks enterprise performance" (Ferreira & Tong, 2026). Cross-functional work is where the handoffs and judgment calls live. It runs below the waterline.

The vault doesn't fix that by being big, or by being used by everyone. Some knowledge should only reach the person who's replacing the executive who held it, when they need it. Proper access matters more than wide access.

And it's never finished. A vault starts going stale the day it launches. It fills only one way, through listening to people and reviewing what they said.

Here's one small practice I'd start with. Before someone retires, sit them down with the vault. Have them ask it the questions their successor is likely to ask, and grade the answers.