Why AI Agents Need Knowledge Graphs

Andreas Kollegger has spent 15+ years building the graph database category at Neo4j. His finding: comparing documents in pairs multiplies the questions agents can answer from the same data.

Oct 14, 2026 36:28

Andreas Kollegger

Expert Insights

Andreas Kollegger is Director of Applied AI Research at Neo4j, where he has worked for more than 15 years, since it was a small Swedish open-source Java project. He had his first graph epiphany as an embedded systems engineer at NASA, and it resurfaced when he moved into medical informatics.

His conclusion is that chunked documents answer roughly one question each, and the harder answers live in the connections between them.

Key Points:

- Why all databases are "just fancy caches"

- How graph pattern matching differs from vector similarity search

- Why comparing documents in pairs multiplies the number of answerable questions

- What the Panama Papers show about hidden patterns in connected data

- Why unfocused extraction produces a complete but noisy graph

- The three stages of question-driven development

- What a context graph is, and why agents need onboarding like new hires

When you bring somebody on, you don't just give them a desk and a pen and paper and say, 'Here's your job. Good luck. Get it done.' You onboard them.

— Andreas Kollegger, Director of Applied AI Research at Neo4j

Monterail Team Analysis

- Confirm you have a graph problem first. If most answers sit inside single chunks, vector search may be enough. Graphs pay off when questions depend on how documents connect.

- Set up evals before extraction. Directional evals are enough to start, and they show whether each stage is heading the right way.

- Test on a slice before processing everything. Andreas's example is about a megabyte from a gigabyte-scale corpus. It caps token spend while you learn what the data holds.

- Chunk along the structure documents already have. Legal, regulatory and medical documents come with curated sections that work better than custom chunking.

- Scope agent tools by role. An enterprise can end up with hundreds of tools over one graph, and each agent should see only the ones for its domain.

- Keep the context window lean. Anything the agent doesn't need raises drift risk on long-horizon tasks, and a distilled graph slice can replace a hundred raw chunks.

- Define accountability before running multiple agents. Decide which agent owns what, and how disagreements get settled.