This is a follow-up to “From Second Brains to LLM-Wiki Knowledge Bases: The Curator.” You don’t need to have read it, but if you are curious about what The Curator actually is, that is the place to start.
The day before yesterday, Monday, I was proceeding with the testing and setup of The Curator, an open-source app that builds a local LLM-powered knowledge base from your own documents. A kind of private NotebookLM that can also grow from your interactions and dialogues with it based on the LLM-wiki concept by Andrej Karpathy.
While doing some initial testing on the first set of documents I had ingested in my private, curation-focused knowledge base, I casually discovered that some key names and concepts were missing, though they were clearly mentioned in the documents that I had uploaded earlier.
Data was going in and then vanishing without a trace. The system was quietly consuming what I fed it, with no warning and no error message. From the outside, everything looked fine. The interface looked normal. The app behaved as if it was working. Underneath, some information was disappearing into a black hole.
I started questioning the system. Literally.
I tried re-ingesting, but when I inspected the specific information, some of it was still not there. Something was not working. So, instead of sending a tech report or giving up I decided to ask the very AI that came with The Curator app to self-diagnose its condition and investigate what could be causing it.
Here’s what it replied:
Honestly, I can’t fully diagnose this from the wiki alone because **the wiki doesn’t contain documentation of the ingestion system itself**—how it works, what it’s supposed to do, what rules it follows, or where it might fail.
What I *can* observe from the evidence:
**The ingestion seems to have a design rule like:**
- “Extract atomic concepts from sources and create pages for them”
- “Create entity pages for people mentioned *as examples*”
- “But don’t necessarily create: source summaries, originator entities, or parent/consolidation concepts”
In other words, it was saying that while there were probably clear instructions to do part of the work, something else may have been missing from them that allowed for all that relevant data to slip through.
That’s when I contacted Dr. Rezun - the author/developer of The Curator app, and reported to him what had surfaced, where I thought he should look, and why I thought this was an important matter to pay attention to.
In less than 24 hours Dr Rezun fixed the issue, updated the code, pushed out a new release with many improvements and wrote me back:
N.B.: I am not a coder. I do not use Terminal. I do not know Bash. I have learned to use a little bit of GitHub only in the last 18 months, mostly because AI has made it easier for me to understand what I am doing one small step at a time.
By any conventional measure, he was the expert and I was the user. Yet, unlike what used to happen in the past of software development, before the introduction of AI, I, the user, was seeing things the developer should have seen before me.
Moral: Now that AI makes everything so easy, it is easy to trust that what “looks” or “appears” to be good, is actually good. The surface, the appearance is there, but sometimes not the infrastructure below it. And unless you personally inspect it, there’s no easy way to realize this.
AI Builds Convincing Surfaces
AI can generate code that looks like working code. It can generate articles that sound finished. It can build workflows, interfaces, summaries, images, plans and explanations that feel complete.
But the surface is not the structure.
Something can look finished and still be weak inside. The weak points do not always show up immediately. The unusual cases, the hidden assumptions, the parts that break only when real people use the system, often stay out of sight until someone looks for them carefully.
This is not a complaint about AI. It is a description of what AI is very good at, and what it still needs help with. AI can make things appear correct very quickly. But appearing correct is not the same as being reliable. That gap between “looks right” and “is right” makes all of the difference.
The key concept here is: As AI makes convincing surfaces easier and cheaper to produce, the ability to inspect what sits underneath becomes more important. The danger is that AI creates things that often look useful and right before they have been properly tested.
Credentials Don’t Predict Who Can Vet
Dr. Rezun has a PhD. He has built and tested systems, and he has taught other people how to think about AI systems in professional contexts. He is not careless, and The Curator is not a sloppy app. It is an ambitious and genuinely useful piece of software built by someone who clearly understands the problem he is trying to solve. I am 100% grateful for the work he is doing and for the app he has built.
Nonetheless, he likely did not see this issue of improper ingestion in his app because it was too close to it. It goes without saying that builders and verifiers see different things.
The person who builds something knows what the system is supposed to do. That knowledge is useful, but it can also become a blind spot. When you know the intended path, you may not notice what happens when someone enters from the side, uses the system differently, or pushes it into a situation you did not fully anticipate.
The verifier starts from another place. He does not live inside the system’s assumptions. He approaches it from the outside and asks simple questions: What happens if I do this? Where does this go? Why did this disappear? Why is there no warning? What changed between this test and the previous one?
I had none of Dr. Rezun’s credentials. What I had was time, curiosity, patience, and the habit of not letting go when I notice that something feels wrong. I also had AI tools he provided me with, that helped me trace problems I could not yet describe in technical language. That turned out to be more than useful.
Moral: Today, credentials do not always predict who can vet. I suspect that it is the habit of looking beyond the surface of things, consistently, over time, that says who can.
From Content Curator to System Verifier
Curation is much more than finding good articles and passing them along. True curation requires making hard choices and taking responsibility for them. You search, compare, filter, reject, organize, contextualize and then take responsibility for what you recommend.
If you think of it, that is already a form of verification, even if it has rarely been called that.
A good curator asks: Is this worth my reader’s attention? Is it reliable? What is missing? What does it connect to? What should be treated with caution? What is the author not saying? What should my reader know before trusting this?
So, what I am realizing now through this experience is that these same curatorial abilities apply directly to AI systems, apps, workflows and knowledge bases. The object changes, but the operation stays the same.
A system verifier does the same thing in another territory. He runs an app and asks: does this actually do what it claims? Where does it fail? What happens when I try something slightly different? What is the gap between the interface and the reality underneath it?
I guess I have been training for this for a very long time. I didn’t know I was doing it. But that is what curation has always required: looking beyond the surface, noticing patterns, recognizing weak signals, asking better questions, and not confusing appearance and features with actual value.
The New Trust Signal: Who Checked This?
Most people still evaluate AI-built systems the way they evaluate any professional product: by reputation and appearance. Who built it? How polished does it look? How many people are using it? Who is talking about it?
These shortcuts worked reasonably well when producing a convincing surface required deep skill, time, money and effort. They become less reliable when the surface itself gets easier and cheaper to produce.
Now the surface is no longer strong evidence of what is underneath, that changes the question.
the question now is not just the classical:
“Who built this?” but:
“Who checked this, and do I trust the person who checked it?”
I think there will be people, in specific fields, whose reputation rests on the accuracy of their vetting. Their track record will be based on saying “this is solid” when something is solid, and “here is what I would not trust” when something is not.
That is a different kind of authority from the kind built by publishing consistently and accumulating followers. It is harder to fake and slower to build. But it is also more durable, precisely because it is harder to fake.
My claim is that in the near future, in some niches, “vetted by X” may work like a small Michelin star. Not because X is famous, but because X has shown, again and again, that his judgment can be trusted.
A Michelin inspector does not cook. He eats, compares, notices, remembers and evaluates with disciplined attention. His value is not in the meal he produces. It is in the reliability of his judgment over time.
The same may happen in many expert fields. The territory is wider than software: anything that can be generated quickly, articles, business plans, market reports, tool recommendations, learning materials, can also be accepted too quickly.
A trusted vetter will likely be a person who has built the habit of carefully checking whatever comes his way. And this will become a unique, rare, in-demand value. Whoever becomes very good at vetting gains a lot of authority and trust.
What This Means for Experts
If you are an expert in a specific field, your advantage in the AI age is probably not going to come from generating more content faster. That race is already crowded, and AI can produce articles, summaries, lists, comparisons, code, images and polished explanations at a speed no human can match.
On the other hand, what AI cannot easily suggest or recommend is what is worth trusting.
That is where your real advantage may be.
Your advantage is the taste / questioning ability you have built over time. The patterns you recognize. The small warning signals you notice. The bad smell you notice when something looks good but does not feel right. The scars that still burn from past mistakes. The memory of what happened the last time someone tried a similar solution and it worked.
This is what many experienced experts underestimate. They think their value is in what they know. But in the AI age, their value may be even more in what they can verify.
The stronger trust signal today is something more specific than just knowing stuff. Here is how this new trust signal sounds:
“I checked this. I tested this. I compared it with other options. I found this weak point. I would trust this part, but not that one. This looks good on the outside, but here is where I would be careful.”
This is the kind of work already being done by people such as Ruben Hassid (everything you need to know to use Claude as a non-tech), as well as by Karo (Product with Attitude) (helping others create products and tools with AI), and Karen Spinner (indie developer / entrepreneur creating apps with AI).
And it is way stronger than publishing yet another polished opinion on whatever you are an expert in.
Become a Trusted Verifier: Leave a Trail
The way to become a trusted verifier, before everyone starts calling themselves one, is to leave a trail.
Pick one tool, one claim, one method, one AI output, one popular idea in your field and inspect it carefully. Show what you tested, what you expected, what worked, what failed, what surprised you, what you would not recommend, and where the thing breaks.
This is how the credentials of a verifier are built: through repeated public acts of careful checking.
A great verifier earns trust by becoming visibly reliable over time. You can do this with tools, articles, workflows, reports, or AI-generated advice on any topic. Anything that asks for people’s trust can be verified.
The more AI fills the world with things that look complete, the more valuable the person becomes who can say, in plain language: “I went inside. I checked. Here is what I found.”
That is the guide function.
Helping others understand what deserves trust, what needs caution, and what should probably be left alone.
In a world full of highly convincing surfaces, the person who learns to look under the hood becomes very hard to replace.
Paradata
Intent / Why: Highlight the strongly re-emerging value of vetting and verifying in the age of AI, as a lived personal experience.
Process: I am using a “Distant Writing” (see Luciano Floridi) approach. In a 2.5 hrs focussed session, I wrote down - as they came to mind - all of the relevant facts, ideas and feelings I have had about The Curator app, the process of installing and configuring my first LLM-wiki knowledge base, and everything I noticed in the process.
Nine key themes emerged. I wondered whether they could be mixed together into one organic piece or if the appropriate thing would be to pick one and use a few of the others as supporting infrastructure where appropriate. That’s when I invited Claude to critically look at my notes and nine themes and suggest multiple output directions that could come from them. Then I went to ChatGPT and did the same.
Out of all the possible options that emerged I chose the one that appeared to me to be the strongest (meaning most valuable in terms of highlighting a new non-obvious insight), the newly emerging value of vetting and verification across sectors and interests. Verifying.
From here on, I remixed elements from the different outline proposals that had emerged and then resubmitted sequentially to both AIs to generate a first full draft. Once Claude produced one, I asked ChatGPT to vet it and suggest where to critically revise it. After deciding what to keep and what to let go or change, I went back to Claude, and did the same. Then I did another round.
When the multi-revision draft emerged I started using that as a canvas for my ideas. Where the draft expressed my experiences and ideas in a way that sounded like me, I left things as they were. Where they did not feel like words I could have said, I edited or rewrote. 45% stayed, 55% went. This is how I arrived at this final article you are reading now.
Time: Jotting down memories and impressions: 2.5 hrs. Analysis by AI and co-exploration of themes: 1.5 hrs. Distant writing and multiple revisions: 2 hrs. Cover design: 1 hour. Formatting: 0.5 hrs. Text vetting and verifying: 3 hrs. Grammar and syntax corrections: 1.5 hours. Total time: apx. 12 hrs and not counting all of the hours invested in installing, configuring and building the knowledge base inside The Curator app (10+).
Tools: Claude Cowork, ChatGPT 5.5
Cover image: Design by Robin Good executed by ChatGPT
What I Can Do For You
I help experts and consultants who are building a new online career gain the authority, credibility, and visibility they initially lack.
I do this by teaching them how to become top trusted curators in their area of expertise.
For this purpose I’ve created a free, focused 55-minute video workshop that walks you through:
Why curated content formats are so useful for building credibility and authority
When to use them and what requirements they have
An updated list of real-world examples of curated formats at work
The specific tools you need to curate
The actual key steps that transform researching and writing into curation
The 11 typical mistakes novice curators make
One-to-One Audit & Strategic Advice
For experts and consultants looking for ways to:
a) Create Value and Build Trust for Non-Writers (Curation)
I help you identify and master your own personal ways to gain authority and value by curating insights, research, news, resources and tools in your field of interest.
b) Improve Credibility and Trust
I review your content, positioning and goals to identify the best editorial strategies to build trust and credibility around your focus.
c) Positioning and Personal Branding
I analyze your market positioning to identify key strengths and weaknesses. I help you redefine a strategy to differentiate yourself from the competition while increasing the practical value you bring to your readers.
Available in two tiers: a) for those just starting and b) for those who have been publishing for more than a year, but are not seeing results.
Follow a path with a heart.
The time is now.
From Koh Samui (TH)
Robin Good