AI Data Organization

How to organize data in the AI era and turn files into usable systems

Drawn from my "Data Organization and Knowledge Base Building in the AI Era" foundations class: from the four Markdown syntaxes, carding, and linking to controlled vocabulary, turning your files into a system both people and AI can use.

Published 2026-03-09 | Last updated 2026-03-09

What this article is about

Late 2022 was the era of the "chat bot" explosion: AI could only chat and answer questions online, like a brilliant consultant who could only text you. 2026 is the era of the "working bot" explosion: AI can go straight into your computer, look at your files, and edit your documents. That raises a new question: can AI get in, find its way around, and actually use your files?

Who this is for

· People whose files are scattered across the cloud, the desktop, and various note apps, so they can't even find things themselves when they need them
· People starting out with ChatGPT or Claude who want AI to read their material but don't know what that material should look like
· People who want to start building a personal knowledge base but are still stuck choosing tools and methods

What you can take away

· A standard for judging what makes a file actually "usable"
· Four Markdown syntaxes that cover 80–90% of knowledge management needs
· Three core ideas (carding, linking, and controlled vocabulary), plus a copy-paste organizing prompt

One metaphor runs through the whole pieceIf someone walked into your office today to help you work, could they get started right away? Or are the files scattered everywhere, so they can't find a thing? What this class teaches is exactly that: how to set up an office AI can actually work in.

The whole method comes down to five ideas

Many people assume data organization means learning a big suite of software. In fact, this class covers just five ideas from start to finish, and the tools are only the means to serve those five ideas.

Idea 1Data cleaning

Clean the formatting into a form AI reads well. Aim for clean plain text first, pretty second.

Idea 2Markdown format

Currently the most AI-readable text format, and also the default format most AIs reply to you in.

Idea 3Carding

Cut large files into small units with clear boundaries, so they are easy to combine and reuse.

Idea 4Linking

Build cause and connection between cards, preserving the thread of your thinking.

Idea 5Metadata and controlled vocabulary

Describe a file's attributes and standardize your wording, turning the whole knowledge base into a system AI can operate.

OutcomeThree stages

First you can use tools to organize knowledge, then you can design data systems, and finally you become a designer who lets AI operate the knowledge system directly.

Why Obsidian: the file is the system

The course uses Obsidian for the demos because it leaves data sovereignty fully in the user's hands: every note you write is a .md plain-text file stored on your own computer.

Its most important trait is this: delete Obsidian entirely today, and your data is still there. As long as the folder exists, your knowledge base exists.

The trade-off of an all-in-one platform

Full-featured, quick to pick up, and good for team collaboration, it suits people who want to do everything inside a single platform. The trade-off is that your data lives on the platform's servers; moving out means relying on an export tool, and the formatting may not survive.

The trade-off of local plain text

Your data lives as plain-text files in your own folder, and that same folder can be opened by Obsidian, a desktop Agent, or any AI. The tool is just a swappable front end. The trade-off is that you organize the structure and manage syncing yourself. This class takes this route, because it is the friendliest to AI.

Each path fits different situations; it is a question of dividing up the work. The core shift in mindset this class teaches is that in the AI era, the file itself can be the system: if ChatGPT is good today, plug in ChatGPT; if Claude is stronger tomorrow, switch to Claude, because the format is open and the sovereignty stays with you.

Four syntaxes cover 80–90% of needs

Markdown is a syntax that uses a few special symbols to tell AI (and various tools) the structure of a document. Learn just four of them and you can cover 80–90% of knowledge management needs.

# Level-one heading (hash + space = section hierarchy) #tag-name (hash without a space = classification tag) [[Name of another document]] (double brackets = card link) - Bulleted point (hyphen + space = list)

The first two are the easiest to confuse: a hash followed by a space is a heading, a hash with no space is a tag. The heading tells AI "this section is a self-contained block of content"; the tag lets you and AI quickly filter for files of the same kind.

A practical detail: it is best to put tags in the first few lines of the file. Obsidian finds them wherever they are, but AI reads the beginning of a document more carefully, so putting tags up front is the safer bet.

The lowest bar for beginnersIf you have no idea where to start, just do one thing first: paste all your current copy, draft plans, and work notes into a single folder and save them. Messy is fine. First give AI an office it can walk into and work in, and then ask AI to help you tidy it up.

Carding: drawing content boundaries around your knowledge

Carding is the single most central idea in knowledge base design: splitting large blocks of information into small units that can stand on their own and be recombined and reused.

Why is it needed? Because AI has a trait called attention dilution. Hand it an entire three- or four-hundred-page PDF and it reads the early pages clearly, but as its memory fills up toward the end it starts skimming, and content easily gets skipped. The fix is to cut the long file into small units with clear boundaries: AI sees the table of contents and grasps the overall structure, sees a heading and knows the scope of that section, without reading cover to cover.

A card can be cut coarse or fine; that is its granularity. A whole article as one card is coarse; each chapter as a card is medium; each individual knowledge point as its own file is fine. How fine you actually go depends on how much precision your work needs and on the AI's context length. Taking early-2026 Claude as a reference, only start thinking about splitting once a card passes about 3,000 words. The most practical approach is to draw boundaries with chapter headings: however many chapters a file has is how many cards it holds, and you do not need a separate file for each card.

The payoff of carding is in "building with blocks." Once your data has clear boundaries and headings, you can give an instruction like this:

Take the origin of Plan A and the execution steps of Plan B, add the client's feedback from the Meeting C notes, and combine them into a new report.

This kind of cross-file recombination only works, with AI finding the right content accurately, once your data has been carded. It is also the digital version of the Zettelkasten (slip-box) method: the sociologist Niklas Luhmann used 90,000 physical cards to support a large body of high-quality work; in the AI era, the job of "querying and combining" moves from human hands to an Agent.

Controlled vocabulary: keeping tags from sprawling

Keep a knowledge base long enough and you hit an inevitable problem: the words you use for the same thing keep multiplying. To describe "suitable for beginners," one day you write "#elementary," another day "#novice," another day "#easy-to-grasp," another day "#basic." Four words for the same thing splits your tag system into four, and search starts to break down. When a team shares a knowledge base, it gets worse.

Controlled vocabulary is a simple practice: decide in advance which words are allowed. For example, "difficulty" gets only three options: basic, advanced, professional. No matter who writes it or when, difficulty is described with those three words, so searching "#basic" pulls up every introductory piece.

The good news is you don't have to design this up front. Tag things loosely at first, and once you have accumulated a while, hand the cleanup to AI. Save a standing instruction in ChatGPT or Claude's project mode:

You are my knowledge base organizing assistant. When I paste in content: 1. Convert it to Markdown, using hashes to set the heading hierarchy 2. Assign suitable tags (#tag format) 3. Put the output inside a code block If I say "analyze tags": Review the files we've organized so far, find where tags are inconsistent, and suggest a way to unify them. If I say "dictionary file": Build a controlled-vocabulary dictionary from the current tags, define the scope of each tag, and specify which single synonym to use.

The dictionary file is the tool that puts controlled vocabulary into practice (for why changing one word makes AI search a different region of its database, see Why prompt templates are sometimes good and sometimes bad). Once you've defined "three difficulty levels," then whether a file says novice, entry-level, beginner, or easy, AI will consistently replace it with "basic."

From by hand to Agent: three stages of evolution

Once you know the ideas, the practice has three evolving stages, matching three levels of effort.

Stage 1Markdown by hand

Type the syntax yourself in Obsidian, file by file. Full control and you learn the syntax along the way, but it gets time-consuming at volume.

Stage 2Web AI + prompts

Paste a file into a web-based AI and ask it to convert the format and set tags. Saves the manual work, but you still paste files one at a time.

Stage 3Desktop Agent

Let AI read your folder directly and process every file in batch. You just give instructions and review the results.

Stage two has a little trick few people know: add one line to the end of your prompt, "please wrap the output in a code block." Because the AI interface renders Markdown syntax into pretty formatting, copying directly often gets you only the rendered plain text with all the syntax gone. Ask for a code block, click copy in the top-right corner, and you get the full syntax intact.

Stage three is where a desktop Agent (such as Claude's desktop app or Google's Agent tools) lives on your computer and organizes folders in batch directly. Turning a three-hour course transcript into a tens-of-thousands-of-words teaching manual, renaming a whole batch of screenshots by their content, and pulling a blog series into a local database are all things this stage can do.

Do two things before you practiceBefore giving AI high-privilege batch processing, make sure your important files are backed up, and have it work on a separate test folder. Don't touch your most important work files first. Once the process runs reliably, then point it at your real folder.

How to practice

Three things are enough to take from the whole class: AI can't read minds, so give it material it can trace; your files are your system, and if the format is good enough any tool can read them; AI knows association but not cause and effect, so you have to write in your insights yourself.

  1. Open a folder and drop in all your notes, drafts, and plans, so AI has an office to walk into first.
  2. Pick three to five documents and organize them with the four syntaxes: split into sections with headings, add tags, and add one link you've spotted yourself.
  3. Save the organizing prompt above into a ChatGPT or Claude project, then hand later files to AI to organize by the rules.
  4. After building up a while, say "analyze tags" and "dictionary file" to the AI to establish your controlled vocabulary.

Your office can be messy at first; what matters is letting AI in. Then, step by step, organize that office into a system that works well for both of you.

AI Data OrganizationKnowledge ManagementMarkdownZettelkastenControlled VocabularyObsidianKnowledge Base