I’ve been trying out Obsidian.md and thinking a lot about personal knowledge management systems over the past couple months.
Personal Knowledge Management
Put simply, personal knowledge management is your system for note-taking. You could imagine many different ways of doing this. Some people keep a series of notebooks, organized by topic. Some people keep a mish-mosh of Google Docs, scraps of paper, and emails sent to themselves without any discernible organization system – this was me until recently. And some German sociologists keep meticulously annotated and organized slips of paper stored in a series of boxes, but more on that later.
There is a growing movement of people trying to improve PKM capabilities with software. The two that seem to be referenced the most in the space are Obsidian.md and Roam Research. Beyond that there’s Notion, though it’s more structured and geared toward team collaboration rather than personal knowledge management. And beyond that are the more straightforward note-taking apps, like OneNote, EverNote, and Google Keep, among many others, which can be effective but lack PKM-specific innovations.
Obsidian
Obsidian has a fairly simple primary feature set, shown in this screenshot.
- On the left panel there is a familiar file tree with folders and notes.
- All writing goes in files called notes.
- In the center there are the note(s) you’re working on.
- Notes are written in Markdown.
- You can create “internal links” from one note to another note by putting the other note’s title in two sets of braces, eg.
[[Obsidian]].- These look like regular link text unless your cursor is over it.
- On the right panel there is a list of note mentions from these links.
Just take notes
Right away, I’ll say that Obsidian has a really nice interface for simply writing Markdown. Obsidian automatically renders the Markdown as you write (eg. beginning a line with # makes the font larger, since that indicates a heading) giving you a seamless and aesthetically pleasing loop of write - read - write again. The dark theme, the use of whitespace, and the fluid Markdown rendering gives me a joy when I write that I haven’t found quite as much in other tools.
The zen of writing in Obsidian is a big draw that’s led me to realize just how important taking notes is, any way you can, whatever your system is.
It shouldn’t surprise me, it’s a similar power to coding. You encode some knowledge then you can build upon it. As Bret Victor says in his essay Learnable Programming, the ability to “see the state” and “create by reacting” are essential elements to building – reacting to what’s there facilitates the iteration loop.
With note-taking, what you’re building is a body of information, and writing ideas or observations down helps you write the next idea or observation down or refine that idea. Plus, something unique happens in the simple action of writing something down – you remember it better.
Networked knowledge
As you create notes and link them in Obsidian you’re not just creating a stack of separate pieces of paper but rather a tangible network of knowledge.
This network is shown most clearly in the graph view that Obsidian offers.
Each node represents a note, and each line shows that those notes are connected via at least one internal link (those links with the double square brackets, eg. [[Obsidian]]). With this feature, you can visually explored your idea graph, and perhaps see connections in these paths you hadn’t considered before.
The interface lets you play with how this graph gets visualized. Here’s a zoomed-in snapshot of the nodes surrounding the Obsidian, in one particular configuration.
It’s interesting what ends up next to each other. All these nodes are related via some path through my personal knowledge graph. Their proximity means something.
You can also see a more focused view of the context any note exists in via a “local” graph view, like this one for my Obsidian note.
The fact that every time you add a thought to Obsidian you’re building out this graph has proven to be very addictive for me. I’m building something important, personal, perhaps with emergent properties. As a person who likes building things, this motivates me to jump back in and get thoughts down. And given my earlier realization that taking notes in any way is a majority of the battle, this motivation is well-appreciated.
This context is also helpful when authoring a note. It’s another interesting way to create by reacting. You’re writing in your note on Obsidian and you see that nearby on the graph view is the note for Functional Programming. Perhaps you could synthesize some new connection from this, further building your body of knowledge.
A note’s immediate context is shown to you in another way within the note view, so you don’t have to rely on the graph view for all your contextualization. On the right side panel you can see all the notes that have linked mentions to the note you’re authoring, plus a little snippet of the text surrounding the mention.
Perhaps as I’m writing in my Obsidian note I glance at these mentions and see a snippet from my daily note from 5 days ago that says:
… just finding it an absolute joy to write in [[Obsidian]], it really keeps me coming …
And I’m thereby inspired to further explore the joy and zen of writing in Obsidian, which turns into a section in this blog post.
Daily notes
Obsidian has a built-in “daily notes” concept. There’s a straightforward button on the left panel that opens (creating if necessary) a new note with the current date as the title (using ISO 8601 format, thank goodness).
I really like this feature. Most of my writing goes in these daily notes. I’ve found that I really want to capture thoughts in the moment as they come with as little friction as possible, and being only a button click (or configured hotkey) away from being in a note and ready to write something down has meant I end up doing it more. If you look back at the snapshots of my knowledge graph you’ll see I have a filter that makes all my daily notes appear in red, and if you look really close you can see most edges have a daily note at one end.
One thing this daily note-oriented organization style gives me is a peek into the chronological context of my thoughts – I can see when I had a particular thought. I can see that I was thinking about Obsidian a lot over the month of August, 2022. I can see that I was thinking about Obsidian the same day I was thinking about Functional Programming. Perhaps some of this contextualization will lead to those interesting syntheses.
Dreaming of more
These features of Obsidian are really cool, but I’m left with a big appetite that isn’t fully satisfied.
Take the example of writing in my “Obsidian” note, and how nice it is that I’m informed by all the mentions it has while I’m writing. The problem with that example is that I don’t write in my “Obsidian” note. I do almost all of my writing in the day’s daily note, and the mentions there aren’t very helpful.
Trying to examine why I like taking notes this way, I’ve come up with this: it’s the closest way to how my brain builds knowledge. I don’t necessarily open some long-running note on some thing, some entity, like “Obsidian”. I have ideas and observations as time goes on, and those new ideas end up enriching my understanding of that entity. Then, really, my knowledge on a subject is encoded solely in the context of ideas and observations I’ve had referring to that subject.
So according to this hypothesis, it’s these ideas and observations that are the most important things for me to capture, and the subjects themselves in a way exist only in relation to them. It’s at this point I discovered Zettelkasten.
Zettelkasten
The German sociologist I mentioned earlier was Niklas Luhmann. He was an incredibly prolific scientific writer active in the mid 20th century, and he credited this productivity to his unique note-taking system he called Zettelkasten. It involved writing down a thought on a slip of paper (“zettel”) and organizing it into a box (“kasten”). These slips would be given a unique identifier, and could reference other slips by their identifiers. This way, you could organize related thoughts together, and traverse your graph of thoughts by following these references.
The manual effort required to maintain this system was worth it to Luhmann for the benefit of having the ability to traverse related thoughts, and it proved itself in his productivity. He could pull out a note from the box, pull out related notes to that note, and related notes to them, and as easy as that he’d have a critical mass of ideas and observations, ready to be worked into some publishable form. This networking of information is the part of Zettelkasten that seems to have taken hold quite strongly in the PKM-verse.
Another part doesn’t seem to have been embraced so strongly. When Luhmann wrote something on a slip he wrote exactly one thought. This is called the Principle of Atomicity: “each Zettel only contains one unit of knowledge and one only.” Which means the network’s nodes are “single units of knowledge”, or perhaps we can call them “atomic thoughts”. It’s a little hard to intuitively determine what exactly the definition of an atomic thought is, but it’s sufficient to me to say that it’s something way less than an entire Wikipedia article on some entity or concept.
There are a lot of articles and discussion out there talking about Zettelkasten in tools like Obsidian and Roam. But looking around at these discussions, I see violations of this principle. I see an embracing of writing longer-form notes on entire subjects and entities.
So I’m left wondering: in its most basic features, does Obsidian encourage you to write and link atomic thoughts? Or is it geared more toward subjects, entities, whose notes may contain many thoughts?
Rethinking the atoms
Writing notes on subjects and entities, and writing longer-form publishable Markdown files like this blog post, is really nice in Obsidian. These end up being the atoms of Obsidian, the basic building blocks of the experience and the nodes in the knowledge network graph.
My thesis here is that these atoms are too big for a personal knowledge management system. I believe to more closely work alongside the brain, and to unlock the power of the knowledge graph, the atoms that the system embraces need to be Zettels, these atomic thoughts of Luhmann’s. This doesn’t have to be at the expense of having larger molecules represented in the system, but they need to come somewhere behind the first-class citizen of the atomic thought.
So what would a system, using Zettels as atoms, offering an interface at that level of modeling that’s as simple as possible without loss of generality, look like? Here’s a compilation of ideas I’ve had.
Context should be lightweight, fluid, and flexible
Right now in Obsidian I can open up a new note and simply write a thought, making sure to end it before I end up writing two thoughts. But some aspects of Obsidian add a bit of friction to that.
Where do I put this new note? Should I have to categorize my thought before writing it into some rigid hierarchical context in the file viewer? Should I have to open up my note on Obsidian to write a new section saying “today I had this observation about Obsidian”? What price do I pay for not having categorized it, or having had to categorize it into some rigid context (like as a section in the Obsidian note)?
If I want to capture chronological context, should I have to lump the multiple thoughts I have throughout the day into a single daily note? Again, what price do I pay for this contextualization?
Or should I be able to simply start writing a thought. Put it into the ether, like depositing a thought into Dumbledore’s Pensieve. The system could capture the one known bit of context for me: the chronological context. All other context is optional and flexibly applied or changed later. You don’t have a growing series of ugly notes named “Untitled 1”, “Untitled 2” clogging up an otherwise well-organized file tree. You don’t have a number of edges in your graph all pointing to “Obsidian”, but really wanting to point to specific ideas and observations you captured within that note. You have a system that embraces atomic thoughts in the form they come to you in.
Having lightweight context reduces the overhead of writing – even writing a title for a thought is some amount of friction, especially before you’ve fully written out the thought. I want to be able to go from having a thought to having recorded it in as little time as possible, and I don’t want to be punished for this.
Context should be expressive
So now I can write a Zettel with very little overhead, and the context I apply is optional. When I do choose to apply context, what kinds of context should I be able to apply?
Currently in Obsidian you can apply these types of context:
- Position in the file tree hierarchy
- Directional link relationships with other notes
Of course, other types of relationships exist between thoughts and entities. Thought X derives from Thought Y. Thought X refines Thought Y. Thought X synthesizes Thoughts Y and Z. Thought X is an observation about Entity Y. Thoughts X, Y, and Z compose Blog Post B.
Right now all of these would simply look like the same untyped links between notes. I’m not convinced an explosion of types here would be worth its weight, but I’m not convinced that reducing everything to the same type is appropriate either. And again, applying any of these should continue to be optional to keep context lightweight and flexible.
What if we reconsider just the relationships gleaned from the file tree. Right now there’s a root folder, and folders can contain both notes and other folders. Notes in the same folder all have an implicit relationship with each other: “sibling”. They also have a child/parent relationship with the containing folder. I’ve found it annoying that I can’t refer to folders themselves in my mentions – when I have a folder that contains all the notes pertaining to a project I really want to link to that folder when I’m mentioning the project elsewhere, but instead I have to create some additional note within the folder that ends up having the same name as the folder. If we made the folders themselves notes, and made the types of relationships notes could have more expressive by including “sibling” and “child/parent” relationships, we could reconstruct the file tree viewer while enriching the knowledge graph with these new types of links.
Context should be powerful
At this point I’ve got a network of atomic thoughts with rich relationships. I’ve built a big hill of knowledge, and there’s gold in that hill. I need the right tools to mine it.
In Obsidian, these tools are:
- The file tree view, which shows me the hierarchical context of any note
- The graph view, which shows me a collective view of the reference relationships between notes
- The linked mentions section of the right sidebar, which shows me a single note’s reference relationships
Even with the current contextual expressivity that Obsidian gives you, what other ways could context be shown back to us? What about showing the collection of all notes that are within an edge distance of 2 of the note you’re working on, as opposed to the local graph view that only shows an edge distance of 1? What about, as we mentioned before, also incorporating the hierarchical relationships into the graph view?
I have a very strong hunch that if we applied interesting graph algorithms, like with Neo4J Graph Data Science-style tooling, to such rich data we’d be able to gain novel insights into our knowledge graph (“mine the gold”). What about something as simple as seeing a ranking of nodes by number of references? What about seeing that ranking within some time period?
This could also lead the way to new, interesting, context-informed experiences. I have this image in my head of a capturing interface sitting on top of all this. It’s just a textbox at the bottom-center of a page. Above it, ordered by chronological context and fading into the top of the window, are the previous Zettels you’ve written. As you write, suggestions for conceptually related thoughts and entities fade in and out of view on the sides. You have the option to swiftly formalize a link between what you’re writing and those other thoughts. When you create a new paragraph, you start a new Zettel and your previous paragraph gets push upward. A “conceptually follows” link is automatically created between your new thought and the thought you just finished, which you could swiftly undo if you’ve moved onto another line of thinking.
You know, maybe
Of course, all of these ideas are easy enough to throw out there. It’s difficult to imagine what the specifics look like, what design choices would need to be made for this to be viable at all. I’m not sure if a system like this would end up being so noisy it’s unreadable, or so expressive it’s unlearnable.
Also, it would be a shame to throw out what Obsidian excels at – writing longer-form markdown files like this blog post. Ideally we’d be able to combine systems, keep everything good from Obsidian while expanding into the smaller atomic modeling. But again, not super clear how to do that.
But it’s all certainly captured my imagination the past couple months. I really feel like there’s gold in the hills, waiting to be mined.
I will say that after researching the PKM space for this post, Roam looks like it’s got valuable stuff for me. It states “As powerful as a graph database” right at the top of its marketing site. Next step is to cough up $15 and try it for a month.