For many years I’ve bounced around using different tools to save information that I might need later. I’ve used MS OneNote, Evernote, Workflowy, Dynalist, Notion, several other hosted and self-hosted wiki systems, and probably many other things.
If I had to name a favorite out of all those, I’d go with Workflowy. It’s a super simple text outliner. You start with a single top level page. Each page is a list of items, and can each have a nested sub-list, with effectively unlimited depth. But you can also focus on any node so that it becomes a page in itself. Dynalist is very much the same, but you get multiple lists and can add images, fancy formatting, check boxes, all kinds of other groovy features. On the surface it sounds a million times better, but with all those bells and whistles, I felt like I was losing the elegant simplicity of Workflowy.
But I digress.
One thing I didn’t like about most of those systems is that someone else owns your data. And if it’s an online system, it might be hard to access offline. Some of the wiki systems do run locally, but then you might lose the online functionality.
Some months ago I came up with a system that I am now totally sold on. It’s super simple and involves only github (or gitlab or bitbucket or a self-hosted git repo or whatever) and markdown files.
The System
Honestly, the system itself is so simple that as I start to describe it, it seems almost so obvious I’m second guessing why I even have to describe it. But I did go through a few iterations before I got it down just how I like it.
Top Level
Start by creating a git repo (either online or locally). Create a folder called docs
and a README.md
file. In that main file create and maintain a bullet list of links. This is your index to all the top level pages in your wiki. Each link should be to another markdown document that lives in the docs
folder. Here’s a sample:
# My Personal Wiki
- [Stuff to Buy](docs/stuff_to_buy.md)
- [Movies to Watch](docs/movies_to_watch.md)
- [Birthdays](docs/birthdays.md)
- [Projects](docs/projects.md)
The reason for the top level docs
folder is because when you first go to your wiki, it’s going to show a list of all the files and then render your README.md
file below that. If you have a ton of documents in the root of your project, your readme is going to scroll right off the page. With this setup, there will always only be two top level items, the folder and your readme. This is what it looks like when you go directly to the repo:
Main Documents
Now, within your docs
folder, create a markdown file for each item in the index. A nice trick I do here is to create a link back to the main readme file as the first line in the file. For example:
[Parent](../README.md)
# Stuff to Buy
- Food
- Drink
- Pencils
- Paper
- Lamborghini
Now you’ll have a link back to where you came from. Here’s that in action:
Of course, you can link out to other content here too:
[Parent](../README.md)
# Movies to Watch
- [The Godfather](https://www.imdb.com/title/tt0068646/?ref_=fn_al_tt_1)
- [The Big Lebowski](https://www.imdb.com/title/tt0118715/?ref_=fn_al_tt_1)
You can also do fancy stuff like include images, or even make tables:
[Parent](../README.md)
# Birthdays
| Who | When | Gift |
|-------|----------|-----------|
| Paw | 01/01/20 | Whiskey |
| Maw | 05/08/23 | Flowers |
| Sis | 08/09/42 | Gift Card |
Which gives you:
Sub-categories
Now you can start making folders for sub-categories. For example, the projects page looks like this:
[Parent](../README.md)
# Projects
- [Project A](project_a/index.md)
- [Project B](project_b/index.md)
The Project A link goes to project_a/index.md
and similarly for Project B. And both of those files will link back to docs/projects.md
as their parent:
[Parent](../projects.md)
# Project A
This is all about Project A
Naming these index.md
rather than README.md
breaks the symmetry between the top level and lower levels, but that’s fine with me.
And of course, those projects can contain additional documents or even nested folders. And as long as you include those parent links at the top of the page and use those for navigating back, you’ll always be viewing rendered markdown files.
This is just one way of setting up sub-categories. Totally up to you how you want to organize it. That’s the beauty of it.
Benefits
Things I love about this system:
- Your content is available on line from wherever you can log into github. But at the same time, you can always keep a complete local backup on any computer you want. Changes are just a pull away. There’s no PITA “export your data in json format” kind of process. Keep it up to date all the time. You can even do a local push/pull via a cronjob once a day or more or less often.
- Online content is nicely rendered in HTML on the web, but also completely readable and editable on your local system. Of course, if you are using a simple text editor, you might miss out on the clickable links – and images.
- Editing content on line is one click away. Just click the pencil icon as you’re viewing the markdown file and you’re in edit mode. Save and you’re back to an HTML view.
- If you do end up in an offline situation you can still edit your local files. If you wind up with a conflict, you’re not at the mercy of the tool in how it’s going to resolve it. It’s a straight up git merge, conflict resolution workflow that you’re already familiar with. Full control.
- Of course, it’s git, so every single change to every file you make is a commit, tracked, revert-able, never lost.
- Get sick of github? Pull locally, push it to gitlab instead. Or wherever.
- No special tools, frameworks, or other systems needed. Just markdown files.
Overall, I’ve really come to rely on this system. I’m constantly thinking of things to add to it. Every time I figure something out or find some interesting bit of data I know I’ll want to come back to, it goes in the wiki. I find myself actively using it all the time.
Potential Improvements
One of the things I’ve thought about is automating some of the work. The main readme and sub-category index files could potentially be auto-generated. It wouldn’t be rocket science. Iterate over each markdown file and make a listing. And for each folder, look for the index file in that folder and add that to the listing. Then re-write that readme/index file with the updated listing. Recursively travel through the folders. If you had a script or tool that did this, it could be run via github actions perhaps.
If I were just editing locally, I could easily set up an editor command to create the file and link to it. But I find myself using the wiki more often online than locally. So I have to keep thinking about all this.
Example
Here’s a link to the sample wiki I created just for this post. (My real personal wiki is private.)
https://github.com/bit101/wiki_example
Feel free to fork it or just copy the idea and make it better. If you come up with any bright ideas on how to improve it, please share back.
Thinking more about that github action idea. Here’s my current thoughts:
1. Run it on every commit.
2. Get a list of every markdown file in the commit.
3. Find every local (non-http) link in each markdown file.
4. Does that file exist?
5. Yes. OK, move on.
6. No. Create it.
The resultant work flow:
1. Edit your markdown file.
2. Create a link to a non-existent file.
3. Save it, which causes a commit, which runs the action, which creates the file you just linked to.
4. Now click on the link you just created, which opens the file that was just created via the action.
This is effectively what I’ve been doing for the last year. One of our concerns at our company is the ability to maintain client notes in a portable format that everyone can access, but at the same time does not reside in another vendors “cloud”. So using a repo for my notes allows me to grant easy access for my colleagues, and keep it within ours, just like our code.
To leverage quick access, I set up a vhost on my local apache install, and serve up the content in the directory with Apache mod markdown (https://github.com/hamano/apache-mod-markdown) so that I can pop into my notes and view them easily (a la Github).
You basically just described most static-site generators! Of which, my favorite is Mkdocs.
Great post 🙂
I made basically the system outlined in this article in 2018: https://github.com/cblgh/monotome
What I use to index everything is a simple node script, https://github.com/cblgh/monotome/blob/master/monotome/bin/generate.js
Somewhat recently, I also added the ability for the system to recognize backlinks (one article linking to another) https://github.com/cblgh/monotome#backlinks
You are describing de Zettelkasten method.
Try this using Zettlr or Obsidian.
I’ve arrived to a similar setup for my personal wiki, but I was missing a metadata layer on top of the notes (for instance tags). Finally I found something that works almost perfectly for this purpose: obsidian.md
Like the rest of the tools in your stack (git, markdown, text editor) it does not come with any strings attached, it’s free and serves one purpose (at least in my case)
Adding backlinking functionality is slick. Its an important feature once you get used to having it, and something most similar tools do not yet have.
The cadillac of these tools is roamresearch.com, but things like org-roam mode in Emacs have added backlinking, and I suspect more to to come.
I’m curious if you have thought about surfacing your content via a different frontend app and something like elasticsearch on the backend for indexing and advanced search?
Hi Keith,
there is neat little project called https://docsify.js.org which renders the markdown files into a website, maybe you can use that to improve your idea. I use it in my company for technical documentation. I created an example for you at https://github.com/vigorcrust/docsify-example . In public github projects you can use the functionality of Github Pages to publish it like https://vigorcrust.github.io/docsify-example/#/ . This adds nice features like full text search, Edit button, offline availability through PWA on smartphones and computer with Chrome. I really like your idea about using Github Actions to generate the index file – this bugs me in my setup as well.
I suggest you try obsidian.md
yeah, I’ve tried it. lots of bells and whistles, but I prefer the simplicity of vim and github.
Thank you for this idea. It helps me a lot. Also I’ve learned how to use relative path correctly, this problem has bothered me for a long time. You can watch [wiki_docs/pull/2/files](https://github.com/Gaotianhe/wiki_docs/pull/2/files).
I wrote this just want to say thank you.
I used a Webstorm IDE, it has a feature named “customize your code template”, so you can customize a sub markdown file with your ‘[parent](../somename.md)’ , is it so simple? For the “Potential Improvements” above , I think it has to use HTML and javascript to implement all things reffer in this article instead of using markdown files, overall we use markdown is for the elegant simplicity. Ha ha.