I believe that knowledge management tied to a development platform is a bad idea. Development isn't the only thing I do in computers and knowledge management should be able to store stuff from all other activities (including cooking recipes or reading notes to read on a mobile phone).
I don't love tiddlywiki but haven't found a better alternative yet. I'd love something easy to configure but powerful like the wiki and bug tracker on the Fossil SCM, but not tied to a specific platform. Also, Markdown/text/html because no vendor should own my thoughts. Obsidian[3] and Notion are in my "to check" list.
I agree and think strong emphasis should be on how a knowledge base saves its files, which should be very interoperable.
I build my own little DSLs and have my own experimental
knowledge base studio going, where I can edit things using
my editors (Sublime, VSCode and Vim), view/query/visualize
in my web browser, and also view/edit in a spreadsheet web
interface using Handsontable.
The secret trick is all the DSLs use a thing I work on
called Tree Notation (some use an even dumber version
simply called Grid Notation). It's a plain text notation
that just defines lists of words (aka cells) on a line (aka
a node or row) and if you indent a line it becomes a child
of the parent line (like python) providing support for tree
structures and scope. That's the basics, and then on top of
that you can design all sorts of DSLs with types and very
different parsing strategies than traditional languages.
But that's still a relatively traditional language. I have
some languages which generally you want to use a spreadsheet
interface to write programs in, where you can just plop a
tree anywhere on the sheet, and then start writing trees.
Those ones are kind of like having a canvas for your project
instead of a folder with files. Makes for a good base for
building DSLs for simulations.
Sorry, I digress, but my purpose in bringing this up is that
if anyone is working on personal knowledge base software
like the OP, Tree Notation is a pretty useful thing, if you
can figure it out. I've got it figured out but not very good
at explaining it.
Tree Notation looks very interesting. It also looks like just another version of S-expressions (but with indentation instead of parens). That's not a bad thing, though! S-expressions aren't used nearly as much as they should be, and indentation syntax can be much clearer than parens.
What's most interesting is the language design DSL that makes it easy to put together a new tree language. I wonder if that can be implemented in Lisp, possibly with an indentation-sensitive syntax like Wisp[1] so that it retains most of Tree Notation's features.
However, the magic happens when you remove the parens
and go to 2 and 3 dimensions. Think of Javascript,
which has 20 million users, versus Excel, which has 2
billion^1.
To paraphrase Twain, removing one thing can be the difference
between a lightning bug and lightning.
Could you give an example (or a link) that demonstrates this higher-dimensional syntax? Are there things you can express with it that couldn't be expressed in S-expressions?
> Are there things you can express with it that couldn't be expressed in S-expressions?
No. But that may be like asking "is there something I can do with the x86 that I couldn't do with any Turing Machine?" Pragmatically speaking when you factor in UX and time there are things that are dramatically faster with Tree Notation versus S-Expressions.
Lots of things that are muscle memory to Excel users (drag to fill, copy/paste blocks, move columns and rows, et cetera), would be very ugly/hard to do if we were using paren based S-Expression notation.
This is fascinating. Don't suppose you'd be willing to share your dsl definitions, or can recommend any collections? Seems like a great layer to have in a modular knowledge system.
I definitely hope to share more in the future but at the
moment busy focusing with the team on a huge refactor of our
codebase at the day job (which includes a fun new DSL editable in a spreadsheet).
When we ship that and I take a breather hope to give the
Tree Notation stuff a refresh and publish some of the newer
langs.
Just saw https://www.youtube.com/watch?v=vn2aJA5ANUc. It looks great! I am excited that Handsontable works for you in this context. Please let me know if you'd need any help with it.
It probably is a bad idea in a way, but in my own personal experience I have found the biggest hurdle to a knowledge management system being actually used and up to date is _actually using it_, and having it just there in front of me in my IDE would probably go a long way to help with that.
I recently switched to Obsidian for my desktop pile-of-notes - so far it's pleasantly compliant with everything else I've done so far, and hasn't done anything to screw up my notes or make it hard to understand in other systems (e.g. Notable has a... "unique" folder system). Definitely recommend giving it a shot.
+1 for Joplin, I moved over from Evernote because I was worried about Evernotes future and I have been pretty happy with Joplin. It's not quite as featured as Evernote but it is more stable and reliable.
The implications of this cannot be overstated. Utilizing VS Code as a platform -- at least for a dev -- makes dendron feel intuitive and flexible enough to customize to my hearts content.
I felt/feel the same way, which kept me from using Foam for a very long time.
That being said, I've now used nearly everything (TiddlyWiki, Obsidian, even Emacs with Org-Roam) and have recently transitioned to Foam. Foam, for me, is the best option I've used.
A couple of reasons:
- Setup and maintenance are effectively nil, and customizing it is extremely easy. This was the big drawback of org-roam for me. It seems extremely powerful, but even after initial setup, it was just a headache to keep it maintained (and this is not me simply hating products that require config. I'm not particulary sensitive to upkeep--I'm typing this on a Surface Pro 3 running Linux, I'm used to needing to fix things.)
- Graph view/general UI. A big feature for me is the graph view, similar to Roam, and the associated UI (autogenerated backlinks, etc.). Foam is fantastic here. The general note taking experience is much better than TiddlyWiki (which I think is a fantastic tool as well) and I have to think less about constructing a knowledge graph, it just happens. Also, due to the VS Code ecosystem, there are a ton of nice extensions available for diagramming, embedding images, writing in LaTeX, etc. that make note taking nice without introducing overhead.
- Privacy/open source. This is perhaps counter-intuitive, but Foam has been one of the better options on this front. It's all markdown/text/html, as you say, but it's also fully open source, unlike something like Obsidian (which has a fantastic out-of-the-box writing experience, btw). I run Foam on VSCodium, a community-driven project that rebuilds VS Code into freely-licensed binaries with all of MS's telemetry turned off. The Foam maintainers have been open and encouraging about making sure Foam works in VSCodium ecosystem: https://github.com/foambubble/foam/issues/26
My whole setup now is more or less run out of Foam. I push up notes to a Git repo, which triggers a deploy of a little web app I have that renders a site built out of my notes, allowing me to do things like view my todo's or access information from my phone/other devices. It's been really easy to set up and maintain, and the Foam community has consistently impressed me.
One of the absolute most critical features of my own knowledge base is that it is entirely in my control. Hence it would not be possible to use an online platform.
- As Derek Sivers say: Apps come and go, we want to use our knowledge base til we die
- I need to know nobody is looking over my shoulders. A bit like social cooling. I will not be honest if I might be sharing.
You mention Obsidian which is definitely a good candidate. But hey, I just use a folder of markdown files. I can use the editor best suited for the task I am doing. So I both use Obsidian, VS Code, normal text editor and whatever floats my boat.
I (respectfully) disagree completely. All source code is knowledge. All the content that goes in to a product when you're shipping it is knowledge. Managing a project is knowledge management. Managing multiple projects is knowledge management. Managing a company is knowledge management.
I've never seen a tool (other than, arguably, an Operating System) that does all of this comfortably.
The data format is literally a SQLITE DB, which is the most used DB on the planet.
Binaries are available for all the major OS's and many "alternative" ones.
I use Fossil-SCM for my personal notes. Works great. I even run many websites with Fossil-SCM as the wiki is fabulous for a super easy to deploy and super easy to edit website, even if there is zero code in the VCS portion.
I also use fossil on USB sticks for my legal information on my death. I include fossil binaries for every platform, so whenever I'm dead my next of kin can just plug in the USB stick I update every year and hand to them and double click and boom, fossil-scm opens with a VCS of all my files and documents, financials, etc, a wiki with next steps, etc. Easy peasy, and they get everything they need to take care of all the stuff to handle my estate, in a nice easy to use manner, with basically no work on my part.
Me too. My case is a little different though. I realized the tickets database is a generic sqlite database you can use as you wish, but comes with a web server, syncing, and version control out of the box. It was simple to create a CGI app in my language of choice that gave me a web interface to my notes system but without having to implement my own authentication. If I want to work locally using Emacs, I can. If I want a full or partial backup in text format, it's a regular sqlite dump of the tickets database.
Having only tried Fossil for this briefly: do you have any recommendations? Viewing is pretty simple, and I agree about the portability / SQLite is a great fallback, but modifying is complex enough to keep me from adopting it.
E.g. I can't find a way to add files/images/make commits in the UI at all. I can make wiki pages or tickets in there (honestly, the wiki probably covers 99% of what I want, aside from it not being very streamlined to say the least), but I haven't found a way to add files except via `fossil add` on the CLI. Have I just missed it?
Otherwise, the main things keeping me from being more interested is that it's a large blob -> harder to sync, and there don't seem to be any mobile-friendly implementations (e.g. no app that just runs it). A folder of markdown files and images/etc to embed via `` is dramatically easier and more flexible on both of those fronts... but I do miss having a history.
Which requires running a fossil server that you can access anywhere, i.e. it has a custom and unique syncing system. That is... a rather significant requirement, compared to using a general file-syncing thing that you may already have.
But yes, you have a good point - once you have a server running, syncing is very efficient.
Fully agree. What about Gollum Wiki[1] ? I have been using it happily for years. It used to be Github wikis, but they diverged apparently. It uses git as backend. You're in full control of your data. Love it.
When you have a family. A wife who barely knows how to create bookmarks in a browser and kids who only use their phone you need something extremely simple.
So you need to " scale " from 1 person to a household and take into account that all information you put in there, financials, recipes, addresses, etc. etc.. need to be ubersimple, working on a phone, auto sync etc... and need to continue working even when something would happens to yourself.
So that is why i standardized the knowledge management system in our household on OneNote (2016 client for the laptop).
This is because everyone understands office.
The API both the COM and the graph API also lets you create , update, read, etc... so you can write basically everything you can think of e.g. auto put bank transaction on the correct place in the correct table in the correct place in the taxonomy.
Thinking about the structure of a family and everything in there is also interesting and I had some larger revisions on that.
This resonates with me. People like to talk about simplicity, and I get that, but to some people simplicity means “just use org mode”, which is a whole different axis of simplicity from “my partner can use it without redirecting their tech-related-frustration anger at me”.
There’s yet another axis of complexity/simplicity, which is that trying all these different note-taking systems means any time I want to look up a note, I have to check half a dozen different places. Recently I’ve been dropping everything except for code snippets into apple notes, and I’ve had a much easier time finding things later.
I use Fossil-SCM for that. 100% OSS, it's a single Sqlite DB file for backups, etc. It's totally fine being offline or online and totally fine being de-synchronized for years and catching up.
I put my legal and financial info on a USB stick with the fossil binaries every year, and hand them out to next of kin. Whenever I die they just plug in the USB and click the appropriate fossil binary and bam, they get everything with a web interface, files and wiki with instructions included.
I’ve always shied away from OneNote, finding it cluttered and as overwhelming as someone else’s heavily dogeared and bookmarked notebook, but you have my attention.
Do you have any breakdowns on how you conquered WAF and the API?
So just $whatever = New-Object -ComObject OneNote.Application and then lookat $whatever |get-member to see what you can do for members and properties.
---
I tried several things. I needed it because the amount of stuff around a family is just too much to keep in my brain especially with kids who needing a lot of help with their school and trying to also put some fun in it like " nice netflix series you want to share" but also serious stuff like " households tasks, bedtimes etc"
I think I could write a book on what i put in there.
Budget is also interesting, after 20 years I decide on the following structure:
A = income
B = budget per household member (so 4 posts)
C = everything that needs to be discussed (vacation budget, savings, subscriptions, stuff we want to buy this year, etc)
D = everything that we do not need to discuss (mortgage, food, all kinds of city taxes, healthcare etc)
That makes it really simple: only the points in C are needed to discuss "want to spend 20.000 on vacation Or want to spend 20.000 in savings" .
Because I noticed that mosts of the costs stay the same.
Many of these "topics" then tie in or have an overlap with the knowledge management topics from another perspective. So e.g. the budget post C.Subscription.XboxLiveGold ties into culture_and_free_time.games and where we just have fun pages on various game consoles and tips for games we play or we want to share with each other or " favorite games" .
What it also means is that when the thing grows at a certain moment you see a parallel in the taxonomy you put in your local organisation with external " all knowledge ever" for instance on wikipedia portal > categories etc or any of the " all knowledge" systems like dewey or libray systems.
And in many cases you write the thing that has a linkage to already being in a place in other generic taxonomies e.g. when i my son has an exam on e.g. French Grammer we put some stuff in there to undertand these lessons, but this " structure " is also already on wikipedia. But the structure on wikipedia is just too much too handle. So I think that the end-situation for larger parts of the personal household knowledge management system overlaps with the structure found in " all knowledge ever" systems. So im in the phase that I try to look at the taxonomy of these systems e.g. wikipedia and also e.g .structure "mathemetics in the taxonomy that is on the wikipedia portal page for mathematics" (but slower since i only need what is in there what my kid has currently on school). The younger kid can then leverage on this and read up on this. Since we already made it for the older kid.
But I feel that exchanges between "external taxonomies" and " household taxonomies" have not yet been establised. E.g. I also have all my " stuff" in there like " household applicances" (warranties, but also comparison sheets, instructions etc) but the overal taxonomy and the " wording" is matchable with e.g. shops who sell household appliances. If there would be a taxonomy standardization on all of these things it would enable exchange information on meta level.
OneNote is feature rich (specially the availability in iOS) ... but what eventually made me drop it was the unavailability of a reliable desktop app in Linux
Can I just say that this is a space I’ve been deeply interested in for a long long time (I have a terrible memory, and latched on to tech early), and it blows my mind that very few (if any) of these solutions use a graph database as the back-end.
I understand the argument that it’s likely overkill for <10000 entries, but for me it’s been wildly freeing. Especially in cases where the abstraction layer gets out of the way and allows for free-form connections between items. No more rigid db schemas, primary keys, or trying to cram “one or many” connections in to a column.
I think I come at this problem the opposite way to everyone that actually builds these products/tools. I want to mindmap and brainstorm all of the topics, and then take notes on them - preferably with a time component to that. Even more than that, I want to be able to take notes on the relationships between nodes, since that's where the real gold tends to be.
Mindmap tools generally don't work for this because they want a single topic for each map, and all the words on the one page. KM/KB tools don't work because I can't just draw the relationships between the points, I have to go in and actually create notes that I then get to visualise.
I really wanted to like Obsidian, but the way it treats folders puts me right off. I tried Neo4j Bloom, but there are UI difficulties and it's not really great for notes. I'd build it myself but I have no idea where I'd even start on a UI.
I'm the same way. Mindmaps are by far the best tooling for how I think. They're also just not portable. I keep trying. It would be awesome if mind maps could be united with a console somehow, so that I could inject things from "wherever".
I've spent a lot of time caring about KM/KB and can't settle on anything that survives more than 2 years. What do you do today?
I once built a mind map with a “MUSH” interface (text-based, where you traverse the map “on ground level” as a character).
You avoid information overload by showing the synopsis as you “enter” a node, then you could say `e[xamine] here` to view the whole text.
Traversal is quicker than you’d think, and it would likely work well on mobile devices.
It was years ago, but I still think about making that in to something “real”.
I keep everything in a combination of OneNote and Google Sheets, which I link back to Salesforce with the Salesforce connector for Google Sheets. It is... not perfect, but spreadsheets were the only way I could find to quickly get the information down that I wanted to.
This is the combination of things that originally led to my loving TagSpaces. I wanted local files that I could edit, have folders of varying file types, and include documents in the folders, opening as needed. Part of the issue was that I then wanted to link to, and from varying files... basically a knowledge base that was independent on data types.
Turns out that image capture, and document sync was the least of my problems. The mind maps, image data not being OCRed, and other things became a pain.
I actually considered using a graph database for our highly connected notes app[1], but in the end went with PostgreSQL because I was following the advice of "go with the technology you know".
It would definitely be interesting to go back and try something like Neo4j, but we've actually managed to wrangle postgres for our graph while still maintaining pretty good performance.
Something I find intriguing about Foam is that it's mainly a recommendation for how to stitch together existing tools into a nice integrated workflow, rather than a new piece of software built from scratch.
I wrote a blog post about this idea of "Software as Curation", and how we might imagine more software being built in this way:
Foam is a great FOSS clone of Roam. If you like to write text files, like markdown, and like to link among the text, then definitely try Foam and Roam-- they are superb for organization. And if you use emacs, the similar tool is org-roam for emacs org mode.
I used all of them and settled on dendron. Because Dendron makes the same links and nodes as roam and all the others. But it is still also hierarchal. After so many years of hierarchy I can't let it go so this hybrid is perfect for me.
I am a logseq user. The nice things about it include:
- all your usual backlinks, graph views, etc.
- saves files as plaintext
- md or org mode files
- operates on / syncs / saves plaintext files
- edit locally, or sync with a git repo
- accessible from mobile (via your github repo)
So I use this with org-mode files, and on my laptop I edit files locally (I have it pointed at a git repo), which is very reliable. Then the phone simply reads from (and writes to) the same git repo.
Since it runs on git, the interoperability of the notes is really nice. If I edit the same page on 2 machines or delete a file, it's just a simple merge conflict that I can use my favorite git merge tool to fix.
Not to mention the developer is extremely active and helpful.
I customized a jekyll template[1] to publish my Obsidian knowledge base and increase the discoverability of notes, which could be better in the app. Also, I wasn't comfortable being tied to the "publish" feature that is still in development.
I just recently started trying to get my note-taking together and build my own personal knowledge base (or at least stop trying to remember everything and instead store it somewhere to refer back to).
After trying a handful of the usually mentioned tools, I've been very happy with Trilium Notes [1], which seems to share some of the same concepts as this project, but providing rich-text (including embedded images).
As much as I like the flexibility that flat files, markdown, or Org-mode provide, I also really like the extra built-in functionality that comes along with Trilium.
I'm not wild about having the data tied into a single program, but the data is at least accessible and documented (if primarily through the open source code), so I know my data's not entirely locked up.
Trilium says macOS isn't supported as the author doesn't have a Mac, but it works 95% like you'd expect with a few small quirks.
I posted this in a reply to another comment, but in case it gets buried, as of last month Foam is also published to OpenVSX, meaning you can run it on VSCodium if VS Code's telemetry and licensing give you pause:
To people who maintain a personal knowledge base: how often do you check older entries of it? And how useful is the graph view as compared to traditional list view note taking grouped by tags?
My base has somewhere around 200k lines of text. I'm pretty sure some of these are almost never read. But I do open old entries/pages some times. I see no problem in writing a note that I never read. The writing is itself useful, in fact, as it helps me learn and organize information.
(FYI: I use a personal wiki based on Vim (with wiki.vim [0]) and a markdown flavor.)
I take notes in an atomic way (other than clipping interesting articles). Because they are atomic, many of the notes are linked in one manner or the other (either via "concepts", "daily log" etc.). So whenever I revisit one of the "concepts" node, I visit these atomic notes.
Also, obsidian has a "open a random note" button. Thus I open a random note once in a while to see if I can learn (or relearn) something + to see if that note is still relevant for me.
A data point: 11 years of note taking → 1,860 notes.
I've been keeping notes for work in a structured way for quite some time.
I log everything that I might want to recall later (project/app ideas, active projects, meeting minutes, phone call logs, people/companies I interacted with).
Typically I use two methods for retrieval:
(1) full text search: I simply search for a keyword; most common access
(2) via backlinks: mostly for people when I need to figure out when I last talked to a person and what we agreed upon. I link the person in each meeting minute so that I get a list for links on the person's page.
Frequency of access:
- For active projects, pretty much each (work) day
- For older notes, infrequently, maybe once a month. The older a note becomes, the less likely it is that I will have to access it.
I started out with Circus Ponies Notebook, migrated to Scrivener, and two years ago started working on my own app, Life Notes[0] (I really wanted links and backlinks and the ability to keep regular files/PDFs next to my notes).
I want a FOSS knowledge management application that supports quick-linking, backlink browsing, a graph visualisation, and tagging. I'm yet to find something that doesn't leave one of these features out, or has some other deal breaker (not FOSS, can't backup, no markdown support, not cross platform, etc).
This looks like it might fill that void, but I don't like that it's based on tying together VSCode extensions. I want something standalone, dependable, and usable purely off-line if need be.
By quicklinking I mean you can quickly link between pages, usually by using some kind of keyboard shortcut or shorthand. e.g. The [[ in Confluence has become common in most tools. (You type double square bracket and sometimes it has a prompt with recent pages, or you just type the page name and close with two more closing square brackets).
Neat, but fails in the same way Obsidian et al do: files are not a good abstraction for atoms of knowledge. What if knowledge isn't textual, or is only partially textual?
Notion [0] and recently released Craft [1] are both good personal-wiki applications that support lots of content types directly (from images and videos, to even things like tables and kanban-style boards for Notion), and have good mobile solutions.
Unfortunately, both are proprietary solutions that require paid subscriptions for most of their features and their own special syncing software. Craft also only runs on Apple devices at the moment (they say they're working on a web app).
Unfortunately OCR is in my experience not as smooth as within Evernote or Google Keep. When I take a picture using the OneNote app on Android, OCR never works. You have to use the separate Office Lens App to upload an image and move it to the correct place in your note later. Also: I've got quite a few old PDF docs containing scans of contracts as images. Google or Evernote enablefull text search everywhere, OneNote cannot search for text in these PDFs files
.
Notenik[0] is a FOSS native Mac app for note taking that I have benefited from tremendously.
It has a few UX quirks to it, but the developer is highly active, responsive and friendly. Its strengths lie in its scripting and templating features (check out the site for more info about them) that turn the app into something like a static site generator with a GUI.
Hype on pkm bases on zettelkasten. So pkm should support backlink. And notes should not live in seperate folders. Relation of note should be provided by links. Notes or parts of notes can be embedded to others. These are basics. And rest is plugins, ux. But these are basics. Of course this philospy also regrets vendorlock. Thats why they mostly use markdwon. These are the requirement. Backlink, embedding, markdown, graph view of notes relation. Who achieve that on multi-platform will win the game. Obsidian is on the way. But their users are generally developers. So they want always new features. So the markdwon format started to lose the feature of interoperatibilty. Needs are that simple. And all solutions on some point will need an mobile app. Because in use case you will need your pkm with you allways.
I use this tool everyday and with some customization it has been very helpful for me. It helps that I also use VS Code for my job so its nothing to have an extra window open. I don't have to worry about folder and file structure and just write. The VS Code search is robust and fast.
The customizations I use are just for auto creating daily notes and including timestamps and other metadata.
Foam is also easily incorporated into static sites, although I have not yet used that feature.
Been using and loving foam since it came out. Decided to use backlinks as a tagging system to take advantage of VSCodes / the extensions backlinks display.
Wish the main mode of navigation was visual clusters of text + used key shortcuts like vimium.
Also wish the graph wasn’t newly generated each time. At the moment the graph is essentially pointless, with no long-term meaning communicated by the structure. There’s an open issue on that iirc
I've found that I write a lot of notes but much less frequently read them. Seems so much of my time goes to writing notes that I don't have much time to read them.
Still I believe it's good that I write them because I assume it clarifies my thoughts and makes it easier to remember the thoughts I had because I wrote them down.
Since this thread is has a lot of note taking enthusiast, going to also suggest https://dendron.so;.
Dendron is a scalable note taking tool built on top of VSCode. It's similar to foam but focuses on a hybrid approach for managing notes notes with both roam styled back links and dendron’s very own hierarchy system. Dendron also helps users selectively publish any subset of your notes as a dendron style digital garden.
Disclaimer: I'm the author and I'm happy to provide any questions or help. You can also drop by our discord community if you just want to talk note taking: https://discord.gg/AE3NRw9
Foam doesn't take a very prescriptivist approach. A flat directory is the path of least resistance but you can put things in to separate directories. Personally, I store all of my daily notes in a separate folder and everything else in the root
Foam is a good solution if your not writing code on vscode. Our you should have second vscode somehow. KM should be seperate from my development enviroment.
And that's why it has only reached adoption inside a very particular demographic, which may or may not be a selling point for you as a user (customer?) but definitely has an impact on the community and the usage of the tool.
Personally, I like having my dev environment and my note-taking environment be the same. I finely tune VS Code to do what I want and I miss those features when I'm editing elsewhere. If I setup configuration or plugin that's non-optimal for note-taking, I can adjust that in the workspace settings
I don't love tiddlywiki but haven't found a better alternative yet. I'd love something easy to configure but powerful like the wiki and bug tracker on the Fossil SCM, but not tied to a specific platform. Also, Markdown/text/html because no vendor should own my thoughts. Obsidian[3] and Notion are in my "to check" list.
[1] https://tiddlywiki.com/
[2] https://fossil-scm.org/home/doc/trunk/www/index.wiki
[3] https://obsidian.md/