Tiddlywiki: A Database for Your Notes (2022): rpdillon.net

Tiddlywiki: A Database for Your Notes (2022)

2022-02-15

Tiddlywiki's name suggests it is some sort of toy, and that impression is somewhat reinforced when the first major hurdle to using it is...saving your changes. It's as though Tiddlywiki's design is intentionally trying to convey a sense of "please, whatever you do, don't take me seriously." This is extremely unfortunate, and is part of the inspiration for Notedeck.

After using Tiddlywiki for a few years, my conclusion is almost the exact opposite. Tiddlywiki is a serious piece of software that is well-maintained, tested, and has a large user base. But why prefer it over any other software? There are a lot of reasons I prefer it, but in this post, I'd like to highlight its flexibility, specifically in the same way a database like sqlite is flexible. Because Tiddlywiki really is a database, albeit one that is document-oriented.

Tiddlers are Structured Documents

"Tiddlers", as Tiddlywiki calls them, are really just JavaScript objects. For those that aren't familiar, you can think of a JavaScript object like a dictionary. You can look up words (called fields, keys, or attributes...they're all the same thing), and find their definition (called values). To visualize this a bit better, we can use JavaScript Object Notation (JSON) to see their structure. Here's an example of the tiddler that contains this post (with only the first sentence of the post, because recursion):

{
  "created": "20220208233147027",
  "creator": "rpdillon",
  "text": "Tiddlywiki's name suggests it is some sort of toy, and that impression is somewhat reinforced when the first major hurdle to using it is...saving your changes.",
  "tags": "Draft Writing",
  "title": "Tiddlywiki: A Database for Your Notes",
  "modified": "20220209182511988",
  "modifier": "rpdillon"
}

Here, you can see it has several attributes that Tiddlywiki uses to store and track it: created, creator, text, tags, title, modified, and modifier, each with a value indicating the contents of that attribute.

The interesting thing here is that Tiddlywiki doesn't only allow its own fields to be added; you can add any fields you want. If you want to add a field called summary to some tiddlers, you can. This means you can attach arbitrary metadata (data about the data) to tiddlers. This turns out to be really useful, because you can use all that metadata to query for tiddlers you're interested in.

Querying Tiddlers with Filter Expressions

Tiddlywiki provides a query language for tiddlers called filter expressions. This allows you to do things like search for tiddlers whose text attribute contains the word "copyright" ([regexp:text[copyright]]), for tiddlers created in 2022 ([regexp:created[^2022]]), or for tiddlers that are tagged with "Draft" ([tag[Draft]]).

To play around with this interactively, Tiddlywiki provides a search tool in the $:/AdvancedSearch tiddler that has four tabs, the last of which is called "Filter", which accepts a filter expression. Results of the filter expression are provided as you type, which is a fast way to get comfortable with how filter expressions work.

Applications of Filter Expressions

I highly recommend playing with the interactive search, but that's not my most common use of filter expressions. My main use of filter expressions is inside various calls to macros and widgets that then perform some action on the selected set of tiddlers.

Organizing Notes into a Hierarchy

As the title of this post suggests, much of the power of Tiddlywiki comes from using it like a database of notes. This allows you to create a topic, like Recipes as a "hub" for your recipes. Once you create the Recipes tiddler, you might want different sections, like Breakfast food, Breads, Desserts, and Dinners. Creating those tiddlers and tagging them with Section allows you to use the $list widget to display a link to each section:

<$list filter="[tag[Section]]">
</$list>

If you're familiar with programming in other languages, this is Tiddlywiki's version of a "for" loop, which iterates over all the tiddlers that pass the filter. Right now, this call does nothing, because there is nothing inside the widget's tag. But it's easy to embed widgets inside other widgets, so we can add a link using the $link widget.

<$list filter="[tag[Section]]">
  <$link/>
</$list>

One thing to note here is that the $link widget allows you to specify a tiddler to link to, but the default is currentTiddler, which the $list widget binds for you as it loops over the tiddlers specified by the filter. This makes the syntax for this common case very simple. In fact, listing links is such a common use case in Tiddlywiki that Tiddlywiki includes a macro to do exactly that called list-links:

<<list-links "[tag[Section]]">>

Quick Access to Recent/Favorite Notes

Filters can be passed to the $list widget, which can then tranclude them. One way I use this is with a "Recent" note that aggregates the last few days of my worklogs, since I tend to access my notes most frequently shortly after they are created. This filter selects the last 5 most-recent notes tagged Journal and displays their title (with a link) and their content:

<$list filter="[tag[Journal]!sort[created]limit[5]]">
  <h3><$link/></h3>
  <$transclude mode="block"/>
</$list>

A similar approach can be used to create "favorites". By tagging notes with Favorite, for example, a list of favorites is easy to create using list-links:

<<list-links "[tag[Favorite]]">>

One thing to note here: because notes can be tagged with multiple tags, they can exist in multiple heirarchies simultaneously, so a note can, for example, be both a part of a section and a favorite.

Finding Backlinks

A common feature of many note systems is the ability to not only follow links forwards from where the link appears to the content it references, but to follow links in reverse, from the content back to the place where it was linked. These are called backlinks, and Tiddlywiki has a built-in filter operator that finds them. Combining this with the now-familiar list-links macro, it's trivial to get a list of backlinks for the current tiddler:

<<list-links "[is[current]backlinks[]]">>

This first narrows the filter to only the current tiddler using the is operator, and then expands the selection to any tiddler that links back to the current one. Without the is[current] filter first, the list will include all tiddlers that link to other tiddlers.

Filters as Conditionals

Coming from a programming background, I was amazed to find that Tiddlywiki has no apparent means to implement branching logic. That is, a way to say "If this tiddler has the tag Draft, then add a red banner to the top of it." Tiddlywiki can indeed implement this sort of logic, and it does it using our old friends: the $list widget and filters! The insight here is that a loop over zero elements never executes, so by putting the conditions into the filter and passing that to a $list, Tiddlywiki will only execute whatever is inside the $list widget if the filter returns 1 or more results.

So, to continue with the example, we can add a banner to the current tiddler, only if it is tagged Draft:

<$list filter='[is[current]tag[Draft]]'>
@@background-color:red;color:white;padding:3px;
''THIS IS A DRAFT''
@@
</$list>

This can be used to implement lots of conditionals, like only adding a backlinks section if some backlinks exist. Notedeck uses this to implement its Log behavior, which transforms a tiddler into a logging tiddler if it is tagged with Log.