Blogdown with submodules

August 22, 2022

Update December 16, 2022: I’ve since decided to remove my notes section from the site. My notes formatting was by nature rough and imprecise, which kept causing all sorts of problems with the site’s rendering. The details below on how to utilize a submodule within Blogdown should still be applicable. I still take notes in Markdown.

Here’s a walkthrough on how to configure Blogdown to have a segregated section of content that’s tracked in an entirely separate git (or rather, Github) repository that in turn is handled as a submodule by the Blogdown repository.

It’s useful in cases where there’s a need to maintain the core Blogdown site in one repository and some distinct body of content in another repository.

Why? #

I’m a big note-taker, and I’ve tried all sorts of different digital note-taking systems over the years. I’ve tried Evernote, I’ve tried Google Notes, I’ve tried Microsoft OneNote, and a whole bunch more.

One thing I never really liked about note platforms is how dependent I’d be on their service providers if I were to use those systems. I’ve always been wary of making my notes vulnerable – which I spend a lot of time and effort on – to the whims of some third party that I have no influence over.

Eventually, I landed on taking notes in Markdown. It’s simple, it’s lightweight, and most importantly, it’s portable. At its core, Markdown is just a bunch of text files.

Taking notes in Markdown means my notes are never locked into some proprietary format; I can simply copy and paste them from one place to another and still use them just as effectively.

I often take notes across many different devices (not all of those devices need access to this site’s code), so to keep them widely accessible, I stash them in a Github repository.

Since the notes were written in Markdown and also tracked in a Github repository, I realized it should be feasible to treat that repository as a submodule to the Blogdown repository that this site is built on. Making this work would accomplish two things for me:

1. It’d make my notes super accessible, both to me and anyone else interested in my notes.

2. It’d help me continue to build upon the content of this site almost effortlessly. For whatever reason, I find writing blog posts extremely tough. Maybe it’s the hangup with getting the content right and polished. Notes on the other hand flow effortlessly – as I consume books, articles, videos, I jot down notes. Sometimes in rough shorthand. And I have no shame with sharing rough notes. They might be ugly, but I find them useful.

How? #

The process of turning a notes repository into a content submodule within a Blogdown site is fairly straightforward.

Blogdown sites contain their content within the content/ directory. Each directory within that top-level content directory is a separate collection of content.

The top level of my content/ directory for this site for instance looks like this:

$ tree -L 1 content
content
├── _index.md
├── about
├── blog
├── disclosure.md
├── food
├── form
├── license.md
├── notes
├── patents
├── resources
└── talk

Where about is the content for the About section, Blog is the content for the Blog section, and so on.

Typically, with a Blogdown or any other statically generated site, all the content is maintained within a single repository.

In my case, what I’ve done is I’ve excised the notes section from the main Blogdown repository, and instead, set it up as a submodule.

To do so, I ran the following command:

git submodule add <url to my notes repository> <path for where to put submodule>

Where <url to my notes repository> points to my already-existing notes repository, and <path for where to put submodule> led to contents/notes. This command clones the remote repository as a submodule within the Blogdown repository.

It’s possible to set a submodule to track a particular branch. In my case, I’ve left it to default to tracking the main branch of my notes repository.

Whenever I now navigate from my Blogdown directory into my contents/notes, I’m effectively switching git repositories. The higher level Blogdown is one repository, and the notes directory is another repository. Each repository can be handled like any conventional git repository, including being able to checkout branches, make changes, commit and merge changes, and so on.

One modification I did have to make to my notes repository to get it to play nicely was to introduce an _index.md file at the top-level, structured similarly to any other _index.md file in any other content section.

I also had to add a pointer to the notes in the config.yaml file in my Blogdown so it can be reached from the navigation bar:

  header:
    ...
    - name: Notes
      title: Notes
      url: /notes/
      weight: 2
    ...

Additionally, the individual notes, the distinct Markdown files, have to massaged slightly into a particular form that fits with the expected Blogdown structure. Fortunately, this isn’t terribly different from how I was doing things before.

The main changes to the notes are:

1. Each individual note has to reside at a path of the form <note name>/index.md. Whatever the <note name> is, that will eventually become the path to the note. For instance, a note Markdown file for iterm2 would exist in the notes repository at the path iterm2/index.md, which eventually would be rendered at the path notes/iterm2/ (which can be tweaked with a YAML slug – see the next point’s snippet).

2. The top section of each note Markdown file should contain some YAML markup to define particular attributes of the note.

---
title:  
author:  
date: ''
slug: []
categories:
  -  
tags:
  -  
summary:  
---

Whenever the notes repository changes, the Blogdown repository is aware that some change occurred. To pull down those notes changes and have them reflect in the main repository and in turn on the site, I simply have to navigate to the Blogdown repository and run a commit for the submodule change, and push that commit.

In my setup, doing so causes my Netlify implementation to build and deploy the changes. Within a matter of moments, my notes section should update and reflect the latest set of notes.

My workflow #

These days, my workflow looks something like this:

1. I take notes in Markdown, tracked within my notes repository. The notes are saved to a path of the form <note name>/index.md. The top of the Markdown file includes some markup. When I’m done with my notes, I push them to my main branch.

2. Once I’m ready to push my notes to my site, I navigate to my Blogdown repo, navigate to content/notes, run a git pull to grab the latest state of the main branch of my notes. This step may not be necessary if I’ve been working on one device the whole time, but that’s rarely the case for me.

3. I navigate up to my Blogdown repo, and commit and push the latest notes (usually something like git commit -am "update notes" && git push). This should then deploy my latest notes to my site.

And that’s it.

Netlify gotchas #

I use Netlify to deploy my site.

For Netlify to be able to access a submodule’s content so it can deploy everything properly, the submodule must be a public repository, and the submodule specifications at .git/config and .gitmodules must utilize the Github https repo path, not the ssh path.

A note on my notes #

I make no guarantees about my notes. They are precisely what I’ve labeled them – notes. That means they’re very rough around the edges and typically not fit for easy consumption. Their purpose is to help me stash knowledge in a retrievable form so I don’t have to bother with committing that content to memory.

The roughness of my notes is the main distinction between blog and notes – the blog has some polish and is readable, the notes are rough and raw.