Generating the Homepage

generator goldsmith web

The first iterations of go back to the early 2000s, back when I was still in school. Back then, the site was entirely hand-written (proudly in Notepad!) and relied extensively on HTML framesets to reduce boilerplate code. Over the years, as frames fell out of favor due to issues with search engines and bookmarks, I had to rethink this approach.

This website has been rebuilt more times than I care to count. The stack evolved from using Wordpress, to ConcreteCMS, to Pelican, to Hexo, to Hugo before briefly pausing on Metalsmith. Static site generation seemed to be the way to go (and yes, of course my Wordpress site was hacked) for simplicity and portability. One problem I ran into was that most of the static site generators placed seemingly arbitrary restrictions on site structure. To enumerate a few of the issues:

Out of all the static site I generators evaluated, Metalsmith came the closest to what I needed. I could finally begin to see a path forward to a homepage structured the way I wanted. Unfortunately, Metalsmith was quite temperamental due shortcuts added in the pipeline design in what I can only assume to be an attempt to make the generator easier to use. It became clear to me that I needed to build my own tooling to get the results I could be truly satisfied. Goldsmith, my Golang-based static generator was born in late 2015.


Here is a sampling of some hard requirements I relied on to guide me through implementation:


Let’s just dive into the code. Many details have been omitted for the sake of brevity; let’s just focus flow for now:

func build(contentDir, buildDir, cacheDir string, devMode bool) {

    // ...

    errs := goldsmith.Begin(contentDir).
        Chain(syndicate.New(baseUrl, "Feed").WithFeed("posts", feedConfig)).

    for _, err := range errs {


  1. Recursively scan all the files in the contentDir and start pushing them through the pipeline:

  2. Enable caching in cacheDir for supported plugins; this helps avoid reprocessing of unchanged resources:

  3. Enable cleanup of extra files found in the buildDir that are not build output:

  4. Chain the rule plugin to enable per-directory rules, which are specified in a rules.toml file:


    Rules can skip files matching a patterns set. To prevent .git directories from being published, we can use the drop directive:

    accept = [

    Rules can also help specify metadata on files that we do not want to modify directly (such as the file for a project). We can instead match the file and apply properties to it externally:

    accept = ["goldsmith/"]
    props.Collection = "ProjectsActive"
    props.Description = "Static pipeline-based website generator written in Go."
    props.GitHub = "goldsmith"
    props.Tags = ["generator", "golang", "goldsmith", "mit license", "web"]
  5. Repositories on GitHub are expected to contain the project description inside of a file. We want to rename all instances of to and ensure that each project directory will eventually contain an index.html file. I added a simple indexRenamer helper plugin to automate this:


    The implementation of this one-off plugin is shown below:

    type indexRenamer struct{}
    func (*indexRenamer) Name() string {
        return "indexRenamer"
    func (*indexRenamer) Initialize(context *goldsmith.Context) error {
        return nil
    func (*indexRenamer) Process(context *goldsmith.Context, inputFile *goldsmith.File) error {
        dir := path.Dir(inputFile.Path())
        inputFile.Rename(path.Join(dir, ""))
        return nil
  6. Chain the frontmatter plugin to parse and strip out any metadata prefixed onto our files:


    This metadata consists of arbitrary properties and can be defined in several formats. The TOML frontmatter for this page is:

    Date = 2023-10-10
    Description = "Exploring the uniquely custom process of generating this website with Goldsmith."
    Tags = ["web", "goldsmith", "generator"]
  7. Chain the markdown plugin to convert all Markdown files to HTML fragments:


    At this stage, these are not yet documents, but merely excerpts that must have header and footer markup templates applied to them before they become functional pages. This plugin uses the goldmark Markdown parser, which supports GitHub extensions, syntax highlighting and more. The desired features can be optionally configured by passing in a goldmark instance:

    gm := goldmark.New(
        goldmark.WithExtensions(extension.GFM, extension.Typographer),
  8. Push a condition filter that ensures that pipeline manipulation within its scope only happens when the page is not being generated in devMode (preparing content for deployment to the web server):


    When generating the site for deployment, we chain the absolute plugin to convert all relative URLs to absolute ones using the baseUrl ( prefix. This is required to ensure that all relative references are resolvable when the site’s content is displayed in an RSS reader. We also apply the minify plugin to reduce the site’s content to the minimal possible size without impacting behavior. We do not want to do this in development mode since it makes debugging the code a lot tricker! Once these two steps are completed, we pop off the condition filter and resume unconditional processing.

  9. Chain the syndicate plugin to generate feeds for selected articles (like this one):

    Chain(syndicate.New(baseUrl, "Feed").WithFeed("posts", feedConfig)).

    Only pages with the Feed metadata property with the value posts are considered during feed generation. We must also provide a basic site configuration and declare the desired feed formats.

    feedConfig := syndicate.FeedConfig{
        Title:       "FooSoft Productions",
        Url:         baseUrl,
        AuthorName:  "Alex Yatskov",
        AuthorEmail: "",
        RssPath:     "feeds/posts.xml",
        AtomPath:    "feeds/posts.atom",
        JsonPath:    "feeds/posts.json",
        ItemConfig: syndicate.ItemConfig{
            TitleKey:        "Title",
            CreatedKey:      "Date",
            DescriptionKey:  "Description",
            ContentFromFile: true,
  10. Chain the collection plugin to group related pages into lists based on the value of the Collection metadata property:


    The resulting collections are ordered using a custom comparer function which groups items first by Date and then Title properties:

    collectionComparer := func(i, j *goldsmith.File) bool {
        if iData, iOk := i.Prop("Date"); iOk {
            if jData, jOk := j.Prop("Date"); jOk {
                return iData.(time.Time).UnixNano() > jData.(time.Time).UnixNano()
        if iData, iOk := i.Prop("Title"); iOk {
            if jData, jOk := j.Prop("Title"); jOk {
                return strings.Compare(iData.(string), jData.(string)) < 0
        return strings.Compare(i.Path(), j.Path()) < 0
  11. Conditionally chain the index plugin to generate directory index pages for paths that do not match the **/*.gohtml pattern:


    These template files are consumed later in the pipeline and we don’t need to include them in the build output. Newly generated index pages are assigned the metadata configured in the plugin:

    indexMeta := map[string]any{
        "Layout": "index",
  12. Chain the tags plugin to generate tag indices from page metadata:


    By default, this plugin builds lists based on the contents of the Tags property. Some additional configuration may be provided to to specify what metadata is assigned to newly generated tag index pages.

    tagMeta := map[string]any{
        "Area":   "tags",
        "Layout": "tag",
  13. Chain the forward plugin to generate stub pages for content on the homepage that has been moved to a different URL:


    We must specify the URL mapping as well as any additional metadata that should be assigned to newly generated forwarder pages:

    forwardMeta := map[string]any{
        "Layout": "forward",
        "Title":  "File Moved",
    forwardPathMap := map[string]string{
        "mangle/index.html":                   "/projects/mangle/",
        "projects/argwrap/index.html":         "/projects/vim-argwrap/",
        "projects/kanji-frequency/index.html": "/posts/kanji-frequency/",
        "projects/yomichan-chrome/index.html": "/projects/yomichan/",
        "yomichan/index.html":                 "/projects/yomichan/",
  14. Chain the layout plugin to apply Go HTML templates to our page HTML fragments:


    By default, the plugin searches for the Layout metadata property to decide which template to apply. All metadata associated with a file can be referenced from within these templates. The templates themselves are stored in *.gohtml files which are consumed by the pipeline along with all the other inputs. For example, the top-level template for the current page looks as follows:

    {{define "page"}}
    {{template "header" .}}
    {{template "footer" .}}
  15. Chain the thumbnail plugin to automatically generate thumbnails for all processed image files:


    By default, this produces PNGs thumbnails resized to 128x128 pixels. The filename is automatically determined by appending -thumb.png to the source image filename after stripping the extension. For example, foo.png becomes foo-thumb.png.

  16. Conditionally chain the livejs plugin in development mode to inject JavaScript to refresh the page when a change is detected:


    This script continuously queries the server to detect changes. It should never be enabled on the homepage once it has been deployed.

  17. Conditionally chain the minify plugin once more to also minify the HTML templates we applied a couple of steps ago:


    This plugin is chained twice so that both the RSS output (which is spun-off earlier in the pipeline) as well as the full site are minified.

  18. Finally, write out all the generated files to the buildDir directory.


    Today, over 400 files are generated by Goldsmith and output into the target directory. This process is highly parallelized and takes just over 100ms to complete with caching enabled and 130ms for a full rebuild. The results are fully deterministic, which makes inspecting content changes between website deployments easy.

Error Handling

If a plugin reports an error while processing a file, the file stops propagating through the pipeline. A list of errors is returned upon calling End. These errors contain the name of the file in question as well as the faulting plugin.

Directory Structure

One unique thing about the on-disk organization of is that projects are actually included as submodules within the website repository. This makes it easy to keep the project’s up to date with the data shown on the generated homepage: it’s literally the same file! I view this as an invaluable feature for keeping the project brand separate from where the project is hosted.

├── build
├── cache
├── content
│   ├──
│   ├── CNAME
│   ├── css
│   │   └── main.css
│   ├── gohtml
│   │   ├── components.gohtml
│   │   └── content.gohtml
│   ├── go.mod
│   ├── img
│   │   ├── alex.png
│   │   └── brand.svg
│   ├──
│   ├── js
│   │   └── main.js
│   ├── portfolio
│   │   ├── amaze
│   │   ├── arenanet
│   │   ├──
│   │   ├── microsoft
│   │   └── rules.toml
│   ├── posts
│   │   ├── decrapifying-the-twitter-timeline
│   │   ├── fixing-firefox-video-playback-in-fedora-37
│   │   ├──
│   │   ├──
│   │   ├── installing-diablo-ii-on-linux
│   │   ├── kanji-frequency
│   │   ├── rules.toml
│   │   ├── saying-goodbye-to-vimscript
│   │   ├── squashing-all-commits-in-a-git-topic-branch
│   │   ├── sunsetting-the-yomichan-project
│   │   └── working-with-patches-in-git
│   ├── projects
│   │   ├── anki-connect
│   │   ├── ankijoy
│   │   ├── btac
│   │   ├── goldsmith
│   │   ├── goldsmith-components
│   │   ├── goldsmith-samples
│   │   ├── guid.nvim
│   │   ├── hlm2-wad-extract
│   │   ├── homemaker
│   │   ├──
│   │   ├── jmdict
│   │   ├── lazarus
│   │   ├── mangle
│   │   ├── md2vim
│   │   ├── mdview
│   │   ├── meganekko
│   │   ├── metacall
│   │   ├── moonfall
│   │   ├── restaurant-search
│   │   ├── revolver.nvim
│   │   ├── rules.toml
│   │   ├── scrawl
│   │   ├── tetrys
│   │   ├── vfs
│   │   ├── vim-argwrap
│   │   ├── yomichan
│   │   ├── yomichan-anki
│   │   ├── yomichan-import
│   │   ├── zero-epwing
│   │   └── zero-epwing-go
│   ├── rules.toml
│   ├── tags
│   │   └──
│   └── ttf
│       ├── MaterialIcons-Regular.ttf
│       ├── RobotoMono-Regular.ttf
│       ├── Roboto-Regular.ttf
│       └── RobotoSlab-Regular.ttf
├── go.mod
├── go.sum
├── main.go


While this may seem like there are a lot of steps required to build this site, the actual cognitive load is very low. The statically generated output for is a perfect match for my expectations and requires no hacks. Iteration is fast and adding new features is a breeze. If this seems interesting, take a look at the goldsmith-samples repository for some simple examples to get started!