notes | journal | about | archive | sourcehut | github

building a crude static site generator in common Lisp

Update September 2019: I think this post is kind of silly (which is how I inevitably feel about anything I do after more than 6 months), but maybe someone somewhere will like it. This code eventually morphed into a static site generator that I used for this website for some time.


I'm learning Common Lisp from the famous "Gentle book". I can't recommend this book enough, and it's also available for free online.

Anyway, as a break from my studies, I decided to test my rudimentary knowledge of CL by building a static website generator. I also wanted to test the language a bit as well, testing the "code is data, data is code" maxim by putting the website content and styling into the same file as the generator; the source code itself includes the content on each webpage. For obvious reasons this isn't the best way to make a static site generator – actually, a lot of things about this program are horrible. But as a CL beginner I learned a lot from making it, so maybe it will help out other beginners.

Before I get started, here are a few disclaimers:

Design Goals

Before writing the program, we have to decide what we want it to do.

Basically, we want a program that can define shared elements for our webpages, so that every new page we add will have the same styling, menu, etc. This is what static site generators like Hugo and Jekyll do, except they do it with more complexity and options. For now, let's forget all of that and build an example website.

The example site I built has two pages, "index.html" and "contact.html".

Here is the code for the "index.html" webpage:

  <HTML>

  <HEAD>

  <TITLE>Site Title</TITLE>

  </HEAD>

  <div id="menu">
      <ul>
          <li><a href="contact.html">Contact</a></li>
          <li><a href="index.html">Home</a></li>
      </ul>
  </div>
  <BODY BGCOLOR="FFF000">

  <CENTER><IMG SRC="https://barkpost-assets.s3.amazonaws.com/wp-content/uploads/2013/05/63.jpg" ALIGN="BOTTOM"> </CENTER>

  <HR>
  This is a test homepage.<hr>
   <B><I>This is the footer. Be sure to <a href="contact.html">Contact Me</a>.
    </I></B>

  </BODY>

  </HTML>

Here is the code for the "contact.html" webpage:

  <HTML>

  <HEAD>

  <TITLE>Site Title</TITLE>

  </HEAD>

  <div id="menu">
      <ul>
          <li><a href="contact.html">Contact</a></li>
          <li><a href="index.html">Home</a></li>
      </ul>
  </div>
  <BODY BGCOLOR="FFF000">
   <H1>This is a Header</H1>  
   <H2>This is a Medium Header</H2>  
   Send me mail at <a href="mailto:support@yourcompany.com">  
                support@yourcompany.com</a>.  
   <P> This is a new paragraph!      
   <B>
   <I>This is the footer. Be sure to <a href="contact.html">Contact Me</a>.
    </I></B>

  </BODY>

  </HTML>

If you compare the two webpages, you'll notice that they are built with the same HTML in the header and the footer. The only difference is the "content", or body of each page. We can use this structure in our program.

Before we start defining the structure of our program, I want to point out one drawback of designing according to this structure. Notice the footer in the webpage? If we were making our own website, we'd probably want to include a slightly different footer for the contact page, because it links to itself, which is silly. This would be easy to accomplish by writing a separate part of the program to build the contact page, but this simple structure (header, per-page content, and footer) won't allow it.

Building a Webpage

Now we're ready to start building the program. The most obvious starting point is to store the header, footer, and webpage content in variables. However, this is frustrating if you don't like escaping every quotation mark found in standard HTML:

  <div id=\"menu\">
      <ul>
          <li><a href=\"contact.html\">Contact</a></li>
          <li><a href=\"index.html\">Home</a></li>
      </ul>
  </div>
  <BODY BGCOLOR=\"FFF000\">

  etc...

It's tedious.

Instead, let's use something called a reader macro; this is one of those things where I'm going to skip explaining, and just steal someone else's implementation. Let Over Lambda has a simple macro, Sharp Greater Than, that we can use to input HTML into our program as-is. It works like this: instead of typing quotation marks when you're designating a string, type "#>" followed by a phrase that you'll use to signal the end of the string, for example END-OF-THE-STRING. Here's how I defined the footer in my lisp program:

  (defparameter footer #>EOF
  <hr>
   <B><I>This is the footer. Be sure to <a href="contact.html">Contact Me</a>.
    </I></B>
  </BODY>
  </HTML> 
  EOF)

I used the phrase "EOF" to designate the start and finish of the string, so that I could type whatever I wanted between them. Of course, you couldn't include EOF anywhere in the webpage or the string would end prematurely, and you'd get errors.

Use the same technique to put the header into a variable called "header". Now create a temporary variable to hold the content for the "index.html" webpage. In a minute, we'll store the content of our pages in a tree so the program can automatically cycle through them and build the website, but for now lets store it in a variable called "index.html".

  (defparameter index.html "This is a test homepage.")

Now we have three strings that we need to concatenate and write to a file. Practical Common Lisp has a good chapter on files and file i/o, but if you don't want to get into that stuff yet, Richard Socher provides a simple writetofile function that we can use. It takes two strings name and content, and writes content to the file located at name in the filesystem. For example, the following code writes "Snow dusts a forest of hoodoos" (and a newline) to the file "hoodoo.txt" in my home directory:

  (writetofile "/home/wrycode/hoodoo.txt" "Snow dusts a forest of hoodoos~%")

Now we can test a "manual" build of one of the webpages, before we start abstracting things away in our program. Try building a webpage using "writetofile" and the "concatenate" function:

  (writetofile "/home/yourusername/location/for/index.html" 
   (concatenate 'string header index.html footer))

Make sure to modify the "name" argument so that the webpage gets built where you want. Also, notice the 'string in concatenate - that simply tells the function that it's dealing with strings.

Finishing the Program

I mentioned earlier that we were going to store the different webpages in a table. In lisp an association list, or alist for short, is really a table. I'm not going to into too much detail about that, because it's covered very well in Common LISP: A Gentle Introduction to Symbolic Computation, Chapter 6.

The following code stores the content of the two webpages in an alist with the keys being symbols that represent the filenames:

  (defparameter *pages* '(
      (index.html "This is a test homepage.")
      (contact.html #>EOF
  <H1>This is a Header</H1>
  <H2>This is a Medium Header</H2>
  Send me mail at <a href="mailto:support@yourcompany.com">
               support@yourcompany.com</a>.
  <P> This is a new paragraph!
   EOF)))

The alist is stored in the global variable *pages*. Note that footer and header are also global variables in this program. I used a normal double-quoted string for the content of the homepage, but contact.html had quotes so I used the #> read macro, just like in the header and footer.

Now, we need to write code to cycle through *pages* and build and write the code for each webpage using the commands from earlier.

First, a function to build each webpage, which takes the body content of the page as its only argument:

  (defun build-page (page-content)
    (concatenate 'string header page-content footer))

build-page outputs a string containing the entire file, so if we call it with "This is a test homepage." as the argument, it will correctly build the index.html file.

The next function, build-site is a little messier, but I left it like this so it can be fiddled with. At the end of the post I'll list some simple improvements that could be made to the program.

  (defun build-site ()
    (dolist (page *pages*)
      (let ((page-content (car (rest page))) (page-name (car page)))

        (writetofile 
        (concatenate 'string *siteloc* (string-downcase (string page-name))) 
        (build-page page-content))
        )))

The dolist loops through every element in pages. It binds the current element to the variable page and executes the forms below it before moving to the next element. In our case, the list will only execute twice, because we only have two pages.

The let form is for setting the temporary variables page-content and page-name. Note that page-name is actually a symbol, not a string, because I chose to write the table with symbols as the keys instead of strings. This will matter in a moment.

After the temporary variable bindings, let proceeds with the writetofile form. For the name parameter, we are constructing a filename out of the site's base directory and the page name. Remember how page-name is a symbol? string fixes that, and I chose to downcase it as well (it doesn't matter on the World Wide Web, but local file systems are case-sensitive, so the menu links in our header and the contact link in our footer would be broken for local viewing otherwise). Make sure to define *siteloc* before running the code yourself.

The content argument to writetofile is pretty self-explanatory; it calls the build-page function we just built with the page-content variable, and outputs the complete HTML webpage for writetofile to use.

That's the whole program! I've moved on to my next project, but here are some ideas for changing or extending it: