Home

You’re Doing That Wrong is a journal of various successes and failures by Dan Sturm.

Creating Unique Footnotes with MultiMarkdown Metadata

Footnotes. Writers love ’em. But if you’re not paying proper attention when creating them, you can quickly make a mess of your website or blog. In fact, until very recently, this site had broken footnote links all over the place. I’m going to pretend like none of you noticed, and tell you about how I fixed the problem, gently glossing over my past stupidity.

Each post on this site starts as a MultiMarkdown document in Sublime Text 2. When it’s complete, I preview it in Marked, then copy the HTML code and paste it into a new Squarespace post.

The Problem

Thing is, since I’m creating the post locally on my Mac, with no reference to the site as a whole, Marked has no way of knowing which footnote IDs have already been used, and which have not. Therefore each post labels its first footnote fn:1 and subsequent footnotes increase by 1, as you’d expect. The problem occurs when viewing the home page that shows 10 posts on a single page, each with their own fn:1. When you click on the [1] link in the body text, where do you think it’s going to take you? That’s right, to the first footnote it finds with the label fn:1, regardless of which post it was intended to link to [1].

Now, Marked has a setting in its preference pane called Use Random Footnote IDs which is used “to avoid conflicts when multiple documents are displayed on the web.” So this is already a solved problem, right? Probably, but there’s a part of my brain that feels uneasy about using randomly generated IDs. I’m sure using this feature would solve my problem and everything would be fine, but since I’m a nut, I want to control exactly how these unique footnotes are created.

Enter MultiMarkdown. MultiMarkdown includes a number of enhancements to the original Markdown syntax [2], including support for customizable metadata fields [3]. So, I created a custom metadata key called footnote:. Here’s what my boilerplate MultiMarkdown header looks like [4] :

Title:            
Author:         Dan Sturm      
Date:           %snippet:ddate%  
Marked Style:   ds_doc  
footnote:         

Now, whatever keyword I choose to add after the footnote: label will show up in the HTML header as [5].

Make With The Magic

The next thing I needed was a script to scan the HTML for the metadata key, and add it to all the footnote IDs, turning fn:1 into fn:my-footnote-key1.

import sys  
import re  
import subprocess


#   Open the file and convert it to searchable text  

fileText = open (sys.argv[1], "r+")  
searchText = fileText.read()


#   Pull the MultiMarkdown metadata key from the header  

mmdkey = re.search("(?<=\"footnote\" content=\")(.*)(?=\"/>)", searchText, re   .I)  
mmdkey = mmdkey.group()


#   Create the new footnote IDs  

fnnew = ("fn:", mmdkey)  
fnrefnew = ("fnref:", mmdkey)  

fnnew = "".join(fnnew)  
fnrefnew = "".join(fnrefnew)


#   Swap the footnote IDs for the new ones  

fnfix = re.sub("(fn:)", fnnew, searchText)  
fnreffix = re.sub("(fnref:)", fnrefnew, fnfix)


#   Strip HTML Header and Body tags. Copy the result to Clipboard  

def setClipboardData(data):  
  p = subprocess.Popen(['pbcopy'], stdin=subprocess.PIPE)  
  p.stdin.write(data)  
  p.stdin.close()  
  retcode = p.wait()  

fixStripped = re.sub("(?s).*(\n)\n", "", fnreffix)  
fixStripped = re.sub("(?s)\n\n\n().*", "", fixStripped)  
setClipboardData(fixStripped)


#   Write the updated code to the same file  

# fileText.seek(0)  
# fileText.write(fnreffix)  
# fileText.truncate()  
# fileText.close()  

Click here to download the script.

Now that we have that out of the way, let’s talk about how to actually use this code. I’m using a Hazel [6] rule pointed at a folder called Create Unique Footnotes. It looks like this:

The Hazel Rule

Simply put, it watches the folder for a new file and, when it sees one, it runs the script, displays a confirmation message, then trashes the now-unnecessary file, leaving the cleaned up, new code on the clipboard for easy pasting into your CMS of choice. So, when I like what I see in Marked and I’m ready to post, I hit CMD+S and save the HTML file to my Create Unique Footnotes folder, head over to my site, and hit CMD+V into a new post.

A Small Rabbit Hole

There are a few specific things I want to point out, just to make sure we’re clear.

The new HTML code is copied to the clipboard by the Python script, not by the Hazel rule. My workflow involves pasting HTML code into Squarespace, not uploading an HTML file and, since my MultiMarkdown file is the master copy of the post, that’s why I’m deleting the HTML file after running the script.

The HTML file being used as input for the script is a complete HTML document with a header, body tags, the whole shebang. I don’t need those extra bits when I’m pasting the code into Squarespace, so the Python script removes them before placing the code on the clipboard. In fact, the code on the clipboard looks exactly like what you see when you toggle the Switch to HTML Source View button in Marked [7] while previewing your MultiMarkdown document.

Another important note; keen observers will notice the last four lines of the Python script are commented out. That code is there for people who actually want a fully processed HTML document with fixed footnote IDs. Un-commenting-out [8] those four lines will write the fixed code over the original HTML document, while maintaining the header, et cetera. If you plan on using this workflow, you’ll want to remove the step in the Hazel rule that throws away the HTML document. I’d suggest you change that part of the rule to move the new HTML file to the Desktop as a sort of visual confirmation that the file has been updated.

I Believe Some Recognition is in Order

When I initially conceived of this idea I hadn’t the slightest idea how to best go about tackling it. I am not a programmer in any sense of the word. A Twitter conversation with my pal Sid O’Neill revealed that REGEX is the method by which I could find and replace items in the code. I don’t know how to do that, so…

Patterns is a Mac app for creating Regular Expressions that I’ve heard a number of people compliment in the past. It lets you see a live preview of the current pattern matches and, when you’ve got the selection you want, will generate the appropriate code for you; Python in my case. Completely indispensable and only $2.99.

Next, of course I have to thank Stack Overflow from whence I acquired some REGEX knowledge and code. Patterns comes with a great Reference Sheet for REGEX commands, but some of the descriptions still left me befuddled. Stack Overflow also linked me to…

This article by Gabe at Macdrifter, where I got the code that places the HTML on the clipboard. It was exactly what I needed, so I took it. And I would have gotten away with it if it wasn’t for you pesky kids and…nevermind.

In any case, high-fives all around. Go team.


  1. Some of what I’m describing is obfuscated by the use of Bigfoot for my footnotes, specifically the numeral for each footnote, but the problem remains. Let’s move on.  ↩

  2. Like footnotes, for one.  ↩

  3. You can learn all about MultiMarkdown Metadata here.  ↩

  4. Always invoked via TextExpander.  ↩

  5. Obviously, I need to choose a unique key for each post or this whole exercise is pointless.  ↩

  6. You do use Hazel, don’t you? Good.  ↩

  7. It’s in the upper right corner of your document, next to the fullscreen button.  ↩

  8. That’s so not a word, is it?  ↩

Converting MultiMarkdown to HTML with TextExpander

If you’re here, reading this site, chances are pretty good that you use and love MultiMarkdown as much as I do. That, or you’re at least fairly curious and have a plenty of free time.

With more and more applications supporting Markdown natively, the need to convert the text to HTML is decreasing in frequency. However, the need isn’t completely gone. Sometimes you just need some good old fashioned HTML.

A Hammer For Every Season

(Or Some Other Metaphor That Actually Makes Sense)

There are almost as many ways to get your MultiMarkdown into HTML as there are applications that support Markdown.

I’ve created build systems for Sublime Text to convert MultiMarkdown documents to HTML files. I’ve got a build system that lets Marked do the heavy lifting for me. For the times I don’t need a full MultiMarkdown document, just a small snippet of text, I’ve got Brett Terpstra’s killer MultiMarkdown Service Tools.

It’s probably due to the myriad of tools at my disposal that I only recently discovered I’m unable to use Brett’s OS X Services in Sublime Text 2. I’ll admit, I didn’t try very hard, but after Control + Clicking, trying my keyboard shortcuts, and doing a bit of searching online, I quickly gave up and decided to build a tool I knew would work.

To TextExpander We Go!

When I need a system-wide tool that works in any application, activated by a few quick keys, the answer is almost always TextExpander. With its ability to act as an intermediary between text and scripts, TextExpander is the hammer that always gets the job done [1].

The snippet:

#!/bin/bash
pbpaste | /usr/local/bin/mmd

Just like my other text processing scripts, proper use involves selecting the text to be processed, copying it to the clipboard, and invoking the snippet, which I’ve bound to the command ;mmd. Now, any time I need to convert MultiMarkdown text into HTML, without the hassle of saving files and opening specific applications, I’ve got a quick, universal keyboard command I can use. Bringing me one step closer to an application agnostic workflow.

Hooray!


  1. Yes, I’m sticking with the hammer metaphor. I’m in too deep to turn back now.  ↩

Launching Marked from Sublime Text 2

I do pretty much all my MultiMarkdown writing anymore in Sublime Text 2. I’ve come to rely on Marked for previewing my files with a variety of custom CSS files, depending on the type of project I’m writing.

At the moment, the most annoying part of this workflow is the time it takes to open Marked and locate the MultiMarkdown file I’m currently working on in Sublime Text. I think I’ve been spoiled by the speed of using Sublime Text’s Goto Anything feature (Command + P) for opening files.

To speed things along, I wrote a new build system for Sublime Text that launches the active document in Marked. Now previewing my active file is as easy as invoking the build command which, for me, is still Command + B.

The (insanely simple) code [1]:

{  
"shell": "true",  
"cmd": ["open -a marked \"$file\""]  
}  

More non-rocket-science, but a big time saver in my world.


  1. This build system is only for OS X.  ↩

SP-MMD Cheaters

Since Brett Terpstra gave us Cheaters, I’ve been populating the app with the various tools I use frequently.

I’ve finally gotten around to creating a cheat sheet for my MultiMarkdown Screenplay syntax. I expect it to be used by precisely one person. Me.

However I’m posting it here to serve as a shorter explanation of the way I write, to spare you from reading the whole back story. It uses the same modified CSS file I created for the Fountain cheet sheet.

Here’s the SP-MMD Cheaters page.

And here’s the cheat sheet in a web-friendly view for curious passers-by.

My Hazel CMS

It should be no surprise to anyone familiar with the app, that Noodlesoft’s Hazel is amazing. Today I setup an automated system using Dropbox, Automator, and Hazel to process MultiMarkdown documents into HTML, give them web-ready filenames, and upload them to my website.

Everything on this site starts as a MultiMarkdown text file. I preview the page in Marked, and when its ready to go live, I save a copy of the file in a folder called _1_ready_, which has 3 Hazel rules applied.

Ready

First things first, the text file needs to be converted to an HTML document. I’ve used the Run Shell Script action to call the mmd command.

html_from_new.png

Pretty straight forward. To add some flexibility, the second Hazel rule for the _1_ready_ folder checks for changes to preexisting text files, and processes them using the same bash script. That saves me from having to delete and re-copy a file to make an update.

html_from_updated.png

The last rule for the _1_ready_ folder renames the HTML file, making it entirely lowercase, replaces the spaces with underscores, and moves the new file to a folder called _2_go_.

rename_and_move.png

It’s important to make sure the name element in the with patern: section is set to lowercase, and the replace text dialog is used to swap the spaces for underscores.

Go

The _2_go_ folder automatically uploads any file it finds to the root of my site. As with _1_ready_, the final files are left in the folder to more easily make changes later. Additionally, I keep a copy of my Blog index page and rss XML in the _2_go_ folder so I can quickly update the main page with links to the new posts.

upload_to_ftp.png

To upload the files, the Hazel rule calls an Automator Workflow that uses the Transmit [1] upload action to log into my site (stored as a favorite), and drop everything in the root folder, overwriting files if necessary.

transmit_automator.png

Dropbox

The best part about this whole workflow is Dropbox. Both the _1_ready_ and _2_go_ folders are in my Dropbox, giving me the ability to drop in files from my iPhone, iPad, etc. With apps like TextExpander and Nebulous Notes, there’s no reason I can’t create, and post entirely from an iOS device. Obviously I’ll need a Mac, running and online, but the flexibility of this workflow is well worth the cost of a dedicated system.

Needless to say, I’m incredibly excited to have this new capability, and I can’t wait to see what other workflow magic I can create with Hazel


  1. Many thanks to Macdrifter for recommending Transmit. I love this app.  ↩

Update: I've since revised my upload method to use a Bash script, rather than Transmit. It's much faster and more efficient, so if this idea interests you, you should definitely check it out.

MultiMarkdown Build Systems for Sublime Text 2

When I started using Sublime Text 2 as my primary text editor, last year sometime, I created a build system to more quickly process my MultiMarkdown files. Since I couldn’t find a preexisting MultiMarkdown build system in the Sublime Text forum, it’s probably a safe bet that others might find it useful for me to post mine.

Since all of my writing is based on MultiMarkdown and varying CSS files, I use the same build system for screenplays, blog posts, presentations, etc. When I created the initial build system I was doing the majority of my writing on a Windows 7 machine. Since that time, I have retired all of my Windows computers [1] and created a new build system for OS X (10.7.3).

Here are both build systems:

OS X

{
"shell": "true",
"path": "/usr/local/bin",
"cmd": ["mmd \"$file\""]
}

Windows

{
"shell": "true",
"cmd": "multimarkdown -b \"$file\"",
"cmd": "\"\"${file/\\.txt/\\.html/}\"\""
}

The last line in the Windows build system is a launch command that will open the processed document in your default HTML application. Since I have Marked on my Mac, I decided to omit the launch command from the OS X version and pick my viewer on a per-file basis.


  1. For a number of reasons, I’m still required to use a VM of Win 7 on my Mac via Parallels.  ↩

My MultiMarkdown SPMD Workflow

MultiMarkdown and SPMD

Back in August when Stu Maschwitz proposed Screenplay Markdown (formerly SPMD, now Fountain), I got very excited. After giving Stu way more input than he needed, I decided I wanted to use something like SPMD sooner rather than later.

Due to my lack of any real programming skills, I decided the quickest way to get something usable was to continue writing in MultiMarkdown and use simple CSS to transform my output into something that looked like a screenplay.

My solution is in no way as robust or polished as Stu’s SPMD proposal, but having used it for a few months now I find it suits my needs pretty well. You can find my SP-MMD(?) CSS file here.

Why MultiMarkdown?

There are a few reasons I chose to use MultiMarkdown instead of standard Markdown syntax. The first reason is it’s inclusion of metadata and the ability to create a “complete document” rather than just an HTML “snippit”.

From the MultiMarkdown user guide:

If you include metadata in your document (with two exceptions), then you will generate a complete document. If you don’t include metadata, then you will instead generate a “snippet.” The snippet will just include the relevant portion of HTML, but will not include the <head> or <body> elements.

The metadata features give me a nice looking header to all my text files and, more importantly, creates a complete HTML document that’s ready for the web. You may be asking, “But why does it need to be ready for the web? It’s a screenplay.” As much as I love writing in my text editor, eventually I will need to preview the final output of the properly formatted document.

Previewing

Ideally to view the final document I’d use something like Brett Terpstra’s awesome Marked app. But, unfortunately, I do the majority of my writing on a Windows 7 machine (please don’t email me [1] ). Regardless of the operating system on the machine itself, I like the idea of doing my previewing in a browser for two reasons. First, let’s face it, any computer you sit down at these days is going to have at least one browser installed, and that’s just one less app I need to worry about installing for my workflow to function properly. The second reason I user the browser for previewing is print support. MultiMarkdown supports some great text to pdf conversion tools through LaTeX, but the simpler solution is simply to print to PDF directly from the browser preview.

Another benefit of creating an HTML document from my text file is the ability to quickly upload it to my website and send a link to people. Having that online backup saves my ass when I’m away from a computer and someone who has lost their copy needs to read my script ASAP. Of course I always have my iPhone with me and I can easily resend the PDF via Dropbox, but that takes longer than just emailing or texting a URL.

CSS

The other huge advantage to using MultiMarkdown is the CSS metadata key. While Markdown is capable of using standard HTML code within the document, I much prefer the cleanliness and simplicity of the MultiMarkdown implementation.

Example. To link my CSS file in a standard Markdown file, I can use the line:

<style type="text/css"> <!-- @import url("http://dansturm.com/ds_script.css"); -->    </style>

Not exactly something I love to see at the top of all of my text files. With MultiMarkdown it’s as simple as:

CSS: http://dansturm.com/ds_script.css

Much better. And being that it’s cleaner to read, if I want to switch to a different CSS file for different formatting, I can more quickly and easily modify the URL.

My SP-MMD Syntax

The next problem MultiMarkdown solved was the fact that I would simply be using existing Markdown syntax elements in ways they were never intended, rather than creating my own language from the ground up. MultiMarkdown solves that problem by having additional syntax features not found in Markdown.

Standard MultiMarkdown *italics* and **bold** syntax is preserved. There is currently no support for underline text.

Basic Action elements of the script have no additional syntax associated with them (just as in Stu’s SPMD) and only required basic margin adjustments to the <p> tag to create the correct looking format.

For other elements of a screenplay I need the ability to have things left, center, and right justified. I appropriated the H1, H2, and H3 tags for this purpose. Examples:

For things like scene headings (left justified via H1), I write:

# Int. Coffee Shop - Day

Since scene headings should be written in CAPS, we can use the magic of CSS and the text-transform: uppercase command to automatically capitalize the line. It may not seem like a big deal, but it’s automating small things like this that allow me to keep my mind on the words and not the formatting.

Elements needing center alignment like the script’s title (NOT characters or dialog), use the H2 tag:

## Fancy Screenplay
## By Dan Sturm

Centered text is different from left or right justified text in that it is not subject to auto capitalization. I felt it’s content needed to be flexible.

And, as expected, right aligned text is written as:

### Cut to

Now for dialog. I needed control over both Character and Dialog elements separately, but I didn’t want each to require different syntax modifiers. As luck would have it, MultiMarkdown’s support for Definition Lists fit this need perfectly. To write dialog the syntax is:

Dan
: Is this guy completely insane?

Using just a colon before the dialog (and no spaces following the character) gives the character name the <dt> tag, and the dialog receives the associated <dd> tag. Again, by way of the text-transform: uppercase command, our character name is automatically capitalized. Two script elements defined by a single syntax character. Just what I was looking for.

To add a parenthetical to a line of dialog, I used the code-block designation of surrounding the line with backticks. I wish additional syntax wasn’t necessary for parentheticals, but at the moment this is the best way I could get the result I needed. Example:

Dan
`(pointing)`
: Is this guy completely insane?

Since there’s really no built in tool for pagination of HTML files, I felt a need to create a tool to force page breaks. A horizontal rule is all that’s required to force a page break. The forced page breaks are only visible when the document is sent to print, but I think the horizontal rule does a decent enough job of communicating the page break in the browser.

Known Issues

Since I put this little project together in a couple hours one night, when it reached the point of being “usable”, I stopped working on the code and went back to writing my script. As expected, there are a few known issues that may or may not be deal breakers for you.

The first issue is related to margins. When printing from the browser, I cannot print all the way to the edge of the paper. If there’s a way to create a full bleed PDF from a browser, I haven’t found it yet. Why is this a problem? My CSS originally assumed a full 8.5“ x 11” piece of paper from which to base it’s margin adjustments for each element. Printing from the browser scales that full size image down to something closer to 8“ x 10.5” ruining the rest of the formatting. I’ve attempted to compensate for those margins, but since I’m not 100% sure how large the margins the browser is adding actually are, it’s just guesswork. I plan to revisit this problem in the next rev of the CSS.

The next issue is one that doesn’t really affect me in the writing I personally do, but may be an issue for others. When creating parentheticals within a line of dialog (i.e. not immediately after the character and immediately before the dialog) the resulting margin is incorrect. Since I don’t currently have a method for determining where in the dialog the parenthetical occurs, there is no workaround for this issue. Example of writing that creates a problem:

Dan
: Is this guy completely insane?
`(pointing)`
: Or is he just full of it?

Again, something I plan on revisiting in the future.

Currently Unsupported

In my implementation there is currently no support for the following screenplay elements:

  • Dual Dialog
  • Sections and Synopses
  • Underlined Text

Pain Vs Flexibility

If you still think this whole idea is crazy and way more painful than the associated benefit, allow me to share a few tips and benefits.

As any good writing nerd, I use TextExpander (and Breevy on the PC) to alleviate some of the pain of the syntax. Page breaks via horizontal rules are added by typing “hhrl”, which creates 5 asterisks. I use “iint” to create “# INT. (FILL-BOX) - (FILL-BOX)”. I have about a dozen snippets that automate frequently used screenplay syntax (and correct typos). Another great benefit of using TextExpander is the ability to have a snippet turn CAPS on and off as needed. And if I somehow manage to override the CAPS on any particular line, the CSS will correct my mistake for me. Again, another small tool that helps keep my mind on the words.

Another reason I love this process may be a bit of blasphemy to many of you, but since it makes my life easier, it may do the same for some of you. At the very bottom of my scripts is usually a page break (horizontal rule) followed by a list of links to other relevant files the customer may need to see. Here I add links to storyboards, budgets, video reference, etc. And since it follows a page break, it’s easy enough to leave out of the printed version by designating a page range in the print dialog box. I love having a linked list of important files directly attached to the screenplay because, frankly it’s one less email I have to receive from the customer who loses their copy of those associated files.

No, I’m Not Crazy

In summary, yes, this is a really weird way to write a script and yes, it probably seems like more work that it’s worth, but I really believe the benefits and flexibility of MultiMarkdown and CSS greatly outweigh the hassles, even if my current implementation is far from ideal. If you think this mess will work for you, feel free to use, copy, steal, or modify my CSS file. And I’ll be sure to let you know if I ever get around to making v0.02.

To close, here’s how a sample script would look in my SP-MMD format.

Title: Fancy Script
Author: Dan Sturm
Date:
CSS: http://dansturm.com/ds_script.css

## FANCY SCRIPT
## By Dan Sturm

* * * * *

# EXT. GAS STATION - DAY

A bright hot day. A truck pulls up to the pump. The DRIVER steps out and heads inside.

The PASSENGER lights a cigarette and waits.

Eventually, the DRIVER bursts out the gas station door, GUN in one hand, MONEY in the other.

Driver
: Start the car!

Passenger
`(under his breath)`
: No shit.

### CUT TO:

# INT. NEXT SCENE - DAY

And Here’s what the above script looks like when formatted


  1. Borrowed, with love, from the great Marco Arment.  ↩