Authoring Markdown with Zotero - My Workflow

This post describes an authoring workflow that combines the simplicity of markdown (for writing) with the power of a reference manager (for citing and generation of a bibliography).

This is my current writing workflow:

1. Collect references, articles, books etc. via Zotero, my citation manager. You might call this the “research phase”.
2. Edit article/text/book using markdown editor.
3. Zotero exports your references to a file (continuously in background)

4. Use make to compile markdown to final result:

   4a. pandoc reads your markdown source file
   4b. pandoc reads the bibliography file
   4c. pandoc formats the bibliography entries following the desired citation style, described in a .csl file
   4d. pandoc creates desired output format(s), like docx or html.

You like more details? Then follow along…

Why Markdown (sometimes) isn’t enough

I don’t need to advertise the advantages of markdown as free,
lightweight, simple yet powerful-enough approach to authoring arbitrary content like articles, blogposts (like this one!) or even books.

But: Markdown is optimized for writing, not for citing. If you refer to other sources, like websites, blogs, books or articles in your writing, you want to systematically reference these sources. An example:

See the original article about Markdown
[1] for further info.

In markdown source, this would be the following:

See the original article about Markdown ([@gruberDaringFireballMarkdown]) for further info.


For a blog post, one could simply include the hyperlink, but for any
printed publication a real reference is more appropriate, often
required by publishers. Such references in your bibliography will look similar to the following example:

[1] J. Gruber, “Daring Fireball: Markdown.” https://daringfireball.net/projects/markdown/.

In case you write just a single article, this isn't a big problem: Manually add your sources at the end of your text, and you’re done. But if you (like myself) publish regularly, you might want to have a central repository of your references including publication dates, publisher info, links etc. That’s where Zotero comes into play:

Zotero

Zotero is a free, open-source and user-friendly tool to help you collect, organize and cite references,
like articles, books or research papers (see [2])

Zotero is available for all major platforms (Win, Mac, Linux).

You can sync your reference library across multiple devices and add new references either by ISBN, DOI or similar ways. For online publications, powerful browser plugins are available to add any webpage to your library, together with a snapshot for offline viewing!

Required Zotero Plugins

You need to add BetterBibTeX (short bbt, see [3]) to your Zotero installation.
bbt can export your Zotero library (or a subset of it) to an arbitrary file system location - and keep this export updated! Wheneve you add a new reference to Zotero, this exported file gets updated too!

Adding References to a Text

I like the editing experience of VS-Code, mainly for its rich plugin universe. One of these plugins is the Zotero-Citation-Picker:
It gives you a search box and retrieves results directly from your Zotero library.

When you have found the correct reference, the plugin enters the unique citation key at the current cursor location.

Compile the Bibliography

Now you have a markdown text containing several of these “citation keys” - but still no list of references, the so called bibliography*.

That’s where the next player enters the field: pandoc (see [4] ).

Compile Output with Pandoc

“If you need to convert files from one markup format into another,
pandoc is your swiss-army knife.”

Although you could use pandoc directly from the command line, I prefer good old make, as I cannot remember parameters and options. Look at my Makefile:

OUTPUT=out
SOURCE=content.md # name of your markdown source file

CITATIONSTYLE=ieee.csl
BIBLIO=~/Dropbox/zotero-export/zorg-biblio.bib
TARGET=$(OUTPUT).docx $(OUTPUT).md

PANDOC=pandoc \
   --from=markdown \
   -s --bibliography $(BIBLIO) \
   --citeproc \
   --csl=$(CITATIONSTYLE) \
   --metadata link-citations=true \
   $(SOURCE)

# create docx
$(OUTPUT).docx: $(SOURCE) $(METADATA)
   $(PANDOC) --output=$(OUTPUT).docx


# create github flavored markdown
$(OUTPUT).md: $(SOURCE) $(METADATA)
   $(PANDOC) --to=gfm --output=$(OUTPUT).md

all: $(TARGET)

clean:
   rm -rf $(OUTPUT).docx $(OUTPUT).md

.PHONY: all clean $(TARGET)
.DEFAULT_GOAL := all

(This Makefile is available as Github-gist)

Yes, I know: One could wrap pandoc into a tiny Docker container - but pandoc is easy to install and is incredibly fast. A Docker container would remove the dependency to pandoc, but add the overhead of a running container.

When you try this on your own, please fulfill the following
prerequisites:

• install pandoc
• add the citation-style csl-file to your current directory (I didn’t manage to keep this file in any central location - help or suggestions would be appreciated)
• add the Makefile to the current directory
• update the BIBLIO variable in the makefile to point to your Zotero export location.

Image Credit

Splash image by Trent Erwin on Unsplash.

Bibliography

This bibliography has been auto-generated by the workflow shown above.

[1] J. Gruber, “Daring Fireball: Markdown.”
https://daringfireball.net/projects/markdown/.

[2] “Zotero | Your personal research assistant.”
https://www.zotero.org/.

[3] E. Heyns, “Better BibTeX for Zotero.” Dec-2021.
https://retorque.re/zotero-better-bibtex/

[4] “Pandoc - About pandoc.” https://pandoc.org/.

Comments

[Anton Rhein]

Thank you for this article, I am using a similar workflow 👍🏻.
I wrote my bachelor thesis using pandoc-markdown. But I wasn't able to mitigate LaTeX entirely, especially when dealing with mathematic formulas 😉.