Skip to main content

Happy Git and GitHub for the useR

34 Make a GitHub repo browsable

The unreasonable effectiveness of GitHub browsability. One of my favorite aspects of GitHub is the ability to inspect a repository’s files in a browser. Certain practices make browsing more rewarding and can postpone the day when you must create a proper website for a project. Perhaps indefinitely.

34.1 Be savvy about your files

Keep files in the plainest, web-friendliest form that is compatible with your main goals. Plain text is the very best. GitHub offers special handling for certain types of files:

  • Markdown files, which may be destined for conversion into, e.g., HTML
  • Markdown files named README.md
  • HTML files, often the result of compiling Markdown files
  • Source code, such as .R files
  • Delimited files, such as CSVs and TSVs
  • PNG files

34.2 Get over your hang ups re: committing derived products

Let’s acknowledge the discomfort some people feel about putting derived products under version control. Specifically, if you’ve got an R Markdown document foo.Rmd, it can be knit() to produce the intermediate product foo.md, which can be converted to the ultimate output foo.html. Which of those files are you “allowed” to put under version control? Source-is-real hardliners will say only foo.Rmd but pragmatists know this can be a serious bummer in real life. Just because I can rebuild everything from scratch, it doesn’t mean I want to.

The taboo of keeping derived products under version control originates from compilation of binary executables from source. Software built on a Mac would not work on Windows and so it made sense to keep these binaries out of the holy source code repository. Also, you could assume the people with access to the repository have the full development stack and relish opportunities to use it. None of these arguments really apply to the foo.Rmd --> foo.md --> foo.html workflow. We don’t have to blindly follow traditions from the compilation domain!

In fact, looking at the diffs for foo.md or foo-figure-01.png can be extremely informative. This is also true in larger data analytic projects after a make clean; make all operation. By looking at the diffs in the downstream products, you often catch unexpected changes. This can tip you off to changes in the underlying data and/or the behavior of packages you depend on.

This chapter explores cool things GitHub can do with various file types, if they happen to end up in your repo. I won’t ask you how they got there.

34.3 Markdown

You will quickly discover that GitHub renders Markdown files very nicely. By clicking on foo.md, you’ll get a decent preview of foo.html. Yay! You should read GitHub’s own guide on how to leverage automatic Markdown rendering.

Exploit this aggressively. Make Markdown your default format for narrative text files and use them liberally to embed notes to yourself and others in a repository hosted on Github. It’s an easy way to get pseudo-webpages inside a project “for free”. You may never even compile these files to HTML explicitly; in many cases, the HTML preview offered by GitHub is all you ever need.

34.4 R Markdown

What does this mean for R Markdown files? Keep intermediate Markdown. Or only render to Markdown. Commit both foo.Rmd and foo.md, even if you choose to .gitignore the final product, e.g. foo.html or foo.pdf or foo.docx. From September 2014, GitHub renders R Markdown files nicely, like Markdown, and with proper syntax highlighting, which is great. But, of course, the code blocks just sit there un-executed, so my advice about keeping Markdown still holds.

If your target output format is not Markdown, you want YAML frontmatter that looks something like this for .Rmd:

---
title: "Something fascinating"
author: "Jenny Bryan"
date: "`r format(Sys.Date())`"
output:
  html_document:
    keep_md: TRUE
---

or like this for .R:

#' ---
#' title: "Something fascinating"
#' author: "Jenny Bryan"
#' date: "`r format(Sys.Date())`"
#' output:
#'   html_document:
#'     keep_md: TRUE
#' ---

The keep_md: TRUE part says to keep the intermediate Markdown. In RStudio, when editing .Rmd, click on the gear next to “Knit HTML” for YAML authoring help.

Since 2016, rmarkdown offers a custom output format for GitHub-flavored markdown, github_document. Read about R Markdown workflows for explicit examples of how to use this. If Markdown is your target output format, your YAML can be even simpler and look like this for .Rmd:

---
output: github_document
---

or like this for .R:

#' ---
#' output: github_document
#' ---

For a quick, stand-alone document that doesn’t fit neatly into a repository or project (yet), make it a Gist. Example: Hadley Wickham’s advice on what you need to do to become a data scientist. Gists can contain multiple files, so you can still provide the R script or R Markdown source and the resulting Markdown, as I’ve done in this write-up of Twitter-sourced tips for cross-tabulation. I’ve collected YAML examples for all the above scenarios in a gist.

34.5 README.md

You probably already know that GitHub renders README.md at the top-level of your repo as the de facto landing page. This is analogous to what happens when you point a web browser at a directory instead of a specific web page: if there is a file named index.html, that’s what the server will show you by default. On GitHub, files named README.md play exactly this role for directories in your repo.

Implication: for any logical group of files or mini project-within-your-project, create a sub-directory in your repository. And then create a README.md file to annotate these files, collect relevant links, etc. Now when you navigate to the sub-directory on GitHub the nicely rendered README.md will simply appear. The GitHub repo that backs the gapminder data package has a README in the data-raw subdirectory that explains exactly how the package data is created. In fact, it is generated programmatically from README.Rmd.

Some repositories consist solely of README.md. Examples: Jeff Leek’s write-ups on How to share data with a statistician or Developing R packages. I am becoming a bigger fan of README-only repos than gists because repo issues trigger notifications, whereas comments on gists do not.

If you’ve got a directory full of web-friendly figures, such as PNGs, you can use code like this to generate a README.md for a quick DIY gallery, as Karl Broman has done with his FruitSnacks. I did same for all the fantastic O RLY book covers made by The Practical Dev.

I have also used this device to share Keynote slides on GitHub (mea culpa!). Export them as PNGs images and throw ’em into a README gallery: slides on file organization and some on file naming.

34.6 Finding stuff

OK these are pure GitHub tips but if you’ve made it this far, you’re obviously a keener.

  • Press t to activate the file finder whenever you’re in a repo’s file and directory view. AWESOME, especially when there are files tucked into lots of subdirectories.
  • Press y to get a permanent link when you’re viewing a specific file. Watch what changes in the URL. This is important if you are about to link to a file or to specific lines. Otherwise your links will break easily in the future. If the file is deleted or renamed or if lines get inserted or deleted, your links will no longer point to what you intended. Use y to get links that include a specific commit in the URL.

34.7 HTML

If you have an HTML file in a GitHub repository, simply visiting the file shows the raw HTML. Here’s a nice ugly example:

No one wants to look at that. You can provide this URL to rawgit.com to serve this HTML more properly and get a decent preview.

You can form two different types of URLs with rawgit.com:

2018-10-09 update: RawGit announced that it is in a sunset phase and will soon shut down. They recommended: jsDelivr, GitHub Pages, CodeSandbox, and unpkg as alternatives.

This sort of enhanced link might be one of the useful things to put in a README.md or other Markdown file in the repo.

You may also want to check out this Chrome extension or GitHub & BitBucket HTML Preview, though recently I’ve more success with rawgit.com. (Neither work with private GitHub repos, which is all the more reason to keep intermediate markdown files for HTML, as described above.)

Sometimes including HTML files will cause GitHub to think that your R repository is HTML. Besides being slightly annoying, this can make it difficult for people to find your work if they are searching specifically for R repos. You can exclude these files or directories from GitHub’s language statistics by adding a .gitattributes file that marks them as ‘documentation’ rather than code. See an example here.

34.8 Source code

You will notice that GitHub does automatic syntax highlighting for source code. For example, notice the coloring of this R script. The file’s extension is the primary determinant for if/how syntax highlighting will be applied. You can see information on recognized languages, the default extensions and more at github/linguist. You should be doing it anyway, but let this be another reason to follow convention in your use of file extensions.

Note you can click on “Raw” in this context as well, to get just the plain text and nothing but the plain text.

34.9 Delimited files

GitHub will nicely render tabular data in the form of .csv (comma-separated) and .tsv (tab-separated) files. You can read more in the blog post announcing this feature in August 2013 or in this GitHub help page.

Advice: take advantage of this! If something in your repo can be naturally stored as delimited data, by all means, do so. Make the comma or tab your default delimiter and use the file suffixes GitHub is expecting. I have noticed that GitHub is more easily confused than R about things like quoting, so always inspect the GitHub-rendered .csv or .tsv file in the browser. You may need to do light cleaning to get the automagic rendering to work properly. Think of it as yet another way to learn about imperfections in your data.

Here’s an example of a tab delimited file on GitHub: lotr_clean.tsv, originally found here (nope, IBM shut down manyeyes July 2015).

Note you can click on “Raw” in this context as well, to get just the plain text and nothing but the plain text.

34.10 PNGs

PNG is the “no brainer” format in which to store figures for the web. But many of us like a vector-based format, such as PDF, for general purpose figures. Bottom line: PNGs will drive you less crazy than PDFs on GitHub. To reduce the aggravation around viewing figures in the browser, make sure to have a PNG version in the repo.

Examples:

  • This PNG figure just shows up in the browser
  • A different figure stored as PDF produces the dreaded, annoying “View Raw” speed bump. You’ll have to click through and, on my OS + browser, wait for the PDF to appear in an external PDF viewer. 2015-06-19 update: since I first wrote this GitHub has elevated its treament of PDFs so YAY. It’s slow but it works.

Hopefully we are moving towards a world where you can have “web friendly” and “vector” at the same time, without undue headaches. As of October 2014, GitHub provides enhanced viewing and diffing of SVGs. So don’t read this advice as discouraging SVGs. Make them! But consider keeping a PNG around as emergency back up for now.

34.11 Other document formats

You may also have a document you want others to be able to browse and interact with, but it is not in the markdown format. Fortunately, the open-source Pandoc program, written by John MacFarlane, allows you to convert a range of formats into markdown, including the widely used .docx format.

When you click the Knit button in RStudio it is actually Pandoc which performs the final conversion to HTML or Microsoft Word (.docx) formats. If you are willing to use the command-line, you can perform the opposite conversion (eg .docx to .md), commonly retaining features such as headings, tables, equations and even figures.

As some boilerplate, running in Windows PowerShell pandoc --extract-media .\media -f docx .\example.docx -t markdown_github -o example_image.md converts a word document called example.docx to markdown, and extracts the images into a directory which corresponds to a filepath in the newly created example.md document. A full list of supported formats and example code for conversions are available at https://pandoc.org/.

You can also perform simple conversions to GitHub-flavored markdown from different markdown flavours (Pandoc supports markdown_mmd, markdown_php_extra and markdown_strict) from within RStudio. To do so you need to rename the file by changing the extension (eg from foo.md to foo.Rmd), then open the renamed file in RStudio and add the following text to the top of the document.

---
output: github_document
---

You can then click on “Knit” then “Knit to github document” to perform the conversion. See Output format for more details of controlling output formats with the YAML frontmatter.

34.12 Linking to a ZIP archive of your repo

The browsability of GitHub makes your work accessible to people who care about your content but who don’t (yet) use Git themselves. What if such a person wants all the files? Yes, there is a clickable “Download ZIP” button offered by GitHub. But what if you want a link to include in an email or other document? If you add /archive/master.zip to the end of the URL for your repo, you construct a link that will download a ZIP archive of your repository. Click here to try this out on a very small repo:

https://github.com/jennybc/lotr/archive/master.zip

Go look in your downloads folder!

34.14 Let people correct you on the internet

They love that!

You can create a link that takes people directly to an editing interface in the browser. Behind the scenes, assuming the click-er is signed into GitHub but is not you, this will create a fork in their account and send you a pull request. When I click the link below, I am able to actually commit directly to master for this repo.

CLICK HERE to suggest an edit to this page!

Here’s what that link looks like in the Markdown source:

[CLICK HERE to suggest an edit to this page!](https://github.com/jennybc/happy-git-with-r/edit/master/workflows-make-github-repo-browsable.Rmd)

and here it is with placeholders:

[INVITATION TO EDIT](<URL to your repo>/edit/master/<path to target source file>)

AFAIK, to do that in a slick automatic way across an entire repo/site, you need to be using Jekyll or some other automated system. But you could easily handcode such links on a small scale.