Reproducible Research in Practice

Neil Shephard

n.shephard@sheffield.ac.uk

Research Software Engineer, Department of Computer Science, University of Sheffield

Scan This

ns-rse.github.io/reproducible_research_example

Who Am I/What Do I Do?

Background

• BSc Zoology and Genetics
• MSc Genetic Epidemiology
• Genetic Statistician
• Medical Statistician
• Data Scientist
• 🎉 Research Software Engineer 🎉

Skills

• Linux, R, Python, Bash
• Literate Programming
• Reproducible Research
• Open Research
• Free Open Source Software
• FAIR Principles

Before we get into the details I thought it might be useful to explain who am I am and why I’m here to talk to you today.

I started my academic career studying Zoology and Genetics at Undergraduate then moved onto a Masters in Genetic Epidemiology, both here at the University of Sheffield quite a few years ago. After graduating I spent about eight or nine years as a Genetic Statistician trying to work out what genes are involved in complex diseases at various Universities, Manchester, Western Australia and then returning to Sheffield. I then shifted careers to work as a Medical Statistician for about nine years followed by four years in industry as a Data Scientist where I didn’t actually do much data science but did learn a lot about software development which is how I’ve ended up back at the University of Sheffield as a Research Software Engineer.

A common theme from my Masters on-wards is that I have used computers to write programmes and analyse data. I’ve taught myself Linux system administration and from a rudimentary training on Fortran, C and Stata have gone on to learn Bash, R and Python as well as a few other bits and pieces of different languages.

Very early on in my career I developed an interest in literate programming and reproducible research which is why Aneta has invited me her today to talk to you and I’ll expand on these topics and show you some work I’m currently engaged in and how I undertake it in a reproducible manner.

Reproducible Research is one of the foundational aspects of Open Research as by ensuring our work is reproducible and open to review it improves confidence in its accuracy and reliability.

Reproducible Research

• Fundamental to the scientific method.
• Traditionally hand-written notebooks with methods shared in publications.
• Modern equivalent is code…
  • Software
  • Scripts
    • Data cleaning
    • Analysis, graphs & tables
    • Numbers and statistics

Reproducibility is fundamental to the scientific method, researchers have to record what they have done and what happened in their experiments.

Traditionally this will have been done by hand in notebooks with methods and results shared in publications but as with many aspects of the modern world they have transitioned to the digital world and the modern equivalent is code. People write software to analyse their data, whether that is a small survey of a few hundred people, or images obtained from Atomic Force Microscopy which is one of the projects I’ve worked on for the past couple of years. Even statistical analysis which involves data cleaning, analyses to produce graphs and tables summarising responses and statistics about a study is still code.

All this code needs to be legible and understandable to others, and that includes your future self, as you can guarantee that after leaving code that was written in a hurry for a few weeks or longer when you return to it you will often find it is hard to understand what you were doing or why you wrote a particular piece of code.

Literate Programming - Origins

a computer program is given as an explanation of how it works in a natural language, such as English, interspersed (embedded) with snippets of macros and traditional source code, from which compilable source code can be generated.¹

Literate Programming - Donald Knuth

This leads me onto the topic of literate programming and RMarkdown and other languages like it such as the newer Quarto and Emacs’ Org-mode are literate programming languages which may be an unfamiliar term so I’ve borrowed the definition from Wikipedia.

The idea was originally proposed and developed the computer scientist Donald Knuth, and he literally wrote the book on it.

1. Wikipedia - Literate programming

Literate Programming - Better Programmes

My programs are not only explained better than ever before; they also are better programs, because the new methodology encourages me to do a better job.¹

Literate Programming - Donald Knuth

Whilst originating in computer science the principle has seen widespread adoption in research and data science because it encourages reproducible research and facilitates open access to research.

Also, as Knuth found when he adopted the practice it improved the quality of the programmes he wrote, and I’ve found the same is true when writing documents myself since adopting the practice over 20 years ago.

1. Knuth (1984)

What’s Involved?

• Logical, ordered directory structure.
• Scripts/code.
• Version control of code.
• Reproducible environment.

It can seem overwhelming learning a new paradigm or framework for undertaking work. I often found that people want to get on and do the work rather than take a step back and assess how they are going about it but its well worth taking the time out to review and assess your working practices and see how they can be improved.

I’m going to walk you through a project that I’ve been working on that is looking to improve the prediction of Thyroid Cancer. It’s not my own work, it’s the topic for a PhD for a clinician Dr Ovie Edafe who is a specialist in neck cancers and related surgery.

There are four broad areas that I’ll cover that hopefully align with the content you have already covered or will be covering in the coming weeks.

The first step is having an organised structure to your directories having data organised in a consistent and structured manner makes it considerably easier to work with.

You then need to write code or scripts, which are still code, that defines the steps you undertake in cleaning and tidying your data and getting it ready for analysis before another series of scripts are used to run the analysis.

All of the code that is written should be version controlled which allos you to track the history of the code and see what changes have been made over time. If a mistake is made it is possible to revert back to a previous state where things did work. Version control and tools such as Git and GitHub make it much easier to collaborate with others on research too.

One final step is having a reproducible environment under which your work is undertaken. This means the specific versions of packages that you have used for analysis should be recorded and used again in the future. In R this can be done via the renv package. Its not without problems but the situation today is far better than it used to be.

Directory Structure

• Important to keep a copy of the raw data as received.
• Data always needs cleaning.
• ~80% of data analysis is data cleaning/wrangling data!
• Always keep a record of changes you make to the data.
• Separate directories for different data types and script/markdown files.
• Use consistent filename nomenclature. ¹

When undertaking research its imperative to keep a copy of the raw data as received. This might be data from sensors that have captured data and if you work with this sort of data then its likely to be fairly well structured, but can also be the results of surveys that have been captured and stored in databases and then exported for you to work with. Depending on how well the data capture was setup you will almost invariably have to clean your data, in my experience about 80% of data analysis is cleaning the data and wrangling it such as deriving variables like age from date of birth, and on that note if you ever do want to analyse age as a variable try and capture date of birth as you can easily derive the age in years or categorise it into bins, but if you only ask people to say what age band they are in you can’t go back the other way. Common errors in data cleaning are having string characters in variables that are meant to be numeric, or having categorical variables where people haven’t entered the correct field and so you have to then work out how to tidy this which I’ll come on to.

The key thin is that its important to keep a record of the changes you make and these will be in R scripts or Markdown/Quarto files. And it therefore makes sense to separate your data from your scripts.

1. Jenny Bryan - How to Name Files

Directory Structure - Thyroid Cancer Prediction

❱ tree -d -L 2
[4.0K Mar 19 12:36]  .
├── [4.0K Nov 22 14:57]  ./data
│   ├── [4.0K Jan  9 13:32]  ./data/csv
│   └── [4.0K Feb 29 17:27]  ./data/r
├── [4.0K Mar 14 17:36]  ./docs
│   ├── [   7 Jan  5 14:45]  ./docs/data -> ../data
│   ├── [4.0K Feb 29 15:38]  ./docs/modelling_cache
│   ├── [4.0K Mar 14 17:21]  ./docs/modelling_files
│   ├── [   4 Jan  5 14:44]  ./docs/r -> ../r
│   └── [4.0K Mar 14 17:21]  ./docs/_site
├── [4.0K Oct 19 11:40]  ./inst
├── [4.0K Nov 16 10:54]  ./log
├── [4.0K Oct 19 11:24]  ./node_modules
├── [4.0K Oct 19 12:39]  ./pages
├── [4.0K Oct 12 10:43]  ./papers
├── [4.0K Nov  9 11:44]  ./quarto
│   └── [4.0K Oct 19 13:12]  ./quarto/_site
├── [4.0K Mar 14 16:59]  ./r
├── [4.0K Mar 14 17:28]  ./renv
│   ├── [4.0K Nov 16 10:57]  ./renv/library
│   └── [4.0K Feb 29 14:28]  ./renv/staging
├── [4.0K Jan 31 11:00]  ./_site
│   ├── [4.0K Dec 19 16:36]  ./_site/data_files
│   ├── [4.0K Feb 15 15:15]  ./_site/docs
│   ├── [4.0K Dec 19 16:36]  ./_site/modelling_files
│   └── [4.0K Jan 25 17:22]  ./_site/site_libs
└── [4.0K Feb  1 17:47]  ./tmp

29 directories

Key top-level directories

• data/ - holds data.
• docs/ - my Quarto files.
• r/ - my R-scripts.

Here is an example of the directory structure of one of the repositories I have setup for this work.

The output only shows the nesting two directory levels deep, but you can see that at the top level I have a number of directories, not all of them are relevant, but the key ones are listed on the right, there is the data/ directory which will hold all of the data, there are two sub-directories csv for raw CSV files and r which holds cleaned data in R’s byte format which speeds up loading and preserves all the labelling and categorical variables.

The docs/ directory contains my Quarto source files that I write, both Quarto and RMarkdown will make a bunch of other directories when they are rendering your pages which are some of those listed such as _site but you don’t need to worry about those.

The final key directory that I have is r/ which holds the R scripts that I write.

Directory Structure - data/

❱ tree data
[4.0K Nov 22 14:57]  data
├── [4.0K Jan  9 13:32]  data/csv
│   ├── [ 20K Jan  9 13:31]  data/csv/sheffield_thyroid_nodule.csv
│   └── [393K Nov  2 11:08]  data/csv/Thy3000_DATA_LABELS_Raw.csv
└── [4.0K Feb 29 17:27]  data/r
    ├── [ 30K Mar 14 11:12]  data/r/clean.rds
    ├── [136K Feb 29 17:56]  data/r/elastic_net.RData
    ├── [136K Feb 29 17:56]  data/r/lasso.RData
    ├── [129K Feb 29 17:56]  data/r/rf.RData
    ├── [137K Feb 29 17:58]  data/r/svm.RData
    └── [ 14K Feb 29 17:56]  data/r/xgboost.RData

3 directories, 9 files

• data/csv/*.csv - CSV files
• data/r/*.rds - R files
• data/r/*.RData - R files

The data/csv directory holds the raw data and there are two files the raw data sheffield_thyroid_nodule.csv and the data dictionary Thy3000_DATA_LABELS_Raw.csv.

The data/r directory holds the cleaned data in clean.rds and the results of fitting various different types of models to the data in the .RData files.

Directory Structure - docs/

❱ l docs | grep -v "~"
drwxr-xr-x neil neil 4.0 KB Thu Mar 14 17:36:04 2024 .
drwxr-xr-x neil neil 4.0 KB Thu Mar 28 16:35:45 2024 ..
.rw-r--r-- neil neil  10 B  Thu Feb 15 15:16:48 2024 .gitignore
.rw-r--r-- neil neil  12 KB Thu Jan 25 15:35:49 2024 .modelling.qmd
drwxr-xr-x neil neil 4.0 KB Thu Mar 14 17:15:43 2024 .quarto
.rw-r--r-- neil neil 483 B  Thu Mar 14 16:59:28 2024 _quarto.yml
drwxr-xr-x neil neil 4.0 KB Thu Mar 14 17:21:20 2024 _site
.rw-r--r-- neil neil 408 B  Thu Mar 14 16:57:10 2024 about.qmd
.rw-r--r-- neil neil 1.2 KB Thu Mar 14 16:57:10 2024 citations.qmd
lrwxrwxrwx neil neil   7 B  Fri Jan  5 14:45:10 2024 data ⇒ ../data
.rw-r--r-- neil neil  33 KB Thu Mar 14 16:57:10 2024 data.qmd
.rw-r--r-- neil neil 363 B  Thu Mar 14 16:57:10 2024 index.qmd
.rw-r--r-- neil neil 569 B  Thu Mar 14 16:57:10 2024 links.qmd
.rw-r--r-- neil neil 2.0 KB Thu Mar 14 16:57:10 2024 literature.qmd
.rw-r--r-- neil neil  32 KB Thu Mar 14 17:36:04 2024 modelling.qmd
drwxr-xr-x neil neil 4.0 KB Thu Feb 29 15:38:53 2024 modelling_cache
.rw-r--r-- neil neil 3.5 KB Thu Mar 14 17:31:46 2024 modelling_elastic_net.qmd
drwxr-xr-x neil neil 4.0 KB Thu Mar 14 17:21:18 2024 modelling_files
.rw-r--r-- neil neil 5.4 KB Thu Mar 14 17:31:46 2024 modelling_lasso.qmd
.rw-r--r-- neil neil 3.6 KB Thu Mar 14 17:31:46 2024 modelling_random_forest.qmd
.rw-r--r-- neil neil 7.0 KB Thu Mar 14 17:31:46 2024 modelling_tidymodel.qmd
lrwxrwxrwx neil neil   4 B  Fri Jan  5 14:44:09 2024 r ⇒ ../r
.rw-r--r-- neil neil  18 KB Thu Mar 14 16:57:10 2024 references.bib
.rw-r--r-- neil neil 2.7 KB Thu Mar 14 16:57:10 2024 reproducibility.qmd
.rw-r--r-- neil neil  17 B  Thu Mar 14 16:59:28 2024 styles.css

The docs/ directory holds my Quarto files which all end in .qmd. I don’t like having large files to navigate so split my code and papers into a number of smaller files that do specific things and include these from the master document which is index.qmd. There is a references.bib file which contains citations so we can include citations easily. The _quarto.yml files is a key file which defines how the Quarto project is setup and rendered but I won’t cover that in detail here.

Directory Structure - r/

❱ l r | grep -v "~"
drwxr-xr-x neil neil 4.0 KB Thu Mar 14 16:59:28 2024 .
drwxr-xr-x neil neil 4.0 KB Thu Mar 28 16:35:45 2024 ..
.rw-r--r-- neil neil  28 KB Thu Mar 14 16:56:18 2024 clean.R
.rw-r--r-- neil neil 684 B  Thu Mar 14 16:59:28 2024 master.R
.rw-r--r-- neil neil  15 KB Thu Feb 29 14:40:13 2024 modelling.R
.rw-r--r-- neil neil 2.6 KB Thu Mar 14 16:59:28 2024 simulate_data.R
.rw-r--r-- neil neil 233 B  Thu Jan 25 17:02:10 2024 summary.R
.rw-r--r-- neil neil 2.9 KB Thu Mar 14 16:57:10 2024 tidymodel.R

The r/ directory holds my R scripts, the key one is master.R which holds the main commands that setup the R environment, load libraries and set relative directories. The clean.R file does what it says on the tin, it cleans the raw data and saves it in R format. There are then scripts for modelling and simulating data.

Now that we’ve covered how I organise my data we can start looking at some code, but before I move on are there any questions about this aspect?

Scripts/Code - master.R

## Filename    : master.R
## Description : Master file that controls running of all subsequent scripts.
library(dplyr)
library(forcats)
library(Hmisc)
library(lubridate)
library(tidymodels)
library(tidyverse)
library(vip)

## Set directories based on current location
base_dir <- getwd()
data_dir <- paste(base_dir, "data", sep = "/")
csv_dir <- paste(data_dir, "csv", sep = "/")
r_dir <- paste(data_dir, "r", sep = "/")
r_scripts <- paste(base_dir, "r", sep = "/")

## Clean the data
source(paste(r_scripts, "clean.R", sep = "/"))


## Simulate data
source(paste(r_scripts, "simulate_data.R", sep = "/"))

## Run Statistical models
source(paste(r_scripts, "tidymodel.R", sep = "/"))

The master.R script doesn’t do a great deal, it loads libraries and then as the comments show, and its important to get into the habit of putting comments in scripts to help other people, including your future self, it then sets the working directory. It does this in a reproducible manner by first asking the operating system what the current directory is using the getwd() command and then appends various directory paths, such as data and saving it in the data_dir variable, this in turn has csv appended to it and saved in the csv_dir which points to the nested directory where our data files live.

You should get in the habit of using relative directories so that the code runs on different computers/systems, if you had hard coded the path based on where you store the files on your computer the script won’t run on any other computer until someone changes that value, relative paths get around that limitation.

After setting directories a number of scripts are called using the source() command and within it I just paste the name of the script onto the r_scripts variable which holds the path to the r/ directory. The clean.R script is first called, followed by the simulate_data.R and then the tidymodels.R.

Scripts/code - r/clean.R

## Filename : clean.R
## Description : Load and clean raw data, saving as a .RData file for subsequent analyses.

## Read the data and convert to tibble
df_raw <- read_csv(paste(csv_dir, "Thy3000_DATA_LABELS_Raw.csv", sep = "/"))
df_raw <- as_tibble(df_raw)
## Rename columns using dplyr::rename()
df_raw <- dplyr::rename(df_raw,
  record_id = "Record ID",
  data_access_group = "Data Access Group",
  study_id = "Study ID",
  date_referral = "1.1 Date of referral",
  clinic_recruiting = "1.2 Which clinic was the patient recruited from?",
  clinic_recruiting_other = "If Other",
  date_clinic = "1.3. The date the patient was seen in clinic",
  referral_source = "1.4 Referral source",
  referral_source_other = "If Other, please specify",
  two_week_wait_referral = "1.4.1 If GP, was it 2-week wait referral?",
  presentation = "1.5. Presentation",
  presentation_complete = "Complete?...12",
  age = "2.1. Age of the patient when seen in clinic",
  bmi = "2.2. Body Mass Index of patient",
  smoking_status = "2.3. Smoking status",
  ...
  )

Its important to script the work you do on the code, it provides evidence of the changes that have been made to clean the code up and allows the work to be fully reproducible. Such an audit trail is useful as it allows mistakes to be corrected. This and other R and Quarto Markdown files are version controlled using Git which I’ll come onto shortly.

This script, r/clean.R does the hard work of cleaning/tidying the data.

It loads the data in and then renames a bunch of variables because spaces in names are a pain and all the numerical prefixes are pointless.

Scripts/code - r/clean.R (cont.)

## Convert dates to elapsed dates using lubridate
df <- df |>
  mutate(
    date_referral = lubridate::dmy(date_referral),
    date_clinic = lubridate::dmy(date_clinic),
    date_initial_management_decision = lubridate::dmy(date_initial_management_decision),
    date_treatment = lubridate::dmy(date_treatment),
    routine_review_date_last_seen = lubridate::dmy(routine_review_date_last_seen)
  )

## Convert variables that are meant to be numeric but aren't
df <- df |>
  dplyr::mutate(
    bmi = as.numeric(bmi),
  )

• Tidyverse
  • {dplyr}
  • {lubridate}
  • R for DataScience (2e)

Another important step to working with data is to ensure the dates are correct and recognised by the software, the strings people typically enter these as are not something that computers can do calculations on, instead software uses elapsed times and takes as epoch time which is the number seconds that have elapsed from midnight on 1970-01-01 and so internally these are stored as numeric values but they can be represented as human readable dates. But the advantage of having them stored as epoch time is that you can then perform calculations on them and work out periods of time.

Often unless data capture has been done very carefully there will be errors in numerical variables such as including characters which means the whole column for something that is meant to be numeric such as Body Mass Index (bmi) are not the correct type, they’re strings rather than actual numbers. We can use the as.numeric() function here with the Tidyverse package dplyr::mutate() to replace these.

Its worth mentioning the Tidyverse which is an “opinionated collection of R packages for data science” that make working with data in R considerably easier than the base functions. I would highly recommend the book.

Perhaps the biggest and best feature of the Tidyverse is the pipe, here I’m using the base pipe |> which allows commands to be written in a Western-style of grammar, reading from left to right and top down rather than in base R where you would nest functions within each other.

This vastly improves the readability of the code.

Scripts/code - r/clean.R (cont.)

check_cols <- c(
  "previous_neck_irradiation",
  "incidental_imaging",
  "retrosternal",
  "palpable_lymphadenopathy",
  "nodule_rapid_growth",
  "thyroid_function_3months",
  "ultrasound",
  "elastography",
  "ct_neck",
  "mri_neck",
  "iodine_scan",
  "nodule_fna",
  "core_biopsy",
  "routine_review_ultrasound",
  "routine_review_fna",
  "routine_review_patient_signposting_information"
)
df <- df %>%
  dplyr::mutate(across(
    all_of(check_cols),
    ~ dplyr::recode(.x,
      "Yes" = 1,
      "No" = 0,
      "Not Known" = NA_real_
    )
  ))

• thyroid-cancer-prediction/r/clean.R

Another common task in cleaning data is to convert string variables such as Yes / No / Not Known to numerical binary variables. Typically there will be lots of columns on which you want to do this and rather than write the same code out lots there are a lot of neat ways of repeating commands across a range of variables.

Here we save all the variables we want to convert in the check_cols variable and then within the dplyr::mutate() function we apply the same function dplyr::recode() across() all_of() this list, replacing Yes with 1, No with 0 and Not Known with R’s internal NA value.

Currently it has 704 lines so I can’t tal you through all of it but because this is all open if you want to look at the code you can do and I’ve included the link on this slide.

Scripts/code - r/clean.R (cont.)

## Finally save the data
saveRDS(df, file = paste(r_dir, "clean.rds", sep = "/"))

• thyroid-cancer-prediction/r/clean.R

Finally though we save the df data frame in the r_dir directory that was defined in the master.R script.

I can re-run this at any time if for example there was an error in my code, or more patients have been collected and the dataset needs analysing again or conversely if someone withdraws from the study I could add lines to r/clean.R to remove them from the dataset that is being analysed.

Quarto

• Cleaned data used by code in docs/*.qmd
• Website summarising the work and modelling.

That is a quick run-through of the code used for cleaning the data but I’ve not shown you any actual summaries yet because that is all done in the Quarto markdown language and the work for that resides in the docs/ directory in files ending in .qmd.

I won’t cover how to set this up as you will be learning how to do this with RMarkdown but there are a number of pages and each has its own file.

Each of these files has Markdown and code chunks and we’ll take a look at these now.

See thyroid-cancer-prediction/docs for examples and walk through.

Demonstration of rendering these locally, maybe even publish!

Version Control

• Keep fine-grained records of changes to code.
• Regular saves of work.
• Backup and collaboration via GitHub / GitLab
• Organised and consistent system, avoids…

I’ve mentioned that the files for this work are version controlled, hands up anyone who has heared of Version Control?

Keep your hands up if you’ve used version control?

Ok, thank you.

Version control has been around for a long time, you may be familiar with editions of books as one example or with Wikipedia where the changes and edits that people make are tracked

Version Control

Version Control

Two repositories on GitHub

• ns-rse/thyroid-cancer-prediction - repository I started to show colleague how to use Git and R.
• mdp21oe/multicenter-thyroid - repository by Ovie to re-write his paper in Quarto Markdown.

For this work we’ve established two Git repositories which are backed up and shared on GitHub, the first I setup when I started the project to document work and I’ve run you through the structure of that.

The second is because my collaborator, Ovie, wanted to re-write a paper he had written in Word to publish the analysis he had undertaken in SPSS in a more reproducible manner.

thyroid-cancer-prediction

• ns-rse/thyroid-cancer-prediction
• Understand the data and the work required
  • Version controlled.
  • Literate (tables, graphs and numbers direct from data).
  • Uses BibTeX to include & format citations.
  • Renders to HTML.
  • Hosted on GitHub Pages

multicenter-thyroid

• mdp21oe/multicentre_thyroid
• Re-write of draft in Quarto.
  • Version controlled.
  • Literate (tables, graphs and numbers direct from data).
  • Uses BibTeX to include & format citations.
  • Renders to HTML.
  • Generate PDF or Word documents.

This repository contains the work Ovie has undertaken to re-do his analysis and re-write his paper which had been drafted using Quarto.

In undertaking this the work is version controlled, literate in so much as tables, graphs and numbers in the text are derived directly from the data rather than being retyped into a Word or Google document which is highly error prone. It leverages BibTeX. The document renders to HTML and could easily be published as a website via GitHub pages if we wanted to, but as you can see in this example it provides simple buttons in the preview to make PDF or Word documents.

multicenter-thyroid (cont.)

• Ovie has said..

It really makes you think about what numbers, statistics you are calculating and the tables and graphs you are including at each step and ensures they are what you want or mean them to be. It’s much better than pointing and clicking in SPSS.

Ovie has found this approach to be really useful and has said…

When you take the time to slow down, think about and understand what you are doing it often gives a deeper understanding to the work you are undertaking instead of doing things by rote because you were taught that to get a p-value for a t-test you point-and-click here, there and there.

Demonstration of rendering the website locally.

Collaboration and The Long Term View

• Code changes over time.
• Including packages you use in R.
• Will the code you run today run in 6 months, a year, or five years?
• Will the code you run on your computer run on someone elses?

Reproducibility in computing has a very challenging issue as the code you use, in this context the R packages that are used to clean the data and produce the tables and slides, are continually evolving and change over time. Normally changes are back-wards compatible which means newer versions still run code written under older versions but that isn’t always the case and sometimes there are breaking changes.

How can you ensure that the code you run today will run in 6 months, a year or five years time?

Another challenge which can compound this is that different operating systems often have subtle differences in terms of the answers that they give or the packages that are available for them.

There are however some ways to deal with this problem.

renv to the rescue

• {renv} package helps with this.
• Creates a “Virtual Environment” and installs packages there in the renv/ directory.
• Records what versions of each package are used in renv.lock file.
• Starting R in this directory reloads these libraries

The solution to this is a R package called {renv} which is used for creating and managing virtual environments.

Installs packages within the renv/ directory and when you start R loads them from there.

Keeps a record of packages and versions in the renv.lock file and when R starts it tries to load these packages. If you’re on a new system that doesn’t have these installed it will install them.

renv.lock

{
  "R": {
    "Version": "4.3.2",
    "Repositories": [
      {
        "Name": "RStudio",
        "URL": "https://cran.rstudio.com"
      }
    ]
  },
    ...
    "dplyr": {
      "Package": "dplyr",
      "Version": "1.1.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Requirements": [
        "R",
        "R6",
        "cli",
        "generics",
        "glue",
        "lifecycle",
        "magrittr",
        "methods",
        "pillar",
        "rlang",
        "tibble",
        "tidyselect",
        "utils",
        "vctrs"
      ],
      "Hash": "fedd9d00c2944ff00a0e2696ccf048ec"
    },
    ...
    "ggplot2": {
      "Package": "ggplot2",
      "Version": "3.5.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Requirements": [
        "MASS",
        "R",
        "cli",
        "glue",
        "grDevices",
        "grid",
        "gtable",
        "isoband",
        "lifecycle",
        "mgcv",
        "rlang",
        "scales",
        "stats",
        "tibble",
        "vctrs",
        "withr"
      ],
      "Hash": "52ef83f93f74833007f193b2d4c159a2"
    },
    ...
}

It’s not that interesting to look at but this is an example of sections from a renv.lock file.

It’s in a format called JavaScript Object Notation or JSON for short.

At the top is the version of R that is used and that is followed by sections which indicate the versions of each package, the repository from whihc it was installed and its requirements along with a “hash” which is a unique way of verifying the integrity of the package. Such hashes are used all over the place and your mobile phones will use them each time they install and update a package.

Summary

• Reproducible Research is a good thing.
• Improves integrity, transparency and quality of research.
• Makes sharing work easier.
• May require learning a new skill set.
• Sometimes tools and languages you use will change.
• Its fun though! 😉

In summary, reproducible research is a good thing, it improves the integrity, transparency and quality of your research and makes sharing it easier.

This may require learning a new skill set but these are useful skills you can take forward in life and use in many different areas.

Over time the tools you use will change, when I started working in this manner I used LaTeX then shifted to RMarkdown and these days use Quarto. But once you’ve understood how these systems work its easy to adapt to new ones.

But overall I personally find working in this manner fun, it very often involves thinking deeply about what you are doing because computers only do what you tell them and by writing code to do your analysis you can easily see if, or should I say when, you make mistakes and its considerably easier to correct them.

Quarto & Data Science Learning Community

The official Quarto documentation is really good.

• Get Started
• Markdown Basics
• Guide with many sections on Presentations, Website, Books and more.

Data Science Learning Community (dslc.io)

I think most of the course material you’re learning is focused on RMarkdown but everything you learn can be applied to Quarto which is really the next iteration or generation of RMarkdown. If you’re keen to give it a try then the documentation is really good.

Perhaps also of interest when you have questions about things which aren’t working as you’d expect or how to do something is the execellent Data Science Learning Community (dslc.io) (need R4DS community) which is a great place to join book clubs to work through things like R for Data Science (2e) and over 50 books clubs or ask questions in their Slack channel.

SheffieldR User Group

sheffieldr.github.io

sheffieldr.github.io

If you’re interested in learning more about the many wonderful things you can do with R then you might be interested in the SheffieldR User Group that meets roughly every month with presentations and talks on different packages and how to do different things with R.

Its meant to be diverse, friendly and welcoming to users of all abilities.

Reproducible Research in Practice