Author: Kirill Müller
Date: 2023-11-05

# Finding files in project subdirectories

#### Kirill Müller

#### 2023-11-05

The rprojroot package solves a seemingly trivial but annoying problem that occurs sooner or later in any largish project: How to find files in subdirectories? Ideally, file paths are relative to the project root.

Unfortunately, we cannot always be sure about the current working directory: For instance, in RStudio it’s sometimes:

- the project root (when running R scripts),
- a subdirectory (when building vignettes),
- again the project root (when executing chunks of a vignette).

    basename(getwd())

    ## [1] "vignettes"

In some cases, it’s even outside the project root.

This vignette starts with a very brief summary that helps you get started, followed by a longer description of the features.

## TL;DR

What is your project: An R package?

    rprojroot::is_r_package

    ## Root criterion: contains a file "DESCRIPTION" with contents matching "^Package: "

Or an RStudio project?

    rprojroot::is_rstudio_project

    ## Root criterion: contains a file matching "[.]Rproj$" with contents matching "^Version: " in the first line

Or something else?

    rprojroot::has_file(".git/index")

    ## Root criterion: contains a file ".git/index"

For now, we assume it’s an R package:

    root <- rprojroot::is_r_package

The `root` object contains a function that helps locating files below the root of your package, regardless of your current working directory. If you are sure that your working directory is somewhere below your project’s root, use the `root$find_file()` function. In this example here, we’re starting in the `vignettes` subdirectory and find the original `DESCRIPTION` file:

    basename(getwd())

    ## [1] "vignettes"

    readLines(root$find_file("DESCRIPTION"), 3)

    ## [1] "Package: rprojroot"
    ## [2] "Title: Finding Files in Project Subdirectories"
    ## [3] "Version: 2.0.4"

There is one exception: if the first component passed to `find_file()` is already an absolute path. This allows safely applying this function to paths that may be absolute or relative:

    path <- root$find_file()
    readLines(root$find_file(path, "DESCRIPTION"), 3)

    ## [1] "Package: rprojroot"
    ## [2] "Title: Finding Files in Project Subdirectories"
    ## [3] "Version: 2.0.4"

You can also construct an accessor to your root using the `root$make_fix_file()` function:

    root_file <- root$make_fix_file()

Note that `root_file()` is a function that works just like `$find_file()` but will find the files even if the current working directory is outside your project:

    withr::with_dir(
      "../..",
      readLines(root_file("DESCRIPTION"), 3)
    )

    ## [1] "Package: rprojroot"
    ## [2] "Title: Finding Files in Project Subdirectories"
    ## [3] "Version: 2.0.4"

If you know the absolute path of some directory below your project, but cannot be sure of your current working directory, pass that absolute path to `root$make_fix_file()`:

    root_file <- root$make_fix_file("C:\\Users\\User Name\\...")

As a last resort, you can get the path of standalone R scripts or vignettes using the `thisfile()` function:

    root_file <- root$make_fix_file(dirname(thisfile()))

The remainder of this vignette describes implementation details and advanced features.

## Project root

We assume a self-contained project where all files and directories are located below a common root directory. Also, there should be a way to unambiguously identify this root directory. (Often, the root contains a regular file whose name matches a given pattern, and/or whose contents match another pattern.) In this case, the following method reliably finds our project root:

- Start the search in any subdirectory of our project
- Proceed up the directory hierarchy until the root directory has been identified

The Git version control system (and probably many other tools) use a similar approach: A Git command can be executed from within any subdirectory of a repository.

### A simple example

The `find_root()` function implements the core functionality. It returns the path to the first directory that matches the filtering criteria, or throws an error if there is no such directory. Filtering criteria are constructed in a generic fashion using the `root_criterion()` function, the `has_file()` function constructs a criterion that checks for the presence of a file with a specific name and specific contents.

    library(rprojroot)

    # List all files and directories below the root
    dir(find_root(has_file("DESCRIPTION")))

    ##  [1] "DESCRIPTION"      "LICENSE"          "LICENSE.md"       "NAMESPACE"
    ##  [5] "NEWS.md"          "R"                "README.Rmd"       "README.md"
    ##  [9] "TODO.md"          "_pkgdown.yml"     "codecov.yml"      "cran-comments.md"
    ## [13] "inst"             "man"              "revdep"           "rprojroot.Rcheck"
    ## [17] "rprojroot.Rproj"  "tests"            "vignettes"

#### Relative paths to a stable root

Here we illustrate the power of rprojroot by demonstrating how to access the same file from two different working directories. Let your project be a package called `pkgname` and consider the desired file `rrmake.R` at `pkgname/R/rrmake.R`. First, we show how to access from the `vignettes` directory, and then from the `tests/testthat` directory.

##### Example A: From `vignettes`

When your working directory is `pkgname/vignettes`, you can access the `rrmake.R` file by:

1. Supplying a pathname relative to your working directory. Here’s two ways to do that:

    rel_path_from_vignettes <- "../R/rrmake.R"
    rel_path_from_vignettes <- file.path("..", "R", "rrmake.R") ##identical

2. Supplying a pathname to the file relative from the root of the package, e.g.,

    rel_path_from_root <- "R/rrmake.R"
    rel_path_from_root <- file.path("R", "rrmake.R") ##identical

This second method requires finding the root of the package, which can be done with the `has_file()` function:

    has_file("DESCRIPTION")

    ## Root criterion: contains a file "DESCRIPTION"

So, using rprojroot you can specify the path relative from root in the following manner:

    # Specify a path/to/file relative to the root
    rel_path_from_root <- find_root_file("R", "rrmake.R", criterion = has_file("DESCRIPTION"))

##### Example B: From `tests/testthat`

When your working directory is `pkgname/tests/testthat`, you can access the `rrmake.R` file by:

1. Supplying a pathname relative to your working directory.

    rel_path_from_testthat <- "../../R/rrmake.R"

Note that this is different than in the previous example! However, the second method is the same…

2. Supplying a pathname to the file relative from the root of the package. With rprojroot, this is the exact same as in the previous example.

    # Specify a path/to/file relative to the root
    rel_path_from_root <- find_root_file("R", "rrmake.R", criterion = has_file("DESCRIPTION"))

##### Summary of Examples A and B

Since Examples A and B used different working directories, `rel_path_from_vignettes` and `rel_path_from_testthat` were different. This is an issue when trying to re-use the same code. This issue is solved by using rprojroot: the function `find_root_file()` finds a file relative from the root, where the root is determined from checking the criterion with `has_file()`.

Note that the follow code produces identical results when building the vignette and when sourcing the chunk in RStudio, provided that the current working directory is the project root or anywhere below. So, we can check to make sure that rprojroot has successfully determined the correct path:

    # Specify a path/to/file relative to the root
    rel_path_from_root <- find_root_file("R", "rrmake.R", criterion = has_file("DESCRIPTION"))

    # Find a file relative to the root
    file.exists(rel_path_from_root)

    ## [1] FALSE

### Criteria

The `has_file()` function (and the more general `root_criterion()`) both return an S3 object of class `root_criterion`:

    has_file("DESCRIPTION")

    ## Root criterion: contains a file "DESCRIPTION"

In addition, character values are coerced to `has_file` criteria by default, this coercion is applied automatically by `find_root()`. (This feature is used by the introductory example.)

    as_root_criterion("DESCRIPTION")

    ## Root criterion: contains a file "DESCRIPTION"

The return value of these functions can be stored and reused; in fact, the package provides 12 such criteria:

    criteria

    ## $is_rstudio_project
    ## Root criterion: contains a file matching "[.]Rproj$" with contents matching "^Version: " in the first line
    ##
    ## $is_r_package
    ## Root criterion: contains a file "DESCRIPTION" with contents matching "^Package: "
    ##
    ## $is_remake_project
    ## Root criterion: contains a file "remake.yml"
    ##
    ## $is_pkgdown_project
    ## Root criterion: one of
    ## - contains a file "_pkgdown.yml"
    ## - contains a file "_pkgdown.yaml"
    ## - contains a file "pkgdown/_pkgdown.yml"
    ## - contains a file "inst/_pkgdown.yml"
    ##
    ## $is_renv_project
    ## Root criterion: contains a file "renv.lock" with contents matching ""Packages":\s*\{"
    ##
    ## $is_projectile_project
    ## Root criterion: contains a file ".projectile"
    ##
    ## $is_quarto_project
    ## Root criterion: contains a file "_quarto.yml"
    ##
    ## $is_git_root
    ## Root criterion: one of
    ## - contains a directory ".git"
    ## - contains a file ".git" with contents matching "^gitdir: "
    ##
    ## $is_svn_root
    ## Root criterion: contains a directory ".svn"
    ##
    ## $is_vcs_root
    ## Root criterion: one of
    ## - contains a directory ".git"
    ## - contains a file ".git" with contents matching "^gitdir: "
    ## - contains a directory ".svn"
    ##
    ## $is_testthat
    ## Root criterion: directory name is "testthat" (also look in subdirectories: `tests/testthat`, `testthat`)
    ##
    ## $from_wd
    ## Root criterion: from current working directory
    ##
    ## attr(,"class")
    ## [1] "root_criteria"

Defining new criteria is easy:

    has_license <- has_file("LICENSE")
    has_license

    ## Root criterion: contains a file "LICENSE"

    is_projecttemplate_project <- has_file("config/global.dcf", "^version: ")
    is_projecttemplate_project

    ## Root criterion: contains a file "config/global.dcf" with contents matching "^version: "

You can also combine criteria via the `|` operator:

    is_r_package | is_rstudio_project

    ## Root criterion: one of
    ## - contains a file "DESCRIPTION" with contents matching "^Package: "
    ## - contains a file matching "[.]Rproj$" with contents matching "^Version: " in the first line

### Shortcuts

To avoid specifying the search criteria for the project root every time, shortcut functions can be created. The `find_package_root_file()` is a shortcut for `find_root_file(..., criterion = is_r_package)`:

    # Print first lines of the source for this document
    head(readLines(find_package_root_file("vignettes", "rprojroot.Rmd")))

    ## [1] "---"
    ## [2] "title: \"Finding files in project subdirectories\""
    ## [3] "author: \"Kirill Müller\""
    ## [4] "date: \"`r Sys.Date()`\""
    ## [5] "output: rmarkdown::html_vignette"
    ## [6] "vignette: >"

To save typing effort, define a shorter alias:

    P <- find_package_root_file

    # Use a shorter alias
    file.exists(P("vignettes", "rprojroot.Rmd"))

    ## [1] TRUE

Each criterion actually contains a function that allows finding a file below the root specified by this criterion. As our project does not have a file named `LICENSE`, querying the root results in an error:

    # Use the has_license criterion to find the root
    R <- has_license$find_file
    R

    ## function (..., path = ".")
    ## {
    ##     find_root_file(..., criterion = criterion, path = path)
    ## }
    ## <environment: 0x135adac48>

    # Our package does not have a LICENSE file, trying to find the root results in an error
    R()

    ## [1] "/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpDwUyx7/Rbuildf23e3bfc9008/rprojroot"

### Fixed root

We can also create a function that computes a path relative to the root at creation time.

    # Define a function that computes file paths below the current root
    F <- is_r_package$make_fix_file()
    F

    # Show contents of the NAMESPACE file in our project
    readLines(F("NAMESPACE"))

This is a more robust alternative to `$find_file()`, because it fixes the project directory when `$make_fix_file()` is called, instead of searching for it every time. (For that reason it is also slightly faster, but I doubt this matters in practice.)

This function can be used even if we later change the working directory to somewhere outside the project:

    # Print the size of the namespace file, working directory outside the project
    withr::with_dir(
      "../..",
      file.size(F("NAMESPACE"))
    )

The `make_fix_file()` member function also accepts an optional `path` argument, in case you know your project’s root but the current working directory is somewhere outside. The path to the current script or `knitr` document can be obtained using the `thisfile()` function, but it’s much easier and much more robust to just run your scripts with the working directory somewhere below your project root.

## `testthat` files

Tests run with `testthat` commonly use files that live below the `tests/testthat` directory. Ideally, this should work in the following situation:

- During package development (working directory: package root)
- When testing with `devtools::test()` (working directory: `tests/testthat`)
- When running `R CMD check` (working directory: a renamed recursive copy of `tests`)

The `is_testthat` criterion allows robust lookup of test files.

    is_testthat

    ## Root criterion: directory name is "testthat" (also look in subdirectories: `tests/testthat`, `testthat`)

The example code below lists all files in the hierarchy test directory. It uses two project root lookups in total, so that it also works when rendering the vignette (sigh):

    dir(is_testthat$find_file("hierarchy", path = is_r_package$find_file()))

    ## [1] "DESCRIPTION"     "a"               "b"               "c"
    ## [5] "hierarchy.Rproj"

### Another example: custom testing utilities

The hassle of using saved data files for testing is made even easier by using rprojroot in a utility function. For example, suppose you have a testing file at `tests/testthat/test_my_fun.R` which tests the `my_fun()` function:

    my_fun_run <- do.call(my_fun, my_args)

    testthat::test_that(
      "my_fun() returns expected output",
      testthat::expect_equal(
        my_fun_run,
        expected_output
      )
    )

There are two pieces of information that you’ll need every time `test_my_fun.R` is run: `my_args` and `expected_output`. Typically, these objects are saved to `.Rdata` files and saved to the same subdirectory. For example, you could save them to `my_args.Rdata` and `expected_output.Rdata` under the `tests/testthat/testing_data` subdirectory. And, to find them easily in any contexts, you can use rprojroot!

Since all of the data files live in the same subdirectory, you can create a utility function `get_my_path()` that will always look in that directory for these types of files. And, since the testthat package will look for and source the `tests/testthat/helper.R` file before running any tests, you can place a `get_my_path()` in this file and use it throughout your tests:

    ## saved to tests/testthat/helper.R
    get_my_path <- function(file_name) {
      rprojroot::find_testthat_root_file(
        "testing_data", filename
      )
    }

Now you can ask `get_my_path()` to find your important data files by using the function within your test scripts!

    ## Find the correct path with your custom rprojroot helper function
    path_to_my_args_file <- get_my_path("my_args.Rdata")

    ## Load the input arguments
    load(file = path_to_my_args_file)

    ## Run the function with those arguments
    my_fun_run <- do.call(my_fun,my_args)

    ## Load the historical expectation with the helper
    load(file = get_my_path("expected_output.Rdata"))

    ## Pass all tests and achieve nirvana
    testthat::test_that(
      "my_fun() returns expected output",
      testthat::expect_equal(
        my_fun_run,
        expected_output
      )
    )

For an example in the wild, see the `test_sheet()` function in the readxl package.

## Summary

The rprojroot package allows easy access to files below a project root if the project root can be identified easily, e.g. if it is the only directory in the whole hierarchy that contains a specific file. This is a robust solution for finding files in largish projects with a subdirectory hierarchy if the current working directory cannot be assumed fixed. (However, at least initially, the current working directory must be somewhere below the project root.)

## Acknowledgement

This package was inspired by the gist “Stop the working directory insanity” by Jennifer Bryan, and by the way Git knows where its files are.