future.tests: Continuous Integration on GitHub Actions

Henrik Bengtsson

We can use continuous integration (CI) services such as GitHub Action and Travis CI to automatically validate future backends via the future.tests test suite.

Here is an example of a .github/workflow/future.tests.yaml file that configures GitHub Actions to check several backends via the future.tests test suite.

on: [push, pull_request]

name: future_tests

jobs:
  future_tests:
    if: "! contains(github.event.head_commit.message, '[ci skip]')"    

    timeout-minutes: 30
    
    runs-on: ubuntu-20.04

    name: future.plan=${{ matrix.future.plan }}

    strategy:
      fail-fast: false
      matrix:
        future:
          - { plan: 'cluster'                             }
          - { plan: 'multicore'                           }
          - { plan: 'multisession'                        }
          - { plan: 'sequential'                          }
          - { plan: 'future.batchtools::batchtools_local' }
          - { plan: 'future.callr::callr'                 }

    env:
      GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
      RSPM: https://packagemanager.rstudio.com/cran/__linux__/focal/latest
      R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
      ## R CMD check
      _R_CHECK_LENGTH_1_CONDITION_: true
      _R_CHECK_LENGTH_1_LOGIC2_: true
      _R_CHECK_MATRIX_DATA_: true
      _R_CHECK_CRAN_INCOMING_: false
      ## Specific to futures
      R_FUTURE_RNG_ONMISUSE: error
      
    steps:
      - uses: actions/checkout@v3

      - uses: r-lib/actions/setup-pandoc@v2

      - uses: r-lib/actions/setup-r@v2
        with:
          r-version: release

      - name: Query R package dependencies
        run: |
          install.packages('remotes')
          saveRDS(remotes::dev_package_deps(dependencies = TRUE), ".github/depends.Rds", version = 2)
          writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version")
        shell: Rscript {0}

      - name: Cache R packages
        uses: actions/cache@v3
        with:
          path: ${{ env.R_LIBS_USER }}
          key: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-${{ hashFiles('.github/depends.Rds') }}
          restore-keys: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-

      - name: Install R package system dependencies (Linux)
        if: runner.os == 'Linux'
        env:
          RHUB_PLATFORM: linux-x86_64-ubuntu-gcc
        run: |
          Rscript -e "remotes::install_github('r-hub/sysreqs')"
          sysreqs=$(Rscript -e "cat(sysreqs::sysreq_commands('DESCRIPTION'))")
          sudo -s eval "$sysreqs"

      - name: Install R package dependencies
        run: |
          remotes::install_deps(dependencies = TRUE)
          install.packages(".", repos=NULL, type="source")  ## needed by parallel workers
        shell: Rscript {0}
          
      - name: Install 'future.tests' and any backend R packages
        run: |
          remotes::install_cran("future.tests")
          if (grepl("::", plan <- "${{ matrix.future.plan }}") && nzchar(pkg <- sub("::.*", "", plan))) install.packages(pkg)
        shell: Rscript {0}

      - name: Session info
        run: |
          options(width = 100)
          pkgs <- installed.packages()[, "Package"]
          sessioninfo::session_info(pkgs, include_base = TRUE)
        shell: Rscript {0}
    
      - name: Check future backend '${{ matrix.future.plan }}'
        run: |
          R CMD build --no-build-vignettes --no-manual . 
          R CMD INSTALL *.tar.gz 
          Rscript -e future.tests::check --args --test-plan=${{ matrix.future.plan }}

      - name: Upload check results
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: ${{ runner.os }}-r${{ matrix.future.plan }}-results
          path: check

For real-world examples, see the GitHub repositories of future, future.batchtools, and future.callr.