Internal r&d and contract r&d are the two basic forms of r&d in organizations.

An R package can be viewed as a set of functions, of which only a part are exposed to the user. In this blog post we shall concentrate of the functions that are not exposed to the user, so called internal functions: what are they, how does one handle them in one’s own package, and how can one explore them?

Internal functions 101

What is an internal function?

It’s a function that lives in your package, but that isn’t surfaced to the user. You could also call it unexported function or helper function; as opposed to exported functions and user-facing functions.

For instance, in the usethis package there’s a

## Error in base_and_recommended(): could not find function "base_and_recommended"

6 function that is not exported.

# doesn't work
library("usethis")
base_and_recommended()

## Error in base_and_recommended(): could not find function "base_and_recommended"

usethis::base_and_recommended()

## Error: 'base_and_recommended' is not an exported object from 'namespace:usethis'

# works
usethis:::base_and_recommended()

##  [1] "base"       "boot"       "class"      "cluster"    "codetools" 
##  [6] "compiler"   "datasets"   "foreign"    "graphics"   "grDevices" 
## [11] "grid"       "KernSmooth" "lattice"    "MASS"       "Matrix"    
## [16] "methods"    "mgcv"       "nlme"       "nnet"       "parallel"  
## [21] "rpart"      "spatial"    "splines"    "stats"      "stats4"    
## [26] "survival"   "tcltk"      "tools"      "utils"

As an user, you shouldn’t use unexported functions of another package in your own code.

Why not export all functions?

There are at least these two reasons:

• In a package you want to provide your user an API that is useful and stable. You can vouch for a few functions, that serve the package main goals, are documented enough, and that you’d only change with great care if need be. If your package users rely on an internal function that you decide to ditch when re-factoring code, they won’t be happy, so only export what you want to maintain.

• If all packages exposed all their internal functions, the user environment would be flooded and the namespace conflicts would be out of control.

Why write internal functions?

Why write internal functions instead of having everything in one block of code inside each exported functions?

When writing R code in general there are several reasons to write functions and it is the same within R packages: you can re-use a bit of code in several places (e.g. an epoch converter used for the output of several endpoints from a web API), and you can give it a self-explaining name (e.g.

## Error in base_and_recommended(): could not find function "base_and_recommended"

7). Any function defined in your package is usable by other functions of your package (unless it is defined inside a function of your package, in which case only that parent function can use it).

Having internal functions also means you can test these bits of code on their own. That said if you test internals too much re-factoring your code will mean breaking tests.

To find blocks of code that could be replaced with a function used several times, you could use the

## Error in base_and_recommended(): could not find function "base_and_recommended"

8 package whose planned enhancements include highlighting or printing the similar blocks.

When not to write internal functions?

There is a balance to be found between writing your own helpers for everything and only depending on external code. You can watch this excellent code on the topic.

Where to put internal functions?

You could save internal functions used in one function only in the R file defining that function, and internal functions used in several other functions in a single utils.R file or specialized utils-dates.R, utils-encoding.R files. Choose a system that helps you and your collaborators find the internal functions easily, R will never have trouble finding them as long they’re somewhere in the R/ directory. 😉

Another possible approach to helper functions when used in several packages is to pack them up in a package such as Yihui Xie’s

## Error in base_and_recommended(): could not find function "base_and_recommended"

9. So then they’re no longer internal functions. 😵

How to document internal functions?

You should at least add a few comments in their code as usual. Best practice recommended in the tidyverse style guide and the rOpenSci dev guide is to document them with roxygen2 tags like other functions, but to use

usethis::base_and_recommended()

0 to prevent manual pages to be created.

#' Compare x to 1
#' @param x an integer
#' @noRd
is_one <- function(x) {
  x == 1
}

The keyword

usethis::base_and_recommended()

1 would mean a manual page is created but not present in the function index. A confusing aspect is that you could use it for an exported, not internal function you don’t want to be too visible, e.g. a function returning the default app for OAuth in a package wrapping a web API.

#' A function rather aimed at developers
#' @description A function that does blabla, blabla.
#' @keywords internal
#' @export
does_thing <- function(){
 message("I am an exported function")
}

Explore internal functions

You might need to have a look at the guts of a package when wanting to contribute to it, or at the guts of several packages to get some inspiration for your code.

Explore internal functions within a package

Say you’ve started working on a new-to-you package (or resumed work on a long forgotten package of yours 😉). How to know how it all hangs together? You can use the same methods as for debugging code, exploring code is like debugging it and vice versa!

One first way to understand what a given helper does is looking at its code, from within RStudio there are some useful tools for navigating functions. You can then search for occurrences of its names across R scripts. These first two tasks are static code analysis (well unless your brain really executes R code by reading it!). Furthermore, a non static way to explore a function is to use

usethis::base_and_recommended()

2 inside it or inside functions calling it.

Another useful tool is the in development

usethis::base_and_recommended()

3 package. Let’s look at the cranlogs source code.

map <- pkgapi::map_package("/home/maelle/Documents/R-hub/cranlogs")

We can see all defined functions, exported or not.

str(map$defs)

## Error in base_and_recommended(): could not find function "base_and_recommended"

0

We can see all calls inside the package code, to functions from the package and other packages.

## Error in base_and_recommended(): could not find function "base_and_recommended"

1

## Error in base_and_recommended(): could not find function "base_and_recommended"

2

We can filter that data.frame to only keep calls between functions defined in the package.

## Error in base_and_recommended(): could not find function "base_and_recommended"

3

## Error in base_and_recommended(): could not find function "base_and_recommended"

4

That table can help understand how a package works. One could combine that with a network visualization.

## Error in base_and_recommended(): could not find function "base_and_recommended"

5

In this interactive visualization one sees three exported functions (triangles), with only one that calls internal functions. Such a network visualization might not be that useful for bigger packages, and in our workflow is limited to

usethis::base_and_recommended()

3's capabilities (e.g. not memoised functions)… but it’s at least quite pretty.

Explore internal functions across packages

Looking at helpers in other packages can help you write your own, e.g. looking at a package elegantly wrapping a web API could help you wrap another one elegantly too.

Bob Rudis wrote a very interesting blog post about his exploration of R packages “utility belts” i.e. the utils.R files. We also recommend our own blog post about reading the R source.

Conclusion

In this post we explained what internal functions are, and gave a few tips as to how to explore them within a package and across packages. We hope the post can help clear up a few doubts. Feel free to comment about further ideas or questions you may have.