%\VignetteIndexEntry{tinytest by example} \documentclass[11pt]{article} \usepackage{enumitem} \usepackage{xcolor} % for color definitions \usepackage{sectsty} % to modify heading colors \usepackage{fancyhdr} \setlist{nosep} % simpler, but issue with your margin notes \usepackage[left=1cm,right=3cm, bottom=2cm, top=1cm]{geometry} \usepackage{hyperref} \definecolor{bluetext}{RGB}{0,101,165} \definecolor{graytext}{RGB}{80,80,80} \hypersetup{ pdfborder={0 0 0} , colorlinks=true , urlcolor=blue , linkcolor=bluetext , linktoc=all , citecolor=blue } \sectionfont{\color{bluetext}} \subsectionfont{\color{bluetext}} \subsubsectionfont{\color{bluetext}} % no serif=better reading from screen. \renewcommand{\familydefault}{\sfdefault} % header and footers \pagestyle{fancy} \fancyhf{} % empty header and footer \renewcommand{\headrulewidth}{0pt} % remove line on top \rfoot{\color{bluetext} tinytest \Sexpr{packageVersion("tinytest")}} \lfoot{\color{black}\thepage} % side-effect of \color{}: lowers the printed text a little(?) \usepackage{fancyvrb} % custom commands make life easier. \newcommand{\code}[1]{\texttt{#1}} \newcommand{\pkg}[1]{\textbf{#1}} \let\oldmarginpar\marginpar \renewcommand{\marginpar}[1]{\oldmarginpar{\color{bluetext}\raggedleft\scriptsize #1}} % skip line at start of new paragraph \setlength{\parindent}{0pt} \setlength{\parskip}{1ex} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \title{Tinytest by example} \author{Mark van der Loo} \date{\today{} | Package version \Sexpr{packageVersion("tinytest")}} \begin{document} \DefineVerbatimEnvironment{Sinput}{Verbatim}{fontshape=n,formatcom=\color{graytext}} \DefineVerbatimEnvironment{Soutput}{Verbatim}{fontshape=sl,formatcom=\color{graytext}} \newlength{\fancyvrbtopsep} \newlength{\fancyvrbpartopsep} \makeatletter \FV@AddToHook{\FV@ListParameterHook}{\topsep=\fancyvrbtopsep\partopsep=\fancyvrbpartopsep} \makeatother \setlength{\fancyvrbtopsep}{0pt} \setlength{\fancyvrbpartopsep}{0pt} \maketitle{} \thispagestyle{empty} \tableofcontents{} <>= options(prompt=" ", continue = " ", width=75, tt.pr.color=FALSE) library(tinytest) @ \subsection*{Introduction} This document provides a number of real-life examples on how \pkg{tinytest} is used by other packages. The examples aim to illustrate the purpose of testing functions and serve as a complement to the technical documentation and the `using \code{tinytest}' vignette. There is a section for each function. Each section starts with a short example that demonstrates the core purpose of the function. Next, one or more examples from packages that are published on CRAN are shown and explained. Sometimes a few lines of code were modified or deleted for brevity. This is indicated with comment between square brackets, e.g. <>= ## [this is an extra comment, only for this vignette] @ This document is probably not interesting to read front-to-back. It is more aimed to browse once in a while to get an idea on how \pkg{tinytest} can be used in practice. Package authors are invited to contribute new use cases so new users can learn from them. Please contact the author of this package either by email or via the \href{http://github.com/markvanderloo/tinytest}{github repository}. \newpage{} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{\code{expect\_equal}} R objects are described by the data they contain and the attributes attached to them. For example, in the vector \code{c(x=1,y=2)}, the data consist of the numbers \code{1} and \code{2} (in that order) and there is a single attribute called \code{names}, consisting of the two strings \code{"x"} and \code{"y"} (in that order). The \code{expect\_equal} function tests whether both the data and the attributes of two objects are the same. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_equal(1,1) expect_equal(1, c(x=1)) @ Numbers do not have to be exactly the same to be equal (by default). <<>>= 0.9-0.7-0.2 expect_equal(0.9-0.7-0.2,0) expect_equal(0.9-0.7-0.2,0, tolerance=0) @ <>= options(prompt=" ", continue = " ", width=75) @ Here is an example from the \pkg{stringdist} package. This package implements various methods to determine how different two strings are. In this test, we check one aspect of the `optimal string alignment' algorithm. In particular, we test if it correctly counts the switch of two adjacent characters as a single operation. <>= expect_equal(stringdist("ab", "ba", method="osa"), 1) @ The \pkg{benchr} package is a package to time R code, and it uses \code{expect\_equal} to extensively check the outputs. Here are a few examples. <>= b <- benchr::benchmark(1 + 1, 2 + 2) m <- mean(b) expect_equal(class(m), c("summaryBenchmark", "data.frame")) expect_equal(dim(m), c(2L, 7L)) expect_equal(names(m), c("expr", "n.eval", "mean", "trimmed", "lw.ci", "up.ci", "relative")) expect_equal(class(m$expr), "factor") expect_equal(levels(m$expr), c("1 + 1", "2 + 2")) expect_true(all(sapply(m[-1], is.numeric))) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_equivalent}} This function ignores the attributes when comparing two R objects. Two objects are equivalent when their data are the same. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_equivalent(1,1) expect_equivalent(1, c(x=1)) @ <>= options(prompt=" ", continue = " ", width=75) @ The \pkg{validate} package offers functions to define restrictions on data, and then confront the data with them. The function \code{values} extracts the boolean results in the form of a matrix with specific row- and column names. In the example below we are only interested in testing whether the \emph{contents} of the matrix is computed correctly. <>= v <- validator(x > 0) d <- data.frame(x=c(1,-1,NA)) expect_equivalent(values(confront(d,v)), matrix(c(TRUE,FALSE,NA)) ) @ The \pkg{anytime} package translates text data into data/time format (\code{Date} or \code{POSIXct}). Here, a test is performed to equivalence, to ignore the timezone label that is attached by anytime but not by \code{as.Date}. <>= refD <- as.Date("2016-01-01")+0:2 expect_equivalent(refD, anydate(20160101L + 0:2)) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_identical}} This is the most strict test for equality. The best way to think about this is that two objects must be byte-by-byte indistinguishable in order to be identical. The differences can be subtle, as shown below. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= La <- list(x=1); Lb <- list(x=1) expect_identical(La, Lb) a <- new.env() a$x <- 1 b <- new.env() b$x <- 1 expect_identical(a,b) @ Here, \code{La} and \code{Lb} are indistinguishable from R's point of view. They only differ in their location in memory. The environments \code{a} and \code{b} \emph{are} distinguishable since they contain an explicit identifier which make them unique. <<>>= print(a) print(b) @ Another difference with \code{expect\_equal} and \code{expect\_equivalent} is that \code{expect\_identical} does not allow any tolerance for numerical differences. <>= options(prompt=" ", continue = " ", width=75) @ The \code{stringdistmatrix} function of \pkg{stringdist} computes a matrix of string dissimilarity measures between all elements of a character vector. Below, it is tested whether the argument \code{useNames="none"} and the legacy (deprecated) argument \code{useName=FALSE}. <>= a <- c(k1 = "aap",k2="noot") expect_identical(stringdistmatrix(a,useNames="none") , stringdistmatrix(a,useNames=FALSE)) @ The \pkg{wand} package can retrieve MIME types for files and directories. This means there are many cases to test. In this particular package this is done by creating two lists, one with input and one with expected results. The tests are then performed as follows: <>= list( ## [long list of results removed for brevity] ) -> results fils <- list.files(system.file("extdat", package="wand"), full.names=TRUE) tst <- lapply(fils, get_content_type) names(tst) <- basename(fils) for(n in names(tst)) expect_identical(results[[n]], tst[[n]]) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_null}} The result of an operation should be \code{NULL}. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_null(iris$hihi) expect_null(iris$Species) @ <>= options(prompt=" ", continue = " ", width=75) @ This function is new in version 0.9.7 and not used in any depending packages yet. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_true}, \code{expect\_false}} The result of an operation should be precisely \code{TRUE} or \code{FALSE}. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_true(1 == 1) expect_false(1 == 2) @ <>= options(prompt=" ", continue = " ", width=75) @ The \pkg{anytime} package converts many types of strings to date/time objects (\code{POSIXct} or \code{Date}). Here is a part of it's \pkg{tinytest} test suite. <>== ## Datetime: factor and ordered (#44) refD <- as.Date("2016-09-01") expect_true(refD == anydate(as.factor("2016-09-01"))) expect_true(refD == anydate(as.ordered("2016-09-01"))) expect_true(refD == utcdate(as.factor("2016-09-01"))) expect_true(refD == utcdate(as.ordered("2016-09-01"))) @ Note that \code{==} used here has subtly different behavior from \code{all.equal} used by \code{expect\_equal}. In the above case, \code{==} does not compare time zone data, which is not added by \code{as.Date} but is added by \code{anytime}. This means that for example <>= expect_equal(anydate(as.factor("2016-09-01")), refD) @ would fail. The \pkg{ulid} package uses \code{expect\_true} to verify the type of a result. <>= x <- ULIDgenerate(20) expect_true(is.character(x)) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_message}} Expect that a message is emitted. Optionally you can specify a regular expression that the message must match. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_message(message("hihi")) expect_message(message("hihi"), pattern = "hi") expect_message(message("hihi"), pattern= "ha") expect_message(print("hihi")) @ <>= options(prompt=" ", continue = " ", width=75) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_warning}} Expect that a warning is emitted. Optionally you can specify a regular expression that the warning must match. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_warning(warning("hihi")) expect_warning(warning("hihi"), pattern = "hi") expect_warning(warning("hihi"), pattern= "ha") expect_warning(1+1) @ <>= options(prompt=" ", continue = " ", width=75) @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_error}} Expect that an error is emitted. Optionally you can specify a regular expression that the error must match. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_error(stop("hihi")) expect_error(stop("hihi"), pattern = "hi") expect_error(stop("hihi"), pattern= "ha") expect_error(print("hoho")) @ <>= options(prompt=" ", continue = " ", width=75) @ The \pkg{ChemoSpec2D} package implements exploratory methods for 2D-spectrometry data. Scaled data has negative values, so one cannot take the logarithm. The function \code{centscaleSpectra2D} must eject an error in such cases and this is tested as follows. <>= # Check that log and centering cannot be combined expect_error( centscaleSpectra2D(tiny, center = TRUE, scale = "log"), "Cannot take log of centered data") @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{expect\_silent}} Sometimes a test is only run to check that the code does not crash. This function tests that no warnings or errors are emitted when evaluating it's argument. <>= options(prompt="R> ", continue = " ", width=75) @ <<>>= expect_silent(print(10)) expect_silent(stop("haha")) @ <>= options(prompt=" ", continue = " ", width=75) @ The \pkg{validate} package defines an object called a \code{validation}, which is the result of confronting a dataset with one or more data quality restrictions in the form of rules. A \pkg{validation} object can be plotted, but this would crash with an error in a certain edge case. Here is a test that was added in response to a reported issue. <>= data <- data.frame(A = 1) rule <- validator(A > 0) cf <- confront(data, rule) expect_silent(plot(rule)) expect_silent(plot(cf)) @ The \pkg{lumberjack} package creates log files that track changes in data. In one test it is first tested whether a file has been generated, next it is tested whether it can be read properly. This is also an example of programming over test results, since the file is deleted if it exists. <>= run("runs/multiple_loggers.R") simple_ok <- expect_true(file.exists("runs/simple_log.csv")) expect_silent(read.csv("runs/simple_log.csv")) if (simple_ok) unlink("runs/simple_log.csv") @ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newpage{} \section{\code{ignore}} Ignore allows you to not record the result of a test. It is not used very often. Its use is probably almost exclusive to \pkg{tinytest} where it is used while testing the expectation functions. The following result is not recorded (note placement of brackets!) <>= ignore(expect_equal)(1+1, 2) @ The \pkg{digest} package computes hashes of R objects. It uses \code{ignore} in one of it's files. <>= mantissa <- gsub(" [0-9]*$", "", x.hex) ignore(expect_true)(all( sapply( head(seq_along(mantissa), -1), function(i){ all( grepl( paste0("^", mantissa[i], ".*"), tail(mantissa, -i) ) ) } ) )) @ \newpage \begin{thebibliography}{5} \bibitem{anytime} \href{https://cran.r-project.org/package=anytime}{anytime} D. Eddelbuettel (2019) \emph{Anything to `POSIXct' or `Date' Converter}. R package version 0.3.3.5 \bibitem{benchr} \href{https://CRAN.R-project.org/package=benchr}{benchr} Arttem Klevtsov (2019) \emph{High Precise Measurement of R Expressions Execution Time}. R package version 0.2.3-1. \bibitem{ChemoSpec2D} \href{https://cran.r-project.org/package=cyclocomp}{ChemoSpec2D} B.A. Hanson (2019) \emph{Exploratory Chemometrics for 2D Spectroscopy} R package version 0.3.166 \bibitem{digest} \href{https://cran.r-project.org/package=digest}{digest} D. Eddelbuettel (2019) \emph{Create Compact Hash Digests of R Objects} R package version 0.6.20 \bibitem{stringdist} \href{https://cran.r-project.org/package=stringr}{stringdist} M. van der Loo (2014). \emph{The stringdist package for approximate string matching}. The R Journal 6(1) 111-122 \bibitem{ulid} \href{https://cran.r-project.org/package=ulid}{ulid} B. Rudis (2019) \emph{Generate Universally Unique Lexicographically Sortable Identifiers}. R package version 0.3.0 \bibitem{validate} \href{https://cran.r-project.org/package=validate}{validate} M. van der Loo, E. de Jonge and P. Hsieh (2019) \emph{Data Validation Infrastructure for R}. R package version 0.2.7 \bibitem{wand} \href{https://cran.r-project.org/package=wand}{wand} B. Rudis (2019) \emph{Retrieve `Magic' Attributes from Files and Directories} R package version 0.5.0 \end{thebibliography} \end{document}