This vignette is made for users that are already familiar with the basic condiments workflow described in the first vignette. Here, we will show how to modify the default parameters for the first two steps of the workflow
# For analysis
library(condiments)
library(slingshot)
set.seed(21)We rely on the same toy dataset as the first vignette
data("toy_dataset", package = "condiments")
df <- toy_dataset$sd
rd <- as.matrix(df[, c("Dim1", "Dim2")])
sds <- slingshot(rd, df$cl)By default, the topologyTest function requires only two inputs, the sds object and the condition labels. To limit run time for the vignette, we also change the default number of permutations used to generate trajectories under the null by setting the rep argument to \(10\) instead of the default \(100\). As such, the test statistics might be more variable.
top_res <- topologyTest(sds = sds, conditions = df$conditions, rep = 10)## Generating permuted trajectories## Running KS-mean testknitr::kable(top_res)| method | thresh | statistic | p.value |
|---|---|---|---|
| KS_mean | 0.01 | 0 | 1 |
The topologyTest function can be relatively slow on large datasets. Moreover, when changing the method used to test the null hypothesis that a common trajectory should be fitted, the first permutation part of generating rep trajectories under the null is identical. Therefore, we allow users to specify more than one method and one value of the threshold. Here, we will use both the Kolmogorov-Smirnov test test(Smirnov 1939) and the classifier-test(Lopez-Paz and Oquab 2016).
top_res <- topologyTest(sds = sds, conditions = df$conditions, rep = 10,
methods = c("KS_mean", "Classifier"),
threshs = c(0, .01, .05, .1))## Generating permuted trajectories## Running KS-mean test## Running Classifier testknitr::kable(top_res)| method | thresh | statistic | p.value |
|---|---|---|---|
| KS_mean | 0 | 0.0070000 | 1 |
| KS_mean | 0.01 | 0.0000000 | 1 |
| KS_mean | 0.05 | 0.0000000 | 1 |
| KS_mean | 0.1 | 0.0000000 | 1 |
| Classifier | 0 | 0.3816667 | 1 |
| Classifier | 0.01 | 0.3600000 | 1 |
| Classifier | 0.05 | 0.3266667 | 1 |
| Classifier | 0.1 | 0.2633333 | 1 |
To see all methods avaible, use /tmp/RtmpLEltKs/Rinst2e85f640eefc26/condiments/help/topologyTest and look at the methods argument.
For all methods but the KS test, additional paramters can be specified, using a custom argument: args_classifier, args_wass or args_mmd. See the help file for given test more information on those parameters. For example, since the default test based on the wasserstein distance and permutation test is quite slow, we can pass a fast argument.
top_res <- topologyTest(sds = sds, conditions = df$conditions, rep = 10,
methods = "wasserstein_permutation",
args_wass = list(fast = TRUE, S = 100, iterations = 10^2))## Generating permuted trajectories## Running wassertsein permutation testknitr::kable(top_res)| method | thresh | statistic | p.value |
|---|---|---|---|
| wasserstein_permutation | NA | 1.521573 | 0.72 |
For now, the first part of the topologyTest has been designed for parallelisation using the BiocParallel package. For example, to run with 4 cores, you can run the following command
library(BiocParallel)
BPPARAM <- bpparam()
BPPARAM$progressbar <- TRUE
BPPARAM$workers <- 4
top_res <- topologyTest(sds = sds, conditions = df$conditions, rep = 100,
parallel = TRUE, BPPARAM = BPPARAM)
knitr::kable(top_res)The tests for the second test are much less compute-intensive, therefore there is no parallelisation. However, the other changes introduce in the previous section are still possible
prog_res <- progressionTest(sds, conditions = df$conditions)
knitr::kable(prog_res)| lineage | statistic | p.value |
|---|---|---|
| All | 5.506366 | 0 |
dif_res <- differentiationTest(sds, conditions = df$conditions)## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .
##
## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .knitr::kable(dif_res)| pair | statistic | p.value |
|---|---|---|
| 1vs2 | 0.7017117 | 0 |
prog_res <- progressionTest(sds, conditions = df$conditions, method = "Classifier")
knitr::kable(prog_res)| lineage | statistic | p.value |
|---|---|---|
| All | 0.6116216 | 0.0004817 |
dif_res <- differentiationTest(sds, conditions = df$conditions, thresh = .05)## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .
##
## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .knitr::kable(dif_res)| pair | statistic | p.value |
|---|---|---|
| 1vs2 | 0.6887387 | 0 |
prog_res <- progressionTest(sds, conditions = df$conditions, method = "Classifier",
args_classifier = list(method = "rf"))## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .knitr::kable(prog_res)| lineage | statistic | p.value |
|---|---|---|
| All | 0.5305405 | 0.1914889 |
dif_res <- differentiationTest(sds, conditions = df$conditions)## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .
##
## note: only 1 unique complexity parameters in default grid. Truncating the grid to 1 .knitr::kable(dif_res)| pair | statistic | p.value |
|---|---|---|
| 1vs2 | 0.6566667 | 1.5e-06 |
For all of the above procedures, it is important to note that we are making multiple comparisons. The p-values we obtain from these tests should be corrected for multiple testing, especially for trajectories with a large number of lineages.
That said, trajectory inference is often one of the last computational methods in a very long analysis pipeline (generally including gene-level quantification, gene filtering / feature selection, and dimensionality reduction). Hence, we strongly discourage the reader from putting too much faith in any p-value that comes out of this analysis. Such values may be useful suggestions, indicating particular features or cells for follow-up study, but should generally not be treated as meaningful statistical quantities.
If some commands and parameters are still unclear after going through this vignette, do not hesitate to open an issue on the condiments Github repository.
sessionInfo()## R version 4.1.0 (2021-05-18)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 20.04.2 LTS
##
## Matrix products: default
## BLAS: /home/biocbuild/bbs-3.13-bioc/R/lib/libRblas.so
## LAPACK: /home/biocbuild/bbs-3.13-bioc/R/lib/libRlapack.so
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_GB LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] parallel stats4 stats graphics grDevices utils datasets
## [8] methods base
##
## other attached packages:
## [1] caret_6.0-88 lattice_0.20-44
## [3] viridis_0.6.1 viridisLite_0.4.0
## [5] RColorBrewer_1.1-2 ggplot2_3.3.3
## [7] tidyr_1.1.3 dplyr_1.0.6
## [9] slingshot_2.0.0 TrajectoryUtils_1.0.0
## [11] SingleCellExperiment_1.14.0 SummarizedExperiment_1.22.0
## [13] Biobase_2.52.0 GenomicRanges_1.44.0
## [15] GenomeInfoDb_1.28.0 IRanges_2.26.0
## [17] S4Vectors_0.30.0 BiocGenerics_0.38.0
## [19] MatrixGenerics_1.4.0 matrixStats_0.58.0
## [21] princurve_2.1.6 condiments_1.0.0
## [23] knitr_1.33
##
## loaded via a namespace (and not attached):
## [1] colorspace_2.0-1 deldir_0.2-10
## [3] ellipsis_0.3.2 class_7.3-19
## [5] XVector_0.32.0 proxy_0.4-25
## [7] spatstat.data_2.1-0 farver_2.1.0
## [9] prodlim_2019.11.13 fansi_0.4.2
## [11] lubridate_1.7.10 sparseMatrixStats_1.4.0
## [13] codetools_0.2-18 splines_4.1.0
## [15] polyclip_1.10-0 jsonlite_1.7.2
## [17] pROC_1.17.0.1 kernlab_0.9-29
## [19] spatstat.linnet_2.1-1 spatstat.sparse_2.0-0
## [21] compiler_4.1.0 assertthat_0.2.1
## [23] Matrix_1.3-3 htmltools_0.5.1.1
## [25] tools_4.1.0 igraph_1.2.6
## [27] gtable_0.3.0 glue_1.4.2
## [29] GenomeInfoDbData_1.2.6 RANN_2.6.1
## [31] reshape2_1.4.4 Rcpp_1.0.6
## [33] spatstat_2.1-0 jquerylib_0.1.4
## [35] vctrs_0.3.8 nlme_3.1-152
## [37] DelayedMatrixStats_1.14.0 iterators_1.0.13
## [39] timeDate_3043.102 gower_0.2.2
## [41] xfun_0.23 stringr_1.4.0
## [43] Ecume_0.9.1 lifecycle_1.0.0
## [45] goftest_1.2-2 zlibbioc_1.38.0
## [47] MASS_7.3-54 scales_1.1.1
## [49] ipred_0.9-11 spatstat.core_2.1-2
## [51] spatstat.utils_2.1-0 yaml_2.2.1
## [53] gridExtra_2.3 pbapply_1.4-3
## [55] sass_0.4.0 rpart_4.1-15
## [57] stringi_1.6.2 highr_0.9
## [59] randomForest_4.6-14 foreach_1.5.1
## [61] e1071_1.7-6 BiocParallel_1.26.0
## [63] lava_1.6.9 rlang_0.4.11
## [65] pkgconfig_2.0.3 bitops_1.0-7
## [67] evaluate_0.14 purrr_0.3.4
## [69] tensor_1.5 labeling_0.4.2
## [71] recipes_0.1.16 transport_0.12-2
## [73] tidyselect_1.1.1 plyr_1.8.6
## [75] magrittr_2.0.1 R6_2.5.0
## [77] generics_0.1.0 DelayedArray_0.18.0
## [79] DBI_1.1.1 pillar_1.6.1
## [81] withr_2.4.2 mgcv_1.8-35
## [83] survival_3.2-11 abind_1.4-5
## [85] RCurl_1.98-1.3 nnet_7.3-16
## [87] tibble_3.1.2 crayon_1.4.1
## [89] utf8_1.2.1 spatstat.geom_2.1-0
## [91] rmarkdown_2.8 grid_4.1.0
## [93] data.table_1.14.0 ModelMetrics_1.2.2.2
## [95] digest_0.6.27 munsell_0.5.0
## [97] bslib_0.2.5.1Lopez-Paz, David, and Maxime Oquab. 2016. “Revisiting Classifier Two-Sample Tests.” Arxiv, October, 1–15. http://arxiv.org/abs/1610.06545.
Smirnov, Nikolai V. 1939. “On the Estimation of the Discrepancy Between Empirical Curves of Distribution for Two Independent Samples.” Bull. Math. Univ. Moscou 2 (2): 3–14.