Back to Multiple platform build/check report for BioC 3.21:   simplified   long
ABCD[E]FGHIJKLMNOPQRSTUVWXYZ

This page was generated on 2024-12-24 11:46 -0500 (Tue, 24 Dec 2024).

HostnameOSArch (*)R versionInstalled pkgs
nebbiolo1Linux (Ubuntu 24.04.1 LTS)x86_64R Under development (unstable) (2024-10-21 r87258) -- "Unsuffered Consequences" 4754
palomino7Windows Server 2022 Datacenterx64R Under development (unstable) (2024-10-26 r87273 ucrt) -- "Unsuffered Consequences" 4472
lconwaymacOS 12.7.1 Montereyx86_64R Under development (unstable) (2024-11-20 r87352) -- "Unsuffered Consequences" 4426
kjohnson3macOS 13.7.1 Venturaarm64R Under development (unstable) (2024-11-20 r87352) -- "Unsuffered Consequences" 4381
kunpeng2Linux (openEuler 22.03 LTS-SP1)aarch64R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences" 4373
Click on any hostname to see more info about the system (e.g. compilers)      (*) as reported by 'uname -p', except on Windows and Mac OS X

Package 673/2274HostnameOS / ArchINSTALLBUILDCHECKBUILD BIN
evaluomeR 1.23.0  (landing page)
José Antonio Bernabé-Díaz
Snapshot Date: 2024-12-23 13:40 -0500 (Mon, 23 Dec 2024)
git_url: https://git.bioconductor.org/packages/evaluomeR
git_branch: devel
git_last_commit: b2ae8f5
git_last_commit_date: 2024-10-29 10:35:48 -0500 (Tue, 29 Oct 2024)
nebbiolo1Linux (Ubuntu 24.04.1 LTS) / x86_64  OK    OK    WARNINGS  UNNEEDED, same version is already published
palomino7Windows Server 2022 Datacenter / x64  OK    OK    WARNINGS    OK  UNNEEDED, same version is already published
lconwaymacOS 12.7.1 Monterey / x86_64  OK    OK    WARNINGS    OK  UNNEEDED, same version is already published
kjohnson3macOS 13.7.1 Ventura / arm64  OK    OK    WARNINGS    OK  UNNEEDED, same version is already published
kunpeng2Linux (openEuler 22.03 LTS-SP1) / aarch64  OK    OK    WARNINGS  


CHECK results for evaluomeR on kunpeng2

To the developers/maintainers of the evaluomeR package:
- Allow up to 24 hours (and sometimes 48 hours) for your latest push to git@git.bioconductor.org:packages/evaluomeR.git to reflect on this report. See Troubleshooting Build Report for more information.
- Use the following Renviron settings to reproduce errors and warnings.
- If 'R CMD check' started to fail recently on the Linux builder(s) over a missing dependency, add the missing dependency to 'Suggests:' in your DESCRIPTION file. See Renviron.bioc for more information.
- See Martin Grigorov's blog post for how to debug Linux ARM64 related issues on a x86_64 host.

raw results


Summary

Package: evaluomeR
Version: 1.23.0
Command: /home/biocbuild/R/R/bin/R CMD check --install=check:evaluomeR.install-out.txt --library=/home/biocbuild/R/R/site-library --no-vignettes --timings evaluomeR_1.23.0.tar.gz
StartedAt: 2024-12-24 06:37:14 -0000 (Tue, 24 Dec 2024)
EndedAt: 2024-12-24 06:43:37 -0000 (Tue, 24 Dec 2024)
EllapsedTime: 383.3 seconds
RetCode: 0
Status:   WARNINGS  
CheckDir: evaluomeR.Rcheck
Warnings: 3

Command output

##############################################################################
##############################################################################
###
### Running command:
###
###   /home/biocbuild/R/R/bin/R CMD check --install=check:evaluomeR.install-out.txt --library=/home/biocbuild/R/R/site-library --no-vignettes --timings evaluomeR_1.23.0.tar.gz
###
##############################################################################
##############################################################################


* using log directory ‘/home/biocbuild/bbs-3.21-bioc/meat/evaluomeR.Rcheck’
* using R Under development (unstable) (2024-11-24 r87369)
* using platform: aarch64-unknown-linux-gnu
* R was compiled by
    aarch64-unknown-linux-gnu-gcc (GCC) 14.2.0
    GNU Fortran (GCC) 14.2.0
* running under: openEuler 24.03 (LTS)
* using session charset: UTF-8
* using option ‘--no-vignettes’
* checking for file ‘evaluomeR/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘evaluomeR’ version ‘1.23.0’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... INFO
Depends: includes the non-default packages:
  'SummarizedExperiment', 'MultiAssayExperiment', 'cluster', 'fpc',
  'randomForest', 'flexmix', 'RSKC', 'sparcl'
Adding so many packages to the search path is excessive and importing
selectively is preferable.
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘evaluomeR’ can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... NOTE
File
  LICENSE
is not mentioned in the DESCRIPTION file.
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘kableExtra’
  All declared Imports should be used.
Packages in Depends field not imported from:
  ‘RSKC’ ‘sparcl’
  These packages need to be imported from (in the NAMESPACE file)
  for when this namespace is loaded but not attached.
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... NOTE
getMetricRangeByCluster: no visible global function definition for
  ‘%>%’
getMetricRangeByCluster: no visible binding for global variable
  ‘cluster’
getMetricsRelevancy: no visible global function definition for ‘RSKC’
kmeansruns: no visible global function definition for ‘pairs’
kmeansruns: no visible global function definition for ‘calinhara’
kmeansruns: no visible global function definition for ‘dudahart2’
plotMetricsCluster: no visible global function definition for
  ‘as.dendrogram’
rskcCBI: no visible global function definition for ‘RSKC’
speccCBI: no visible global function definition for ‘specc’
Undefined global functions or variables:
  %>% RSKC as.dendrogram calinhara cluster dudahart2 pairs specc
Consider adding
  importFrom("graphics", "pairs")
  importFrom("stats", "as.dendrogram")
to your NAMESPACE file.
* checking Rd files ... OK
* checking Rd metadata ... WARNING
Rd files with duplicated alias 'getMetricRangeByCluster':
  ‘getMetricRangeByCluster.Rd’ ‘getMetricsRelevancy.Rd’
* checking Rd cross-references ... NOTE
Found the following Rd file(s) with Rd \link{} targets missing package
anchors:
  getDataQualityRange.Rd: SummarizedExperiment
  getMetricsRelevancy.Rd: RSKC
  getOptimalKValue.Rd: ExperimentList, SummarizedExperiment
  globalMetric.Rd: flexmix, SummarizedExperiment
  metricsCorrelations.Rd: SummarizedExperiment
  plotMetricsBoxplot.Rd: SummarizedExperiment
  plotMetricsCluster.Rd: SummarizedExperiment
  plotMetricsClusterComparison.Rd: SummarizedExperiment
  plotMetricsMinMax.Rd: SummarizedExperiment
  plotMetricsViolin.Rd: SummarizedExperiment
  quality.Rd: SummarizedExperiment, pam
  qualityRange.Rd: SummarizedExperiment, pam
  qualitySet.Rd: SummarizedExperiment, pam
  stability.Rd: SummarizedExperiment, pam, ExperimentList
  stabilityRange.Rd: SummarizedExperiment, pam, ExperimentList
  stabilitySet.Rd: SummarizedExperiment, pam, ExperimentList
Please provide package anchors for all Rd \link{} targets not in the
package itself and the base packages.
* checking for missing documentation entries ... WARNING
Undocumented code objects:
  ‘clusterbootWrapper’ ‘standardizeQualityData’
  ‘standardizeStabilityData’
All user-level objects in a package should have documentation entries.
See chapter ‘Writing R documentation files’ in the ‘Writing R
Extensions’ manual.
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... WARNING
Undocumented arguments in Rd file 'quality.Rd'
  ‘...’

Undocumented arguments in Rd file 'qualityRange.Rd'
  ‘...’

Undocumented arguments in Rd file 'qualitySet.Rd'
  ‘...’

Undocumented arguments in Rd file 'stability.Rd'
  ‘...’

Undocumented arguments in Rd file 'stabilityRange.Rd'
  ‘...’

Undocumented arguments in Rd file 'stabilitySet.Rd'
  ‘...’

Functions with \usage entries need to have the appropriate \alias
entries, and all their arguments documented.
The \usage entries must correspond to syntactically valid R code.
See chapter ‘Writing R documentation files’ in the ‘Writing R
Extensions’ manual.
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of ‘data’ directory ... OK
* checking data for non-ASCII characters ... OK
* checking LazyData ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testAll.R’
  Running ‘testAnalysis.R’
  Running ‘testCBI.R’
  Running ‘testMetricsRelevancy.R’
  Running ‘testQuality.R’
  Running ‘testStability.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking running R code from vignettes ... SKIPPED
* checking re-building of vignette outputs ... SKIPPED
* checking PDF version of manual ... OK
* DONE

Status: 3 WARNINGs, 4 NOTEs
See
  ‘/home/biocbuild/bbs-3.21-bioc/meat/evaluomeR.Rcheck/00check.log’
for details.


Installation output

evaluomeR.Rcheck/00install.out

##############################################################################
##############################################################################
###
### Running command:
###
###   /home/biocbuild/R/R/bin/R CMD INSTALL evaluomeR
###
##############################################################################
##############################################################################


* installing to library ‘/home/biocbuild/R/R-4.5.0-devel_2024-11-24/site-library’
* installing *source* package ‘evaluomeR’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
Loading required namespace: evaluomeR
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (evaluomeR)

Tests output

evaluomeR.Rcheck/tests/testAll.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> 
> data("rnaMetrics")
> 
> dataFrame <- stability(data=rnaMetrics, k=4, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 4
> dataFrame <- stabilityRange(data=rnaMetrics, k.range=c(2,4), bs=20, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
> assay(dataFrame)
     Metric    Mean_stability_k_2  Mean_stability_k_3  Mean_stability_k_4 
[1,] "RIN"     "0.825833333333333" "0.778412698412698" "0.69625"          
[2,] "DegFact" "0.955595238095238" "0.977777777777778" "0.820833333333333"
> # Metric    Mean_stability_k_2  Mean_stability_k_3  Mean_stability_k_4
> # [1,] "RIN"     "0.825833333333333" "0.778412698412698" "0.69625"
> # [2,] "DegFact" "0.955595238095238" "0.977777777777778" "0.820833333333333"
> dataFrame <- stabilitySet(data=rnaMetrics, k.set=c(2,3,4), bs=20, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
> 
> dataFrame <- quality(data=rnaMetrics, cbi="kmeans", k=3, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 3
Processing metric: DegFact(2)
	Calculation of k = 3
> assay(dataFrame)
     Metric    Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore 
[1,] "RIN"     "0.724044583696066" "0.68338517747747"  "0.420502645502646"
[2,] "DegFact" "0.876516605981734" "0.643613928123002" "0.521618857725795"
     Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size
[1,] "0.627829396038413"  "4"            "8"            "4"           
[2,] "0.737191191352892"  "8"            "5"            "3"           
> # Metric    Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore
> # [1,] "RIN"     "0.420502645502646" "0.724044583696066" "0.68338517747747"
> # [2,] "DegFact" "0.876516605981734" "0.643613928123002" "0.521618857725795"
> # Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size
> # [1,] "0.627829396038413"  "4"            "4"            "8"
> # [2,] "0.737191191352892"  "8"            "5"            "3"
> dataFrame <- qualityRange(data=rnaMetrics, k.range=c(2,4), seed = 20, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
> assay(getDataQualityRange(dataFrame, 2))
  Metric    Cluster_1_SilScore  Cluster_2_SilScore  Avg_Silhouette_Width
1 "RIN"     "0.619872562681118" "0.583166775069983" "0.608402004052639" 
2 "DegFact" "0.664573423022171" "0.675315791048653" "0.666587617027136" 
  Cluster_1_Size Cluster_2_Size Cluster_position                 
1 "11"           "5"            "1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2"
2 "13"           "3"            "1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2"
  Cluster_labels
1 ""            
2 ""            
> # Metric    Cluster_1_SilScore  Cluster_2_SilScore  Avg_Silhouette_Width Cluster_1_Size
> # 1 "RIN"     "0.583166775069983" "0.619872562681118" "0.608402004052639"  "5"
> # 2 "DegFact" "0.664573423022171" "0.675315791048653" "0.666587617027136"  "13"
> # Cluster_2_Size
> # 1 "11"
> # 2 "3"
> assay(getDataQualityRange(dataFrame, 4))
  Metric    Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore 
1 "RIN"     "0.348714574898785" "0.420502645502646" "0.674226581940152"
2 "DegFact" "0.59496499852177"  "0.521618857725795" "0.600198799385732"
  Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size
1 "0.433333333333333" "0.463905611516569"  "5"            "4"           
2 "0.759196481622952" "0.634170498361632"  "3"            "3"           
  Cluster_3_Size Cluster_4_Size Cluster_position                 
1 "4"            "3"            "1,1,1,1,1,4,4,4,3,3,3,3,2,2,2,2"
2 "5"            "5"            "4,4,4,4,4,1,1,1,3,3,3,3,3,2,2,2"
  Cluster_labels
1 ""            
2 ""            
> # Metric    Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore
> # 1 "RIN"     "0.420502645502646" "0.674226581940152" "0.433333333333333"
> # 2 "DegFact" "0.759196481622952" "0.59496499852177"  "0.600198799385732"
> # Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size
> # 1 "0.348714574898785" "0.463905611516569"  "4"            "4"            "3"
> # 2 "0.521618857725795" "0.634170498361632"  "5"            "3"            "5"
> # Cluster_4_Size
> # 1 "5"
> # 2 "3"
> dataFrame1 <- qualitySet(data=rnaMetrics, k.set=c(2,3,4), all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
> 
> 
> dataFrame <- metricsCorrelations(data=rnaMetrics, getImages = FALSE, margins = c(4,4,11,10))

Data loaded.
Number of rows: 16
Number of columns: 3


> assay(dataFrame, 1)
               RIN    DegFact
RIN      1.0000000 -0.9744685
DegFact -0.9744685  1.0000000
> 
> 
> dataFrame <- stability(data=rnaMetrics, cbi="kmeans", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> dataFrame <- stability(data=rnaMetrics, cbi="clara", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> dataFrame <- stability(data=rnaMetrics, cbi="clara_pam", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> dataFrame <- stability(data=rnaMetrics, cbi="hclust", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> dataFrame <- stability(data=rnaMetrics, cbi="pamk", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> dataFrame <- stability(data=rnaMetrics, cbi="pamk_pam", k=2, bs=100, all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> #dataFrame <- stability(data=rnaMetrics, cbi="rskc", k=2, bs=100, all_metrics = TRUE, L1 = 2, alpha=0, getImages = FALSE)
> 
> # Supported CBIs:
> evaluomeRSupportedCBI()
[1] "kmeans"    "clara"     "clara_pam" "hclust"    "pamk"      "pamk_pam" 
[7] "rskc"     
> 
> dataFrame <- qualityRange(data=rnaMetrics, k.range=c(2,10), all_metrics = FALSE, getImages = FALSE)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
	Calculation of k = 6
	Calculation of k = 7
	Calculation of k = 8
	Calculation of k = 9
	Calculation of k = 10
Processing metric: DegFact(2)
	Calculation of k = 2
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
	Calculation of k = 6
	Calculation of k = 7
	Calculation of k = 8
	Calculation of k = 9
	Calculation of k = 10
> dataFrame
ExperimentList class object of length 9:
 [1] k_2: SummarizedExperiment with 2 rows and 8 columns
 [2] k_3: SummarizedExperiment with 2 rows and 10 columns
 [3] k_4: SummarizedExperiment with 2 rows and 12 columns
 [4] k_5: SummarizedExperiment with 2 rows and 14 columns
 [5] k_6: SummarizedExperiment with 2 rows and 16 columns
 [6] k_7: SummarizedExperiment with 2 rows and 18 columns
 [7] k_8: SummarizedExperiment with 2 rows and 20 columns
 [8] k_9: SummarizedExperiment with 2 rows and 22 columns
 [9] k_10: SummarizedExperiment with 2 rows and 24 columns
> 
> #dataFrame <- stabilityRange(data=rnaMetrics, k.range=c(2,8), bs=20, getImages = FALSE)
> #assay(dataFrame)
> 
> 
> proc.time()
   user  system elapsed 
 15.028   0.686  15.738 

evaluomeR.Rcheck/tests/testAnalysis.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> 
> 
> data("rnaMetrics")
> plotMetricsMinMax(rnaMetrics)
There were 17 warnings (use warnings() to see them)
> plotMetricsBoxplot(rnaMetrics)
Warning messages:
1: Use of `data.melt$variable` is discouraged.
ℹ Use `variable` instead. 
2: Use of `data.melt$value` is discouraged.
ℹ Use `value` instead. 
> cluster = plotMetricsCluster(ontMetrics, scale = TRUE)
> plotMetricsViolin(rnaMetrics)
Warning messages:
1: Use of `data.melt$variable` is discouraged.
ℹ Use `variable` instead. 
2: Use of `data.melt$value` is discouraged.
ℹ Use `value` instead. 
3: Use of `data.melt$variable` is discouraged.
ℹ Use `variable` instead. 
4: Use of `data.melt$value` is discouraged.
ℹ Use `value` instead. 
> plotMetricsViolin(ontMetrics, 2)
Warning messages:
1: Use of `data.melt$variable` is discouraged.
ℹ Use `variable` instead. 
2: Use of `data.melt$value` is discouraged.
ℹ Use `value` instead. 
3: Use of `data.melt$variable` is discouraged.
ℹ Use `variable` instead. 
4: Use of `data.melt$value` is discouraged.
ℹ Use `value` instead. 
> 
> stabilityData <- stabilityRange(data=rnaMetrics, k.range=c(3,4), bs=20, getImages = FALSE, seed=100)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 3
	Calculation of k = 4
> qualityData <- qualityRange(data=rnaMetrics, k.range=c(3,4), getImages = FALSE, seed=100)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DegFact(2)
	Calculation of k = 3
	Calculation of k = 4
> 
> kOptTable <- getOptimalKValue(stabilityData, qualityData, k.range=c(3,4))
Processing metric: RIN

	Maximum stability and quality values matches the same K value: '3'

Processing metric: DegFact

	Maximum stability and quality values matches the same K value: '3'

> kOptTable
   Metric Stability_max_k Stability_max_k_stab Stability_max_k_qual
1     RIN               3            0.8901389            0.6278294
2 DegFact               3            1.0000000            0.7371912
  Quality_max_k Quality_max_k_stab Quality_max_k_qual Global_optimal_k
1             3          0.8901389          0.6278294                3
2             3          1.0000000          0.7371912                3
> 
> 
> df = assay(rnaMetrics)
> k.vector1=rep(5,length(colnames(df))-1)
> k.vector2=rep(2,length(colnames(df))-1)
> 
> plotMetricsClusterComparison(rnaMetrics, k.vector1=k.vector1, k.vector2=k.vector2)
> plotMetricsClusterComparison(rnaMetrics, k.vector1=3, k.vector2=c(2,5))
> plotMetricsClusterComparison(rnaMetrics, k.vector1=3)
> 
> x = as.data.frame(assay(rnaMetrics))
> 
> # Multi metric clustering
> a = clusterbootWrapper(data=x[c("RIN", "DegFact")], B=100,
+                    bootmethod="boot",
+                    cbi="kmeans",
+                    krange=2, seed=100)
> a$bootmean # 0.8534346 for "RIN"
[1] 0.8306667 0.9233683
> mean(a$bootmean) # 0.8534346 for "RIN"
[1] 0.8770175
> stab = stability(data=x, k=2, bs=100, seed=100)

Data loaded.
Number of rows: 16
Number of columns: 3


Processing metric: RIN(1)
	Calculation of k = 2
Processing metric: DegFact(2)
	Calculation of k = 2
> assay(stab$stability_mean) # 0.8534346 for "RIN"
     Metric    Mean_stability_k_2 
[1,] "RIN"     "0.853434523809524"
[2,] "DegFact" "0.872830988455988"
> 
> proc.time()
   user  system elapsed 
 15.560   0.510  16.089 

evaluomeR.Rcheck/tests/testCBI.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> 
> 
> evaluomeRSupportedCBI()
[1] "kmeans"    "clara"     "clara_pam" "hclust"    "pamk"      "pamk_pam" 
[7] "rskc"     
> 
> 
> dataFrame <- stability(data=ontMetrics, cbi="kmeans", k=3, all_metrics=FALSE, bs=100)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 3
Processing metric: AROnto(2)
	Calculation of k = 3
Processing metric: CBOOnto(3)
	Calculation of k = 3
Processing metric: CBOOnto2(4)
	Calculation of k = 3
Processing metric: CROnto(5)
	Calculation of k = 3
Processing metric: DITOnto(6)
	Calculation of k = 3
Processing metric: INROnto(7)
	Calculation of k = 3
Processing metric: LCOMOnto(8)
	Calculation of k = 3
Processing metric: NACOnto(9)
	Calculation of k = 3
Processing metric: NOCOnto(10)
	Calculation of k = 3
Processing metric: NOMOnto(11)
	Calculation of k = 3
Processing metric: POnto(12)
	Calculation of k = 3
Processing metric: PROnto(13)
	Calculation of k = 3
Processing metric: RFCOnto(14)
	Calculation of k = 3
Processing metric: RROnto(15)
	Calculation of k = 3
Processing metric: TMOnto(16)
	Calculation of k = 3
Processing metric: TMOnto2(17)
	Calculation of k = 3
Processing metric: WMCOnto(18)
	Calculation of k = 3
Processing metric: WMCOnto2(19)
	Calculation of k = 3
> assay(dataFrame)
      Metric     Mean_stability_k_3 
 [1,] "ANOnto"   "0.711599421597794"
 [2,] "AROnto"   "0.834242802235359"
 [3,] "CBOOnto"  "0.836200447888132"
 [4,] "CBOOnto2" "0.836200447888132"
 [5,] "CROnto"   "0.80871022609772" 
 [6,] "DITOnto"  "0.802620378293628"
 [7,] "INROnto"  "0.813132039213596"
 [8,] "LCOMOnto" "0.995402775270891"
 [9,] "NACOnto"  "0.705135779579475"
[10,] "NOCOnto"  "0.902528819875511"
[11,] "NOMOnto"  "0.793513639960901"
[12,] "POnto"    "0.660145923222329"
[13,] "PROnto"   "0.960518110441289"
[14,] "RFCOnto"  "0.765127486244089"
[15,] "RROnto"   "0.960518110441289"
[16,] "TMOnto"   "0.862955680341511"
[17,] "TMOnto2"  "0.953719590152899"
[18,] "WMCOnto"  "0.85715656831332" 
[19,] "WMCOnto2" "0.904134166028688"
> 
> #dataFrame <- stabilityRange(data=ontMetrics, cbi="rskc", k.range=c(3,4), all_metrics=TRUE, bs=100, L1=2)
> #assay(dataFrame)
> 
> #dataFrame <- stabilitySet(data=ontMetrics, k.set=c(3,4), bs=100, cbi="rskc", all_metrics=TRUE, L1=2)
> #assay(dataFrame)
> 
> #dataFrame <- quality(data=ontMetrics, cbi="rskc", k=3, all_metrics=TRUE, L1=2)
> #assay(dataFrame)
> 
> #dataFrame <- qualityRange(data=ontMetrics, cbi="rskc", k.range=c(3,4), all_metrics=TRUE, L1=2)
> #assay(dataFrame$k_3)
> 
> #dataFrame <- qualitySet(data=ontMetrics, cbi="rskc", k.set=c(3,5), all_metrics=TRUE, L1=2)
> #assay(dataFrame$k_3)
> 
> 
> proc.time()
   user  system elapsed 
 14.194   0.486  14.693 

evaluomeR.Rcheck/tests/testMetricsRelevancy.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> 
> individuals_per_cluster = function(qualityResult) {
+   qual_df = as.data.frame(assay(qualityResult))
+ 
+ 
+   cluster_pos_str = as.character(unlist(qual_df["Cluster_position"]))
+   cluster_labels_str = as.character(unlist(qual_df["Cluster_labels"]))
+ 
+   cluster_pos = as.list(strsplit(cluster_pos_str, ",")[[1]])
+   cluster_labels = as.list(strsplit(cluster_labels_str, ",")[[1]])
+ 
+   individuals_in_cluster = as.data.frame(cbind(cluster_labels, cluster_pos))
+   colnames(individuals_in_cluster) = c("Individual", "InCluster")
+ 
+   return(individuals_in_cluster)
+ }
> 
> data("ontMetrics")
> metricsRelevancy = getMetricsRelevancy(ontMetrics, k=3, alpha=0.1, seed=100)
[1] "No L1 provided. Computing best L1 boundry with 'sparcl::KMeansSparseCluster.permute'"
[1] "Alpha set as: 0.1"
[1] "L1 set as: 2"
> # RSKC output object
> metricsRelevancy$rskc

Input: 
#obs= 80  #feature= 20 
L1= 2  alpha= 0.1

Result:
wbss: 36267.83
trimmed cases: 1 5 13 34 41 68 73 79 2 21 59 67 71
#non-zero weights: 20 
 3 clusters of sizes 31, 26, 23 
Cluster labels: 2 1 2 3 1 3 1 3 2 3 3 2 2 1 3 1 3 1 3 1 1 1 1 2 3 2 2 2 3 1 3 2 2 2 1 3 2 1 1 3 1 1 3 1 2 3 2 1 2 2 2 1 2 2 1 3 1 3 3 2 3 2 3 2 1 2 2 2 3 1 2 2 1 1 2 2 1 3 2 3 
> # Trimmed cases from input (row indexes)
> metricsRelevancy$trimmed_cases
 [1]  1  2  5 13 21 34 41 59 67 68 71 73 79
> # Metrics relevancy table
> metricsRelevancy$relevancy
        metric       weight
1  Description 9.999835e-01
7      DITOnto 3.782486e-03
19     WMCOnto 3.559593e-03
3       AROnto 2.391771e-03
9     LCOMOnto 4.148457e-04
15     RFCOnto 3.485766e-04
12     NOMOnto 2.586262e-04
11     NOCOnto 1.394339e-04
20    WMCOnto2 3.226109e-05
13       POnto 3.168475e-05
18     TMOnto2 1.159899e-05
14      PROnto 7.809770e-06
16      RROnto 7.809770e-06
2       ANOnto 7.428264e-06
4      CBOOnto 6.718581e-06
5     CBOOnto2 6.718581e-06
10     NACOnto 4.589905e-06
8      INROnto 2.615559e-06
17      TMOnto 1.667959e-06
6       CROnto 1.063741e-06
> 
> 
> test = qualityRange(data=ontMetrics, k.range=c(3,3),
+                              seed=13007,
+                              all_metrics=TRUE,
+                              cbi="rskc", L1=2, alpha=0)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing all metrics, 'merge', in dataframe (19)
	Calculation of k = 3
> 
> # Shows how clusters are partitioned according to the individuals
> individuals_per_cluster(test$k_3)
   Individual InCluster
1           3         2
2          42         2
3          26         1
4          79         2
5          41         3
6          66         2
7          53         2
8          76         2
9           6         2
10         68         2
11         74         2
12          7         2
13         30         1
14         57         2
15         69         2
16         48         2
17         80         2
18         45         2
19         61         2
20         49         2
21         55         2
22         52         2
23         50         2
24         16         2
25         70         2
26         28         2
27         13         2
28         24         2
29         60         2
30         40         2
31         64         2
32         11         2
33         19         2
34          1         2
35         38         2
36         58         2
37         29         2
38         54         2
39         37         2
40         62         2
41         34         3
42         51         2
43         71         2
44         43         2
45         25         2
46         77         2
47          4         2
48         36         2
49         14         2
50         20         2
51          9         2
52         35         2
53         17         2
54         23         2
55         46         2
56         59         2
57         33         2
58         73         2
59         63         1
60          8         2
61         65         2
62         10         2
63         67         2
64         21         2
65         47         2
66         15         2
67         12         1
68         31         2
69         75         2
70         56         2
71         22         1
72         18         2
73         32         1
74         44         2
75         27         2
76          5         2
77         39         2
78         72         2
79          2         2
80         78         2
> 
> 
> proc.time()
   user  system elapsed 
 14.177   0.470  14.663 

evaluomeR.Rcheck/tests/testQuality.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> library(RSKC)
> library(sparcl)
> seed = 100
> dataFrame <- quality(data=ontMetrics, cbi="kmeans", k=3)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 3
Processing metric: AROnto(2)
	Calculation of k = 3
Processing metric: CBOOnto(3)
	Calculation of k = 3
Processing metric: CBOOnto2(4)
	Calculation of k = 3
Processing metric: CROnto(5)
	Calculation of k = 3
Processing metric: DITOnto(6)
	Calculation of k = 3
Processing metric: INROnto(7)
	Calculation of k = 3
Processing metric: LCOMOnto(8)
	Calculation of k = 3
Processing metric: NACOnto(9)
	Calculation of k = 3
Processing metric: NOCOnto(10)
	Calculation of k = 3
Processing metric: NOMOnto(11)
	Calculation of k = 3
Processing metric: POnto(12)
	Calculation of k = 3
Processing metric: PROnto(13)
	Calculation of k = 3
Processing metric: RFCOnto(14)
	Calculation of k = 3
Processing metric: RROnto(15)
	Calculation of k = 3
Processing metric: TMOnto(16)
	Calculation of k = 3
Processing metric: TMOnto2(17)
	Calculation of k = 3
Processing metric: WMCOnto(18)
	Calculation of k = 3
Processing metric: WMCOnto2(19)
	Calculation of k = 3
> assay(dataFrame)
      Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore 
 [1,] "ANOnto"   "0.754894925204277" "0.570241066303214" "0.775876285585267"
 [2,] "AROnto"   "0.837074497995987" "0.509946991883709" "0.959264389073384"
 [3,] "CBOOnto"  "0.470708665744913" "0.766630500367533" "0.574451527320666"
 [4,] "CBOOnto2" "0.470708665744913" "0.766630500367533" "0.574451527320666"
 [5,] "CROnto"   "0"                 "0.636126752920544" "0.885055456924709"
 [6,] "DITOnto"  "0.615581638093901" "0.441137593941046" "0.746848044839846"
 [7,] "INROnto"  "0"                 "0.760945813444805" "0.506239463726949"
 [8,] "LCOMOnto" "0.657281417643165" "0.61764525421598"  "0.722333227599342"
 [9,] "NACOnto"  "0.445845264823784" "0.759522276872854" "0.254826579985626"
[10,] "NOCOnto"  "0.363472944618239" "0.898396530127955" "0.742673517080307"
[11,] "NOMOnto"  "0.708789049998754" "0"                 "0.605603643727872"
[12,] "POnto"    "0.755700546488043" "0.737169134813343" "0.651090644844594"
[13,] "PROnto"   "0.770018889790615" "0.636058646833202" "0.56606585120985" 
[14,] "RFCOnto"  "0.672903800663584" "0"                 "0.571360647044581"
[15,] "RROnto"   "0.770018889790615" "0.636058646833202" "0.56606585120985" 
[16,] "TMOnto"   "0.50860642260504"  "0.782948726523096" "0.634534477835837"
[17,] "TMOnto2"  "0.73737171744016"  "1"                 "0.462679160671249"
[18,] "WMCOnto"  "0.868556472442156" "0.369670756071292" "0.763547528087877"
[19,] "WMCOnto2" "0.891854974826074" "0.598522433823083" "0.613618761016468"
      Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size
 [1,] "0.736742918153759"  "12"           "14"           "54"          
 [2,] "0.786971025529677"  "65"           "13"           "2"           
 [3,] "0.72319889705568"   "2"            "63"           "15"          
 [4,] "0.72319889705568"   "2"            "63"           "15"          
 [5,] "0.855322610912838"  "1"            "6"            "73"          
 [6,] "0.553468450386794"  "41"           "33"           "6"           
 [7,] "0.690941232718754"  "1"            "60"           "19"          
 [8,] "0.652913140794165"  "21"           "40"           "19"          
 [9,] "0.661322430756974"  "17"           "58"           "5"           
[10,] "0.879183827500925"  "2"            "75"           "3"           
[11,] "0.668973564992505"  "55"           "1"            "24"          
[12,] "0.67661537075347"   "8"            "14"           "58"          
[13,] "0.668644905329162"  "32"           "24"           "24"          
[14,] "0.635298846489826"  "56"           "1"            "23"          
[15,] "0.668644905329162"  "32"           "24"           "24"          
[16,] "0.710090639489989"  "18"           "56"           "6"           
[17,] "0.724657891719511"  "45"           "16"           "19"          
[18,] "0.828514820105485"  "72"           "6"            "2"           
[19,] "0.870232442430684"  "74"           "4"            "2"           
> # Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size
> # [1,] "ANOnto"   "0.754894925204277" "0.570241066303214" "0.775876285585267" "0.736742918153759"  "12"           "14"           "54"
> # [2,] "AROnto"   "0.837074497995987" "0.509946991883709" "0.959264389073384" "0.786971025529677"  "65"           "13"           "2"
> # [3,] "CBOOnto"  "0.766630500367533" "0.574451527320666" "0.470708665744913" "0.72319889705568"   "63"           "15"           "2"
> # [4,] "CBOOnto2" "0.766630500367533" "0.574451527320666" "0.470708665744913" "0.72319889705568"   "63"           "15"           "2"
> # [5,] "CROnto"   "0.885055456924709" "0.636126752920544" "0"                 "0.855322610912838"  "73"           "6"            "1"
> # [6,] "DITOnto"  "0.615581638093901" "0.441137593941046" "0.746848044839846" "0.553468450386794"  "41"           "33"           "6"
> # [7,] "INROnto"  "0.760945813444805" "0.506239463726949" "0"                 "0.690941232718754"  "60"           "19"           "1"
> # [8,] "LCOMOnto" "0.657281417643165" "0.61764525421598"  "0.722333227599342" "0.652913140794165"  "21"           "40"           "19"
> # [9,] "NACOnto"  "0.759522276872854" "0.445845264823784" "0.254826579985626" "0.661322430756974"  "58"           "17"           "5"
> # [10,] "NOCOnto"  "0.898396530127955" "0.742673517080307" "0.363472944618239" "0.879183827500925"  "75"           "3"            "2"
> # [11,] "NOMOnto"  "0.708789049998754" "0.605603643727872" "0"                 "0.668973564992505"  "55"           "24"           "1"
> # [12,] "POnto"    "0.755700546488043" "0.737169134813343" "0.651090644844594" "0.67661537075347"   "8"            "14"           "58"
> # [13,] "PROnto"   "0.770018889790615" "0.56606585120985"  "0.636058646833202" "0.668644905329162"  "32"           "24"           "24"
> # [14,] "RFCOnto"  "0.672903800663584" "0.571360647044581" "0"                 "0.635298846489826"  "56"           "23"           "1"
> # [15,] "RROnto"   "0.636058646833202" "0.56606585120985"  "0.770018889790615" "0.668644905329162"  "24"           "24"           "32"
> # [16,] "TMOnto"   "0.782948726523096" "0.50860642260504"  "0.634534477835837" "0.710090639489989"  "56"           "18"           "6"
> # [17,] "TMOnto2"  "1"                 "0.73737171744016"  "0.462679160671249" "0.724657891719511"  "16"           "45"           "19"
> # [18,] "WMCOnto"  "0.868556472442156" "0.369670756071292" "0.763547528087877" "0.828514820105485"  "72"           "6"            "2"
> # [19,] "WMCOnto2" "0.891854974826074" "0.598522433823083" "0.613618761016468" "0.870232442430684"  "74"           "4"            "2"
> 
> dataFrame <- quality(data=ontMetrics, cbi="kmeans", k=4)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 4
Processing metric: AROnto(2)
	Calculation of k = 4
Processing metric: CBOOnto(3)
	Calculation of k = 4
Processing metric: CBOOnto2(4)
	Calculation of k = 4
Processing metric: CROnto(5)
	Calculation of k = 4
Processing metric: DITOnto(6)
	Calculation of k = 4
Processing metric: INROnto(7)
	Calculation of k = 4
Processing metric: LCOMOnto(8)
	Calculation of k = 4
Processing metric: NACOnto(9)
	Calculation of k = 4
Processing metric: NOCOnto(10)
	Calculation of k = 4
Processing metric: NOMOnto(11)
	Calculation of k = 4
Processing metric: POnto(12)
	Calculation of k = 4
Processing metric: PROnto(13)
	Calculation of k = 4
Processing metric: RFCOnto(14)
	Calculation of k = 4
Processing metric: RROnto(15)
	Calculation of k = 4
Processing metric: TMOnto(16)
	Calculation of k = 4
Processing metric: TMOnto2(17)
	Calculation of k = 4
Processing metric: WMCOnto(18)
	Calculation of k = 4
Processing metric: WMCOnto2(19)
	Calculation of k = 4
> assay(dataFrame)
      Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  
 [1,] "ANOnto"   "0.569222510427433" "0.552363239306396" "0.584449669565973" 
 [2,] "AROnto"   "0.891757427020894" "0.498602630835942" "0.953766280221553" 
 [3,] "CBOOnto"  "0.682847685112873" "0.475694878561971" "0.418096612044278" 
 [4,] "CBOOnto2" "0.682847685112873" "0.475694878561971" "0.418096612044278" 
 [5,] "CROnto"   "0.615016966742524" "0.931552645421743" "0.460688748724164" 
 [6,] "DITOnto"  "0.621392145232729" "0.589638237470761" "0.512852920317478" 
 [7,] "INROnto"  "0.679354776901229" "0.514845315378322" "0.552323396139528" 
 [8,] "LCOMOnto" "0.563584714383498" "0.565734453969461" "0.526937877760086" 
 [9,] "NACOnto"  "0.507554700154524" "0.763008703189753" "0.0693863149967116"
[10,] "NOCOnto"  "0.363472944618239" "0.712806750183687" "0.368068489789737" 
[11,] "NOMOnto"  "0.796568957921031" "0"                 "0.487448631370323" 
[12,] "POnto"    "0.717551583859045" "0.702605079149018" "0.531828315626997" 
[13,] "PROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586" 
[14,] "RFCOnto"  "0.708660103503223" "0"                 "0.527891770926241" 
[15,] "RROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586" 
[16,] "TMOnto"   "0.527581279093128" "0.772548576303018" "0.756878515673905" 
[17,] "TMOnto2"  "0.593309463294573" "1"                 "0.709314170957853" 
[18,] "WMCOnto"  "0.811550829534933" "0.517887706724764" "0.751527957476758" 
[19,] "WMCOnto2" "0.48724511207104"  "0.806794961402285" "0.613618761016468" 
      Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size
 [1,] "0.717030499002753" "0.600638738086962"  "11"           "4"           
 [2,] "0.614385150712436" "0.813833608784603"  "58"           "7"           
 [3,] "0.462053414220223" "0.5843870090796"    "46"           "18"          
 [4,] "0.462053414220223" "0.5843870090796"    "46"           "18"          
 [5,] "0"                 "0.84502648526675"   "10"           "63"          
 [6,] "0.717462336796908" "0.582143307479606"  "15"           "35"          
 [7,] "0"                 "0.609561353444975"  "46"           "19"          
 [8,] "0.662861247621334" "0.57713748864992"   "19"           "19"          
 [9,] "0.610806402578204" "0.627188990478616"  "23"           "42"          
[10,] "0.711626648649838" "0.600607673118847"  "2"            "51"          
[11,] "0.505810544669573" "0.620956620752701"  "35"           "1"           
[12,] "0.755700546488043" "0.676374911502771"  "14"           "42"          
[13,] "0.546429726628472" "0.623564355956028"  "22"           "23"          
[14,] "0.575667190561062" "0.613856368788046"  "37"           "1"           
[15,] "0.546429726628472" "0.623564355956028"  "22"           "23"          
[16,] "0.56435245544769"  "0.694408411158545"  "15"           "48"          
[17,] "0.516092763511662" "0.725408613137789"  "19"           "16"          
[18,] "0.232935788267106" "0.737070037248562"  "62"           "12"          
[19,] "0.458575230569131" "0.72940235766569"   "4"            "61"          
      Cluster_3_Size Cluster_4_Size
 [1,] "53"           "12"          
 [2,] "2"            "13"          
 [3,] "14"           "2"           
 [4,] "14"           "2"           
 [5,] "6"            "1"           
 [6,] "24"           "6"           
 [7,] "14"           "1"           
 [8,] "23"           "19"          
 [9,] "5"            "10"          
[10,] "24"           "3"           
[11,] "25"           "19"          
[12,] "16"           "8"           
[13,] "12"           "23"          
[14,] "27"           "15"          
[15,] "12"           "23"          
[16,] "5"            "12"          
[17,] "39"           "6"           
[18,] "2"            "4"           
[19,] "2"            "13"          
> # Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  Cluster_4_SilScore   Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size Cluster_4_Size
> # [1,] "ANOnto"   "0.717030499002753" "0.569222510427433" "0.552363239306396" "0.584449669565973"  "0.600638738086962"  "12"           "11"           "4"            "53"
> # [2,] "AROnto"   "0.891757427020894" "0.614385150712436" "0.498602630835942" "0.953766280221553"  "0.813833608784603"  "58"           "13"           "7"            "2"
> # [3,] "CBOOnto"  "0.682847685112873" "0.475694878561971" "0.418096612044278" "0.462053414220223"  "0.5843870090796"    "46"           "18"           "14"           "2"
> # [4,] "CBOOnto2" "0.682847685112873" "0.475694878561971" "0.418096612044278" "0.462053414220223"  "0.5843870090796"    "46"           "18"           "14"           "2"
> # [5,] "CROnto"   "0.931552645421743" "0.615016966742524" "0.460688748724164" "0"                  "0.84502648526675"   "63"           "10"           "6"            "1"
> # [6,] "DITOnto"  "0.621392145232729" "0.589638237470761" "0.512852920317478" "0.717462336796908"  "0.582143307479606"  "15"           "35"           "24"           "6"
> # [7,] "INROnto"  "0.679354776901229" "0.514845315378322" "0.552323396139528" "0"                  "0.609561353444975"  "46"           "19"           "14"           "1"
> # [8,] "LCOMOnto" "0.563584714383498" "0.565734453969461" "0.526937877760086" "0.662861247621334"  "0.57713748864992"   "19"           "19"           "23"           "19"
> # [9,] "NACOnto"  "0.763008703189753" "0.507554700154524" "0.610806402578204" "0.0693863149967116" "0.627188990478616"  "42"           "23"           "10"           "5"
> # [10,] "NOCOnto"  "0.712806750183687" "0.368068489789737" "0.711626648649838" "0.363472944618239"  "0.600607673118847"  "51"           "24"           "3"            "2"
> # [11,] "NOMOnto"  "0.796568957921031" "0.487448631370323" "0.505810544669573" "0"                  "0.620956620752701"  "35"           "25"           "19"           "1"
> # [12,] "POnto"    "0.755700546488043" "0.717551583859045" "0.702605079149018" "0.531828315626997"  "0.676374911502771"  "8"            "14"           "42"           "16"
> # [13,] "PROnto"   "0.808419016380534" "0.406920889282586" "0.546429726628472" "0.636912857924547"  "0.623564355956028"  "22"           "12"           "23"           "23"
> # [14,] "RFCOnto"  "0.708660103503223" "0.527891770926241" "0.575667190561062" "0"                  "0.613856368788046"  "37"           "27"           "15"           "1"
> # [15,] "RROnto"   "0.636912857924547" "0.546429726628472" "0.406920889282586" "0.808419016380534"  "0.623564355956028"  "23"           "23"           "12"           "22"
> # [16,] "TMOnto"   "0.772548576303018" "0.527581279093128" "0.56435245544769"  "0.756878515673905"  "0.694408411158545"  "48"           "15"           "12"           "5"
> # [17,] "TMOnto2"  "1"                 "0.709314170957853" "0.593309463294573" "0.516092763511662"  "0.725408613137789"  "16"           "39"           "19"           "6"
> # [18,] "WMCOnto"  "0.811550829534933" "0.517887706724764" "0.232935788267106" "0.751527957476758"  "0.737070037248562"  "62"           "12"           "4"            "2"
> # [19,] "WMCOnto2" "0.806794961402285" "0.458575230569131" "0.48724511207104"  "0.613618761016468"  "0.72940235766569"   "61"           "13"           "4"            "2"
> 
> dataFrame <- qualityRange(data=ontMetrics, cbi="kmeans", k.range = c(3,4))

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: AROnto(2)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: CBOOnto(3)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: CBOOnto2(4)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: CROnto(5)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: DITOnto(6)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: INROnto(7)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: LCOMOnto(8)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: NACOnto(9)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: NOCOnto(10)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: NOMOnto(11)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: POnto(12)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: PROnto(13)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: RFCOnto(14)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: RROnto(15)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: TMOnto(16)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: TMOnto2(17)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: WMCOnto(18)
	Calculation of k = 3
	Calculation of k = 4
Processing metric: WMCOnto2(19)
	Calculation of k = 3
	Calculation of k = 4
> assay(dataFrame$k_4)
   Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  
1  "ANOnto"   "0.569222510427433" "0.552363239306396" "0.584449669565973" 
2  "AROnto"   "0.891757427020894" "0.498602630835942" "0.953766280221553" 
3  "CBOOnto"  "0.682847685112873" "0.475694878561971" "0.418096612044278" 
4  "CBOOnto2" "0.682847685112873" "0.475694878561971" "0.418096612044278" 
5  "CROnto"   "0.615016966742524" "0.931552645421743" "0.460688748724164" 
6  "DITOnto"  "0.621392145232729" "0.589638237470761" "0.512852920317478" 
7  "INROnto"  "0.679354776901229" "0.514845315378322" "0.552323396139528" 
8  "LCOMOnto" "0.563584714383498" "0.565734453969461" "0.526937877760086" 
9  "NACOnto"  "0.507554700154524" "0.763008703189753" "0.0693863149967116"
10 "NOCOnto"  "0.363472944618239" "0.712806750183687" "0.368068489789737" 
11 "NOMOnto"  "0.796568957921031" "0"                 "0.487448631370323" 
12 "POnto"    "0.717551583859045" "0.702605079149018" "0.531828315626997" 
13 "PROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586" 
14 "RFCOnto"  "0.708660103503223" "0"                 "0.527891770926241" 
15 "RROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586" 
16 "TMOnto"   "0.527581279093128" "0.772548576303018" "0.756878515673905" 
17 "TMOnto2"  "0.593309463294573" "1"                 "0.709314170957853" 
18 "WMCOnto"  "0.811550829534933" "0.517887706724764" "0.751527957476758" 
19 "WMCOnto2" "0.48724511207104"  "0.806794961402285" "0.613618761016468" 
   Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size
1  "0.717030499002753" "0.600638738086962"  "11"           "4"           
2  "0.614385150712436" "0.813833608784603"  "58"           "7"           
3  "0.462053414220223" "0.5843870090796"    "46"           "18"          
4  "0.462053414220223" "0.5843870090796"    "46"           "18"          
5  "0"                 "0.84502648526675"   "10"           "63"          
6  "0.717462336796908" "0.582143307479606"  "15"           "35"          
7  "0"                 "0.609561353444975"  "46"           "19"          
8  "0.662861247621334" "0.57713748864992"   "19"           "19"          
9  "0.610806402578204" "0.627188990478616"  "23"           "42"          
10 "0.711626648649838" "0.600607673118847"  "2"            "51"          
11 "0.505810544669573" "0.620956620752701"  "35"           "1"           
12 "0.755700546488043" "0.676374911502771"  "14"           "42"          
13 "0.546429726628472" "0.623564355956028"  "22"           "23"          
14 "0.575667190561062" "0.613856368788046"  "37"           "1"           
15 "0.546429726628472" "0.623564355956028"  "22"           "23"          
16 "0.56435245544769"  "0.694408411158545"  "15"           "48"          
17 "0.516092763511662" "0.725408613137789"  "19"           "16"          
18 "0.232935788267106" "0.737070037248562"  "62"           "12"          
19 "0.458575230569131" "0.72940235766569"   "4"            "61"          
   Cluster_3_Size Cluster_4_Size
1  "53"           "12"          
2  "2"            "13"          
3  "14"           "2"           
4  "14"           "2"           
5  "6"            "1"           
6  "24"           "6"           
7  "14"           "1"           
8  "23"           "19"          
9  "5"            "10"          
10 "24"           "3"           
11 "25"           "19"          
12 "16"           "8"           
13 "12"           "23"          
14 "27"           "15"          
15 "12"           "23"          
16 "5"            "12"          
17 "39"           "6"           
18 "2"            "4"           
19 "2"            "13"          
   Cluster_position                                                                                                                                                 
1  "1,3,1,3,1,3,4,3,3,3,3,3,3,3,3,3,3,3,4,4,3,3,1,1,3,3,3,3,1,3,3,3,2,3,3,2,3,3,2,3,3,4,3,3,2,3,3,3,4,3,4,4,3,4,3,3,4,1,1,3,3,3,1,4,4,4,1,3,3,3,3,3,3,3,3,3,3,1,3,3"
2  "1,1,1,1,1,2,1,4,1,2,4,1,1,1,2,1,1,4,1,1,1,1,1,1,4,1,1,1,1,1,4,1,1,1,1,1,4,1,1,4,4,1,1,4,1,3,1,1,1,1,1,1,1,1,2,1,1,1,1,4,2,4,1,1,1,1,1,1,2,1,4,1,1,4,1,2,1,1,3,1"
3  "1,1,3,1,3,1,2,2,1,3,2,2,2,2,1,1,1,1,4,1,1,1,1,2,3,1,2,2,1,2,1,1,1,1,1,1,3,2,1,2,3,1,2,1,1,1,1,1,1,3,2,1,1,3,1,2,1,1,3,1,1,1,1,1,3,3,3,1,4,1,3,2,3,1,1,2,1,1,2,1"
4  "1,1,3,1,3,1,2,2,1,3,2,2,2,2,1,1,1,1,4,1,1,1,1,2,3,1,2,2,1,2,1,1,1,1,1,1,3,2,1,2,3,1,2,1,1,1,1,1,1,3,2,1,1,3,1,2,1,1,3,1,1,1,1,1,3,3,3,1,4,1,3,2,3,1,1,2,1,1,2,1"
5  "2,2,2,2,2,3,1,2,2,1,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1,2,2,2,2,2,1,2,2,2,2,2,2,2,1,2,2,2,2,3,2,3,2,2,2,2,2,2,2,2,2,2,2,2,2,1,3,2,2,2,2,2,2,2,2,2,3,2,2,1,2,1,4,2,3,2"
6  "2,1,4,2,4,3,1,2,1,3,2,1,3,2,3,2,2,2,3,2,1,2,2,3,3,2,3,2,2,1,2,1,3,2,1,1,3,2,2,3,3,1,2,2,2,4,2,3,1,3,2,3,1,4,2,2,3,2,3,1,3,2,1,2,1,2,4,2,3,3,3,2,3,3,2,3,2,2,4,2"
7  "1,1,3,1,3,1,2,2,1,3,2,3,2,2,1,1,1,1,4,1,1,1,1,2,1,1,2,2,2,2,1,1,1,1,1,1,3,2,1,2,3,1,2,1,1,1,1,1,1,3,2,1,1,3,1,2,1,1,3,1,1,1,1,1,3,2,3,1,3,1,3,2,3,1,1,2,1,1,2,1"
8  "1,1,4,2,4,3,1,1,2,4,3,2,4,2,4,1,3,1,4,3,1,1,3,3,3,2,3,3,1,2,3,1,3,2,1,1,4,2,2,4,4,1,3,2,1,3,3,3,2,3,2,4,1,4,3,3,4,2,4,1,3,3,2,2,1,3,4,1,4,4,3,2,4,3,1,4,2,2,4,2"
9  "2,2,4,2,4,2,1,1,2,3,4,1,1,1,2,2,2,2,3,1,2,1,2,1,2,2,1,1,1,1,2,2,2,2,2,2,4,1,2,1,4,2,1,2,2,2,2,2,2,4,1,2,2,4,2,1,2,2,3,1,2,2,2,2,4,1,4,2,3,2,3,1,4,2,1,1,2,2,1,2"
10 "3,1,2,3,2,3,2,3,2,2,2,2,2,3,2,3,3,3,4,2,1,3,2,2,2,3,2,4,3,2,2,2,2,2,4,2,3,2,2,2,2,2,2,2,3,2,3,2,2,2,2,3,3,2,2,3,3,2,2,2,3,2,2,3,3,2,2,3,2,3,2,2,2,2,2,2,3,2,2,2"
11 "1,4,1,4,1,3,1,1,1,3,3,4,3,1,4,3,4,3,1,1,4,1,1,3,4,1,1,4,1,1,3,4,3,1,1,3,4,1,3,3,4,1,1,3,4,2,3,3,1,4,1,1,3,1,3,3,1,3,4,4,3,1,1,1,1,1,3,4,4,1,3,1,1,3,1,3,4,3,4,1"
12 "1,2,3,2,1,2,4,2,2,3,3,3,2,2,2,1,2,2,4,4,2,2,1,1,2,2,3,2,1,2,2,2,1,2,2,1,3,2,1,2,3,4,2,2,1,2,2,2,2,3,4,4,2,3,2,3,4,1,3,2,2,2,1,2,4,3,1,2,3,2,3,2,3,2,2,3,2,1,2,2"
13 "1,2,3,2,3,4,3,3,1,4,4,2,4,1,2,4,2,4,1,1,2,1,3,4,2,3,1,2,1,1,4,2,4,3,1,4,2,1,4,4,2,1,3,2,2,2,4,2,1,2,3,1,2,1,2,4,1,4,4,2,4,3,1,1,1,3,4,2,2,1,4,1,1,4,3,4,2,4,2,4"
14 "1,3,1,4,3,1,1,1,1,3,3,4,3,1,4,1,4,1,3,1,3,1,1,3,4,1,1,4,1,1,3,4,3,1,1,3,4,1,3,3,4,1,1,3,3,2,3,3,1,4,1,1,3,1,3,3,1,1,4,4,3,1,1,1,1,3,3,3,4,1,4,1,1,3,1,3,3,1,4,1"
15 "1,2,3,2,3,4,3,3,1,4,4,2,4,1,2,4,2,4,1,1,2,1,3,4,2,3,1,2,1,1,4,2,4,3,1,4,2,1,4,4,2,1,3,2,2,2,4,2,1,2,3,1,2,1,2,4,1,4,4,2,4,3,1,1,1,3,4,2,2,1,4,1,1,4,3,4,2,4,2,4"
16 "2,2,4,2,1,2,2,1,2,3,4,4,4,1,2,2,2,2,2,2,2,1,2,2,2,2,4,1,2,1,2,2,2,1,2,2,4,1,2,1,3,2,4,2,2,2,2,2,2,4,2,2,2,4,2,4,2,2,3,2,2,2,2,2,2,1,1,1,3,2,3,1,4,2,1,4,2,2,1,2"
17 "3,3,4,3,1,3,3,3,2,1,3,4,1,3,3,3,3,3,2,2,2,1,3,2,1,3,3,3,4,3,3,2,2,3,1,2,1,3,2,4,1,2,3,2,3,3,3,3,2,1,1,4,2,1,1,3,1,2,1,3,3,3,2,2,3,4,1,3,3,3,1,3,1,3,3,1,3,3,1,3"
18 "1,1,2,1,3,1,1,1,1,2,1,1,2,1,1,1,1,1,2,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,2,1,1,1,3,1,2,1,1,1,1,1,1,1,1,2,1,2,1,1,2,1,4,1,1,1,1,1,1,1,4,1,2,1,4,1,4,1,1,2,1,1,1,1"
19 "2,2,4,2,3,2,2,2,2,2,2,2,4,2,2,2,2,2,4,2,2,2,2,2,2,2,4,2,2,2,2,2,2,2,2,2,4,2,2,2,3,2,4,2,2,2,2,2,2,4,2,2,2,4,2,2,4,2,1,2,2,2,2,2,4,4,1,2,4,2,1,2,1,2,2,4,2,2,2,2"
   Cluster_labels                                                                                                                                                                                                                          
1  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
2  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
3  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
4  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
5  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
6  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
7  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
8  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
9  "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
10 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
11 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
12 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
13 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
14 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
15 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
16 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
17 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
18 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
19 "3,42,26,79,41,66,53,76,6,68,74,7,30,57,69,48,80,45,61,49,55,52,50,16,70,28,13,24,60,40,64,11,19,1,38,58,29,54,37,62,34,51,71,43,25,77,4,36,14,20,9,35,17,23,46,59,33,73,63,8,65,10,67,21,47,15,12,31,75,56,22,18,32,44,27,5,39,72,2,78"
> # Metric     Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore   Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size Cluster_4_Size
> # 1  "ANOnto"   "0.569222510427433" "0.552363239306396" "0.584449669565973"  "0.717030499002753" "0.600638738086962"  "11"           "4"            "53"           "12"
> # 2  "AROnto"   "0.891757427020894" "0.498602630835942" "0.953766280221553"  "0.614385150712436" "0.813833608784603"  "58"           "7"            "2"            "13"
> # 3  "CBOOnto"  "0.682847685112873" "0.475694878561971" "0.418096612044278"  "0.462053414220223" "0.5843870090796"    "46"           "18"           "14"           "2"
> # 4  "CBOOnto2" "0.682847685112873" "0.475694878561971" "0.418096612044278"  "0.462053414220223" "0.5843870090796"    "46"           "18"           "14"           "2"
> # 5  "CROnto"   "0.615016966742524" "0.931552645421743" "0.460688748724164"  "0"                 "0.84502648526675"   "10"           "63"           "6"            "1"
> # 6  "DITOnto"  "0.621392145232729" "0.589638237470761" "0.512852920317478"  "0.717462336796908" "0.582143307479606"  "15"           "35"           "24"           "6"
> # 7  "INROnto"  "0.679354776901229" "0.514845315378322" "0.552323396139528"  "0"                 "0.609561353444975"  "46"           "19"           "14"           "1"
> # 8  "LCOMOnto" "0.563584714383498" "0.565734453969461" "0.526937877760086"  "0.662861247621334" "0.57713748864992"   "19"           "19"           "23"           "19"
> # 9  "NACOnto"  "0.507554700154524" "0.763008703189753" "0.0693863149967116" "0.610806402578204" "0.627188990478616"  "23"           "42"           "5"            "10"
> # 10 "NOCOnto"  "0.363472944618239" "0.712806750183687" "0.368068489789737"  "0.711626648649838" "0.600607673118847"  "2"            "51"           "24"           "3"
> # 11 "NOMOnto"  "0.796568957921031" "0"                 "0.487448631370323"  "0.505810544669573" "0.620956620752701"  "35"           "1"            "25"           "19"
> # 12 "POnto"    "0.717551583859045" "0.702605079149018" "0.531828315626997"  "0.755700546488043" "0.676374911502771"  "14"           "42"           "16"           "8"
> # 13 "PROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586"  "0.546429726628472" "0.623564355956028"  "22"           "23"           "12"           "23"
> # 14 "RFCOnto"  "0.708660103503223" "0"                 "0.527891770926241"  "0.575667190561062" "0.613856368788046"  "37"           "1"            "27"           "15"
> # 15 "RROnto"   "0.808419016380534" "0.636912857924547" "0.406920889282586"  "0.546429726628472" "0.623564355956028"  "22"           "23"           "12"           "23"
> # 16 "TMOnto"   "0.527581279093128" "0.772548576303018" "0.756878515673905"  "0.56435245544769"  "0.694408411158545"  "15"           "48"           "5"            "12"
> # 17 "TMOnto2"  "0.593309463294573" "1"                 "0.709314170957853"  "0.516092763511662" "0.725408613137789"  "19"           "16"           "39"           "6"
> # 18 "WMCOnto"  "0.811550829534933" "0.517887706724764" "0.751527957476758"  "0.232935788267106" "0.737070037248562"  "62"           "12"           "2"            "4"
> # 19 "WMCOnto2" "0.48724511207104"  "0.806794961402285" "0.613618761016468"  "0.458575230569131" "0.72940235766569"   "4"            "61"           "2"            "13"
> 
> #dataFrame <- qualityRange(data=ontMetrics, cbi="kmeans", k.range = c(3,4), all_metrics=TRUE, getImages = TRUE)
> #assay(dataFrame$k_3)
> # Metric        Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  Cluster_4_SilScore  Avg_Silhouette_Width Cluster_1_Size Cluster_2_Size Cluster_3_Size Cluster_4_Size
> # 1 "all_metrics" "0.560364615463509" "0.768006541644696" "0.761635263968552" "0.343459043619883" "0.730815149196402"  "2"            "70"           "2"            "6"
> 
> #dataFrame <- quality(data=ontMetrics, cbi="kmeans", k=4, all_metrics=TRUE)
> #assay(dataFrame)
> # Metric        Cluster_1_SilScore  Cluster_2_SilScore  Cluster_3_SilScore  Cluster_4_SilScore  Avg_Silhouette_Width
> # [1,] "all_metrics" "0.560364615463509" "0.768006541644696" "0.761635263968552" "0.343459043619883" "0.730815149196402"
> # Cluster_1_Size Cluster_2_Size Cluster_3_Size Cluster_4_Size
> # [1,] "2"            "70"           "2"            "6"
> 
> proc.time()
   user  system elapsed 
 12.445   0.339  12.792 

evaluomeR.Rcheck/tests/testStability.Rout


R Under development (unstable) (2024-11-24 r87369) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(evaluomeR)
Loading required package: SummarizedExperiment
Loading required package: MatrixGenerics
Loading required package: matrixStats

Attaching package: 'MatrixGenerics'

The following objects are masked from 'package:matrixStats':

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Loading required package: GenomicRanges
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname

Loading required package: IRanges
Loading required package: GenomeInfoDb
Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attaching package: 'Biobase'

The following object is masked from 'package:MatrixGenerics':

    rowMedians

The following objects are masked from 'package:matrixStats':

    anyMissing, rowMedians

Loading required package: MultiAssayExperiment
Loading required package: cluster
Loading required package: fpc
Loading required package: randomForest
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 'randomForest'

The following object is masked from 'package:Biobase':

    combine

The following object is masked from 'package:BiocGenerics':

    combine

Loading required package: flexmix
Loading required package: lattice
Loading required package: RSKC
Loading required package: flexclust
Loading required package: grid
Loading required package: modeltools

Attaching package: 'modeltools'

The following objects are masked from 'package:generics':

    fit, refit

Loading required package: sparcl
> library(RSKC)
> library(sparcl)
> 
> dataFrame <- stability(data=ontMetrics, cbi="kmeans", k=3, bs=100)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 3
Processing metric: AROnto(2)
	Calculation of k = 3
Processing metric: CBOOnto(3)
	Calculation of k = 3
Processing metric: CBOOnto2(4)
	Calculation of k = 3
Processing metric: CROnto(5)
	Calculation of k = 3
Processing metric: DITOnto(6)
	Calculation of k = 3
Processing metric: INROnto(7)
	Calculation of k = 3
Processing metric: LCOMOnto(8)
	Calculation of k = 3
Processing metric: NACOnto(9)
	Calculation of k = 3
Processing metric: NOCOnto(10)
	Calculation of k = 3
Processing metric: NOMOnto(11)
	Calculation of k = 3
Processing metric: POnto(12)
	Calculation of k = 3
Processing metric: PROnto(13)
	Calculation of k = 3
Processing metric: RFCOnto(14)
	Calculation of k = 3
Processing metric: RROnto(15)
	Calculation of k = 3
Processing metric: TMOnto(16)
	Calculation of k = 3
Processing metric: TMOnto2(17)
	Calculation of k = 3
Processing metric: WMCOnto(18)
	Calculation of k = 3
Processing metric: WMCOnto2(19)
	Calculation of k = 3
> assay(dataFrame)
      Metric     Mean_stability_k_3 
 [1,] "ANOnto"   "0.711599421597794"
 [2,] "AROnto"   "0.834242802235359"
 [3,] "CBOOnto"  "0.836200447888132"
 [4,] "CBOOnto2" "0.836200447888132"
 [5,] "CROnto"   "0.80871022609772" 
 [6,] "DITOnto"  "0.802620378293628"
 [7,] "INROnto"  "0.813132039213596"
 [8,] "LCOMOnto" "0.995402775270891"
 [9,] "NACOnto"  "0.705135779579475"
[10,] "NOCOnto"  "0.902528819875511"
[11,] "NOMOnto"  "0.793513639960901"
[12,] "POnto"    "0.660145923222329"
[13,] "PROnto"   "0.960518110441289"
[14,] "RFCOnto"  "0.765127486244089"
[15,] "RROnto"   "0.960518110441289"
[16,] "TMOnto"   "0.862955680341511"
[17,] "TMOnto2"  "0.953719590152899"
[18,] "WMCOnto"  "0.85715656831332" 
[19,] "WMCOnto2" "0.904134166028688"
> # Metric     Mean_stability_k_3
> # [1,] "ANOnto"   "0.711599421597794"
> # [2,] "AROnto"   "0.834242802235359"
> # [3,] "CBOOnto"  "0.836200447888132"
> # [4,] "CBOOnto2" "0.836200447888132"
> # [5,] "CROnto"   "0.80871022609772"
> # [6,] "DITOnto"  "0.802620378293628"
> # [7,] "INROnto"  "0.813132039213596"
> # [8,] "LCOMOnto" "0.995402775270891"
> # [9,] "NACOnto"  "0.705135779579475"
> # [10,] "NOCOnto"  "0.902528819875511"
> # [11,] "NOMOnto"  "0.793513639960901"
> # [12,] "POnto"    "0.660145923222329"
> # [13,] "PROnto"   "0.960518110441289"
> # [14,] "RFCOnto"  "0.765127486244089"
> # [15,] "RROnto"   "0.960518110441289"
> # [16,] "TMOnto"   "0.862955680341511"
> # [17,] "TMOnto2"  "0.953719590152899"
> # [18,] "WMCOnto"  "0.85715656831332"
> # [19,] "WMCOnto2" "0.904134166028688"
> 
> dataFrame <- stability(data=ontMetrics, cbi="kmeans", k=5, bs=100)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 5
Processing metric: AROnto(2)
	Calculation of k = 5
Processing metric: CBOOnto(3)
	Calculation of k = 5
Processing metric: CBOOnto2(4)
	Calculation of k = 5
Processing metric: CROnto(5)
	Calculation of k = 5
Processing metric: DITOnto(6)
	Calculation of k = 5
Processing metric: INROnto(7)
	Calculation of k = 5
Processing metric: LCOMOnto(8)
	Calculation of k = 5
Processing metric: NACOnto(9)
	Calculation of k = 5
Processing metric: NOCOnto(10)
	Calculation of k = 5
Processing metric: NOMOnto(11)
	Calculation of k = 5
Processing metric: POnto(12)
	Calculation of k = 5
Processing metric: PROnto(13)
	Calculation of k = 5
Processing metric: RFCOnto(14)
	Calculation of k = 5
Processing metric: RROnto(15)
	Calculation of k = 5
Processing metric: TMOnto(16)
	Calculation of k = 5
Processing metric: TMOnto2(17)
	Calculation of k = 5
Processing metric: WMCOnto(18)
	Calculation of k = 5
Processing metric: WMCOnto2(19)
	Calculation of k = 5
> assay(dataFrame)
      Metric     Mean_stability_k_5 
 [1,] "ANOnto"   "0.53661574785721" 
 [2,] "AROnto"   "0.808877375863211"
 [3,] "CBOOnto"  "0.773161766854306"
 [4,] "CBOOnto2" "0.773161766854306"
 [5,] "CROnto"   "0.747939612559589"
 [6,] "DITOnto"  "0.738901091226716"
 [7,] "INROnto"  "0.804579603939195"
 [8,] "LCOMOnto" "0.703629344931179"
 [9,] "NACOnto"  "0.663958844840551"
[10,] "NOCOnto"  "0.899994756895055"
[11,] "NOMOnto"  "0.758789978458299"
[12,] "POnto"    "0.646480707690646"
[13,] "PROnto"   "0.782307410022412"
[14,] "RFCOnto"  "0.726761185593769"
[15,] "RROnto"   "0.782307410022412"
[16,] "TMOnto"   "0.88221333660635" 
[17,] "TMOnto2"  "0.830282245373099"
[18,] "WMCOnto"  "0.747236615208537"
[19,] "WMCOnto2" "0.752468990321845"
> # Metric     Mean_stability_k_5
> # [1,] "ANOnto"   "0.53661574785721"
> # [2,] "AROnto"   "0.808877375863211"
> # [3,] "CBOOnto"  "0.773161766854306"
> # [4,] "CBOOnto2" "0.773161766854306"
> # [5,] "CROnto"   "0.747939612559589"
> # [6,] "DITOnto"  "0.738901091226716"
> # [7,] "INROnto"  "0.804579603939195"
> # [8,] "LCOMOnto" "0.703629344931179"
> # [9,] "NACOnto"  "0.663958844840551"
> # [10,] "NOCOnto"  "0.899994756895055"
> # [11,] "NOMOnto"  "0.758789978458299"
> # [12,] "POnto"    "0.646480707690646"
> # [13,] "PROnto"   "0.782307410022412"
> # [14,] "RFCOnto"  "0.726761185593769"
> # [15,] "RROnto"   "0.782307410022412"
> # [16,] "TMOnto"   "0.88221333660635"
> # [17,] "TMOnto2"  "0.830282245373099"
> # [18,] "WMCOnto"  "0.747236615208537"
> # [19,] "WMCOnto2" "0.752468990321845"
> 
> dataFrame <- stabilityRange(data=ontMetrics, cbi="kmeans", k.range = c(3,5), bs=100)

Data loaded.
Number of rows: 80
Number of columns: 20


Processing metric: ANOnto(1)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: AROnto(2)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: CBOOnto(3)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: CBOOnto2(4)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: CROnto(5)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: DITOnto(6)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: INROnto(7)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: LCOMOnto(8)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: NACOnto(9)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: NOCOnto(10)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: NOMOnto(11)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: POnto(12)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: PROnto(13)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: RFCOnto(14)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: RROnto(15)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: TMOnto(16)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: TMOnto2(17)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: WMCOnto(18)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
Processing metric: WMCOnto2(19)
	Calculation of k = 3
	Calculation of k = 4
	Calculation of k = 5
> assay(dataFrame)
      Metric     Mean_stability_k_3  Mean_stability_k_4  Mean_stability_k_5 
 [1,] "ANOnto"   "0.711599421597794" "0.661877018484356" "0.53661574785721" 
 [2,] "AROnto"   "0.834242802235359" "0.905679508527523" "0.808877375863211"
 [3,] "CBOOnto"  "0.836200447888132" "0.809715382620901" "0.773161766854306"
 [4,] "CBOOnto2" "0.836200447888132" "0.809715382620901" "0.773161766854306"
 [5,] "CROnto"   "0.80871022609772"  "0.848428661689236" "0.747939612559589"
 [6,] "DITOnto"  "0.802620378293628" "0.801976319968573" "0.738901091226716"
 [7,] "INROnto"  "0.813132039213596" "0.833324929464065" "0.804579603939195"
 [8,] "LCOMOnto" "0.995402775270891" "0.758953924881616" "0.703629344931179"
 [9,] "NACOnto"  "0.705135779579475" "0.679182045909186" "0.663958844840551"
[10,] "NOCOnto"  "0.902528819875511" "0.844518653163586" "0.899994756895055"
[11,] "NOMOnto"  "0.793513639960901" "0.779713596698101" "0.758789978458299"
[12,] "POnto"    "0.660145923222329" "0.795675361207579" "0.646480707690646"
[13,] "PROnto"   "0.960518110441289" "0.790969731730725" "0.782307410022412"
[14,] "RFCOnto"  "0.765127486244089" "0.790802265552443" "0.726761185593769"
[15,] "RROnto"   "0.960518110441289" "0.790969731730725" "0.782307410022412"
[16,] "TMOnto"   "0.862955680341511" "0.904973710968594" "0.88221333660635" 
[17,] "TMOnto2"  "0.953719590152899" "0.868195348078741" "0.830282245373099"
[18,] "WMCOnto"  "0.85715656831332"  "0.854182751568963" "0.747236615208537"
[19,] "WMCOnto2" "0.904134166028688" "0.883417390847072" "0.752468990321845"
> # Metric     Mean_stability_k_3  Mean_stability_k_4  Mean_stability_k_5
> # [1,] "ANOnto"   "0.711599421597794" "0.661877018484356" "0.53661574785721"
> # [2,] "AROnto"   "0.834242802235359" "0.905679508527523" "0.808877375863211"
> # [3,] "CBOOnto"  "0.836200447888132" "0.809715382620901" "0.773161766854306"
> # [4,] "CBOOnto2" "0.836200447888132" "0.809715382620901" "0.773161766854306"
> # [5,] "CROnto"   "0.80871022609772"  "0.848428661689236" "0.747939612559589"
> # [6,] "DITOnto"  "0.802620378293628" "0.801976319968573" "0.738901091226716"
> # [7,] "INROnto"  "0.813132039213596" "0.833324929464065" "0.804579603939195"
> # [8,] "LCOMOnto" "0.995402775270891" "0.758953924881616" "0.703629344931179"
> # [9,] "NACOnto"  "0.705135779579475" "0.679182045909186" "0.663958844840551"
> # [10,] "NOCOnto"  "0.902528819875511" "0.844518653163586" "0.899994756895055"
> # [11,] "NOMOnto"  "0.793513639960901" "0.779713596698101" "0.758789978458299"
> # [12,] "POnto"    "0.660145923222329" "0.795675361207579" "0.646480707690646"
> # [13,] "PROnto"   "0.960518110441289" "0.790969731730725" "0.782307410022412"
> # [14,] "RFCOnto"  "0.765127486244089" "0.790802265552443" "0.726761185593769"
> # [15,] "RROnto"   "0.960518110441289" "0.790969731730725" "0.782307410022412"
> # [16,] "TMOnto"   "0.862955680341511" "0.904973710968594" "0.88221333660635"
> # [17,] "TMOnto2"  "0.953719590152899" "0.868195348078741" "0.830282245373099"
> # [18,] "WMCOnto"  "0.85715656831332"  "0.854182751568963" "0.747236615208537"
> # [19,] "WMCOnto2" "0.904134166028688" "0.883417390847072" "0.752468990321845"
> 
> 
> #dataFrame <- stability(data=ontMetrics, cbi="kmeans", k=3, all_metrics = TRUE, bs=100)
> #assay(dataFrame)
> # Metric        Mean_stability_k_3
> # [1,] "all_metrics" "0.846238406081907"
> 
> #dataFrame <- stability(data=ontMetrics, cbi="kmeans", k=5, all_metrics = TRUE, bs=100)
> #assay(dataFrame)
> # Metric        Mean_stability_k_3
> # [1,] "all_metrics" "0.803322946463351"
> 
> #dataFrame <- stabilityRange(data=ontMetrics, cbi="kmeans", k.range = c(3,5), all_metrics = TRUE, bs=100)
> #assay(dataFrame)
> # Metric        Mean_stability_k_3  Mean_stability_k_4  Mean_stability_k_5
> # [1,] "all_metrics" "0.846238406081907" "0.783588073668732" "0.803322946463351"
> 
> proc.time()
   user  system elapsed 
 23.748   1.077  24.864 

Example timings

evaluomeR.Rcheck/evaluomeR-Ex.timings

nameusersystemelapsed
annotateClustersByMetric1.5620.0951.662
evaluomeRSupportedCBI000
getDataQualityRange0.4410.0360.478
getMetricsRelevancy2.2740.0282.307
getOptimalKValue0.4300.0240.455
globalMetric1.6890.0631.755
metricsCorrelations0.0480.0040.051
plotMetricsBoxplot1.5670.1561.726
plotMetricsCluster0.0560.0040.059
plotMetricsClusterComparison0.4370.0360.473
plotMetricsMinMax0.8490.0590.911
plotMetricsViolin1.0930.0681.163
quality0.1350.0000.135
qualityRange0.3320.0080.341
qualitySet0.0640.0000.065
stability3.1350.0883.230
stabilityRange3.5490.0643.622
stabilitySet0.4900.0120.503