---
title: "A Brief Introduction to TReNA"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A Brief Introduction to TReNA}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

## Overview of TReNA

TReNA is a package for inferring relationships between target genes and their transcription factors. By using TReNA, we can generate hypotheses aroung regulation of gene expression and enable the construction of gene network models. In the most basic sense, TReNA accepts a matrix of gene expression data (that must include the expression levels of transcription factors) and provides two operations using the data. First, it filters putitive transcriptional regulators based on user-specified criteria, such as presence of motifs or footprints. Second, given a target gene and list of transcription factors as predictors, it uses a user-specified solver to perform feature selection. Thus, the standard TReNA workflow is the following:

1. Create a TReNA object using an expression matrix (RNA-seq or microarray) and a desired solver.
2. (optional) Select candidate transcription factors from either motifs or footprints. 
3. Perform feature selection to identify putative transcriptional regulators for the target gene.

The filtering step, while optional, can constrain the solution space and reduce computational time by reducing the number of predictors supplied to the solver. Rather than supplying hundreds of transcription factors that can obscure meaningful relationships between genes, filters allow the user to screen out less likely candidates, leaving only the most likely affectors. 

Starting off, we'll load the TReNA library.

```{r}
suppressMessages(library(TReNA))
```

## Creating a TReNA Object

Running TReNA requires a matrix of gene expression data; there are several example data sets included with the TReNA package. In this case, we will load data from a study on Alzheimer's disease (Nat Sci Data, doi:10.1038/sdata.2016.89), in which 278 samples were obtained for 154 transcription factors.

```{r}
load(system.file(package="TReNA", "extdata/ampAD.154genes.mef2cTFs.278samples.RData"))
```

This gives us a matrix, *mtx.sub*, that contains our gene expression data in units of TMM. Before creating a TReNA object, it is wise to inspect the distribution of the matrix and consider using some sort of data transformation to normalize the mean-variance relationship. We can see, for instance, that our dataset is fairly skewed, with most of the expression values falling close to 0 and only a few values on the higher end of the spectrum.

```{r echo=FALSE, fig.width = 6}
hist(mtx.sub, main = "Expression Matrix Data")
```

The skewed nature of the matrix may profoundly affect our results. To overcome this, we will transform our matrix using the *voom* function in the *limma* package.

```{r}
suppressMessages(library(limma))
mtx.voom <- voom(mtx.sub)$E
```

If we now plot our transformed matrix, we can see that it's been scaled down substantially.

```{r echo=FALSE, fig.width = 6}
hist(mtx.voom, main = "VOOM-Transformed Expression Matrix Data")
```

Moving forward, we will use this transformed data set in the rest of our analyses. That gives us one half of our TReNA object; the other half concerns our choice of solver. TReNA currently supports 9 different solvers. Their associated character strings for use in TReNA are (in no particular order):

1. lasso - an application of *glmnet* using *alpha = 0.9* as the default elastic net parameter; default TReNA solver
2. ridge - an application of *glmnet* using *alpha = 0* as the default elastic net parameter
3. randomForest - an application of *randomForest*, the standard Random Forest function
4. bayesSpike - an application of *vbsr*, the Bayes Spike function
5. sqrtlasso - an application of the *slim* function from the *flare* package, with *q = 2*
6. lassopv - an application of *lassopv*, the P-Value LASSO function
7. pearson - an application of the *cor* function using the default parameters
8. spearman - an application of the *cor* function using *method = "spearman"*
9. ensemble - a combination of 2 or more of the other solvers; by default, it uses all but *bayesSpike* and *sqrtlasso*

Although TReNA defaults to using LASSO as the solver, we recommend doing some critical thinking about which solver most suits your purposes before choosing. Once a solver has been chosen, the TReNA object is simply specified (using Random Forest in this case):

```{r}
trena <- TReNA(mtx.assay = mtx.voom, solver = "randomForest")
```

We have now created an object of the TReNA class, which contains our solver and our assay matrix. 

## Using Filtering to Reduce Predictor Number

TReNA currently supports 3 different candidate transcription factor inputs:

1. FootprintFilter: using a PostgreSQL or SQLite database of footprint data, returns transcription factors with footprints for a given target gene for a specified genomic range
2. VarianceFilter: returns transcription factors with expression variance within a certain range of the target gene's expression variance
3. NullFilter: returns all transcription factors supplied in the matrix

In order to get a list of transcription factor candidates, you must construct the appropriate object by supplying an assay matrix, then call the *getCandidates* function on the candidate inputs. Depending on the input, this method may require other parameters. The simplest input is the NullFilter, which can return candidates with no additional parameters; however, as the name suggests, this just returns all transcription factors, not including the target gene.

```{r}

null.filter <- NullFilter(mtx.assay = mtx.voom)
tfs <- getCandidates(null.filter)
str(tfs)

```

The VarianceFilter requires the specification of a target gene and also allows you to specify a variance size range. It finds the variance of all genes in the assay matrix, then returns all transcription factors with variance within the variance size range of the target gene. For instance, we can create a VarianceFilter and use it to find all transcription factors with variance within 50% of the target gene's variance. This will return both the names of the transcription factors and their variances in a named list. 

```{r}

variance.filter <- VarianceFilter(mtx.assay = mtx.voom)
tf.list <- getCandidates(variance.filter, extraArgs = list("target.gene" = "MEF2C", "var.size" = 0.5))
str(tf.list)

```

The most complex filter is the FootprintFilter, which leverages information from footprint databases in either SQLite or PostgreSQL. This input requires connection to 2 databases: 1) A genome database, which contains information on the location and function of genes; 2) A project database, which contains footprint regions from a specific project. As an illustration of the format and utility of these databases, we have included in *extdata* two SQLite databases demonstrating the required tables and information needed to use the FootprintFilter. Our databases are subsets of larger databases, corresponding to the *MEF2C* gene and a footprinting project in the brain using the Wellington method for determining footprints. The FootprintFilter method for getting candidates also allows for specification of the distance upstream and downstream of a transcription start site to look for footprints; in this case, we'll use 1000 bases for each, the default distance. As with the VarianceFilter, we return a named list, with the names of the canidate transcription factors, plus a table of other information about them taken from the project and genome databases. 

```{r}
footprint.filter <- FootprintFilter(mtx.assay = mtx.voom)
db.address <- system.file(package = "TReNA", "extdata")
genome.db.uri <- paste("sqlite:/",db.address,"genome.sub.db", sep = "/")
project.db.uri <- paste("sqlite:/",db.address,"project.sub.db", sep = "/")
target.gene <- "MEF2C"
tfs <- getCandidates(footprint.filter, extraArgs = list("target.gene" = target.gene,
                                                        "genome.db.uri"=genome.db.uri, 
                                                        "project.db.uri" = project.db.uri,
                                                        "size.upstream" = 1000,
                                                        "size.downstream" = 1000))
str(tfs)
```

The FootprintFilter makes use of another class in the package, the FootprintFinder. This class provides an interface to PostgreSQL and SQLite databases of a specific 2-table schema and has a family of methods for accessing footprint data. To demonstrate some of this functionality, we will use the FootprintFinder to extract footprints from a particular region. In the previous example, we used an SQLite database included in the package; essentially a small snapshot of a larger dataset. More usefully, there are now publically available databases of footprints for use at [bddsrds.globusgenomics.org](). Currently, the sole project database is of brain tissue using the HINT alignment method; this database will remain in perpetuity and will be joined by other methods, issues, and alignments. We can access this information as follows:

```{r}
genome.db.uri    <- "postgres://bddsrds.globusgenomics.org/hg38"  # has gtf and motifsgenes tables
footprint.db.uri <- "postgres://bddsrds.globusgenomics.org/brain_hint"  # has hits and regions tables
fpf <- FootprintFinder(genome.db.uri, footprint.db.uri, quiet=FALSE)
tbl.fp <- getFootprintsInRegion(fpf, "chr5", 88822685, 89011824)
str(tbl.fp)
```

## Solving a TReNA Object

Each of solvers has its own strengths and weaknesses that we will not delve into here, but that are generally detailed within their own packages. To begin with, we will look at the LASSO solver, which is the default solver for a TReNA object. 

Once a user reaches the point of actually solving the TReNA object, a solver has already been specified, but the user must still specify the target gene response and transcription factor predictors of interest. For our purposes, we will obtain our list of transcription factors using a filter on variance, with *"MEF2C"* as our target gene. 

```{r}

trena <- TReNA(mtx.assay = mtx.voom, solver = "lasso")
target.gene <- "MEF2C"
gene.filter <- VarianceFilter(mtx.assay = mtx.voom)
tf.list <- getCandidates(gene.filter, extraArgs = list("target.gene" = target.gene, "var.size" = 0.5))

tbl.out <- solve(trena, target.gene, tf.list$tfs)
head(tbl.out)

```

Each solver will necessarily give a different result, neccessitating some amount of thoughtfulness on the part of the user. For instance, if the goal is to return a sparse list of only the most influential genes, then LASSO may be the best choice. By contrast, the Random Forest solver will return scores for all genes, and will thus result in inferring more relationships.  

## Using an Ensemble Solver Approach

In addition to the individual solvers, TReNA includes an ensemble approach as a solver option. The ensemble solver allows the user to specify a vector of multiple TReNA solvers as part of the *extraArgs* list argument (e.g. *extraArgs = list(solver.list = c("lasso","ridge")))*. Alternatively, the user can specify *solver.list = "default.solvers"* to run all solvers but Bayes Spike and square root LASSO (the most sensitive to outliers and the computationally intensive, respectively). In addition to computing and returning the individual scores of each solver, the ensemble solver generates an "extreme" score (*pcaMax*) based on all solvers and a "concordance" score (*concordance*). These new scores differ in that the pcaMax score tends to give more weight to extreme outlying scores from a particular solver and generally scales from 0 to 15, whereas the concordance score attempts to use more of a "voting" system of which solves consider a feature significant and ranges from 0-1. As an example, we can run the ensemble solver on the same problem we just solved using LASSO. 

```{r}
trena <- TReNA(mtx.assay = mtx.voom, solver = "ensemble")
target.gene <- "MEF2C"
gene.filter <- VarianceFilter(mtx.assay = mtx.voom)
tf.list <- getCandidates(gene.filter, extraArgs = list("target.gene" = target.gene, "var.size" = 0.5))
tbl.out <- solve(trena, target.gene, tf.list$tfs)
tbl.out
```

The ensemble approach is an attractive option for a couple of reasons:

1. Solver choice can be a non-trivial task for a TReNA user, as it requires some working knowledge of the pros and cons of the different methods. The ensemble method alleviates the need to choose one solver. 

2. Different solvers will give different predictions for important transcription factors; the ensemble method provides a way to create a composite score that accounts for all included solvers and can thus be seen as something of an overall metric. 

3. The relationship between transcription factors and target genes is a young area of active investigation. How this relates to the various solvers is an open question. TReNA was born out of a desire to explore these relationships. 

We can optionally pass solver-specific parameters as well using named arguments in the *extraArgs* list argument. For instance, we may want to specify the number of cores to use for the square root LASSO solver and include it in our solver list. The following *solve* command will accomplish that, setting the number of cores to dedicate to square root LASSO as 4:

```{r}

tbl.out.2 <- solve(trena, target.gene, tf.list$tfs, 
                   extraArgs = list(
                     "solver.list" = c("lasso", "ridge", "sqrtlasso", "randomForest",
                                       "lassopv", "spearman", "pearson"),
                     "sqrtlasso" = list("num.cores" = 4)
                   ))
tbl.out.2
```

