Using fmrs package

Farhad Shokoohi

2020-10-27

fmrs package in action

Data generation

The function fmrs.gendata generates a data set from an FMRs model. It has the form

fmrs.gendata(nObs, nComp, nCov, coeff, dispersion, mixProp, rho, umax, ...)

where n is the sample size, nComp is the order of FMRs model, nCov is the number of regression covariates, coeff, dispersion and mixProp are the parameters of regression models, i.e. regression coefficients, dispersion (standard deviation) of the errors (sub-distributions) and mixing proportions, respectively, and rho is the used in the variance-covariance matrix for simulating the design matrix x, and umax is the upper bound for Uniform distribution for generating censoring times.

Depending on the choice of disFamily, the function fmrs.gendata generates a simulated data from FMRs models. For instance, if we choose disFamily = "norm", the function ignores the censoring parameter umax and generates a data set from an FMR model with Normal sub-distributions. However, if we choose disFamily = "lnorm" or disFamily = "weibull", the function generates data under a finite mixture of AFT regression model with Log-Normal or Weibull sub-distributions.

The fmrs.gendata function returns a list which includes a vector of responses $y, a matrix of covariates $x and a vector of censoring indicators $delta as well as the name of sub-distribution of the mixture model.

MLE of FMRs models

The function fmrs.mle in fmrs package provides maximum likelihood estimation for the parameters of an FMRs model. The function has the following form,

fmrs.mle(y, x, delta, nComp, ...)

where y, x and delta are observations, covariates and censoring indicators respectively, and nComp is the order of FMRs, initCoeff, initDispersion and initmixProp are initial values for EM and NR algorithms, and the rest of arguments of the function are controlling parameres. The function returns a fitted FMRs model with estimates of regression parameters, standard deviations and mixing proportions of the mixture model. It also returns the log-likelihood and BIC under the estimated model, the maximum number of iterations used in EM algorithm and the type of the fitted model.

Note that one can do ridge regression by setting a value for tuning parameter of the ridge penalty other than zero in the argument lambRidge.

Variable selection in FMRs models

To do the variable selection we provided the function fmrs.varsel with the form

fmrs.varsel(y, x, delta, nComp, ...)

where penFamily is the penalty including "adplasso", "lasso", "mcp", "scad", "sica" and "hard", and lambPen is the set of tuning parameters for components of penalty. We can run the function fmrslme first and use the parameter estimates as initial values for the function fmrs.varsel.

Choice of tuning parameter

There are two approaches for specifying tuning parameters: Common and Component-wise tuning parameters. If we consider choosing common tuning parameter, we can use the BIC criteria to search through the a set of candidate values in the interval \((0,\lambda_M)\), where \(\lambda_M\) is a prespecified number. The BIC is provided by the function fmrs.varsel for each candidate point and we choose the optimal \(\hat\lambda\), the one that maximizes BIC. This approach will be good for the situations with enough samples sizes. It also reduces the computational burden.

On the other hand, if we consider choosing component-wise tuning parameters we use the following function to search for optimal \((\lambda_1, \ldots, \lambda_K)\) from the set of candidate values in \((0, \lambda_M)\). The function is

fmrs.tunsel(y, x, delta, nComp, ...)

It is a good practice run the function fmrs.mle first and use the parameter estimates as initial values in the function fmrs.tunsel. The function fmrs.mle then returns a set optimal tuning parameters of size nComp to be used in variable selection function. Note that this approach still is under theoretical study and is not proved to give optimal values in general.

Example: finite mixture of AFT regression model (Log-Normal)

We use a simulated data set to illustrate using fmrs package. We generate the covariates (design matrix) from a multivariate normal distribution of dimension nCov=10 and sample size 200 with mean vector \(\bf 0\) and variance-covariance matrix \(\Sigma=(0.5^{|i-j|})\). We then generate time-to-event data from a finite mixture of two components AFT regression models with Log-Normal sub-distributions. The mixing proportions are set to \(\pi=(0.3, 0.7)\). We choose \(\boldsymbol\beta_0=(2,-1)\) as the intercepts, \(\boldsymbol\beta_1=(-1, -2, 1, 2, 0 , 0, 0, 0, 0, 0)\) and \(\boldsymbol\beta_2=(1, 2, 0, 0, 0 , 0, -1, 2, -2, 3)\) as the regression coefficients in first and second component, respectively.

We start by loading necessary libraries and assigning the parameters of model as follows.

## BugReports: https://github.com/shokoohi/fmrs/issues

Using the function fmrs.gendata, we generate a data set from a finite mixture of accelerated failure time regression models with above settings as follow.

Now we assume that the generated data are actually real data. We find MLE of the parameters of the assumed model using following code. Note that almost all mixture of regression models depends on initial values. Here we generate the initial values form a Normal distribution with

##                 Comp.1      Comp.2
## Intercept -1.010955632  2.82934475
## X.1       -1.074204696  2.56103599
## X.2        0.867308452 -1.24031167
## X.3        2.061889483 -2.67956391
## X.4        0.132001311  1.26454498
## X.5       -0.006088159  2.41799216
## X.6        0.036983323 -0.05336414
## X.7       -0.130527645  0.25911727
## X.8       -1.014300698 -0.15142594
## X.9        1.981867545  0.04084794
## X.10      -1.994605789  0.10765316
##         Comp.1   Comp.2
## [1,] 0.9929071 1.367746
##         Comp.1    Comp.2
## [1,] 0.5557073 0.4442927

As we see the ML estimates of regression coefficients are not zero. Therefore MLE cannot provide a sparse solution. In order to provide a sparse solution, we use the variable selection methods developed by Shokoohi et. ale. (2016). First we need to find a good set of tuning parameters. It can be done by using component-wise tuning parameter selection function implemented in fmrs as follows. In some settings, however, it is better to investigate if common tuning parameter performs better.

## An object of class 'fmrstunpar'
##   Finite Mixture of Accelerated Failure Time Regression Models
##     Log-Normal Sub-Distributions 
##   2 Components; adplasso Penalty;

We have used MLE estimates as initial values for estimating the tuning parameters. Now we used the same set of values to do variable selection with adaptive lasso penalty as follows.

##                  Comp.1       Comp.2
## Intercept -1.022469e+00  2.836041642
## X.1       -7.244985e-01  2.539338329
## X.2        4.908385e-01 -1.209699576
## X.3        2.053667e+00 -2.673814191
## X.4        7.642512e-03  1.241205239
## X.5        3.151068e-06  2.406888694
## X.6       -2.006873e-04 -0.001175967
## X.7       -9.977027e-03  0.152058087
## X.8       -7.214355e-01 -0.035671116
## X.9        1.580710e+00  0.002913783
## X.10      -1.742116e+00  0.040724698
##        Comp.1   Comp.2
## [1,] 1.113307 1.381159
##         Comp.1    Comp.2
## [1,] 0.5557073 0.4442927

The final variables that are selected using this procedure are those with non-zero coefficients. Note that a switching between components of mixture has happened here.

##      Comp.1 Comp.2
## X.1  -0.724  2.539
## X.2   0.491 -1.210
## X.3   2.054 -2.674
## X.4   0.008  1.241
## X.5   0.000  2.407
## X.6   0.000 -0.001
## X.7  -0.010  0.152
## X.8  -0.721 -0.036
## X.9   1.581  0.003
## X.10 -1.742  0.041

Therefore the variable selection and parameter estimation is done simultaneously using the fmrs package.