Builds a synthetic twin of x that preserves every marginal distribution
exactly and the correlation matrix to within sampling error, while
containing no cluster structure by construction. With
copula = "gaussian" (the default) all dependence in the twin is Gaussian.
Clustering found in real data but not in its twins must come from structure
beyond the margins and covariance; clustering found equally in both was
never more than the data's shape.
Usage
copula_null(x, copula = c("gaussian", "t"), df = 8, ridge = 1e-06)Arguments
- x
A numeric matrix or data frame (rows = observations, columns = variables), complete cases only, at least two rows and two columns.
- copula
Dependence family of the twin:
"gaussian"(default) or"t". The Gaussian family is the null of the method;"t"supplies a second, deliberately blinded null for stress-testing an exceedance the Gaussian twins have already licensed, and does not replace it.- df
Degrees of freedom of the t copula, a single positive number (default 8; used only when
copula = "t"). Smaller is heavier-tailed;df = 3is a hard stress test,df = 8a moderate one.- ridge
Small value added to the diagonal of the correlation matrix only if it is not positive definite (default
1e-6).
Value
A numeric matrix of the same dimensions as x: one matched-null
draw. Each column contains exactly the values of the corresponding column
of x, rearranged.
Details
The construction is a rank reordering in the tradition of Iman and Conover (1982): draw a Gaussian sample with the data's correlation matrix, then replace each column with the sorted real values laid down in the rank order of the Gaussian column. Every real value is reused exactly once per column, which is why the margins match exactly; the rank correlation is matched exactly and the Pearson correlation to within sampling error.
With copula = "t" the same construction is driven by a multivariate t
sample instead: margins and correlations are preserved as before, but the
twin also carries tail dependence, so extreme values across variables
arrive together, the more strongly the smaller df. A t twin is still a
single population with no clusters. Its use is as a stress test: a verdict
of "exceeds the Gaussian null" that a t twin reproduces was heavy-tailed
dependence, not types.
References
Iman, R. L., & Conover, W. J. (1982). A distribution-free approach to inducing rank correlation among input variables. Communications in Statistics - Simulation and Computation, 11(3), 311-334.
Examples
set.seed(1)
x <- matrix(rnorm(200 * 3), 200, 3) %*% chol(matrix(c(1, .5, .3,
.5, 1, .4,
.3, .4, 1), 3, 3))
twin <- copula_null(x)
# margins identical:
all(sort(twin[, 1]) == sort(x[, 1]))
#> [1] TRUE
# correlations close:
round(cor(x) - cor(twin), 2)
#> [,1] [,2] [,3]
#> [1,] 0.00 -0.02 -0.01
#> [2,] -0.02 0.00 0.03
#> [3,] -0.01 0.03 0.00
# a heavier-tailed twin for stress-testing:
stress <- copula_null(x, copula = "t", df = 3)
all(sort(stress[, 1]) == sort(x[, 1]))
#> [1] TRUE