Adaptive Testing for High-Dimensional Data

被引:0
|
作者
Zhang, Yangfan [1 ]
Wang, Runmin [2 ]
Shao, Xiaofeng [3 ]
机构
[1] Two Sigma Investments, New York, NY USA
[2] Texas A&M Univ, Dept Stat, 3143 TAMU, College Stn, TX 77843 USA
[3] Univ Illinois, Dept Stat, Champaign, IL USA
关键词
Independence testing; Simultaneous testing; Spatial sign; U-statistics; HIGHER CRITICISM; COVARIANCE-MATRIX; 2-SAMPLE TEST; ASYMPTOTIC DISTRIBUTIONS; U-STATISTICS; INDEPENDENCE; COHERENCE; SIGNALS; ANOVA;
D O I
10.1080/01621459.2024.2439617
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
In this article, we propose a class of L-q -norm based U-statistics for a family of global testing problems related to high-dimensional data. This includes testing of mean vector and its spatial sign, simultaneous testing of linear model coefficients, and testing of component-wise independence for high-dimensional observations, among others. Under the null hypothesis, we derive asymptotic normality and independence between L-q -norm based U-statistics for several qs under mild moment and cumulant conditions. A simple combination of two studentized L-q -based test statistics via their p-values is proposed and is shown to attain great power against alternatives of different sparsity. Our work is a substantial extension of He et al., which is mostly focused on mean and covariance testing, and we manage to provide a general treatment of asymptotic independence of L-q -norm based U-statistics for a wide class of kernels. To alleviate the computation burden, we introduce a variant of the proposed U-statistics by using the monotone indices in the summation, resulting in a U-statistic with asymmetric kernel. A dynamic programming method is introduced to reduce the computational cost from O(n(qr)) , which is required for the calculation of the full U-statistic, to O(n (R)) where r is the order of the kernel. Numerical results further corroborate the advantage of the proposed adaptive test as compared to some existing competitors. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] Testing the additional predictive value of high-dimensional molecular data
    Anne-Laure Boulesteix
    Torsten Hothorn
    BMC Bioinformatics, 11
  • [22] Nonparametric Additive Regression for High-Dimensional Group Testing Data
    Zuo, Xinlei
    Ding, Juan
    Zhang, Junjian
    Xiong, Wenjun
    MATHEMATICS, 2024, 12 (05)
  • [23] Testing ARCH effect of high-dimensional time series data
    Li, Xuejiao
    Wei, Shufang
    Yang, Yaxing
    COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION, 2024,
  • [24] On high-dimensional two sample mean testing statistics: a comparative study with a data adaptive choice of coefficient vector
    Soeun Kim
    Jae Youn Ahn
    Woojoo Lee
    Computational Statistics, 2016, 31 : 451 - 464
  • [25] On high-dimensional two sample mean testing statistics: a comparative study with a data adaptive choice of coefficient vector
    Kim, Soeun
    Ahn, Jae Youn
    Lee, Woojoo
    COMPUTATIONAL STATISTICS, 2016, 31 (02) : 451 - 464
  • [26] LoHDP: Adaptive local differential privacy for high-dimensional data publishing
    Shen, Guohua
    Cai, Mengnan
    Huang, Zhiqiu
    Yang, Yang
    Guo, Feifei
    Wei, Linlin
    CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2024, 36 (11):
  • [27] Adaptive banding covariance estimation for high-dimensional multivariate longitudinal data
    Qian, Fang
    Zhang, Weiping
    Chen, Yu
    CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE, 2021, 49 (03): : 906 - 938
  • [28] High-dimensional generalized median adaptive lasso with application to omics data
    Liu, Yahang
    Gao, Qian
    Wei, Kecheng
    Huang, Chen
    Wang, Ce
    Yu, Yongfu
    Qin, Guoyou
    Wang, Tong
    BRIEFINGS IN BIOINFORMATICS, 2024, 25 (02)
  • [29] Data structures and algorithms for high-dimensional structured adaptive mesh refinement
    Grandin, Magnus
    ADVANCES IN ENGINEERING SOFTWARE, 2015, 82 : 75 - 86
  • [30] On the performance of adaptive preprocessing technique in analyzing high-dimensional censored data
    Khan, Md Hasinur Rahaman
    BIOMETRICAL JOURNAL, 2018, 60 (04) : 687 - 702