Can LSH (locality-sensitive hashing) be replaced by neural network?

被引:1
|
作者
Liu, Renyang [1 ,3 ]
Zhao, Jun [4 ]
Chu, Xing [2 ,3 ]
Liang, Yu [2 ,3 ]
Zhou, Wei [2 ,3 ]
He, Jing [2 ,3 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650500, Yunnan, Peoples R China
[2] Yunnan Univ, Sch Software, Kunming 650500, Yunnan, Peoples R China
[3] Yunnan Univ, Engn Res Ctr Cyberspace, Kunming 650500, Yunnan, Peoples R China
[4] Didi Chuxing, Beijing 100000, Peoples R China
基金
中国国家自然科学基金;
关键词
Learned index; Deep learning; Locality-sensitive hashing; KNN; CODES;
D O I
10.1007/s00500-023-09402-3
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the rapid development of GPU (graphics processing unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (deep neural network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (locality-sensitive hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of data sets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
引用
收藏
页码:887 / 902
页数:16
相关论文
共 50 条
  • [31] Fast Access for Star Catalog Based on Locality-Sensitive Hashing
    Zhu H.
    Liang B.
    Zhang T.
    Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University, 2018, 36 (05): : 988 - 994
  • [32] Locality-Sensitive Hashing for Finding Nearest Neighbors in Probability Distributions
    Tang, Yi-Kun
    Mao, Xian-Ling
    Hao, Yi-Jing
    Xu, Cheng
    Huang, Heyan
    SOCIAL MEDIA PROCESSING, SMP 2017, 2017, 774 : 3 - 15
  • [33] On the Problem of p1-1 in Locality-Sensitive Hashing
    Ahle, Thomas Dybdahl
    SIMILARITY SEARCH AND APPLICATIONS, SISAP 2020, 2020, 12440 : 85 - 93
  • [34] An improved method of locality-sensitive hashing for scalable instance matching
    Mehmet Aydar
    Serkan Ayvaz
    Knowledge and Information Systems, 2019, 58 : 275 - 294
  • [35] Fast hierarchical clustering algorithm using locality-sensitive hashing
    Koga, H
    Ishibashi, T
    Watanabe, T
    DISCOVERY SCIENCE, PROCEEDINGS, 2004, 3245 : 114 - 128
  • [36] A Scalable ECG Identification System Based on Locality-Sensitive Hashing
    Chu, Hui-Yu
    Lin, Tzu-Yun
    Lee, Song-Hong
    Chiu, Jui-Kun
    Nien, Cing-Ping
    Wu, Shun-Chi
    2023 45TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE & BIOLOGY SOCIETY, EMBC, 2023,
  • [37] Similar Pair Identification using Locality-Sensitive Hashing Technique
    Lee, Kyung Mi
    Lee, Keon Myung
    6TH INTERNATIONAL CONFERENCE ON SOFT COMPUTING AND INTELLIGENT SYSTEMS, AND THE 13TH INTERNATIONAL SYMPOSIUM ON ADVANCED INTELLIGENT SYSTEMS, 2012, : 2117 - 2119
  • [38] Fast anomaly detection with locality-sensitive hashing and hyperparameter autotuning
    Meira, Jorge
    Eiras-Franco, Carlos
    Bolon-Canedo, Veronica
    Marreiros, Goreti
    Alonso-Betanzos, Amparo
    INFORMATION SCIENCES, 2022, 607 : 1245 - 1264
  • [39] CONSULT: accurate contamination removal using locality-sensitive hashing
    Rachtman, Eleonora
    Bafna, Vineet
    Mirarab, Siavash
    NAR GENOMICS AND BIOINFORMATICS, 2021, 3 (03)
  • [40] A novel locality-sensitive hashing for large scale image retrieva
    Li, Junyi
    Li, Jianhua
    Ni, Bingbing
    Yan, Shuicheng
    Journal of Computational Information Systems, 2012, 8 (23): : 9611 - 9617