Can LSH (locality-sensitive hashing) be replaced by neural network?

被引:1
|
作者
Liu, Renyang [1 ,3 ]
Zhao, Jun [4 ]
Chu, Xing [2 ,3 ]
Liang, Yu [2 ,3 ]
Zhou, Wei [2 ,3 ]
He, Jing [2 ,3 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650500, Yunnan, Peoples R China
[2] Yunnan Univ, Sch Software, Kunming 650500, Yunnan, Peoples R China
[3] Yunnan Univ, Engn Res Ctr Cyberspace, Kunming 650500, Yunnan, Peoples R China
[4] Didi Chuxing, Beijing 100000, Peoples R China
基金
中国国家自然科学基金;
关键词
Learned index; Deep learning; Locality-sensitive hashing; KNN; CODES;
D O I
10.1007/s00500-023-09402-3
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the rapid development of GPU (graphics processing unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (deep neural network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (locality-sensitive hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of data sets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
引用
收藏
页码:887 / 902
页数:16
相关论文
共 50 条
  • [11] DB-LSH 2.0: Locality-Sensitive Hashing With Query-Based Dynamic Bucketing
    Tian, Yao
    Zhao, Xi
    Zhou, Xiaofang
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2024, 36 (03) : 1000 - 1015
  • [12] Optimal Parameters for Locality-Sensitive Hashing
    Slaney, Malcolm
    Lifshits, Yury
    He, Junfeng
    PROCEEDINGS OF THE IEEE, 2012, 100 (09) : 2604 - 2623
  • [13] Locality-sensitive hashing for the edit distance
    Marcais, Guillaume
    DeBlasio, Dan
    Pandey, Prashant
    Kingsford, Carl
    BIOINFORMATICS, 2019, 35 (14) : I127 - I135
  • [14] Using Locality-sensitive Hashing for Rendezvous Search
    Jiang, Guann-Yng
    Chang, Cheng-Shang
    ICC 2023-IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS, 2023, : 1743 - 1749
  • [15] S2JS']JSD-LSH: A Locality-Sensitive Hashing Schema for Probability Distributions
    Mao, Xian-Ling
    Feng, Bo-Si
    Hao, Yi-Jing
    Nie, Liqiang
    Huang, Heyan
    Wen, Guihua
    THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2017, : 3244 - 3251
  • [16] Locality-sensitive hashing of permutations for proximity searching
    Figueroa, Karina
    Camarena-Ibarrola, Antonio
    Valero-Elizondo, Luis
    Reyes, Nora
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2019, 36 (05) : 4677 - 4684
  • [17] Locality-sensitive hashing for finding nearest neighbors
    Slaney, Malcolm
    Casey, Michael
    IEEE SIGNAL PROCESSING MAGAZINE, 2008, 25 (02) : 128 - 131
  • [18] Non-Metric Locality-Sensitive Hashing
    Mu, Yadong
    Yan, Shuicheng
    PROCEEDINGS OF THE TWENTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE (AAAI-10), 2010, : 539 - 544
  • [19] DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor Search
    Wei, Jiuqi
    Peng, Botao
    Lee, Xiaodong
    Palpanas, Themis
    PROCEEDINGS OF THE VLDB ENDOWMENT, 2024, 17 (09): : 2241 - 2254
  • [20] IP.LSH.DBSCAN: Integrated Parallel Density-Based Clustering Through Locality-Sensitive Hashing
    Keramatian, Amir
    Gulisano, Vincenzo
    Papatriantafilou, Marina
    Tsigas, Philippas
    EURO-PAR 2022: PARALLEL PROCESSING, 2022, 13440 : 268 - 284