Can LSH (locality-sensitive hashing) be replaced by neural network?

被引:1
|
作者
Liu, Renyang [1 ,3 ]
Zhao, Jun [4 ]
Chu, Xing [2 ,3 ]
Liang, Yu [2 ,3 ]
Zhou, Wei [2 ,3 ]
He, Jing [2 ,3 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650500, Yunnan, Peoples R China
[2] Yunnan Univ, Sch Software, Kunming 650500, Yunnan, Peoples R China
[3] Yunnan Univ, Engn Res Ctr Cyberspace, Kunming 650500, Yunnan, Peoples R China
[4] Didi Chuxing, Beijing 100000, Peoples R China
基金
中国国家自然科学基金;
关键词
Learned index; Deep learning; Locality-sensitive hashing; KNN; CODES;
D O I
10.1007/s00500-023-09402-3
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the rapid development of GPU (graphics processing unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (deep neural network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (locality-sensitive hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of data sets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
引用
收藏
页码:887 / 902
页数:16
相关论文
共 50 条
  • [41] Maintaining Academic Integrity in Programming: Locality-Sensitive Hashing and Recommendations
    Karnalim, Oscar
    EDUCATION SCIENCES, 2023, 13 (01):
  • [42] Locality-Sensitive Hashing for Efficient Web Application Security Testing
    Ben-Bassat, Ilan
    Rokah, Erez
    PROCEEDINGS OF THE 5TH INTERNATIONAL CONFERENCE ON INFORMATION SYSTEMS SECURITY AND PRIVACY (ICISSP), 2019, : 193 - 204
  • [43] Secure Approximate Nearest Neighbor Search with Locality-Sensitive Hashing
    Song, Shang
    Liu, Lin
    Chen, Rongmao
    Peng, Wei
    Wang, Yi
    COMPUTER SECURITY - ESORICS 2023, PT III, 2024, 14346 : 411 - 430
  • [44] Locality-Sensitive Hashing for Efficient Rendezvous Search: A New Approach
    Jiang, Guann-Yng
    Chang, Cheng-Shang
    IEEE TRANSACTIONS ON COMMUNICATIONS, 2024, 72 (09) : 5674 - 5687
  • [45] Scalable Graph Representation Learning via Locality-Sensitive Hashing
    Chen, Xiusi
    Jiang, Jyun-Yu
    Wang, Wei
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, CIKM 2022, 2022, : 3878 - 3882
  • [46] Collision Frequency Locality-Sensitive Hashing for Prediction of Critical Events
    Kim, Y. Bryce
    Hemberg, Erik
    O'Reilly, Una-May
    2017 39TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC), 2017, : 3088 - 3093
  • [47] Batch-Orthogonal Locality-Sensitive Hashing for Angular Similarity
    Ji, Jianqiu
    Yan, Shuicheng
    Li, Jianmin
    Gao, Guangyu
    Tian, Qi
    Zhang, Bo
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2014, 36 (10) : 1963 - 1974
  • [48] An improved method of locality-sensitive hashing for scalable instance matching
    Aydar, Mehmet
    Ayvaz, Serkan
    KNOWLEDGE AND INFORMATION SYSTEMS, 2019, 58 (02) : 275 - 294
  • [49] Speeding up probabilistic roadmap planners with locality-sensitive hashing
    Rantanen, Mika T.
    Juhola, Martti
    ROBOTICA, 2015, 33 (07) : 1491 - 1506
  • [50] LSH-SMILE: Locality Sensitive Hashing Accelerated Simulation and Learning
    Sima, Chonghao
    Xue, Yexiang
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34