Can LSH (locality-sensitive hashing) be replaced by neural network?

被引:1
|
作者
Liu, Renyang [1 ,3 ]
Zhao, Jun [4 ]
Chu, Xing [2 ,3 ]
Liang, Yu [2 ,3 ]
Zhou, Wei [2 ,3 ]
He, Jing [2 ,3 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650500, Yunnan, Peoples R China
[2] Yunnan Univ, Sch Software, Kunming 650500, Yunnan, Peoples R China
[3] Yunnan Univ, Engn Res Ctr Cyberspace, Kunming 650500, Yunnan, Peoples R China
[4] Didi Chuxing, Beijing 100000, Peoples R China
基金
中国国家自然科学基金;
关键词
Learned index; Deep learning; Locality-sensitive hashing; KNN; CODES;
D O I
10.1007/s00500-023-09402-3
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the rapid development of GPU (graphics processing unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (deep neural network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (locality-sensitive hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of data sets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
引用
收藏
页码:887 / 902
页数:16
相关论文
共 50 条
  • [21] Locality-Sensitive Hashing Optimizations for Fast Malware Clustering
    Oprisa, Ciprian
    Checiches, Marius
    Nandrean, Adrian
    2014 IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTER COMMUNICATION AND PROCESSING (ICCP), 2014, : 97 - +
  • [22] Learnable Locality-Sensitive Hashing for Video Anomaly Detection
    Lu, Yue
    Cao, Congqi
    Zhang, Yifan
    Zhang, Yanning
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (02) : 963 - 976
  • [23] Locality-Sensitive Hashing Without False Negatives for lp
    Pacuk, Andrzej
    Sankowski, Piotr
    Wegrzycki, Karol
    Wygocki, Piotr
    COMPUTING AND COMBINATORICS, COCOON 2016, 2016, 9797 : 105 - 118
  • [24] Fast Redescription Mining Using Locality-Sensitive Hashing
    Karjalainen, Maiju
    Galbrun, Esther
    Miettinen, Pauli
    MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES: RESEARCH TRACK, PT VII, ECML PKDD 2024, 2024, 14947 : 124 - 142
  • [25] Locality-Sensitive Hashing for Chi2 Distance
    Gorisse, David
    Cord, Matthieu
    Precioso, Frederic
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2012, 34 (02) : 402 - 409
  • [26] Kernelized Locality-Sensitive Hashing for Scalable Image Search
    Kulis, Brian
    Grauman, Kristen
    2009 IEEE 12TH INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2009, : 2130 - 2137
  • [28] MIXED-LSH: Reduction of Remote Accesses in Distributed Locality-Sensitive Hashing based on L1-distance
    Koga, Hisashi
    Oguri, Masayuki
    Watanabe, Toshinori
    2012 IEEE 26TH INTERNATIONAL CONFERENCE ON ADVANCED INFORMATION NETWORKING AND APPLICATIONS (AINA), 2012, : 175 - 182
  • [29] Digital Watermarks for Videos Based on a Locality-Sensitive Hashing Algorithm
    Sun, Yajuan
    Srivastava, Gautam
    MOBILE NETWORKS & APPLICATIONS, 2023, 28 (05): : 1724 - 1737
  • [30] Frequent-Itemset Mining Using Locality-Sensitive Hashing
    Bera, Debajyoti
    Pratap, Rameshwar
    COMPUTING AND COMBINATORICS, COCOON 2016, 2016, 9797 : 143 - 155