Compressed full-text indexes

被引:410
|
作者
Navarro, Gonzalo
Makinen, Veli
机构
[1] Univ Chile, Santiago, Chile
[2] Univ Helsinki, FIN-00014 Helsinki, Finland
关键词
algorithms; text indexing; text compression; entropy;
D O I
10.1145/1216370.1216372
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Full-text indexes provide fast substring search over large text collections. A serious problem of these indexes has traditionally been their space consumption. A recent trend is to develop indexes that exploit the compressibility of the text, so that their size is a function of the compressed text length. This concept has evolved into self-indexes, which in addition contain enough information to reproduce any text portion, so they replace the text. The exciting possibility of an index that takes space close to that of the compressed text, replaces it, and in addition provides fast search over it, has triggered a wealth of activity and produced surprising results in a very short time, which radically changed the status of this area in less than 5 years. The most successful indexes nowadays are able to obtain almost optimal space and search time simultaneously. In this article we present the main concepts underlying (compressed) self-indexes. We explain the relationship between text entropy and regularities that show up in index structures and permit compressing them. Then we cover the most relevant self-indexes, focusing on how they exploit text compressibility to achieve compact structures that can efficiently solve various search problems. Our aim is to give the background to understand and follow the developments in this area.
引用
收藏
页数:61
相关论文
共 50 条
  • [21] SEARCHING FULL-TEXT DATABASES
    TENOPIR, C
    LIBRARY JOURNAL, 1988, 113 (08) : 60 - 61
  • [22] FULL-TEXT AND BIBLIOGRAPHIC DATABASES
    TENOPIR, C
    LIBRARY JOURNAL, 1985, 110 (19) : 62 - 63
  • [23] FULL-TEXT INFORMATION RETRIEVAL
    FAY, RJ
    LAW LIBRARY JOURNAL, 1971, 64 (02): : 167 - 175
  • [24] FULL-TEXT DATABASES IN MEDICINE
    SIEVERT, MC
    MCKININ, EJ
    JOHNSON, ED
    JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE, 1995, 46 (10): : 748 - 754
  • [25] Harvesting for full-text retrieval
    Simeoni, F
    Yakici, M
    Neely, S
    Crestani, F
    DIGITAL LIBRARIES: IMPLEMENTING STRATEGIES AND SHARING EXPERIENCES, PROCEEDINGS, 2005, 3815 : 204 - 213
  • [26] VIDEODISCS FOR FULL-TEXT SEARCHING
    SCHIPMA, PB
    ZIEMER, SM
    ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 1982, 183 (MAR): : 26 - CINF
  • [27] General science full-text
    Stoklosa, K
    LIBRARY JOURNAL, 2003, 128 (04) : 129 - 129
  • [28] FULL-TEXT ONLINE RETRIEVAL
    COLBERT, AW
    ONLINE, 1988, 12 (02): : 91 - 91
  • [29] SAGE full-text collections
    不详
    PROGRAM-ELECTRONIC LIBRARY AND INFORMATION SYSTEMS, 2003, 37 (04) : 271 - 271
  • [30] WHERE FULL-TEXT IS VIABLE
    COTTON, PL
    ONLINE REVIEW, 1987, 11 (02): : 87 - 93