High Recall Retrieval Via Technology-Assisted Review

被引:0
|
作者
Gray, Lenora [1 ]
Lewis, David D. [1 ]
Pickens, Jeremy [1 ]
Yang, Eugene [2 ]
机构
[1] Redgrave Data, Chantilly, VA 20151 USA
[2] Johns Hopkins Univ, HLTCOE, Baltimore, MD USA
关键词
text classification; human-in-the-loop; active learning; generative; AI; statistical evaluation;
D O I
10.1145/3626772.3661376
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
High Recall Retrieval (HRR) tasks, including eDiscovery in the law, systematic literature reviews, and sunshine law requests focus on efficiently prioritizing relevant documents for human review. Technology-assisted review (TAR) refers to iterative human-in-theloop workflows that combine human review with IR and AI techniques to minimize both time and manual effort while maximizing recall. This full-day tutorial provides a comprehensive introduction to TAR. The morning session presents an overview of the key technologies and workflow designs used, the basics of practical evaluation methods, and the social and ethical implications of TAR deployment. The afternoon session provides more technical depth on the implications of TAR workflows for supervised learning algorithm design, how generative AI is can be applied in TAR, more sophisticated statistical evaluation techniques, and a wide range of open research questions.
引用
收藏
页码:2987 / 2988
页数:2
相关论文
共 50 条
  • [1] ECIR 2022 Tutorial: Technology-Assisted Review for High Recall Retrieval
    Yang, Eugene
    Pickens, Jeremy
    Lewis, David D.
    ADVANCES IN INFORMATION RETRIEVAL, PT II, 2022, 13186 : 597 - 600
  • [2] Multi-Faceted Recall of Continuous Active Learning for Technology-Assisted Review
    Cormack, Gordon V.
    Grossman, Maura R.
    SIGIR 2015: PROCEEDINGS OF THE 38TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 2015, : 763 - 766
  • [3] Towards Question-based High-recall Information Retrieval: Locating the Last Few Relevant Documents for Technology-assisted Reviews
    Zou, Jie
    Kanoulas, Evangelos
    ACM TRANSACTIONS ON INFORMATION SYSTEMS, 2020, 38 (03)
  • [4] Unbiased Validation of Technology-Assisted Review for eDiscovery
    Cormack, Gordon V.
    Grossman, Maura R.
    Harbison, Andrew
    O'Halloran, Tom
    McManus, Bronagh
    PROCEEDINGS OF THE 47TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2024, 2024, : 2677 - 2681
  • [5] Technology-assisted assessment of spasticity: a systematic review
    Xinliang Guo
    Rebecca Wallace
    Ying Tan
    Denny Oetomo
    Marlena Klaic
    Vincent Crocher
    Journal of NeuroEngineering and Rehabilitation, 19
  • [6] A Review of Technology-Assisted Interventions for Diabetes Prevention
    Grock, Shira
    Ku, Jeong-hee
    Kim, Julie
    Moin, Tannaz
    CURRENT DIABETES REPORTS, 2017, 17 (11)
  • [7] Heuristic Stopping Rules For Technology-Assisted Review
    Yang, Eugene
    Lewis, David D.
    Frieder, Ophir
    PROCEEDINGS OF THE 21ST ACM SYMPOSIUM ON DOCUMENT ENGINEERING (DOCENG '21), 2021,
  • [8] Technology-Assisted Review for Spreadsheets and Noisy Text
    O'Halloran, Tom
    McManus, Bronagh
    Harbison, Andrew
    Grossman, Maura R.
    Cormack, Gordon V.
    PROCEEDINGS OF THE 2023 ACM SYMPOSIUM ON DOCUMENT ENGINEERING, DOCENG 2023, 2023,
  • [9] Engineering Quality and Reliability in Technology-Assisted Review
    Cormack, Gordon V.
    Grossman, Maura R.
    SIGIR'16: PROCEEDINGS OF THE 39TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 2016, : 75 - 84
  • [10] Technology-assisted assessment of spasticity: a systematic review
    Guo, Xinliang
    Wallace, Rebecca
    Tan, Ying
    Oetomo, Denny
    Klaic, Marlena
    Crocher, Vincent
    JOURNAL OF NEUROENGINEERING AND REHABILITATION, 2022, 19 (01)