A Siamese Transformer Network for Zero-Shot Ancient Coin Classification

被引:3
|
作者
Guo, Zhongliang [1 ]
Arandjelovic, Ognjen [1 ]
Reid, David [1 ]
Lei, Yaxiong [1 ]
Buettner, Jochen [2 ]
机构
[1] Univ St Andrews, Sch Comp Sci, St Andrews KY16 9AJ, Fife, Scotland
[2] Max Planck Inst Hist Sci, Boltzmannstr 22, D-14195 Berlin, Germany
关键词
Siamese neural network; matching; deep learning; computer vision; machine learning; low-shot learning; RECOGNITION;
D O I
10.3390/jimaging9060107
中图分类号
TB8 [摄影技术];
学科分类号
0804 ;
摘要
Ancient numismatics, the study of ancient coins, has in recent years become an attractive domain for the application of computer vision and machine learning. Though rich in research problems, the predominant focus in this area to date has been on the task of attributing a coin from an image, that is of identifying its issue. This may be considered the cardinal problem in the field and it continues to challenge automatic methods. In the present paper, we address a number of limitations of previous work. Firstly, the existing methods approach the problem as a classification task. As such, they are unable to deal with classes with no or few exemplars (which would be most, given over 50,000 issues of Roman Imperial coins alone), and require retraining when exemplars of a new class become available. Hence, rather than seeking to learn a representation that distinguishes a particular class from all the others, herein we seek a representation that is overall best at distinguishing classes from one another, thus relinquishing the demand for exemplars of any specific class. This leads to our adoption of the paradigm of pairwise coin matching by issue, rather than the usual classification paradigm, and the specific solution we propose in the form of a Siamese neural network. Furthermore, while adopting deep learning, motivated by its successes in the field and its unchallenged superiority over classical computer vision approaches, we also seek to leverage the advantages that transformers have over the previously employed convolutional neural networks, and in particular their non-local attention mechanisms, which ought to be particularly useful in ancient coin analysis by associating semantically but not visually related distal elements of a coin's design. Evaluated on a large data corpus of 14,820 images and 7605 issues, using transfer learning and only a small training set of 542 images of 24 issues, our Double Siamese ViT model is shown to surpass the state of the art by a large margin, achieving an overall accuracy of 81%. Moreover, our further investigation of the results shows that the majority of the method's errors are unrelated to the intrinsic aspects of the algorithm itself, but are rather a consequence of unclean data, which is a problem that can be easily addressed in practice by simple pre-processing and quality checking.
引用
收藏
页数:34
相关论文
共 50 条
  • [1] A Siamese Transformer Network for Zero-Shot Ancient Coin Classification (vol 9, 107, 2023)
    Guo, Zhongliang
    Arandjelovic, Ognjen
    Reid, David
    Lei, Yaxiong
    Buettner, Jochen
    JOURNAL OF IMAGING, 2024, 10 (03)
  • [2] Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning
    Li, Xiangyu
    Yang, Xu
    Wei, Kun
    Deng, Cheng
    Yang, Muli
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, : 9316 - 9325
  • [3] Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network
    Jia, Dongya
    Tian, Qiao
    Peng, Kainan
    Li, Jiaxin
    Chen, Yuanzhe
    Ma, Mingbo
    Wang, Yuping
    Wang, Yuxuan
    INTERSPEECH 2023, 2023, : 5476 - 5480
  • [4] Knowledge Guided Transformer Network for Compositional Zero-Shot Learning
    Panda, Aditya
    Prasad, Dipti
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (11)
  • [5] A Novel Siamese Network for Few/Zero-Shot Handwritten Character Recognition Tasks
    Elaraby, Nagwa
    Barakat, Sherif
    Rezk, Amira
    CMC-COMPUTERS MATERIALS & CONTINUA, 2023, 74 (01): : 1 - 18
  • [6] Zero-shot Node Classification with Decomposed Graph Prototype Network
    Wang, Zheng
    Wang, Jialong
    Guo, Yuchen
    Gong, Zhiguo
    KDD '21: PROCEEDINGS OF THE 27TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY & DATA MINING, 2021, : 1769 - 1779
  • [7] Zero-shot image classification based on generative adversarial network
    Wei H.
    Zhang Y.
    Beijing Hangkong Hangtian Daxue Xuebao/Journal of Beijing University of Aeronautics and Astronautics, 2019, 45 (12): : 2345 - 2350
  • [8] Adaptive Relation-Aware Network for zero-shot classification
    Zhang, Xun
    Liu, Yang
    Dang, Yuhao
    Gao, Xinbo
    Han, Jungong
    Shao, Ling
    NEURAL NETWORKS, 2024, 174
  • [9] Learning Multipart Attention Neural Network for Zero-Shot Classification
    Meng, Min
    Wei, Jie
    Wu, Jigang
    IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, 2022, 14 (02) : 414 - 423
  • [10] Hybrid routing transformer for zero-shot learning
    Cheng, De
    Wang, Gerong
    Wang, Bo
    Zhang, Qiang
    Han, Jungong
    Zhang, Dingwen
    PATTERN RECOGNITION, 2023, 137