Deep learning assisted diagnosis for assessing the severity of various respiratory infections using chest computed tomography (CT) scan images has gained much attention after the COVID-19 pandemic. Major tasks while building such models require an understanding of the characteristic features associated with the disease, patient-to-patient variations and changes associated with disease severity. In this work, an attention-based convolutional neural network (CNN) model with customized bottleneck residual module (Attn-CNN) is proposed for classifying CT images into three classes: COVID-19, normal, and other pneumonia. The efficacy of the model is evaluated by carrying out various experiments, such as effect of class imbalance, impact of attention module, generalizability of the model and providing visualization of model's prediction for the interpretability of results. Comparative performance evaluation with five state-of-the-art deep architectures such as MobileNet, EfficientNet-B7, Inceptionv3, ResNet-50 and VGG-16, and with published models such as COVIDNet-CT, COVNet, COVID-Net CT2, etc. is discussed.