Enhancing Extractive Question Answering with A CNN-BERT Hybrid Model: Benchmark and Domain-Specific Evaluation

Abstract

Extractive question answering (QA) aims to locate precise answer spans within a text passage for a given question, which is crucial for applications such as virtual assistants, information retrieval, and domain-specific tools. Transformer-based models like BERT have achieved high accuracy on benchmark datasets, yet they often struggle with capturing fine-grained local patterns, domain-specific terminology, and computational efficiency. To address these limitations, we propose a hybrid CNN-BERT architecture that integrates convolutional neural networks with BERT’s contextual embeddings. The CNN layer enhances local feature extraction while BERT captures global semantic context, resulting in richer token representations. Experiments were conducted on the SQuAD 2.0 dataset and a custom banking QA dataset. The proposed model achieves an Exact Match (EM) of 82.5% and F1 score of 85.2% on SQuAD 2.0, outperforming recent transformer-based baselines. On the domain-specific dataset, it reaches 75.8% EM and 78.9% F1, demonstrating robustness in handling specialized vocabulary and unanswerable questions. Ablation studies confirm the importance of multi-scale CNN filters in capturing local patterns. These results indicate that the CNN-BERT hybrid model effectively balances global contextual understanding and local semantic precision, providing a reliable solution for both general and domain-specific QA tasks.