Efficient transformer architecture for sarcasm detection: a study on compression and performance

Abstract

This sarcasm detection is a crucial subtask in natural language processing (NLP) particularly for sentiment analysis and conversational AI. Its complexity lies in interpreting context, tone, and intent beyond literal meanings. Traditional models often struggle to capture such nuances, especially in informal and diverse language settings. Moreover, existing approaches lack computational efficiency and fail to adapt well across different domains. This study evaluates three benchmark datasets—News Headlines, Mustard, and Reddit (SARC)—representing structured, scripted, and conversational sarcasm, respectively. Each dataset poses unique linguistic and contextual challenges. The proposed methodology integrates transformer-based models (RoBERTa and DistilBERT) with context summarization using BART and metadata embedding. A comparative analysis is conducted on both linguistic accuracy and computational efficiency. The novelty lies in aligning sarcasm detection performance with architectural optimization for real-time deployment. Evaluation is conducted using accuracy, F1-score, Jaccard coefficient, precision, and recall. Results show that RoBERTa delivers peak performance, while DistilBERT achieves a 1.74× speedup with competitive results, making it suitable for scalable and efficient sarcasm detection.