Comparative Analysis of Performance and Interpretability of XGBoost and TabNet Models in IDS Using XAI

Abstract

Cyberattacks are becoming a more apparent danger to modern network traffic. Robust and transparent Intrusion Detection Systems (IDS) are increasingly needed to counter this massive wave. The emergence of machine learning (ML) and deep learning (DL) offers a huge advantage in detecting this wave with high precision; nevertheless, they also have a downside due to their secretive nature regarding decision-making algorithms, often referred to as the “black-box” problem. Digital forensics is affected by this nature as well. As a result, XGBoost (ML) and TabNet (DL) were selected and compared based on their performance and interpretability using Explainable AI (XAI) to understand the reasoning behind their decisions with the CIC-IDS-2017 dataset in this study. Both models naturally achieved extremely high classification capabilities, with XGBoost slightly outperforming TabNet in overall detection reliability and in minimizing false negatives. The XAI evaluation additionally uncovered equally valid decision-making logic: TabNet prioritizes connection states and temporal variances, while XGBoost primarily relies on rigid volumetric payload statistics. LIME analysis also affirms that both models maintain high consistency with SHAP global explanations. In summary, XGBoost is recommended as the primary real-time detector due to its superior precision and transparent logic, while TabNet serves as a secondary validator.