Talk
Beyond Downstream Accuracy: A Multi-Axis Framework for Evaluating Synthetic EEG in Seizure Detection
Vanda Barata (Vanda Barata); Ana de Almeida (de Almeida, A.); Luís Nunes (Nunes, Luis);
Event Title
EPIA Conference on Artificial Intelligence (EPIA 2026)
Year (definitive publication)
2026
Language
English
Country
Portugal
More Information
--
Web of Science®

This publication is not indexed in Web of Science®

Scopus

This publication is not indexed in Scopus

Google Scholar

This publication is not indexed in Google Scholar

This publication is not indexed in Overton

Abstract
Seizures are rare events in electroencephalogram (EEG) recordings, and clinical data are scarce and hard to access, making it difficult to train reliable classifiers. Synthetic data augmentation is increasingly used to address this, yet most studies evaluate generators using only utility metrics like accuracy or F1 score, without validating the generated data itself. Such metrics may be misleading under extreme class imbalance and do not assess whether synthetic signals are physiologically plausible or clinically relevant. This paper proposes a multi-axis evaluation framework that evaluates synthetic EEG beyond utility alone, across four complementary dimensions: (1)fidelity, the degree to which generated signals preserve spectral, temporal, and spatial structure; (2)discriminability, how easily a classifier can distinguish real from synthetic data; (3)utility, measured by per-patient Area Under the Precision-Recall Curve (AUPRC) and sensitivity at 95\% specificity, ideally under leave-one-patient-out cross-validation; and (4) privacy, the extent to which generators memorise subject-specific patterns and whether augmentation reduces or amplifies the detector's reliance on patient identity. We apply the framework to three generators (TimeGAN, Conditional Variational Autoencoder, and Latent Diffusion Model) on the CHB-MIT dataset, evaluating only three of the four axes due to computational constraints. We show that even three axes reveal failure modes invisible to utility metrics alone - the generator with the most uniform spectral fidelity does not improve detection, while one with better fidelity in seizure-relevant bands does. Only the joint assessment across axes explains why and correctly identifies which generator yields a meaningful improvement.
Acknowledgements
This work was partially supported by national funds through Funda\c{c}\~{a}o para a Ci\^{e}ncia e a Tecnologia, I.P. (FCT), ISTAR-Iscte Pluriannual Projects UID/04466/2025 (https://doi.org/10.54499/UID/04466/2025)
Keywords
EEG,Epileptic seizure,Anomaly detection,Synthetic data,Time-series,Deep generative models
  • Computer and Information Sciences - Natural Sciences
  • Health Sciences - Medical and Health Sciences

With the objective to increase the research activity directed towards the achievement of the United Nations 2030 Sustainable Development Goals, the possibility of associating scientific publications with the Sustainable Development Goals is now available in Ciência_Iscte. These are the Sustainable Development Goals identified by the author(s) for this publication. For more detailed information on the Sustainable Development Goals, click here.