Eklavya Sarkar | EPFL Graph Search

Eklavya Sarkar is currently a Research Assistant and Doctoral Student at EPFL, working in the Speech and Audio Processing group at Idiap Research Institute, under the supervision of Dr. Mathew Magimai Doss. His project is funded by the NCCR Evolving Languages project, under the Automatic Speech Recognition (ASR) Work Package of the Transversal Task Force (TTF) Technology. Previously, he was a Research Intern in the Biometrics Security and Privacy lab at Idiap, under Dr. Sébastien Marcel, working on the Generation, Detection, and Vulnerability Analysis of Face Recognition Systems to Face Morphing Presentation Attacks, particularly on novel methods involving StyleGAN2.He also worked as an intern at CERN, under Dr. Archana Sharma, on the 'CMS-GEM' collaboration at the CMS Experiment Lab.He was affiliated with Prof. and Nobel Laureate Didier Queloz and supervised by Dr. Daniel Kessler for a project on Exoplanets during high-school, which was selected for the TM-TPE Colloque Transfrontalier 2013, also at CERN.He holds: MSc in Data Science from the University of Bath (2018-19), with a thesis supervised by Dr. Wenbin Li. BSc in Computer Science from the University of Liverpool (2015-18), with a thesis supervised by Dr. Irina Biktasheva. He is a Swiss citizen who grew up in Geneva, Switzerland after moving from New Delhi, India at the age of 10.In his free time he enjoys making and publishing silly indie games made on Unity, Swimming, Skiing, and holds a diploma in Film-Making from Brighton Film School.

Can Self-Supervised Neural Networks Pre-Trained on Human Speech distinguish Animal Callers?

Mathew Magimai Doss, Eklavya Sarkar

Self-supervised learning (SSL) models use only the intrinsic structure of a given signal, independent of its acoustic domain, to extract essential information from the input to an embedding space. This implies that the utility of such representations is not limited to modeling human speech alone. Building on this understanding, this paper explores the cross-transferability of SSL neural representations learned from human speech to analyze bio-acoustic signals. We conduct a caller discrimination analysis and a caller detection study on Marmoset vocalizations using eleven SSL models pre-trained with various pretext tasks. The results show that the embedding spaces carry meaningful caller information and can successfully distinguish the individual identities of Marmoset callers without fine-tuning. This demonstrates that representations pre-trained on human speech can be effectively applied to the bio-acoustics domain, providing valuable insights for future investigations in this field.

ISCA2023

Are GAN-based Morphs Threatening Face Recognition?

Laurent Colbois, Sébastien Marcel, Eklavya Sarkar

Morphing attacks are a threat to biometric systems where the biometric reference in an identity document can be altered. This form of attack presents an important issue in applications relying on identity documents such as border security or access control. Research in generation of face morphs and their detection is developing rapidly, however very few datasets with morphing attacks and open-source detection toolkits are publicly available. This paper bridges this gap by providing two datasets and the corresponding code for four types of morphing attacks: two that rely on facial landmarks based on OpenCV and FaceMorpher, and two that use StyleGAN 2 to generate synthetic morphs. We also conduct extensive experiments to assess the vulnerability of four state-of-the-art face recognition systems, including FaceNet, VGG-Face, ArcFace, and ISV. Surprisingly, the experiments demonstrate that, although visually more appealing, morphs based on StyleGAN 2 do not pose a significant threat to the state to face recognition systems, as these morphs were outmatched by the simple morphs that are based facial landmarks.

IEEE2022

Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering

Mathew Magimai Doss, Eklavya Sarkar

Voice activity detection (VAD) is an important pre-processing step for speech technology applications. The task consists of deriving segment boundaries of audio signals which contain voicing information. In recent years, it has been shown that voice source and vocal tract system information can be extracted using zero-frequency filtering (ZFF) without making any explicit model assumptions about the speech signal. This paper investigates the potential of zero-frequency filtering for jointly modeling voice source and vocal tract system information, and proposes two approaches for VAD. The first approach demarcates voiced regions using a composite signal composed of different zero-frequency filtered signals. The second approach feeds the composite signal as input to the rVAD algorithm. These approaches are compared with other supervised and unsupervised VAD methods in the literature, and are evaluated on the Aurora-2 database, across a range of SNRs (20 to -5 dB). Our studies show that the proposed ZFF-based methods perform comparable to state-of-art VAD methods and are more invariant to added degradation and different channel characteristics.

2022