Researchअनुसंधान
Our overarching goal is to broaden the understanding of sensory interactions from a science and engineering perspective, by analyzing the associated signals. We use critical thinking, systematic experimentation, and in-depth analysis to address the questions we find most interesting. Here are the themes we currently work on.हमारा व्यापक लक्ष्य संवेदी अंतःक्रियाओं से जुड़े संकेतों का विश्लेषण करके उन्हें विज्ञान और इंजीनियरिंग की दृष्टि से बेहतर समझना है। आलोचनात्मक सोच, व्यवस्थित प्रयोग और गहन विश्लेषण के ज़रिए हम उन प्रश्नों पर काम करते हैं जो हमें सबसे रोचक लगते हैं। ये हैं हमारे वर्तमान विषय।
Modeling sensory signal processingसंवेदी संकेत प्रसंस्करण की मॉडलिंग
संवेदी तंत्र दुनिया का सैंपल कैसे लेते हैं, और इससे हम संकेतों की सैंपलिंग के बारे में क्या सीखते हैं।
Our sensory system is operating 24x7, and is an evolutionary engineering marvel. We intend to understand how different sensory systems process signals. This process is more broadly termed as mechano-electro-transduction. Computationally modeling this can provide alternative methods for sampling and representation of continuous-time signals. One topic which we have carved out from this interest is “information-rich” sampling of non-stationary signals. Here, the goal is to sample not for reconstruction of the complete signal but for reconstruction of the information of interest.
Spoken conversation analysisबोली गई बातचीत का विश्लेषण
बातचीत कैसे आगे बढ़ती है, यह समझने के लिए वाणी, चेहरे, साँस, हृदय और पुतली के संकेतों का अध्ययन।
Spoken communication enables us to share thoughts, learn and create knowledge. Often we spend our day immersed in listening to or speaking in a conversation. Some conversations are quick and some go on and on, some start gradually and some end abruptly. Our goal is to explore spoken conversations by capturing and analyzing the speech, facial, breathing, heart and pupil signals of the participating individuals. This is a long-term effort involving building and testing several hypotheses, and gaining scientific insights into this amazing form of information transfer. Strength permitting, we will also analyze this aspect in bird (and animal) communication.
- Breaking the Pair: Evaluating dyadic interaction via speaker switching (SPCOM 2026)
- Accommodation or Drift? Nested controls for probing conversational adaptation in foundation-model embeddings (NeurIPS 2026 Workshop LP4FM)
- Leveraging LSTM models for overlap detection in multi-party meetings (ICASSP 2018)
Speech and language technology for Indian languagesभारतीय भाषाओं के लिए वाणी एवं भाषा प्रौद्योगिकी
हकलाहट, स्वर-सामंजस्य और कम संसाधन वाली भारतीय भाषाओं के लिए डेटासेट और मॉडल।
India speaks in hundreds of languages, and most of them are low-resourced for speech technology. We build datasets and models that respect this diversity: Boli, a multilingual dataset for understanding the stuttering experience; generative models that learn the vowel harmony of Assamese from raw audio; phonology-informed evaluation of multilingual text-to-speech; and, in a SERB-funded project, normative nasalance and acoustic characteristics of North-Eastern languages for language technology and speech therapy.
- Boli: A dataset for understanding stuttering experience and analyzing stuttered speech (ICASSP 2025)
- Deciphering Assamese vowel harmony with Featural InfoWaveGAN (Interspeech 2024)
- Towards a phonology-informed evaluation of multilingual TTS (Interspeech 2026, to appear)
Audio-based brain-computer interfacingऑडियो-आधारित ब्रेन-कंप्यूटर इंटरफ़ेस
ईईजी से पता लगाना कि श्रोता किस वक्ता को ध्यान से सुन रहा है।
Since 2013, several EEG decoding studies have demonstrated correlation between the attended audio signal and the corresponding brain activity captured using EEG. In a recent study, we corroborate this evidence and add new insights. We are looking forward to scale up this effort and design audio-based BCIs.
Audio cognition through behavioral experimentsव्यवहारिक प्रयोगों से श्रवण संज्ञान की समझ
बटन दबाने और प्रतिक्रिया-समय से हम कैसे सुनते हैं, इसका पता चलता है।
Unlike neuroimaging experiments like EEG and fMRI, behavioral experiments capture simple responses such as a button press or reaction time during a live experiment. When carefully captured, these simple responses can be analyzed to understand human cognition. We have explored talker perception, the impact of language on talker perception, and, currently, spatial cognition. We look forward to translating these findings into AI algorithms.
Data analysis for human healthcareमानव स्वास्थ्य के लिए डेटा विश्लेषण
दूरस्थ, कम लागत वाली रोग-जाँच के लिए साँस, खाँसी और वाणी को सुनना।
Do you recall a doctor listening to body sounds using a stethoscope? The art of listening to body sounds is called auscultation and dates back to the 1820s. We aim to extract more information from body sounds using signal processing and machine learning, towards remote, cost-effective, and scalable disease screening. We have analyzed respiratory sounds such as breathing, cough and speech collected during the pandemic from individuals with and without COVID-19, and released them openly as the Coswara dataset. Our ANRF-funded project Melodies of the Lungs takes this towards AI-based point-of-care respiratory healthcare.
Data analysis for climate sciencesजलवायु विज्ञान के लिए डेटा विश्लेषण
मौसम और जलवायु के प्रति जंगल की प्रतिक्रिया को उसकी ध्वनियों से समझना।
The cacophony of sounds in a forest draws contributions from many species, natural phenomena such as wind, rain, and thunder, and man-made vehicles and machines. Building on this observation, we intend to design ways to monitor a forest’s response to changing weather and climate conditions, allowing prediction of adversities. This is a long-term effort involving several hypotheses and sub-problems. We are also analyzing satellite imagery for climate and weather monitoring.
Fundingअनुदान
Prime Minister Early Career Research Grant, ANRF · PI
Core Research Grant, SERB · Co-PI