Research

Our overarching goal is to broaden the understanding of sensory interactions from a science and engineering perspective, by analyzing the associated signals. We utilize critical thinking, systematic experimentation, and in depth analysis, enabling us address questions we find most interesting.

Here are some themes that we currently work on:

Modeling sensory signal processing
Our sensory system is operating 24x7, and is an evolutionary engineering marvel. We intend to understand how different sensory systems process signals. This process is more broadly termed as mechano-electro-transduction. Computationally modeling this can provide alternative methods for sampling and representation of continuous-time signals. One topic which we have carved out from this interest is “information-rich’’ sampling of non-stationary signals. Here, the goal is to sample not for reconstruction of the complete signal but for reconstruction of the information of interest. Our initial work in this direction has proposed methods such as signal dependent sampling Ref, Ref, and extrema-triggered sampling of time-varying signals for instantaneous frequency estimation Ref.
Keywords: time-series analysis, information theory, sparse signal recovery, compressive sensing, time-frequency analysis, physical and neural processes in sensory systems, signal representations

Spoken conversation analysis
Spoken communication enables us to share thoughts, learn and create knowledge. Often we spend our day immersed in listening to or speaking in a conversation. Some conversations are quick and some go on and on, some start gradually and some end abruptly. Some conversations have stayed in our memory for quite some time. Our goal in this work is to explore spoken conversations by capturing and analyzing the speech, facial, breathing, heart and pupil signals of the participating individuals. This is a long-term effort involving building and testing of several hypotheses, and gaining scientific insights into this amazing form of information transfer used by humans. Strength permitting, we also will look forward to analyze this aspect in bird (and animal) communication.
Keywords: real-time multi-channel data acquisition and synchronization, signal processing, machine learning and NNs, psychology-based experiment design, perception and human cognition

Audio-based brain-computer interfacing
Since 2013, several EEG data decoding studies have demonstrated correlation between the attended audio signal and the corresponding brain activity captured using EEG. In a recent study, we corroborate this evidence Ref and add new insights. We are looking forward to scale up this effort and design audio-based BCIs.
Keywords: EEG signal capture and analysis, signal processing, machine learning, audio processing

Understanding audio cognition using behavioral experiments
Unlike neuroimaging experiments like EEG and fMRI, behavioral experiments are designed to capture simple behavioral responses such as button press, reaction time, etc. during a live experiment. Interestingly, these simple responses when carefully captured can be analyzed to understand human cognition. We have explored analyzing aspects such as talker perception Ref1, impact of language on talker perception Ref2, and currently ongoing, spatial cognition. We look forward to continue learning from this direction of work and translate our findings to make AI algorithms.
Keywords: psychology and cognitive sciences, vision and audition, data science and inferential statistics, AI

Data analysis for human healthcare
Do you recall a doctor listening to body sounds using a stethoscope? The art of listening to body sounds is referred to as auscultation and dates back to the 1820s. Our goal is to further the information extraction from body sounds using signal processing and machine learning. The goal is to design remote, cost-effective, and scalable disease screening methodologies. In a recent work, we pursued analysis of respiratory sounds such as breathing, cough and speech collected during the pandemic from individuals with and without COVID-19 Ref, Ref.
Keywords: time-series analysis, signal processing, audio signal processing, machine learning and NNs, inferential statistics, full stack web development, biosciences

Data analysis for climate sciences
The cacophony of sounds in a forest draws contributions from many different species, natural phenomena such as wind, rain, and thunder, and man-made vehicles and machines. Building on this observation, we intend to design ways to monitor forest’s response to changing weather and climate conditions, allowing prediction of adversities. This is a long-term effort involving testing of several hypothesis and solving multiple sub-problems.
Keywords: wildlife sensor design and deployment, data analysis, signal processing and machine learning, ecological sciences, full stack web development

Exploratory data analysis and visualization
Data science, being an interdisciplinary field, brings challenges and opportunities for designing creative ways to visualize data. Sometimes a good visualization can highlight hidden patterns in data and light up new ideas for solving a problem. We recognize this talent and try pulling data from various sources to illustrate some interesting patterns.
Keywords: visualization, sonification, animating data science and AI concepts, linear algebra, full stack web development

… and more as we continue to learn and do.