NeuroPilot translates neural EEG signals (brainwaves) into natural language commands and allows users to control computer functions simply by imagining speech; It is a real-time, end-to-end EEG-to-action pipeline.
With over 5.4 million people in the United States alone having some form of paralysis, limited motor and vocal function is a pervasive barrier for many in leveraging the power of computers. NeuroPilot provides an accessible digital assistant to help support these people and expand the potential of technology to serve everyone.
The impactful technology of NeuroPilot is enabled by a complex yet systematic pipeline. EEG inputs are processed by an RNN that predicts phoneme sequences, which are then decoded to identify probable utterances through a language model; the resulting thought stream is refined into an effective prompt for commanding agentic use of computer instances with Scrapybara. We apply a transfer learning approach that leverages learned features from pretraining on quality invasive data to apply to limited data from an accessible non-invasive BCI electrode headset.
NeuroPilot is the future of digital accessibility and effective human-AI workflows with the power of your mind.
Team - Nicole Ma, Zeyneb Kaya, Harsh Singh