Using Paper-2 functionality to perform live detection

[kubs0ne]

Hello,
I have a question, have you tried to run this detector using live audio recording? I've tried to implement this functionality to your program. Right now i have something like this:

scaler_filename = 'scaler.save'
scaler = joblib.load(scaler_filename)    
p = pyaudio.PyAudio()

CHUNK = 4800 #4800   # samples per frame
FORMAT = pyaudio.paFloat32 # audio format (bytes per sample?)
CHANNELS = 1                 # single channel for microphone
RATE = 8000                 # samples per second
stream = p.open(
    format=FORMAT,
    channels=CHANNELS,
    rate=RATE,
    input=True,
    output=True,
    frames_per_buffer=CHUNK
)
while True:
    data = stream.read(CHUNK)  
    decoded = np.fromstring(data, 'Float32')
    classes = predict_probability(decoded, scaler)

Generally, the program works, but it's very slow and has delay. I think the biggest problem here is the def predict_probability(y, scaler) function which i left unchanged from the em_detection.py program. Did you try any solutions to perform detections from live audio? Also I have a question how could I change the def predict_probability(y, scaler) function to work better with live audio? I tried to extract single mfcc from each chunk of live audio and then perform predictions, but it didn't work.
Kind regards
Kuba

[sheelabhadra]

Hi Kuba,
It's been a while since I worked on this project. I did try to perform detections from live audio using this script. However, I remember not being able to make it work flawlessly. Could you run this script and see if it solves your issue?

For detection from live audio, you have to make predictions over small chunks and that will introduce some delay (probably in the order of milliseconds). My code does this naively by maintaining a running average of the predictions over the small chunks. But, a more diligent way to do this would be to probably use an LSTM.

Thanks,
Sheel

[kubs0ne]

Thank you for your response. Actually I can't run this script because of errors that are caused by incompatibilities between several python packages. I also couldn't run it on python 3.

[sheelabhadra]

That's unfortunate and understandable since the code has been written in Python 2. You could try creating a virtual environment and install the dependencies (cf. requirements.txt) and see if that solves the problem.