Insufficient Memory for HD Decode (1920x1080)

[muman613]

Hello,

I have encoded a new HD stream using AOMenc from raw YUV data. I can use AOMdec to decode the stream so I verified that the stream is decode-able. But when I try to use the stream in the Analyzer I get errors related to memory allocation.

I have added many debug messages to the code so I can trace where the problems are originating from. Here is the debug output from the section where the memory runs out:

decoder.readFrame()
Using AOMedia Project AV1 Decoder v0.1.0-6172-g3a29fdc
Worker: setLayers
Worker: readFrame
worker readFrame()
Cannot enlarge memory arrays. Either (1) compile with  -s TOTAL_MEMORY=X  with X higher than the current value 134217728, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime but prevents some optimizations, (3) set Module.TOTAL_MEMORY to a higher value before the program runs, or (4) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 
Cannot enlarge memory arrays. Either (1) compile with  -s TOTAL_MEMORY=X  with X higher than the current value 134217728, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime but prevents some optimizations, (3) set Module.TOTAL_MEMORY to a higher value before the program runs, or (4) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 
Worker: readFrame
worker readFrame()
Failed to decode frame.: Corrupt frame detected
    Invalid frame size

I found a solution which requires rebuilding the 'inspect.js' component. I modified the 'build_inspection.sh" script which actually makes the JS component with Emscripten. What I did was double the -s TOTAL_MEMORY=268435456 and added the -s ALLOW_MEMORY_GROWTH=1.

When I do this then I am able to decode the HD 1920x1080 stream.

What we are really trying to do here is support the AVS2 decoder which supports the 4kx2k UHD resolution. But I am bumping up against memory limitations at that size.

I would be interested in knowing if anyone has succeeded in loading a UHD 4k image into the AOMAnalyzer.

Thank you,
Michael Uman
Sr. Software Engineer
Sigma Designs, Inc.

[tdaede]

In theory that amount of memory should be enough to decode a 1920x1080 stream, however it's possible something may have changed in the codec recently. I believe increasing TOTAL_MEMORY is preferable to setting ALLOW_MEMORY_GROWTH as the latter is slower, though I have not measured how much.

The analyzer has not been tested on a 4k stream in a long time, however there is no reason it should not work as long as the enscripten memory allocation is large enough (and a 64-bit browser is used)

[muman613]

Thank you for the reply tdaede...

I have determined that there is a limitation in the Emscripten/Electron/NodeJS environment which threatens our ability to load 4kx2k video using our AVS2 decoder component.

Apparently there is a hard limit of ~2GB when allocating the ArrayBuffer object used as the Emscripten HEAP object. What this means is that the maximum HEAP memory available to the decoder is a bit less than 2GB of memory.

I profiled our AVS2 reference decoder when decoding the 4k stream and discovered that it requires a peak memory heap of at least 5GB of memory. Our decoder fails to load when I build with TOTAL_MEMORY set to anything over 2130706432 bytes. The allocation fails and throws a 'RangeError: Unable to allocate buffer' exception.

I have created an Electron project which demonstrates this issue. If anyone would like to verify my findings please check out my github project @ https://github.com/muman613/electron-memory-test .

git clone https://github.com/muman613/electron-memory-test

This project opens a BrowserWindow presents a HTML which contains some instructions and a text entry field which accepts a memory size, a selector which allows specifying memory in Gb/Mb/Kb and a button which attempts the allocation. By default it attempts to allocate 2GB of memory. When the allocate button is pressed and the allocation fails an alert is displayed and status is shown in the HTML.

At this point I don't know if we will be able to support 4kx2K using our AVS2 decoder. Short of re-engineering the AVS2 decoder (not our code, provided as reference decoder)... I hope that someone has some advice on getting around this 2GB limitation.

Thank you,
Michael Uman

[tdaede]

Wow, that's a lot! For comparison, libaom uses ~300MB when decoding 10bit 4k. You may be able to use wasm output, which I believe has a limitation of 4GB (until wasm64 lands). I haven't enabled wasm by default on the analyzer yet, but it's a todo.

[muman613]

@tdaede I agree, we are trying to determine why AVS2 requires that much memory. I am starting an analysis of why 5Gb of memory is needed. I also did profile the AOM av1 decoder and verify your finding that it never needs more than around 300Mb...

Thank you