Update tool_destinations.yaml #196
Conversation
| @@ -542,8 +542,8 @@ featurecounts: {cores: 8, mem: 18} | |||
| feebayes: {cores: 10, mem: 12} | |||
| flashlfq: | |||
| env: | |||
| MONO_GC_PARAMS: max-heap-size=90g | |||
| mem: 90 | |||
| MONO_GC_PARAMS: max-heap-size=64g | |||
There was a problem hiding this comment.
that is strange ... the scheduler would kill the job if it exceeds 50GB (in theory, we have not instructed our scheduler to kill jobs), but Mono would try to reserve 64GB always, no?
There was a problem hiding this comment.
I don't really know, this is outside my field I am afraid. This link suggets that lowering the memory of the application would help (mono/mono#10415). Now I am just trying different settings to see if I can make it work...
There was a problem hiding this comment.
@magnusarntzen but the same tool is working on a smaller file, correct?
Are you using multiple files? Can you use your current and make it smaller.
I'm just trying to find out if this tools needs way more memory than your 90GB? You can also look under the 'i' symbol in your crashed dataset. There is some information about used memory.
There was a problem hiding this comment.
The tool works fine for a smaller dataset of about 250.000 PSMs. This one has 410.000 PSMs and we expect even more in the future.
FlashLFQ uses one input file and there is no way to split this up as it would skew the normalizations across runs.
Here is the output from the 'i'-page regarding memory:
Was OOM Killer active? | No
OOM Control enabled | No
CPU Time | 1.91 hours and 54.36 minutes
Max memory usage (MEM) | 29.1 GB
Max memory usage (MEM+SWP) | 29.1 GB
Memory limit on cgroup (MEM+SWP) | 8.0 EB
Memory limit on cgroup (MEM) | 8.0 EB
Memory softlimit on cgroup | 8.0 EB
Failed to allocate memory count | 0
So it seems the tool does not use the 90Gb available.
This could be a memory mapping issue. Others have reported problems with some memory-intense programs on Mono. Perhaps increase vm.max_map_count to 1000000?
There was a problem hiding this comment.
Mh, seems we could set this on our compute servers: https://www.suse.com/support/kb/doc/?id=000016692
I'm just confused why we need to do this, has no one ever used this tool before on Linux with big files.
There was a problem hiding this comment.
I have never tried myself nor can I find any info about this on the FlashLFQ git page. What I could do is to download the input files for FlashLFQ and try 1) run it natively on a Windows PC and 2) run it on our local linux HPC system with mono.
New try to fix FlashLFQ/mono out of memory issue.
Mono heap size should be larger than FlashLFQ's usage.