Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[vim] massive file descriptors leak #4293

Open
alxn1 opened this issue Mar 8, 2025 · 4 comments
Open

[vim] massive file descriptors leak #4293

alxn1 opened this issue Mar 8, 2025 · 4 comments

Comments

@alxn1
Copy link

alxn1 commented Mar 8, 2025 •
edited
Loading

I'm not sure, but looks like YouCompleteMe do not close sockets, and after some time vim lose ability to open files due to number of file descriptors limit per process. :)

As i understand, this problem is caused by 2 places:

  1. this one: in this place YouCompleteMe create request wrapped into future but do not do anything with it and socket inside is never closed. - this is not right, and in this place python really close socket, but little bit later (when GC is come)

  2. and second one: in this place request_id is removed from internal plugin dictionary, but python side is not know about this, and another request wrapped into future is lost for forever, and this is real memory and socket leak place :)

In addition in this line vim.eval( "request_id" ) should be replaced with int( vim.eval( "request_id" ) ) otherwise request_id will be passed as string into python (as i understand), and request will be not removed from internal ycm dictionary.

Vim and neovim handle plugins with different ways, and looks like for neovim this problem is not fatal. But original vim after some time reaches number of opened fd limit and stop working; only relaunch is help, but sometimes it's annoying :(

ps -lA | grep vim
0 S  1000   84616     650  5  80   0 - 583175 core_s pts/2   00:00:04 vim

lsof -p 84616
COMMAND   PID  USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
vim     84616 alxn1  cwd    DIR   8,32     4096 120713 /home/alxn1/.vim/bundle/YouCompleteMe
vim     84616 alxn1  rtd    DIR   8,32     4096      2 /
vim     84616 alxn1  txt    REG   8,32  4126400   2163 /usr/bin/vim.basic
vim     84616 alxn1  mem    REG   8,32    23816  10825 /usr/lib/python3.12/lib-dynload/_queue.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32   696512  57328 /usr/lib/x86_64-linux-gnu/libssl.so.3
vim     84616 alxn1  mem    REG   8,32   225488  10528 /usr/lib/python3.12/lib-dynload/_ssl.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32  5305304  57327 /usr/lib/x86_64-linux-gnu/libcrypto.so.3
vim     84616 alxn1  mem    REG   8,32    64368  10527 /usr/lib/python3.12/lib-dynload/_hashlib.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32   202904  12905 /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.5
vim     84616 alxn1  mem    REG   8,32    49256  10756 /usr/lib/python3.12/lib-dynload/_lzma.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32    78944  12702 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4
vim     84616 alxn1  mem    REG   8,32    32112  10392 /usr/lib/python3.12/lib-dynload/_bz2.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32    48952  10754 /usr/lib/python3.12/lib-dynload/_json.cpython-312-x86_64-linux-gnu.so
vim     84616 alxn1  mem    REG   8,32   360460  12199 /usr/lib/locale/C.utf8/LC_CTYPE
vim     84616 alxn1  mem    REG   8,32       50  12205 /usr/lib/locale/C.utf8/LC_NUMERIC
vim     84616 alxn1  mem    REG   8,32     3360  12208 /usr/lib/locale/C.utf8/LC_TIME
vim     84616 alxn1  mem    REG   8,32     1406  12198 /usr/lib/locale/C.utf8/LC_COLLATE
vim     84616 alxn1  mem    REG   8,32      270  12203 /usr/lib/locale/C.utf8/LC_MONETARY
vim     84616 alxn1  mem    REG   8,32       48  12202 /usr/lib/locale/C.utf8/LC_MESSAGES/SYS_LC_MESSAGES
vim     84616 alxn1  mem    REG   8,32       34  12206 /usr/lib/locale/C.utf8/LC_PAPER
vim     84616 alxn1  mem    REG   8,32       62  12204 /usr/lib/locale/C.utf8/LC_NAME
vim     84616 alxn1  mem    REG   8,32      127  12197 /usr/lib/locale/C.utf8/LC_ADDRESS
vim     84616 alxn1  mem    REG   8,32       47  12207 /usr/lib/locale/C.utf8/LC_TELEPHONE
vim     84616 alxn1  mem    REG   8,32   170240  12779 /usr/lib/x86_64-linux-gnu/libexpat.so.1.9.1
vim     84616 alxn1  mem    REG   8,32   113000  13112 /usr/lib/x86_64-linux-gnu/libz.so.1.3
vim     84616 alxn1  mem    REG   8,32   625344  38865 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.11.2
vim     84616 alxn1  mem    REG   8,32  2125328  50301 /usr/lib/x86_64-linux-gnu/libc.so.6
vim     84616 alxn1  mem    REG   8,32  9055112  57163 /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0
vim     84616 alxn1  mem    REG   8,32    26688  12833 /usr/lib/x86_64-linux-gnu/libgpm.so.2
vim     84616 alxn1  mem    REG   8,32    38984  12658 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2302
vim     84616 alxn1  mem    REG   8,32   355040  13014 /usr/lib/x86_64-linux-gnu/libsodium.so.23.3.0
vim     84616 alxn1  mem    REG   8,32   174472  17147 /usr/lib/x86_64-linux-gnu/libselinux.so.1
vim     84616 alxn1  mem    REG   8,32   208328  13038 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
vim     84616 alxn1  mem    REG   8,32   952616  50304 /usr/lib/x86_64-linux-gnu/libm.so.6
vim     84616 alxn1  mem    REG   8,32       23  12201 /usr/lib/locale/C.utf8/LC_MEASUREMENT
vim     84616 alxn1  mem    REG   8,32    27028  50290 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
vim     84616 alxn1  mem    REG   8,32      258  12200 /usr/lib/locale/C.utf8/LC_IDENTIFICATION
vim     84616 alxn1  mem    REG   8,32   236616  50298 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
vim     84616 alxn1    0u   CHR  136,2      0t0      5 /dev/pts/2
vim     84616 alxn1    1u   CHR  136,2      0t0      5 /dev/pts/2
vim     84616 alxn1    2u   CHR  136,2      0t0      5 /dev/pts/2
vim     84616 alxn1    3w   REG   8,32        0  47681 /tmp/ycm_f5uowmn3.log
vim     84616 alxn1    4rR  DIR   8,32     4096  11224 /tmp/vNv4FVJ
vim     84616 alxn1    5u  IPv4 296781      0t0    TCP localhost:58206->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1    6u  IPv4 306359      0t0    TCP localhost:34208->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1    7r  FIFO   0,12      0t0 299422 pipe
vim     84616 alxn1    8u  IPv4 305509      0t0    TCP localhost:35128->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1    9r  FIFO   0,12      0t0 299423 pipe
vim     84616 alxn1   10u  IPv4 299435      0t0    TCP localhost:58296->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   11u  IPv4 299434      0t0    TCP localhost:58258->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   12u  IPv4 306326      0t0    TCP localhost:35212->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   13u  IPv4 267851      0t0    TCP localhost:36068->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   14u  IPv4 299450      0t0    TCP localhost:40210->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   15u  IPv4 301494      0t0    TCP localhost:36128->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   16u  IPv4 301524      0t0    TCP localhost:36170->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   17u  IPv4 301445      0t0    TCP localhost:34298->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   18u  IPv4 296801      0t0    TCP localhost:35160->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   19u  IPv4 296834      0t0    TCP localhost:34338->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   20u  IPv4 307802      0t0    TCP localhost:36006->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   21u  IPv4 302776      0t0    TCP localhost:36060->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   22u  IPv4 267840      0t0    TCP localhost:34188->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   23u  IPv4 300271      0t0    TCP localhost:36106->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   24u  IPv4 305568      0t0    TCP localhost:36092->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   25u  IPv4 296876      0t0    TCP localhost:36150->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   26u  IPv4 305540      0t0    TCP localhost:34246->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   27u  IPv4 302793      0t0    TCP localhost:60926->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   28u  IPv4 294737      0t0    TCP localhost:60910->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   29u  IPv4 305547      0t0    TCP localhost:34322->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   30u  IPv4 306569      0t0    TCP localhost:60948->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   32u  IPv4 302764      0t0    TCP localhost:34356->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   33u  IPv4 305553      0t0    TCP localhost:36008->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   34u  IPv4 306501      0t0    TCP localhost:36034->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   35u  IPv4 302775      0t0    TCP localhost:36040->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   36u  IPv4 296841      0t0    TCP localhost:36070->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   37u  IPv4 294719      0t0    TCP localhost:36096->localhost:59079 (CLOSE_WAIT)
vim     84616 alxn1   38u  IPv4 296875      0t0    TCP localhost:36136->localhost:59079 (CLOSE_WAIT)

My vim version is:

vim --version
VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Jan 16 2025 20:13:18)
Included patches: 1-16, 647, 678, 697
Modified by [email protected]
Compiled by [email protected]
Huge version without GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tag_any_white
+arabic            +find_in_path      +mouse_xterm       -tcl
+autocmd           +float             +multi_byte        +termguicolors
+autochdir         +folding           +multi_lang        +terminal
-autoservername    -footer            -mzscheme          +terminfo
-balloon_eval      +fork()            +netbeans_intg     +termresponse
+balloon_eval_term +gettext           +num64             +textobjects
-browse            -hangul_input      +packages          +textprop
++builtin_terms    +iconv             +path_extra        +timers
+byte_offset       +insert_expand     -perl              +title
+channel           +ipv6              +persistent_undo   -toolbar
+cindent           +job               +popupwin          +user_commands
-clientserver      +jumplist          +postscript        +vartabs
-clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +vim9script
+cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           +python3           +virtualedit
+comments          +linebreak         +quickfix          +visual
+conceal           +lispindent        +reltime           +visualextra
+cryptv            +listcmds          +rightleft         +vreplace
+cscope            +localmap          -ruby              +wildignore
+cursorbind        -lua               +scrollbind        +wildmenu
+cursorshape       +menu              +signs             +windows
+dialog_con        +mksession         +smartindent       +writebackup
+diff              +modify_fname      +sodium            -X11
+digraphs          +mouse             -sound             +xattr
-dnd               -mouseshape        +spell             -xfontset
-ebcdic            +mouse_dec         +startuptime       -xim
+emacs_tags        +mouse_gpm         +statusline        -xpm
+eval              -mouse_jsbterm     -sun_workshop      -xsmp
+ex_extra          +mouse_netterm     +syntax            -xterm_clipboard
+extra_search      +mouse_sgr         +tag_binary        -xterm_save
-farsi             -mouse_sysmouse    -tag_old_static
   system vimrc file: "/etc/vim/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffile-prefix-map=/build/vim-EcmdP9/vim-9.1.0016=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fdebug-prefix-map=/build/vim-EcmdP9/vim-9.1.0016=/usr/src/vim-2:9.1.0016-1ubuntu7.6 -DSYS_VIMRC_FILE=\"/etc/vim/vimrc\" -DSYS_GVIMRC_FILE=\"/etc/vim/gvimrc\" -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -lselinux -lsodium -lacl -lattr -lgpm -L/usr/lib/python3.12/config-3.12-x86_64-linux-gnu -lpython3.12 -ldl -lm

And OS:

cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
@alxn1 alxn1 changed the title Massive file descriptors leak [vim] massive file descriptors leak Mar 8, 2025
@alxn1
Copy link
Author

alxn1 commented Mar 9, 2025 •
edited
Loading

No description provided.

@puremourning
Copy link
Member

Thanks for taking the time to look into this. Would you be willing to open a PR?

Would be helpful to get some repro steps if you have them too.

@alxn1
Copy link
Author

alxn1 commented Mar 10, 2025

About steps to reproduce - sockets always leaks, as i understand, this can be reproduced with default settings while you move cursor up/down/left/right in one file with source code. Count of opened sockets in CLOSE_WAIT state will growing and (if standartl vim is used) these sockets never closed withoud vim restart - lsof show it.

About PR - ok, i will try :)

@alxn1 alxn1 mentioned this issue Mar 10, 2025
Open
4 tasks
@alxn1
Copy link
Author

alxn1 commented Mar 10, 2025 •
edited
Loading

I did some additional investigation and found that I was wrong in point 1 of this issue description. Now all is done (PR is ready), and all sockets closing (with some delay, but it's normal, i think).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alxn1 @puremourning