Problem with overwriting variables from .env

[waterstark]

Behaviour

If I restart VS Code and modify the .env file, the new variable values from the .env file do not get applied to my FastAPI application.

Steps to reproduce problem:

1. Go to the vs code settings tab and make sure that the Python:Env File field has the value ${workspaceFolder}/.env
   
2. Create a .env file and add the variable MY_VAR=1.
3. Save the file.
4. Close VS Code by right-clicking the VS Code icon and selecting "Quit".
5. Open VS Code.
6. Change the variable MY_VAR in the .env file to MY_VAR=2.
7. Enter the command echo $MY_VAR in the terminal.

Description

I had to spend around two days troubleshooting an issue where VS Code automatically sets environment variables at terminal launch, causing non-obvious code behavior.

Considering that most of the audience are junior and middle-level developers, I believe many might face this problem. Is there a way to make this feature more explicit, such as prompting the user to enable or disable this setting? Or maybe we can make it opt-in instead of opt-out?

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-07-23 15:10:15.854 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-07-23 15:10:15.854 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-07-23 15:10:15.854 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-07-23 15:10:15.854 [info] Experiment 'pythonTestAdapter' is active
2024-07-23 15:10:15.854 [error] Reading directory to watch failed [Error: ENOENT: no such file or directory, scandir '/Users/mun/code/test/.pixi/envs'
	at Object.readdirSync (node:fs:1515:3)
	at t.readdirSync (node:electron/js2c/node_init:2:11289)
	at /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/out/client/extension.js:2:574598
	at /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/out/client/extension.js:2:574789
	at Array.map (<anonymous>)
	at d.initWatchers (/Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/out/client/extension.js:2:574507)
	at async d.ensureWatchersReady (/Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/out/client/extension.js:2:536827)] {
  errno: -2,
  syscall: 'scandir',
  code: 'ENOENT',
  path: '/Users/mun/code/test/.pixi/envs'
}
2024-07-23 15:10:15.854 [error] Dir "/Users/mun/code/test/.pixi/envs" is not watchable (directory does not exist)
2024-07-23 15:10:15.854 [info] Default formatter is set to charliermarsh.ruff for workspace /Users/mun/code/test
2024-07-23 15:10:15.854 [info] Python interpreter path: ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 15:10:15.860 [info] > pyenv which python
2024-07-23 15:10:15.860 [info] cwd: .
2024-07-23 15:10:16.393 [info] > pixi --version
2024-07-23 15:10:16.393 [error] [Error: spawn pixi ENOENT
	at ChildProcess._handle.onexit (node:internal/child_process:286:19)
	at onErrorNT (node:internal/child_process:484:16)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn pixi',
  path: 'pixi',
  spawnargs: [ '--version' ]
}
2024-07-23 15:10:16.393 [warning] could not find a pixi interpreter for the interpreter at /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 15:10:16.448 [info] > . ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 15:10:16.448 [info] shell: zsh
2024-07-23 15:10:16.603 [info] > /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 15:10:16.603 [info] shell: zsh
2024-07-23 15:10:16.680 [info] Setting environment variable MY_VAR in collection to 1 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:16.680 [info] Prepending environment variable PATH in collection with /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh:/Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:16.680 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:16.680 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:16.680 [info] Prepending environment variable PS1 in collection with test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-07-23 15:10:16.680 [info] Send text to terminal: /usr/bin/python3 /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariablesToFile.py /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh/envVars.txt
2024-07-23 15:10:17.363 [info] Starting Pylance language server.
2024-07-23 15:10:20.290 [warning] could not find a pixi interpreter for the interpreter at /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 15:10:20.294 [info] > . ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 15:10:20.294 [info] shell: zsh
2024-07-23 15:10:20.322 [info] Setting environment variable MY_VAR in collection to 2 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:20.322 [info] Prepending environment variable PATH in collection with /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh:/Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:20.322 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:20.322 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:10:20.322 [info] Prepending environment variable PS1 in collection with test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-07-23 15:14:44.889 [warning] could not find a pixi interpreter for the interpreter at /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 15:14:44.889 [warning] could not find a pixi interpreter for the interpreter at /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 15:14:44.905 [info] > . ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 15:14:44.905 [info] shell: zsh
2024-07-23 15:14:44.905 [info] > . ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 15:14:44.905 [info] shell: zsh
2024-07-23 15:14:44.934 [info] Setting environment variable VSCODE_L10N_BUNDLE_LOCATION in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.934 [info] Prepending environment variable PATH in collection with /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh:/Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.934 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.934 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.934 [info] Prepending environment variable PS1 in collection with test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-07-23 15:14:44.935 [info] Setting environment variable VSCODE_L10N_BUNDLE_LOCATION in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.935 [info] Prepending environment variable PATH in collection with /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh:/Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.935 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.935 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 15:14:44.935 [info] Prepending environment variable PS1 in collection with test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-07-23 15:14:45.129 [info] Discover tests for workspace name: undefined - uri: /Users/mun/Library/Application Support/Code/User/settings.json
2024-07-23 16:52:09.240 [warning] could not find a pixi interpreter for the interpreter at /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/python
2024-07-23 16:52:09.250 [info] > . ~/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/printEnvVariables.py
2024-07-23 16:52:09.250 [info] shell: zsh
2024-07-23 16:52:09.296 [info] Setting environment variable VSCODE_L10N_BUNDLE_LOCATION in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 16:52:09.296 [info] Prepending environment variable PATH in collection with /Users/mun/.vscode/extensions/ms-python.python-2024.10.0-darwin-arm64/python_files/deactivate/zsh:/Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 16:52:09.296 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/mun/Library/Caches/pypoetry/virtualenvs/test-3KCHDMEl-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 16:52:09.296 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-07-23 16:52:09.296 [info] Prepending environment variable PS1 in collection with test-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-07-23 17:09:14.521 [info] Discover tests for workspace name: test - uri: /Users/mun/code/test/config.py
2024-07-23 17:09:20.298 [info] Discover tests for workspace name: test - uri: /Users/mun/code/test/config.py

[anthonykim1]

Can you try adding

"python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"]

into your settings.json and see if you still experience this behavior?
Also make sure to control+shift+p (cmd+shit+p on mac) and type "Python: Clear Cache reload".

Perhaps first try with just "Python: Clear Cache reload" and see if your environment variables from .env gets updated, if that does not happen THEN try to opt out(by adding setting) and reload again.

[andylizf]

Is this related to #23280 ?

[anthonykim1]

@andylizf Are you talking about the feature request that was mentioned

updating environment variables as a part of .env file modification is a feature request. in #23280 ?

[andylizf]

@anthonykim1 Yeah. The issue seems closely related to what I observed in #23280, where environment variables set in the terminal do not update after modifying the .env file. Both issues highlight the persistent nature of environment variables in the terminal even after changes to the .env file or deactivating the Python extension.

This behavior can be confusing, especially since the documentation doesn't clearly outline this.

[anthonykim1]

I've created a debt item on this just now: #24034

It's bit tricky since there are no real official standard regarding .env files, but agreed yes we should have better documentation of what is going on. Let's track this: #24034

[anthonykim1]

Hi @waterstark Would https://code.visualstudio.com/docs/python/environments#_environment-variables help you better understand .env with Python extension? Curious to hear what you think. Thanks

[andylizf]

Thanks for your contribution! It seems that the documentation explains how .env files are used when the extension is active, but doesn't mention what happens after the extension is deactivated or when updates are made to the .env file during this state.

…

On Sep 17, 2024, 2:40 PM -0700, Anthony Kim ***@***.***>, wrote: Hi @waterstark Would https://code.visualstudio.com/docs/python/environments#_environment-variables help you better understand .env with Python extension? Curious to hear what you think. Thanks — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Laubeee]

I have the same issue. Setting the optOut didn't seem to help.

[NickCrews]

I just ran into this error with this flow:

1. set VAR=old_value in .env
2. set VAR=new_value in .env
3. pull my hair out for 30 minutes trying to figure out why my nextjs app, totally unrelated to python, was still using old_val
4. find this thread eventually
5. set the optOut
6. kill all open terminals, restart vscode
7. open a new bash shell, echo $VAR still shows old_val

This is especially frustrating because nextjs resolves env vars in priority of shell above .env files, so even if I set the new val in the .env file, nextjs still picks up the stale value in my shell that ms-python set, and and I seem to be able to find no way to get it to get ms-python to forget that old value.

Can you give us a workaround to get ms-python to forget about all env vars it has cached? Long term, updating the env vars when the .env file changes I would hope would be top priority, without this it is like if I bought an oven and after I turn it on the first time, I will only ever be able to use it at that same temperature.