[RFC]: add support for persisting REPL configuration across sessions

[Snehil-Shah]

Description

This RFC proposes adding support for REPL configuration files to persist user preferences (eg. settings) across each session.

Questions

• What to persist? For starters, we can persist settings, themes, and most of the options that the REPL receives during instantiation like inputPrompt, welcomeMessage, padding etc. I think we should avoid saving options like load and others which are something that is typically used per session.

• If we end up NOT saving anything else than what already can be passed as REPL options, does it make sense to have commands for loading custom configuration files? As loading a configuration or just loading the REPL options is pretty much the same process. In that case, just having a single configuration that remembers user preferences is more than enough, as the user can have scripts with specific options for different use-cases/profiles.

Other

Saving command history is an existing TODO, once we support that maybe we can also auto-persist them. I believe IPython does this.

Checklist

• I have read and understood the Code of Conduct.
• Searched for existing issues and pull requests.
• The issue name begins with RFC:.

[kgryte]

Re: load, save, log. I think these should be capable of persistence. Going back to your profile idea, load can make sense as it would allow, e.g., loading various third-party packages on REPL startup. This might change depending on the "profile". Similarly, for different profiles, I may want history saved to different locations. Ditto for logging.

The only REPL option which doesn't seem sensible for persistence is isTTY.

Re: commands for loading and saving configuration files. Yes, this makes sense. We should add these.

[Snehil-Shah]

But what about the default persistence? Does it make sense persisting load, save, log by default? In general, I would assume people load a file to maybe experiment and doesn't necessarily mean they want it to load it everytime they use the REPL..?

[kgryte]

I am not sure it makes sense to persist settings by default. Instead, I'm thinking a user should explicitly opt-in to persisting the settings for a specific REPL session. And whenever settings are updated, they shouldn't be persisted until a user explicitly asks them to be persisted. Otherwise, any setting experimentation ends up overriding previously saved settings, which may not be desirable.

We could, I suppose, persist settings by default on exit, but it is not clear to me that they should be auto-loaded the next time a user starts a new REPL session. The use case I am imagining is if a user's computer crashes and they want to restart the REPL with the same settings as when the crash happened. This is a "restore" use case.

For the restore use case, we could persist settings just before exit to a file named, e.g., last_session_settings.json. This settings file would not be auto-loaded, but could be explicitly loaded by a user to perform a manual restore.

For auto-loading, a user could explicitly persist settings to a file named startup.json (see also startup.m files for MATLAB). A file named startup.* would always be auto-loaded on startup, unless a user explicitly points to a different configuration file.

To load a specific configuration file on REPL instantiation, a user would be required to pass the configuration file to the REPL constructor via a new REPL option (e.g., config).

[Snehil-Shah]

Behaviors from other REPLs:

• Node & Python: Stores command history in files, namely <home_dir>/.node_repl_history and <home_dir>/.python_history. These are just the commands separated by newline. I don't think they have any mechanism for any sort of configuration or persistence.

• Julia: It has a folder <home_dir>/.julia which I presume is more general purpose than specific to the REPL. Command history is stored in <home_dir>/.julia/logs/repl_history.jl. I believe they also have a startup.jl file where you can run some functions upon REPL startup. Example:

function customize_colors(repl)
    repl.prompt_color = Base.text_colors[:cyan]
end

atreplinit(customize_colors)

This is how you can customize the theme using REPL.prototype methods.

• IPython: Similar to julia, has a folder <home_dir>/.ipython, which contains folders for each profile, the default folder is named profile_default. Command history is stored in an SQLite file. Each profile also has a startup directory, where .py and .ipy (normal py scripts with REPL specific %magic commands) can be stored. These are executed in a lexicographic order upon startup.
  <ipython_profile>/ipython_config.py stores configuration. Example:

c.InteractiveShell.autoindent = True
c.InteractiveShell.colors = 'LightBG'
c.InteractiveShell.confirm_exit = False
c.InteractiveShell.deep_reload = True
c.InteractiveShell.editor = 'nano'
c.InteractiveShell.xmode = 'Context'

c.PromptManager.in_template  = 'In [\#]: '
c.PromptManager.in2_template = '   .\D.: '
c.PromptManager.out_template = 'Out[\#]: '
c.PromptManager.justify = True

c.PrefilterManager.multi_line_specials = True

There is no automatic persistence of changes made in the REPL (like themes). ie, if I change the theme, I would have to change it again the next visit (unless I make a profile configuration).
IPython has a environment variable to control the .ipython directory, and can also be controlled through a command-line option: --ipython-dir=<path>. I don't think there is any mechanism for directory walking to resolve the configuration, all their profiles are hosted in the single .ipython directory.

I think we should stick to the <home_dir>/.stdlib folder to house our default configuration following conventions (not <configdir>)

[Snehil-Shah]

Also, I was wondering if we could extend the reset command to also clear the default config?

[kgryte]

Thanks for the R&D.

Re: path for storing default configuration. I don't think I agree. Personally, I am always a little annoyed when libraries/apps start cluttering my home directory. Not clear to me why we'd want to not use the system configuration directory. Storing configuration is what it's for.

Re: extending reset. Not clear to me why we'd want to clear the default config. Typically, if you run reset, the point is to simply clear history and the command queue, not revert back to "factory settings". We could introduce a separate command (e.g., hardReset) for performing such a reset, but, again, not clear to me the need.

[Snehil-Shah]

Yeah, I am curious why so many services don't use the config dir.
hardReset sounds good in the case when say the user wants back a fresh REPL? Maybe they changed stuff like inputPrompt and certain settings, and now they want to clear and go back to the defaults?

[kgryte]

Update: it looks like FHS does allow for storing files in a user's home directory (ref), but I don't believe this applies on Windows. We could, however, allow a user to specify the configuration directory, overriding default resolution.

Regardless, I think, for walking the file system to find a configuration file, we should begin with local directories and walk until reaching the root directory. Once at the root directory, check for a .stdlib folder. If present, use configuration files stored in that directory. And finally, if not present, check the system configuration directory.

For storing the default and last session configuration files, I think using the resolved system configuration directory should be fine; no need to clutter a user's home directory by default.

[kgryte]

Yeah, hardReset is fine with me.

[kgryte]

Update: seems like a user's home directory is commonly used for two reasons: (1) easy to find and (2) allows modification without needing elevated permissions. The compromise is using ~/.config/stdlib (on MacOS and Linux), which is (a) in a user's home directory and (b) minimizes clutter. On Windows, we rely on environment variables for resolving a configuration directory (with the possibility of not being able to resolve any configuration directory), and I don't think following Linux-based conventions makes sense. In which case, we'd need some sort of fallback.

[kgryte]

May be worthwhile seeing how other REPLs, such as Julia's, handle the Windows case.

[Snehil-Shah]

I was running Julia on windows when writing the above observations..

[kgryte]

@Snehil-Shah Including the configuration file path? I'm a little surprised that APPDATA would not be used.

[Snehil-Shah]

Yes, startup file also in the <home_dir>/.julia folder.

[kgryte]

I believe this is an artifact of the Windows Linux Subsystem. Reading through SO, it seems one possible place this corresponds to is APPDATA.

[Snehil-Shah]

I believe this is an artifact of the Windows Linux Subsystem. Reading through SO, it seems one possible place this corresponds to is APPDATA.

I am sorry, I don't think I follow. Can you elaborate on this..?

[kgryte]

What's the value of <homedir> for you on Windows?

[Snehil-Shah]

The directory where .julia, .ipython resides.

And on windows subsystem for linux:

which similarly also has it's own configuration directories:

[kgryte]

Interesting. Okay, it seems my understanding of Windows path conventions are outdated. Given that you've demonstrated that there's uniformity across Linux, Mac, and Windows, I'll stand down. Defaulting to a ~/.config/stdlib folder in the user's home directory seems fine. We can always change in the future should things evolve or users complain.

[Snehil-Shah]

To summarize, starting from pwd, walking to the root directory, then checking ~/.stdlib folder, and finally ~/.config/stdlib right? And we automatically make the last one?

[kgryte]

In what circumstance would ~/.stdlib be used? If we control where the default is saved, it should always be ~/.config/stdlib, correct?

[Snehil-Shah]

Yeah, maybe if the user adds their own configuration in ~/.stdlib? But yeah, in general by default we should just be using ~/.config/stdlib.

[kgryte]

Typically, users never directly add files to root dot directories. That is managed by the application. I'd think we should just have a canonical location.

[Snehil-Shah]

Makes sense. Let's stick with ~/.config/stdlib for the default config.