Ability to automatically update add-ons

[nvaccessAuto]

Original report

Reported by nvdakor on 2013-05-08 07:08
Hi,
With the launch of the official NvDA add-ons page at addons.nvda-project.org, it became easier to search add-ons from one central location. Although this helps with downloading add-ons for the first time, a user who is running an add-on may find that he or she is running an older version of an add-on. So just like automatic NVDA update feature, there could be a feature where newer versions of add-ons would be downloaded automatically and prompt the user to install it without having to visit the addons site. Thanks.

Remaining work

update channel selection

• Ability to decide the default update channel for all add-ons, and the preferred update channel on a per add-on basis.
• Per-addon update channel selection is done from within the add-on store update tab using a submenu of the actions menu
  • "update channel" submenu items: Default, same channel, any, do not update, stable, beta&dev, beta, dev
• Anything but same, which is the current default, should allow auto updating from external add-ons

Add-on store panel settings

• Combo-box to change the default update channel for all add-ons.
  • Combo-box items: same (default), any, do not update, stable, beta&dev, beta, dev
• Checkbox to include incompatible add-ons when checking for update
• Convert the update system / combobox so users can decide between:
  • notify on updates (default)
  • fully automatic update
  • disabled

[nvaccessAuto]

Comment 1 by blindbhavya on 2014-09-07 08:27
This proposal would be very useful if implemented.
Could you (the devs James Sir) share your thoughts on this?

[nvaccessAuto]

Comment 3 by bhavyashah on 2015-01-01 07:44
Hi,
Please add #4762 to 'blocking'.

[nvaccessAuto]

Comment 5 by fastfinge on 2015-02-19 16:17
As far as I know, we haven't yet had security issues with an addon. But I'm wondering what the security implications of people running out-of-date addons are? Especially as NVDA needs administrator access. This is another reason why, personally, I think having the ability to check for addon updates from within NVDA is kind of important. Even if the addon itself could specify a URL to check for updates, rather than forcing everything through the centralized NVDA addon website, that might be enough to get people updating old addons when, not if, a serious security hole in an addon is discovered. In it's simplest form, it could be a file with a version number, download URL, and file hash. I don't think addons are currently signed via any kind of private key system at the moment, and I'm not sure they urgently need to be. Maybe this isn't a big deal, though, if addons are sandboxed in some way I don't know about?

[nvaccessAuto]

Comment 6 by jteh on 2015-02-20 01:17
First and most important, NVDA does not require administrator access. You should absolutely not be running NVDA with administrator privileges. You also should not disable UAC.

For what it's worth, no one is disputing that add-on updates would be nice. It's just a matter of someone having the time to do it, including integration with the add-ons repository.

[nvaccessAuto]

Comment 7 by nvdakor on 2015-04-26 05:54
Hi,
After some investigation, it appears it might be doable. The biggest challenge is the mechanism to integrate with the add-ons repository. Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class); the issue is how to differentiate between current and the newer add-on (hashing may work).

The big picture could be the following:

1. Add an update button in the add-ons manager if the add-on manifest specifies an update URL. This could be either the repurposed URL field from the current manifest format, or it could be a new field just for this purpose.
2. When the update button is clicked, a routine similar to update check routine from NVDA core would run, attempting to compare hashes and version and so on, which will result in the update UI similar to that of NVDA core.
3. Once the update is downloaded, the add-on installation UI should come up, asking users if they wish to install the new add-on.

Issues to be resolved:

• How can we calculate the first hash for our add-ons?
• How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server. We also need to hear from add-on authors so we can prepare our add-ons to take advantage of this idea in the future.
Thanks.

[nvaccessAuto]

Comment 8 by jteh (in reply to comment 7) on 2015-04-27 03:50
Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

the issue is how to differentiate between current and the newer add-on (hashing may work).

Keep it simple. Just compare the old version with the new version. If it's different, consider it an update. This is all we do for NVDA updates. That doesn't account for someone running a versio that is newer than is available on the site, but if that happens, the user clearly is a developer and/or is working with one.

1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

• How can we calculate the first hash for our add-ons?

See above. We don't need hashes for version identification. We do need hashes for verifying file integrity, but those can be generated automatically.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

• How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

We could do something similar to the current NVDA update check server script. The bigger issue is actually having a database on the server of add-ons and versions. Currently, as I understand it, version information is only listed in the Markdown for the website, which is not something we can reliably use.

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server.

The entire translation and add-ons system is now all running on the NV Access server. Mesar is pretty busy with other things these days and I haven't seen much from him lately, so I'm now handling everything related to this (unless Mesar wants to step in). I have a pretty good understanding of some parts of the system, though am still learning my way around others.

[nvaccessAuto]

Comment 9 by nvdakor (in reply to comment 8) on 2015-04-27 03:59
Replying to jteh:

Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

If you've started on this, then I think we should wait until #3504 is in place before returning to this ticket. Besides, update downloader and installer GUI has issues - for instance, the GUI for update downloader doesn't close automatically once an update has been downloaded and multiple update progress GUI may appear after a failed update is attempted.

1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

Understood.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

Sure. I'd say we the authors need at least three months time to adjust to the new routines once this ticket has been implemented; whatever major version of the add-on we were working on, that version or the version after that can implement needed changes, similar to what happened last fall when we started implementing add-on help functionality.
Thanks.

[nvaccessAuto]

Comment 10 by nvdakor on 2015-10-02 01:40
Hi,
Coming back to this ticket...
Looking at urllib and NvDA Core's update routines ghave me an idea as to how to implement client side, but for version 1, it'll require user intervention.
The proposal is as follows:

1. Add-ons should specify the download URL (perhaps a new key). This can be done easily, as many add-ons come with readme files written in Markdown that specifies URL's for stable and development versions.
2. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on. Perhaps a call to locate the add-on installer size should be specified for dev versions.
3. NVDA will open the URL via urllib.urlopen, parse the URL and look at content length header. Thankfully, because of the add-on file naming convention in use, it is easy to parse the address looking for the version information (a couple calls to strip function).
4. If the version on the server is newer than the installed add-on, assume that a newer version is available and present the update dialog (borrow NVDA Core's update presentation dialog, rewording parts of it to refer to add-ons).
5. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.
6. For steps 4 or 5, if an update is downloaded, save the content length for future update lookup.
7. Until Split out downloader used by updateCheck so it can be used elsewhere #3504 is implemented, NVDA will call os.startfile.
   I'd be happy to write a proof of concept add-on that demonstrates this algorithm.
   Thanks.

[nvaccessAuto]

Comment 11 by jteh (in reply to comment 10) on 2015-10-02 08:29
Replying to nvdakor:

1. Add-ons should specify the download URL (perhaps a new key).

I'm still thinking we should restrict this to the official add-on repository. Also, there are actually two URLs required in your proposed system: the URL for the manifest and the URL for the add-on itself.

1. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on.

As noted in comment:8, I think this is better done with a server script similar to the current update check server. This centralises things and allows for tracking of statistics (though that's of course not required for the first round).

1. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.

This is going to be fragile. Honestly, I think dev versions should just increment the version number (like NVDA snapshots do) if users really are supposed to update.

Thanks.

[josephsl]

Hi all,

@derekriemer and I had a talk about how to implement the server side (client side can be done easily). @nvdaes said on the add-ons mailing list that a file that lists add-on keys and versions would be sufficient, but Derek believes a database would be much better (I agree). The questions are what (database), when (if someone has time), and how (procedure).

CC @jcsteh

[derekriemer]

Hi,
I can open another ticket for this, I'd like an opinion from @jcsteh on this though.
I think when we do this, signing all community add-ons, and only allowing installation of community add-ons until a box is unchecked authorizing that unreviewed add-ons may be installed.
This has these advantages.

1. Add-ons can check whether the new bundle is properly signed.
2. When NVDA Boots, it checks the signature of all add-ons if the allow untrusted add-ons option is unchecked. It refuses to load any untrusted add-ons.
3. When a user tries to install an add-on, if the add-on is not signed, it tells them this is an untrusted andd-on, and prompts them to change their settings if they really want untrusted add-ons.
4. When updates occur, the add-on bundle must be signed before it will install.

[derekriemer]

Also, a post request for this would be desired, with all the add-ons being specified. I.E.
{
"rf": {
"version" : "2.2.2",
"beta" : false,
"hash" : "987987987987987987987987"
},
"ocr" : {
"version" : "2.1",
"beta" : false,
"hash" : "d53929098d550098409834409824..."
},
...
}
with json being returned of the form
{
"rf" : {
"version" : 2.5.0",
"beta" : false,
...
}
}

[rkingett]

Could all the add on's be hosted on Git Hub and then someone uses the Git Hub server to keep the costs down? & would that be an easier way to have NVDA check for updates?

[LeonarddeR]

This seemed to be quite an interesting ticket to read. Here are my thoughts.

1. Although I can see why @jcsteh proposes add-ons can only be updated from a central server, this might cause problems for paid add-ons (such as speech synths) which are not in the community repository since they are not or not entirely open source. Even for non-open source add-ons, I think there should be a pathway for automatic updates. Two alternatives:
   A. Only allow third party unregistered add-ons to be updated from a https source with verified SSL certificate
   B. Cumulatively, Allow add-ons from other sources and their corresponding url's to be registered on the centralised add-on system.

[jcsteh]

I agree we want paid add-ons to get automatic updates. I imagined we would have them registered with our central repository, even though you probably wouldn't be able to actually purchase them from there.

[feerrenrut]

P3. This seems like it would take a large amount of time to implement, and is not currently stopping primary use cases. It would however be very convenient.

@jcsteh Perhaps the priority should be higher if this is considered of strategic importance to NVDA?

[jcsteh]

I think this is something we really do want, but there is just so much other higher priority work that I can't justify making it p2 right now.

[rkingett]

A lot of people have no idea where to get add on updates, though, or
even that an add on has updated, so this would seriously be more than
just a convenience

On 11/13/2016 11:29 PM, Reef Turner wrote:

P3. This seems like it would take a large amount of time to implement,
and is not currently stopping primary use cases. It would however be
very convenient.

@jcsteh https://github.com/jcsteh Perhaps the priority should be
higher if this is considered of strategic importance to NVDA?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#3208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMj7zJ0QpOnkrL1fZnuKML9ZGP0oEp5mks5q9_HAgaJpZM4JXkpI.

[josephsl]

Hi,

After doing some more research and talking with @derekriemer, it appears the best possible server side implementation would be that of a table that includes info such as add-on name, description, version, download link and hash and so on, with another table reserved strictly for add-on updates. When the client looks up add-on updates, the client should specify add-on update channel, then the server should return update info such as version, download link and hash.

For example, suppose we use the URL of the form: https://addons.nvaccess.org/updateCheck?name=addon_channel

For instance, Windows 10 App Essentials dev version could say:
https://addons.nvaccess.org/updateCheck?name=wintenApps-dev

The server should return nothing if the version is up to date or a response similar to how NVDA Core checks for new versions (version, download link, hash).

For add-ons, a new manifest key named "addon_updateChannel" or something similar could be added, with the default value being "None" (no info). However, before the add-on updates database says that the specified add-on does not exist, the server would assume that one wants to update to the latest stable branch. Ideally, a new add-on that does include update name should be used.

Additional comments are appreciated. Thanks.

[derekriemer]

Note from @jcsteh: Don't put description in that table, it's problematic for l10n. We'll need another table for that. We need to make the constraint unique across name-channel pairs.

…

On 1/15/2017 7:19 PM, Joseph Lee wrote: Hi, After doing some more research and talking with @derekriemer <https://github.com/DerekRiemer>, it appears the best possible server side implementation would be that of a table that includes info such as add-on name, description, version, download link and hash and so on, with another table reserved strictly for add-on updates. When the client looks up add-on updates, the client should specify add-on update channel, then the server should return update info such as version, download link and hash. For example, suppose we use the URL of the form: https://addons.nvaccess.org/updateCheck?name=addon_channel For instance, Windows 10 App Essentials dev version could say: https://addons.nvaccess.org/updateCheck?name=wintenApps-dev The server should return nothing if the version is up to date or a response similar to how NVDA Core checks for new versions (version, download link, hash). For add-ons, a new manifest key named "addon_updateChannel" or something similar could be added, with the default value being "None" (no info). However, before the add-on updates database says that the specified add-on does not exist, the server would assume that one wants to update to the latest stable branch. Ideally, a new add-on that does include update name should be used. Additional comments are appreciated. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3208 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFGiveFSOhB9tUP-F64Ygiq614BfXBuwks5rStPOgaJpZM4JXkpI>.

-- ------------------------------------------------------------------------ Derek Riemer * Department of computer science, third year undergraduate student. * Proud user of the NVDA screen reader. * Open source enthusiast. * Member of Bridge Cu * Avid skiier. Websites: Honors portfolio <http://derekriemer.com> Awesome little hand built weather app! <http://django.derekriemer.com/weather/> email me at [email protected] <mailto:[email protected]> Phone: (303) 906-2194

[jcsteh]

Two tables are being suggested here (one for the version info and one for updates), but I think you only need one which is used for both. The unique constraint ensures there's only ever one "current" version for each channel. Even though the channel names can be arbitrary, I suggest the site restrict it to well known names so that we know which to use by default, etc.

[Adriani90]

Actually now we have comfort enough to see whether an Addon has updates or not:

• Incompatibility is triggered if the add-on has not been updatet for a longer time
• Add-on updates are shown on NVDA startup automatically if the user vonfigured this in NVDA add-on store settings
• Add-on updates can be performed through the add-on store.

@CyrilleB79 do you think there is anything else we need in this regard? Or should we close this issue?

[josephsl]

Hi,

I advise keeping this issue open - the crux of the issue (updating add-ons without user intervention as demonstrated by Add-on Updater) is still on the table.

Thanks.

[Adriani90]

Does that behavior not have negative security implications if the add-on author changed something, the add-on becomes insecure, and it is updated automatically without users confirmation? I would expect users should always confirm updates of add-ons as they do for NVDA itself. cc: @gerald-hartig what do you think?

[jscholes]

The initial description of this issue explicitly mentions users being prompted to install add-ons, and doesn't say anything about them being applied without any user intervention. I'd suggest that if someone has a strong case for such a feature, they make it in a separate issue.

[josephsl]

Hi,

Even if an add-on is downloaded and installed automatically, the restart dialog should pop up to mitigate against security implications of automated add-on download and installation. However, as the originator of this issue (raised more than ten years ago), I think the point about confirmation makes sense as a safeguard - the question becomes at which point should the user be prompted to restart NVDA. The issue also touches psychology of human-software interaction - how users may not realize security implications when they respond "yes" when asked to restart the screen reader without giving it some thought.

Based on the latest points raised (as of December 2024), I think it would be best to revisit this issue in a more concrete way a.k.a. creating a new issue with justifications. Even though I think I should close the issue (as the proposal writer), as NV Access marked this for 2025.1 milestone, I think we should hear from NV Access folks on their thoughts.

Thanks.

[CyrilleB79]

Yes, let's hear from NV Access on this topic.

moreover, even without installing add-ons automatically without any user action, there is still a step that can be done to automate add-ons installation before full install:

We can optionally allow NVDA to download add-on updates without any user action. Once the add-ons are downloaded and ready to be installed, NVDA can prompt the user for a confirmation to install the add-ons. For small add-ons and a fast connection, it does not change so much with respect to today's UX. But for big add-ons and/or slow connection, this makes the process smoother.

[Adriani90]

@CyrilleB79 wrote:

We can optionally allow NVDA to download add-on updates without any user action. Once the add-ons are downloaded and ready to be installed, NVDA can prompt the user for a confirmation to install the add-ons. For small add-ons and a fast connection, it does not change so much with respect to today's UX. But for big add-ons and/or slow connection, this makes the process smoother.

I think before introducing this behavior to add-ons, NVDA should provide this for its own installer first.
There would have to be a kind of window that at least shows download progress, or a Windows toast notification showing that something happened in the background.
This is what happens for Windows updates at least.
I think this remaining part of the issue is closely related to #2257.

[Adriani90]

Closing in favor of #17541, and #17162 which focus on remaining parts of the issue. An user interaction is definitely needed in this case.

[seanbudd]

@Adriani90 there's still significant work remaining for this to be completed

[Adriani90]

@seanbudd in terms of transparency, what is still to be done apart from the issues I referenced above? The milestone has been changed several times and there is an official position from NV Access is needed to continue this discussion constructively. Cc:

[gerald-hartig]

@Adriani90 Please refrain from closing issues that are on milestones. In this case you asked for our feedback and then closed the issue before we responded. Was there any particular urgency in needing to close this issue? We appreciate the hard work of our community on managing these issues, but we also need to keep control of the product vision and release process - things that occur through github issues.

[seanbudd]

I've outlined the remaining work in the issue now

[Adriani90]

The original issue as it was written before, didn‘t contain the request for fully automated updates without user interaction, and this is perceived anyway as vulnerability as per discussion above. The part about less intrusive addon updates is covered in the referenced issues above. I thought closing this old issue which is partially fixed, in favor of the newer requests might give a better overview of what users expect. Imo the remaining work in the issue description is better suited in a discussion, but yeah feel free to let this open. I just wanted to make things easier for a better overview.
Note that the part about customizing update channels is covered in #10679

[seanbudd]

More broadly we don't consider an issue titled as "ability to automatically update add-ons" complete until the remaining work outlined in the issue is complete.

[Adriani90]

See also #15072.

[seanbudd]

@Adriani90

Note that the part about customizing update channels is covered in #10679
See also #15072.

this appears to be for updating NVDA itself, not updating add-ons