Deploy 1wallet to Skynet

[kwypchlo]

This pull request adds Deploy to Skynet github action integration for deploying 1wallet to Skynet decentralised cloud storage. It works automatically, detecting pushed changes to wallet related paths.

This pull request relies on github actions. In a case where github actions are not preferred solution, I can provide a manual drop in deploy script that does the same thing although it will require the maintainer to run it potentially on every release.

Example deploy: 04052te3b2jhcugo7e03rabhpjd3ps8n8dmfm8md885pa9u6aof1850

maintainer follow ups

1. github actions have to be enabled for the repository
2. maintainer should add SKYNET_WALLET_REGISTRY_SEED secret to github which will act as a deploy key
   • it has to be cryptographically secure - it will be used to sign all of your deploys to Skynet
   • we recommend either generating it yourself or using one of the random ones from an online generator ie. https://randomkeygen.com/
   • the key can be changed at any time but your resolver skylink (hash for latest deploy) will change so users will need to manually switch to new url
3. once this pull request is merged (steps 1 and 2 should be completed prior to that), github will be set to automatically run an action on change to any wallet related code that takes the latest master branch, builds wallet and deploys it to Skynet. For the first time only, maintainer will need to open the action logs and copy "Resolver Skylink Url" from "Deploy to Skynet" step once it's completed (see example output from one of our own repositories). This url will stay the same after each deploy so you can share it with your users, I usually recommend a small note in README.md.
   • a note here: github action will only run on changes to either code/client/** or scripts/hdnode.js so to trigger it for the first time I recommend doing a small change inside of any file in code/client/** for example add a new line somewhere or use this opportunity to update some npm dependencies; on that note, the yml action configuration file can be also adjusted to trigger on different events so let me know if that's something that you would be interested in adjusting

homescreen integration

Once wallet is successfully set to deploy to Skynet, it is highly recommended to also add support for Homescreen users (please read more at our homescreen webpage about the advantages it brings). It only consists of two steps:

1. update manifest.json by adding a skylink key "skylink": "<55_character_resolver_skylink_here>" - skylink is the 46 or 55 character part of the url (subdomain) from point 3 of previous section (example commit) - unfortunately 1wallet doesn't have manifest file yet so in any case it is recommended to add one according to the spec https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json and link it in index.html file so it can be discovered by browsers
2. add a badge to your readme so users can easily click to add wallet to homescreen (example commit) with markdown code (replace skylink with the resolver skylink from the manifest)

[![Add to Homescreen](https://img.shields.io/badge/Skynet-Add%20To%20Homescreen-00c65e?logo=skynet&labelColor=0d0d0d)](https://homescreen.hns.siasky.net/#/skylink/<skylink-here>)

---

Let me know if there are any questions or concerns you would like to have addressed! I hope the integration goes smooth and easy :)

[polymorpher]

Thanks. I reviewed most Skynet documentations and the video published at EthCC. There are several issues which until resolved, would prevent us making a working integration:

Main issues

1. Lacking the ability to persist data in browser under a fixed domain

With Skynet resolvers, apps can be accessed using a fixed domain name or URI (e.g. 1wallet.xyz). Assuming appropriate integrations with Skynet's DNS/HNS/ENS are completed, the domain name or URI are eventually resolved to a Skylink address pointing to an immutable version of the app hosted on Skynet storage network. The Skylink is translated to a standard URL depending on the portal the user chooses, which by default is the one operated by Siasky.net. The URL is in the form of a subdomain specific to the app. The browser retrieves content from the URL as run the app in the same manner as they do without any Skynet integration.

When the app is updated by the developer, a new immutable version is uploaded to Skynet and the resolver is updated to point to the Skylink of the new version. The new Skylink would result in a different URL under a different subdomain.

However, under the new subdomain, the user's browser would start with a fresh storage. It would not have any local data stored in previous versions. The browser's security policy would also prevent such access pattern due to the change in subdomain. The user experience would not be acceptable if users have to be asked to manually transfer data over. On the other hand, if we move all locally stored data to Skynet, it would involve a large amount of work and a high risk of not being able to achieve a pleasant user experience. Every wallet creates 64MB of data locally, and a user may have many wallets. This situation presents a serious question of who pays for the storage and access of the data, and whether the data can be retrieved efficiently and reliably on-demand. The data cannot be encrypted using Skynet's encrypted storage mechanism since it would tie the access to the storage (hence the wallet) to the user's Skynet private key / seed, which causes 1wallet to lose portability (cannot be easily restored on other devices using solely Google Authenticator export), and is counterproductive to 1wallet's design goal (getting rid of private keys). That means significant additional work needs to be implemented to encrypt the data using 1wallet's own mechanism that ties the decryption to Google Authenticator entry. Although this was already planned for IPFS, replicating the same mechanism on Skynet is non-trivial. See also #63.

2. Lacking ability to respond custom HTTP headers based on routes

App integrating with 1wallet may use an embedded iframe for authorizing actions using their 1wallet. See https://github.com/polymorpher/one-wallet/wiki/App-Integration. One strong reason for using iframe is that it allows the browser to retrieve locally stored data under the iframe's origin inside the iframe. In this case, the required data is the partial proofs stored locally which are necessary for completing actions requested by the app. See https://github.com/polymorpher/one-wallet/wiki/Web-Client:-Data-Stored-Locally for data stored locally. Per wiki, such actions can be connecting the wallet, calling another contract, making payment, or signing a message. An example of such integration is Harmony Safe (MultiSig) based on Gnosis Safe. See https://multisig.harmony.one/ . By default, browsers disallow contents from another origin to be shown inside an iframe, unless specific headers are present. See https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP . However, wallets should not indiscriminately allow such header to be present on every route, otherwise malware and scam sites may load the wallet as an iframe on their own site, pretend that they are the wallets and steal credentials, or manipulate the user (through altered UI or otherwise) to steal their assets. For that reason, the server hosting the wallet client applications need to respond appropriate Content-Security-Policy (CSP) headers based on pattern in (URL) routes.

It appears Skynet does not provide such customization ability to their apps. The protocol has not discussed any specification about headers. Therefore, it would be up to Skynet portal operators to implement such feature, namely advanced routing and header configuration similar to those in Nginx and Apache. It would prove to be a very challenging task since Skynet presumes the configuration of deployed app (at resolved Skylink) must be immutable, but such configurations should be made mutable and easily reconfigurable, since browsers and headers specifications constantly evolve and may change every 1-3 years.

3. Lacking HTTPS support

Skynet documentation explicitly stated so and recommends using "fake" HTTPS bridges such as Cloudflare as a solution. It is understandable that implementing HTTPS on Skynet would be very challenging because HTTPS requires a server to store the private key and sign messages, and Skynet does not have servers at all (or more precisely, servers possessing secret information and can provide custom response to requests using its secret information). As another alternative, Skynet advocates the method of verifying the retrieved content using its SDK, hashes or other client side operations.

As it stands, neither of the alternatives provide a satisfactory level of security, which is of paramount importance to wallets (and even more so for 1wallet). If the user's DNS is hijacked, none of the alternatives offered above would prevent the attacker from directing the user to an altered client and carefully craft an attack that may steal the users' assets.

4. The requirement of an assurance that "APIs and web requests from your app only depend on decentralized protocols that will remain accessible even with future front-end updates"

See https://docs.siasky.net/integrations/homescreen#integrating-with-homescreen . At present, 1wallet must use a non-decentralized Relayer node to initiate transactions and pay for gas. See https://github.com/polymorpher/one-wallet/tree/master/code#relayer . Although any node can be a relayer and the user needs not to trust relayer, this component still remains a non-decentralized element. Until account abstraction (harmony-one/bounties#35) is implemented to allow smart contracts paying for gas, the relayer will remain to be a necessary component in the whole architecture.

Non-critical non-decentralized components used in 1wallet are Binance price APIs, and cached SushiSwap pairs and tokens. They do not affect security and core functionality of the wallet. Moving these components to decentralized network would incur significant amount of work to achieve a similar level of user experience compared to how things are running right now.

5. Lack of auto-update on Homescreen

Aside from the issues above, the lack of auto-update feature from Skynet Homescreen is also concerning. Wallets frequently need patches for both critical issues and feature enhancement. Sometimes the patches are silently delivered multiple times per day. Requiring the user to manually "update" their app for every patch would be a major hassle.

Other issues

After all above issues are resolved, I will also need a custom release and deployment script. I do not use Github action since commits are very frequent. If Github actions were used, I will need to manually select release anyway and perhaps to configure separate branches for that purpose, which seem to be a lot of work without any apparent benefit.

[kwypchlo]

@polymorpher thank you for the in depth review, for some of the concerns I have immediate response but some are more complicated - I will talk to the team and get back to you

[polymorpher]

You are welcome. Let's reopen this when we can move forward

[kwypchlo]

Sorry it took this long to respond. I think I can address most of the concerns right now.

1. Lacking the ability to persist data in browser under a fixed domain

With Skynet resolvers, apps can be accessed using a fixed domain name or URI (e.g. 1wallet.xyz). Assuming appropriate integrations with Skynet's DNS/HNS/ENS are completed, the domain name or URI are eventually resolved to a Skylink address pointing to an immutable version of the app hosted on Skynet storage network. The Skylink is translated to a standard URL depending on the portal the user chooses, which by default is the one operated by Siasky.net. The URL is in the form of a subdomain specific to the app. The browser retrieves content from the URL as run the app in the same manner as they do without any Skynet integration.

When the app is updated by the developer, a new immutable version is uploaded to Skynet and the resolver is updated to point to the Skylink of the new version. The new Skylink would result in a different URL under a different subdomain.

However, under the new subdomain, the user's browser would start with a fresh storage. It would not have any local data stored in previous versions. The browser's security policy would also prevent such access pattern due to the change in subdomain. The user experience would not be acceptable if users have to be asked to manually transfer data over. On the other hand, if we move all locally stored data to Skynet, it would involve a large amount of work and a high risk of not being able to achieve a pleasant user experience. Every wallet creates 64MB of data locally, and a user may have many wallets. This situation presents a serious question of who pays for the storage and access of the data, and whether the data can be retrieved efficiently and reliably on-demand. The data cannot be encrypted using Skynet's encrypted storage mechanism since it would tie the access to the storage (hence the wallet) to the user's Skynet private key / seed, which causes 1wallet to lose portability (cannot be easily restored on other devices using solely Google Authenticator export), and is counterproductive to 1wallet's design goal (getting rid of private keys). That means significant additional work needs to be implemented to encrypt the data using 1wallet's own mechanism that ties the decryption to Google Authenticator entry. Although this was already planned for IPFS, replicating the same mechanism on Skynet is non-trivial. See also #63.

response: skynet has 3 options to keep the url immutable while being able to update content:

• resolver skylinks
• dnslink integration
• handshake domains

While handshake is a third party and we're just supporting the integration, I will not suggest it as a solution here unless you are interested in specifically that - please let me know if so.

Both the below solutions (with resolver skylink being preferred one) will allow users to retain their local browser data even when 1wallet gets updated since the url that users use to access the wallet will not change. Also I'd like to underline that we're not proposing here to replace your wallet domain with skynet but merely to provide alternative decentralised hosting as a part of a progression towards web3 adoption and improvements.

Resolver skylinks is one of the options that we use ourselves to serve our siasky.net homepage. Resolver skylink looks just like a regular skylink but it's a pointer to an immutable skylink. This means that while on every deploy you get an immutable skylink, in an automated deploy process you can point a resolver skylink at a new updated immutable skylink so your users will be able to access the updated version using the resolver skylink that they already have.

Example is out website skylink: AQBG8n_sgEM_nlEp3G0w3vLjmdvSZ46ln8ZXHn-eObZNjA
It does not change so we can share it as is but underneath you can see what it resolves into by using an endpoint:
https://siasky.net/skynet/resolve/AQBG8n_sgEM_nlEp3G0w3vLjmdvSZ46ln8ZXHn-eObZNjA

{"skylink":"jADcpLWPGeiiXjY8VEypeWGbyEiM6u0H4aoGlkxmSd-KVw"}

Dnslink integration is the other solution to make sure that you can share immutable address with your users.

We use dnslink to server https://siasky.tools that is a regular domain but underneath there's an CNAME dns record that points to siasky.net and a dnslink specific _dnslink.siasky.net text dns record that specifies the skylink that should the gateway (siasky.net in that case) resolve when accessing the address.

While dnslink spec if neither final nor official, it is widely adopted in ipfs community and skynet added an integration that works well. Unfortunately currently for https to work with dnslink, you need to use a full proxy like cloudflare to provide the secure ssl certificate. Being able to support dnslink certificates on a portal level is something that we intend to support in future but it's not a focus currently.

1. Lacking ability to respond custom HTTP headers based on routes

App integrating with 1wallet may use an embedded iframe for authorizing actions using their 1wallet. See https://github.com/polymorpher/one-wallet/wiki/App-Integration. One strong reason for using iframe is that it allows the browser to retrieve locally stored data under the iframe's origin inside the iframe. In this case, the required data is the partial proofs stored locally which are necessary for completing actions requested by the app. See https://github.com/polymorpher/one-wallet/wiki/Web-Client:-Data-Stored-Locally for data stored locally. Per wiki, such actions can be connecting the wallet, calling another contract, making payment, or signing a message. An example of such integration is Harmony Safe (MultiSig) based on Gnosis Safe. See https://multisig.harmony.one/ . By default, browsers disallow contents from another origin to be shown inside an iframe, unless specific headers are present. See https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP . However, wallets should not indiscriminately allow such header to be present on every route, otherwise malware and scam sites may load the wallet as an iframe on their own site, pretend that they are the wallets and steal credentials, or manipulate the user (through altered UI or otherwise) to steal their assets. For that reason, the server hosting the wallet client applications need to respond appropriate Content-Security-Policy (CSP) headers based on pattern in (URL) routes.

It appears Skynet does not provide such customization ability to their apps. The protocol has not discussed any specification about headers. Therefore, it would be up to Skynet portal operators to implement such feature, namely advanced routing and header configuration similar to those in Nginx and Apache. It would prove to be a very challenging task since Skynet presumes the configuration of deployed app (at resolved Skylink) must be immutable, but such configurations should be made mutable and easily reconfigurable, since browsers and headers specifications constantly evolve and may change every 1-3 years.

response: when uploading content to skynet, we're including a metadata fields like (internal) directory structure, file names and configurable routing settings like tryfiles and errorpages. Support for that has been added 4 months ago and it's been tested properly. That said if there is any specific configuration that we're lacking in terms of necessary requirements as to how to server an app we might be able to support that too. Could you write down what kind of CSP configuration specifically do you use for 1wallet so we could review that internally?

1. Lacking HTTPS support

Skynet documentation explicitly stated so and recommends using "fake" HTTPS bridges such as Cloudflare as a solution. It is understandable that implementing HTTPS on Skynet would be very challenging because HTTPS requires a server to store the private key and sign messages, and Skynet does not have servers at all (or more precisely, servers possessing secret information and can provide custom response to requests using its secret information). As another alternative, Skynet advocates the method of verifying the retrieved content using its SDK, hashes or other client side operations.

As it stands, neither of the alternatives provide a satisfactory level of security, which is of paramount importance to wallets (and even more so for 1wallet). If the user's DNS is hijacked, none of the alternatives offered above would prevent the attacker from directing the user to an altered client and carefully craft an attack that may steal the users' assets.

response: https://siasky.net and any other skynet portal that serves as a gateway supports full https end to end support when you access a specific skylink.
example: https://0404dsjvti046fsua4ktor9grrpe76erq9jot9cvopbhsvsu76r4r30.siasky.net/

The issues with no https support is only present when using dnslink. We have a roadmap item to support https for dnslink directly on portal level and it's a challenging task to solve correctly. In the meantime using dnslink with https is supported via full proxy like cloudflare and while it's less than ideal, the end to end communication is still done through https channel - user to cloudflare uses a fully secure ssl certificate and cloudflare to siasky.net uses self signed certificate. Still if this is a concern I suggest using the resolver skylink on siasky.net directly for the time being.

1. The requirement of an assurance that "APIs and web requests from your app only depend on decentralized protocols that will remain accessible even with future front-end updates"

See https://docs.siasky.net/integrations/homescreen#integrating-with-homescreen . At present, 1wallet must use a non-decentralized Relayer node to initiate transactions and pay for gas. See https://github.com/polymorpher/one-wallet/tree/master/code#relayer . Although any node can be a relayer and the user needs not to trust relayer, this component still remains a non-decentralized element. Until account abstraction (harmony-one/bounties#35) is implemented to allow smart contracts paying for gas, the relayer will remain to be a necessary component in the whole architecture.

Non-critical non-decentralized components used in 1wallet are Binance price APIs, and cached SushiSwap pairs and tokens. They do not affect security and core functionality of the wallet. Moving these components to decentralized network would incur significant amount of work to achieve a similar level of user experience compared to how things are running right now.

response: I think this is the biggest concern because the responsibility to be backwards compatible lies with the developer. I would like to highlight though that being able to roll back to previous frontend version of a specific app does not guarantee user that the app will be usable if it relies on specific apis but it's an option that we would like to at least give users and raise awareness in web3 app developers.

1. Lack of auto-update on Homescreen

Aside from the issues above, the lack of auto-update feature from Skynet Homescreen is also concerning. Wallets frequently need patches for both critical issues and feature enhancement. Sometimes the patches are silently delivered multiple times per day. Requiring the user to manually "update" their app for every patch would be a major hassle.

response: we are working on improving this situation for homescreen and auto updates will be the default behaviour unless user specifically selects a certain app version, the development is in progress.

Other issues

After all above issues are resolved, I will also need a custom release and deployment script. I do not use Github action since commits are very frequent. If Github actions were used, I will need to manually select release anyway and perhaps to configure separate branches for that purpose, which seem to be a lot of work without any apparent benefit.

response: we already addressed the concern of having a custom deploy script, we've prepared one that we're using for some of our tools already and we're working on releasing an official version in the upcoming weeks once all the testing is finished - the current version is available here https://github.com/SkynetLabs/skynet-mysky/blob/main/utilities/deploy.js

---

@polymorpher I am starting couple of week of a private time off starting next week but let's keep this channel of communication open and as a lead developer on this I'll work on addressing the concerns after I'm back. It's ok to keep the PR closed until then 👍

[polymorpher]

Thanks! Let me review this later this week.

[polymorpher]

Let's review this later this month and get it integrated

[Imanmahmoodi]

Good

[polymorpher]

Sorry, was really backlogged by various other things. Let's aim this for May

[polymorpher]

I think we should have immutable client deployments in the future to prevent hacking

[kwypchlo]

Hey @polymorpher, recently Skynet Labs announced that it is shutting down and while Skynet network and portals infrastructure will stay online, we will not be able to maintain new integrations ourselves. That said I am going to close this pull request for the time being.