reworked uid for linux, debugging and documentation enhancements

Author: sciabarracom

DEBUG.md

@@ -0,0 +1,79 @@
+# Tips to help debugging
+
+## Hidden flags
+
+The `iosdk` has a number of hidden flags useful for development
+
+Global flags
+
+- `--use-default-api-key`: do not generate an api key use the default one
+- `--skip-docker-version`: do not check the docker version
+- `--skip-pull-images`: do not pull images
+- `--skip-open-browser`: no not open the browser
+- `--skip-ide`: do not start the ide
+
+Hidden commands:
+
+- `wskprops`: set the .wskprops to use `wsk` with the server
+- `ide-deploy`: deploy the ide
+- `ide-destroy`: destroy the ide
+- `whisk-deploy`: deploy whisk
+- `whisk-destroy`: destroy whisk
+- `redis-deploy`: deploy redis
+- `redis-destroy`: destroy redis
+
+## Checking ide test results
+
+You can run the test suite of the cli with 
+
+```
+cd iosdk
+make test
+```
+
+If a test fails it may be difficult to understand what is wrong in the output.
+
+The test command saves the test output in `difftest.out`
+
+The `difftest.py` comes to the rescue, you pass the test output and returns a list of failed tests , with an index:
+
+```
+$ python3 difftest.py difftest.out
+0 --- FAIL: ExampleIdeDockerRun (0.00s)
+1 --- FAIL: ExampleStart (1.03s)  
+```
+
+Then you can see what went wrong in a single test passing the index:
+
+```
+$ python3 difftest.py difftest.out 1
+31c31
+< docker run -d -p 3000:3000 --rm --name iosdk-theia -e HOME=/home/project --add-host=openwhisk: 172.17.0.2 -v /tmp/iosdk-test/javascript:/home/project pagopa/iosdk-theia:test 
+---
+> docker run -d -p 3000:3000 --rm --name iosdk-theia --add-host=openwhisk:172.17.0.2 -v /tmp/iosdk-test/javascript:/home/project pagopa/iosdk-theia:test
+```
+
+## Debugging action tests
+
+Bats tests are shell scripts.
+
+The easiest way to debug them is to execute them line by line using VSCode.
+
+First, configure a keybinding to execute a line in the terminal.
+
+Open the `keybindings.json` as [described here](https://code.visualstudio.com/docs/getstarted/keybindings#_advanced-customization) and add this snippet:
+
+```
+{
+  "key": "ctrl+enter",
+  "command": "workbench.action.terminal.runSelectedText",
+  "when": "editorTextFocus"
+}
+```
+
+Open a terminal, change to `admnin/actions` then do a `source debug-src`.
+
+Open a test and send each line to the terminal with control-enter.
+
+
+

DEVEL.md

@@ -5,13 +5,15 @@
 
 # Using IO-SDK as a developer
 
-If you are one of those heroic people wanting to help and you can deal with bugs, crashes and the occasional complete destruction of you computer, you can follow with those instructions. 
+If you are one of those heroic people wanting to help and you can deal with bugs, crashes and the occasional complete destruction of you computer, you can follow with those instructions.
+
+After you read this document, do not forget to [check also this other document](DEBUG.md) for debugging tips.
 
 ## Prerequisites
 
-Supported development platforms are: 
+Supported development platforms are:
 - Linux Ubuntu 18.x
-- Mac OS Catalina 
+- Mac OS Catalina
 - Windows 10 build 2003 with WSL2 and Ubuntu 18.04
 
 Also your workstation needs at least 8gb of memory.
@@ -42,7 +44,7 @@ Windows configuration is a bit more complex. The steps are:
 #### Update to build 2009
 
 More informations [to update windows are here](https://support.microsoft.com/en-us/help/4027667/windows-10-update)
- 
+
 #### Install Ubuntu-18.04 under WSL2
 
 Go into the Microsof Store and seach for Ubuntu. Install version 18.04 and launch it.
@@ -59,7 +61,7 @@ Instructions to install Docker Desktop for Windows [are here](https://docs.docke
 
 #### Use docker for windows as docker backend in WSL 2
 
-This step is critical. 
+This step is critical.
 
 After installation follow [these instructions](https://docs.docker.com/docker-for-windows/wsl/) to use the Docker running in Windows as the Docker to use in WSL.
 
@@ -122,7 +124,7 @@ DO `make build` from the top level to embed the code you modified in the image.
 
 ## Backend Actions
 
-The server side part of the backend are [OpenWhisk](http://openwhisk.apache.org), action written in various languages. 
+The server side part of the backend are [OpenWhisk](http://openwhisk.apache.org), action written in various languages.
 
 Source code is under `admin/actions/src`
 
@@ -134,11 +136,11 @@ You can deploy actions after changes with `make deploy`
 
 The CLI, that is used to launch and initialize the environment is under `iosdk` and it is written in the Go programming language.
 
-To develop the cli, do `cd iosdk` and use `make`. 
+To develop the cli, do `cd iosdk` and use `make`.
 
-The cli embeds the version number of the images to refer to them correctly. 
+The cli embeds the version number of the images to refer to them correctly.
 
-In development, the version is just the branch name. 
+In development, the version is just the branch name.
 
 When you push to master, the CI builds images and pushes to the Docker Hub, so the CLI can retrieve the version stored in Docker Hub.
 

admin/actions/DEBUG.md

@@ -1,4 +1,5 @@
 FROM pagopa/theia:2020.06.08
 ADD ide-plugin /home/theia/plugins/ide-plugin
 RUN sudo find /home/theia -uid 0 -exec chown theia:theia {} \;
+RUN echo 'export PS1="\w\$ "' | sudo tee -a /etc/bash.bashrc
 #RUN cd /home/theia/plugins/ide-plugin && npm install && npm run build

ide/Dockerfile

@@ -170,6 +170,8 @@ func PropagateConfig() {
 
 	fmt.Println("Configuring Whisk")
 	WhiskUpdatePackageParameters("iosdk", ConfigMap())
-	fmt.Println("Configuring IDE")
-	configureIde()
+	if !*skipIde {
+		fmt.Println("Configuring IDE")
+		configureIde()
+	}
 }

iosdk/config.go

@@ -1,5 +1,7 @@
 package main
 
+import "runtime"
+
 // ConfigFile is the config file, change number if you need the project to be re-initialized
 const ConfigFile = "~/.iosdk.v2"
 
@@ -35,3 +37,6 @@ const DefaultWhiskAPIKey = "23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v
 
 // MinDockerMem is the minimum amount of memory required by docker
 const MinDockerMem = (4 * 1000 * 1000 * 1000) - 1
+
+// RuntimeOS to be changed for tests
+var RuntimeOS = runtime.GOOS

iosdk/const.go

@@ -4,7 +4,6 @@ import (
 	"fmt"
 	"os/user"
 	"path/filepath"
-	"runtime"
 )
 
 // IdeDeploy deploys and mounts a folder
@@ -48,7 +47,7 @@ func ideDockerRun(dir string) (err error) {
 	}
 
 	uid := ""
-	if runtime.GOOS == "linux" {
+	if RuntimeOS == "linux" {
 		usr, err := user.Current()
 		LogIf(err)
 		if err == nil {
@@ -57,8 +56,8 @@ func ideDockerRun(dir string) (err error) {
 	}
 
 	command := fmt.Sprintf(`docker run -d -p 3000:3000
-	--rm --name iosdk-theia -e HOME=/home/project -e USER=iosdk %s
-	--add-host=openwhisk:%s %s %s`, uid, *openwhiskIP, mount, image)
+	--rm --name iosdk-theia -e HOME=/home/project 
+	%s %s --add-host=openwhisk:%s %s`, mount, uid, *openwhiskIP, image)
 	Sys(command)
 	return nil
 }

iosdk/ide.go

@@ -9,20 +9,21 @@ func ExampleIdeDockerRun() {
 	DryRunPush("", "172.17.0.2", "641792b3e0112c8fa1896b8944a846dbbab88fe5729f3d464e71475afd9e6057",
 		"", "Error:",
 		"", "172.17.0.2")
+	RuntimeOS = "darwin"
 	fmt.Println(1, ideDockerRun("/tmp"))
 	fmt.Println(2, ideDockerRun("/tmp"))
 	fmt.Println(3, ideDockerRun(""))
 	// Output:
 	// docker pull pagopa/iosdk-theia:test
 	// docker inspect --format={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} iosdk-openwhisk
-	// docker run -d -p 3000:3000 --rm --name iosdk-theia --add-host=openwhisk:172.17.0.2 -v /tmp:/home/project pagopa/iosdk-theia:test
+	// docker run -d -p 3000:3000 --rm --name iosdk-theia -e HOME=/home/project -v /tmp:/home/project --add-host=openwhisk:172.17.0.2 pagopa/iosdk-theia:test 
 	// 1 <nil>
 	// docker pull pagopa/iosdk-theia:test
 	// docker inspect --format={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} iosdk-openwhisk
 	// 2 cannot find openwhisk
 	// docker pull pagopa/iosdk-theia:test
 	// docker inspect --format={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} iosdk-openwhisk
-	// docker run -d -p 3000:3000 --rm --name iosdk-theia --add-host=openwhisk:172.17.0.2 pagopa/iosdk-theia:test
+	// docker run -d -p 3000:3000 --rm --name iosdk-theia -e HOME=/home/project --add-host=openwhisk:172.17.0.2 pagopa/iosdk-theia:test
 	// 3 <nil>
 }
 

iosdk/ide_test.go

@@ -23,6 +23,7 @@ var (
 	skipDockerVersion = kingpin.Flag("skip-docker-version", "Skip check of docker version").Hidden().Default("false").Bool()
 	skipPullImages    = kingpin.Flag("skip-pull-images", "skip pull images").Hidden().Default("false").Bool()
 	skipOpenBrowser   = kingpin.Flag("skip-open-browser", "skip pull images").Hidden().Default("false").Bool()
+	skipIde           = kingpin.Flag("skip-ide", "skip starting ide").Hidden().Default("false").Bool()
 
 	// hidden debug commands
 	debugCmd        = kingpin.Command("debug", "debug").Hidden()
@@ -48,6 +49,8 @@ var (
 
 	// stop
 	stopCmd = kingpin.Command("stop", "Stop Development Environment")
+	// restart
+	restartCmd = kingpin.Command("restart", "Restart Development Environment")
 	// status
 	statusCmd = kingpin.Command("status", "Check Containers Status")
 )
@@ -58,11 +61,15 @@ func parseDebug(cmd string) bool {
 		ConfigLoad()
 		WskPropsSave()
 	case ideDeployCmd.FullCommand():
-		IdeDeploy("")
+		FatalIf(ConfigLoad())
+		IdeDeploy(Config.AppDir)
+		configureIde()
 	case ideDestroyCmd.FullCommand():
 		IdeDestroy()
 	case whiskDeployCmd.FullCommand():
+		FatalIf(ConfigLoad())
 		WhiskDeploy()
+		WhiskUpdatePackageParameters("iosdk", ConfigMap())
 	case whiskDestroyCmd.FullCommand():
 		WhiskDestroy()
 	case redisDeployCmd.FullCommand():

iosdk/main.go

@@ -24,9 +24,11 @@ func Start() error {
 	if err != nil {
 		return err
 	}
-	err = IdeDeploy(Config.AppDir)
-	if err != nil {
-		return err
+	if !*skipIde {
+		err = IdeDeploy(Config.AppDir)
+		if err != nil {
+			return err
+		}
 	}
 	return nil
 }

iosdk/start.go

@@ -48,7 +48,7 @@ func ExampleStart() {
 	// Deploying IDE...
 	// docker pull pagopa/iosdk-theia:test
 	// docker inspect --format={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} iosdk-openwhisk
-	// docker run -d -p 3000:3000 --rm --name iosdk-theia --add-host=openwhisk:172.17.0.2 -v /tmp/iosdk-test/javascript:/home/project pagopa/iosdk-theia:test
+	// docker run -d -p 3000:3000 --rm --name iosdk-theia -e HOME=/home/project -v /tmp/iosdk-test/javascript:/home/project --add-host=openwhisk:172.17.0.2 pagopa/iosdk-theia:test
 	// <nil>
 }
 

iosdk/start_test.go

@@ -9,7 +9,7 @@ iosdk/iosdk init $HOME/tmp-iosdk-test pagopa/io-sdk-javascript --io-apikey=12345
 echo "****** START"
 iosdk/iosdk -v start --skip-pull-images --skip-docker-version --skip-open-browser
 echo "****** BUILD"
-docker exec --user=$UID iosdk-theia bash -c 'cp /home/theia/.wskprops /home/project/.wskprops ; env HOME=/home/project bash /home/project/build.sh'
+docker exec --user=$UID iosdk-theia bash -c 'bash /home/project/build.sh'
 echo "****** STATUS"
 iosdk/iosdk status
 CHECK=ISPXNB32R82Y766F