Rewrite synchronization and test-exec troubleshooting guides

docs/articles/how-detox-works.md

@@ -1,31 +1,47 @@
 # How Detox Works
 
-Detox is an end-to-end testing framework. This means it runs your app on an actual device/simulator and interacts with it just like a real user would. This type of testing can give a lot of confidence in your app and help automate a manual QA process.
+Detox is an end-to-end testing framework. This means it runs your app on an actual device, or a device simulator/emulator and interacts with it just like a real user would. This type of testing can give a lot of confidence in your app and help automate an otherwise tedious manual QA process.
 
 When a Detox test executes, you actually have two different parts running side by side:
 
-- **The mobile app itself**, usually running on a simulator/emulator. A regular native build of your app is installed and executed on the device. Your app is usually built once before the tests start running.
+1. **The mobile app itself**, running on a device or a device simulator/emulator. A regular native-build of your app is installed and executed on the device, orchestrated by native Detox code that is built separately and installed alongside the app itself.
 
-- **The test suite**, running on Node.js, using a test runner like Jest. The tests are normally written in JavaScript. Because the tests are asynchronous in nature (every test line requires to access the app and wait for a response), the tests rely heavily on [`async`-`await`](https://ponyfoo.com/articles/understanding-javascript-async-await).
+2. **The test suite**, running on Node.js, over a test runner like Jest. The tests are normally written in JavaScript, and utilize the JavaScript part of Detox.
 
-The two parts are usually running in separate processes on your machine. It is also possible to run the two parts on different machines. Communication between the two parts takes place over the network using a web socket.
+The two parts are run in separate processes on your machine. It is also possible to run the two parts on different machines. Communication between the two parts takes place over the network using a web socket.
 
-In practice, to make the communication more resilient, both parts are implemented as clients and communicate with a Detox server that acts as proxy. This allows some nice behaviors like allowing one side to disconnect (during a simulator boot for example or app restart) without disconnecting the other side and losing its state.
+In practice, to make the communication more resilient, both parts are implemented as clients and communicate through a Detox server that acts as mediator. Having that server allows for some advantages like allowing one side to disconnect (during a simulator boot for example or app restart) without disconnecting the other side and losing its state.
 
-## How Detox Automatically Synchronizes With Your App
+## Automatic App-State Synchronization
 
-One of the key features of Detox is its ability to automatically synchronize the test execution with your app. The most annoying aspect of end-to-end tests is flakiness—tests sometimes fail without anything changing. Flakiness happens because tests are nondeterministic. Every time a test is running, things take place in a slightly different order inside your app.
+One of Detox's key features is the automatic synchronization of test execution with the app's state. For example, consider the following super-common moment in a test scenario:
 
-Consider a scenario where the app is making multiple network requests at the same time. What is the order of execution? It depends on which request completes first. This is an external concern depending on network congestion and how busy the server is.
+1. Node.js runs test code that effectively tells Detox to tap on the *login* button. Detox sends this tap command to the app.
+2. The app receives the command, the button is pressed, and the login process begins. A secure user session is obtained from the server, and the app navigates to the home screen. The home screen fetches yet even more data from the servers and renders elements with loading animations.
+3. **Detox - being a gray-box testing framework, monitors these changes in the app's state and waits for them to complete. This ensures that the test and the app's current state remain in-sync.**
+4. Detox proceeds to the next action in the test code only after the app is stable (!)
 
-The traditional method of dealing with flakiness is adding various `sleep()`/`waitFor()` commands throughout the test in an attempt to force a certain execution order. This is a bad practice, riddled with fragile magic values that often change if the machine running the tests becomes faster or slower.
+Let’s deep-dive into step #2: So much UI work happens with numerous network requests performed in the background… What is the order of execution of those requests, and how long should you wait until all of them are replied to? How long should you wait until the UI is ready? For the network, it depends on which request completes first, which in turn depends on network congestion and how busy the server is. As for the UI, it depends on the specific test device / machine specs and how busy its processor is.
 
-Detox tries to eliminate flakiness by automatically synchronizing your tests with the app. A test cannot continue to the next command until the app becomes idle. Detox monitors your app very closely in order to know when it’s idle. It tracks several asynchronous operations and waits until they complete. This includes:
+In the traditional black-box (rather than gray-box) testing approach, you normally deal with being blind to the app’s state by adding various `sleep()` / `waitFor()` commands throughout the test in an attempt to force order into the chaos. In step #3, **Detox eliminates the need for that malpractice, and so introduces stability into the otherwise inherently-flaky test world.**
 
-- Keeping track of all network requests that are currently in-flight and waiting until they complete
-- Keeping track of pending animations and waiting until they complete
-- Keeping track of timers and waiting until they expire or are cancelled
-- Keeping track of the React Native operations
+### Operations Detox synchronizes with automatically
+
+- **Network requests** - Detox monitors in-flight requests over the network (waiting for them to be responded).
+
+- **Main thread (native)** - Detox monitors pending native operations on the app's main thread (main dispatch queue and main `NSOperationQueue`).
+
+- **Layout of UI** - Detox monitors UI layout operations. There’s also special support for React Native layout which includes the Shadow Queue where [yoga](https://github.com/facebook/yoga) runs.
+
+- **Timers** - Detox monitors timers (explicit asynchronous delays). There’s special support for JavaScript's `setTimeout`, which is monitored.
+
+- **Animations** - Detox monitors active animations and transitions. There’s special support for React Native animations with the Animated library, and even the popular [react-native-reanimated](https://docs.swmansion.com/react-native-reanimated).
+
+- **React Native JavaScript thread** - Detox monitors pending operations on the JavaScript thread in RN apps.
+
+- **React Native native-modules thread** - Detox monitors pending RN native-module actions executed on its dedicated thread.
+
+- **React Native bridge** - In non-bridge-less apps (i.e. before RN's new-architecture), Detox monitors the React Native bridge and asynchronous messages delivered through it.
 
 :::info
 

docs/config/session.mdx

@@ -76,22 +76,22 @@ If `false`, it is assumed that you will be running it independently via [`detox
 
 ### `session.debugSynchronization` \[number]
 
-Default: `10000`.
+**Enabled by default**, with the value of `10_000`ms.
 
-Tells Detox how long (in milliseconds) to wait for the app to become idle until it starts querying it for more details.
+Tells Detox how long (in milliseconds) to wait for the app to become idle until it starts querying it for more details in order to print out automated-synchronization debugging logs.
 
 ```json
 {
   "session": {
-    "debugSynchronization": 20000
+    "debugSynchronization": 5000
   }
 }
 ```
 
 Detox will be printing the list of busy idling resources every time an action takes more than the specified period, e.g.:
 
 ```plain text
-15:13:07.309 detox[17005] i The app is busy with the following tasks:
+09:41:00.941 detox[1337] i The app is busy with the following tasks:
 • There are 10 work items pending on the dispatch queue: "Main Queue (<OS_dispatch_queue_main: com.apple.main-thread>)".
 • UI elements are busy:
   - Layers pending animations: 96.
@@ -102,8 +102,10 @@ Detox will be printing the list of busy idling resources every time an action ta
 • Run loop "Main Run Loop" is awake.
 ```
 
+<sup>(These are logs generated by Detox for iOS; Detox for Android generates different yet equivalent ones)</sup>
+
 To disable this behavior (i.e. querying the app periodically), set the value to `0`.
 
-Seeing logs like these usually indicates certain issues in your application, as mentioned in the [Troubleshooting Guide](../troubleshooting/synchronization.md).
+Seeing logs like these usually indicates certain issues in your application, as explained in the [Troubleshooting Guide](../troubleshooting/synchronization.md).
 
-For the most detailed information, refer to the DetoxSync (iOS) [Status Documentation](https://github.com/wix-incubator/DetoxSync/blob/master/StatusDocumentation.md).
+For extended, more detailed information on iOS, refer to the `DetoxSync` project's [Status Documentation](https://github.com/wix-incubator/DetoxSync/blob/master/StatusDocumentation.md).

docs/introduction/debugging.mdx

@@ -1,6 +1,9 @@
 import CodeBlock from '@theme/CodeBlock';
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
+import DebuggerAttachWebstorm from './partials/_debugging-attach-webstorm.mdx';
+import DebuggerAttachChrome from './partials/_debugging-attach-chrome.mdx';
+import DebuggerAttachVscode from './partials/_debugging-attach-vscode.mdx';
 import CompilingIOS from './partials/_debugging-native-code-compliing-ios.mdx';
 import CompilingAndroid from './partials/_debugging-native-code-compliing-android.mdx';
 import ConfigIOS from './partials/_debugging-native-code-config-ios.mdx';
@@ -11,50 +14,48 @@ import TroubleshootingAndroid from './partials/_debugging-native-code-troublesho
 
 # How to Debug
 
-## Detox Tests
+In the Detox world, you can debug either Detox itself (i.e. run it step by step), and the tested app. This guide covers both options.
 
-If you need to walk through your Detox tests step by step, add a `debugger` statement
-inside your test to mark a starting point, e.g.:
+## Running Detox Tests Step-by-Step
 
-```diff title="e2e/starter.test.js"
- describe('Example', () => {
-   beforeAll(async () => {
-     await device.launchApp();
-+    debugger;
-   });
-```
+Detox tests can be run step-by-step either using an IDE or by using Chrome debugger.
 
-Now run Detox with that specific test and `--inspect-brk` flag, e.g.:
+Start by running Detox using the Detox CLI alongside the inspection argument (`--inspect-brk`) and the file in which the test resides. For example:
 
 ```bash
 detox test --inspect-brk -c android.emu.debug e2e/starter.test.js
 ```
 
-Assuming you're using Jest, you'll see something like:
+You will see Detox starts and these logs:
 
 ```plain text
-DETOX_CONFIGURATION="android.emu.debug" node --inspect-brk ./node_modules/.bin/jest --config e2e/jest.config.js --runInBand e2e/starter.test.js
 Debugger listening on ws://127.0.0.1:9229/3dedd03b-8896-4ab8-a0a8-1b647abb9c98
 For help, see: https://nodejs.org/en/docs/inspector
 ```
 
+Now you can attach to Detox and tap-in into its execution process.
+
 :::info
 
-To learn more about debugging with `--inspect-brk`, refer to
-[Debugging — Getting Started](https://nodejs.org/en/docs/guides/debugging-getting-started/)
-on the official Node.js website. This tutorial suggests using Google Chrome for debugging,
-but you can also use an IDE to connect to the debugger.
+To learn more about debugging with `--inspect-brk`, refer to [Debugging — Getting Started](https://nodejs.org/en/docs/guides/debugging-getting-started/) on the official Node.js website.
 
 :::
 
-Open `Google Chrome` and go to `chrome://inspect` tab, where you'll see `./node_modules/.bin/jest` as a remote
-target waiting until you click `inspect` to attach to it.
+<Tabs groupId="debuggerAttach">
+  <TabItem value="Webstorm" label="Webstorm" default>
+    <DebuggerAttachWebstorm />
+  </TabItem>
+  <TabItem value="vs-code" label="vs-code">
+    <DebuggerAttachVscode />
+  </TabItem>
+  <TabItem value="Chrome" label="Chrome">
+    <DebuggerAttachChrome />
+  </TabItem>
+</Tabs>
 
-![](../img/inspect-brk.png)
 
-Happy debugging!
 
-## JavaScript application code
+## Debugging JavaScript application code
 
 Use debug configurations of your app that rely on React Native Packager running on port 8081 (or another):
 
@@ -63,7 +64,7 @@ Use debug configurations of your app that rely on React Native Packager running
 
 For the rest of details, please refer to [React Native – Debugging](https://reactnative.dev/docs/debugging).
 
-## Native application code
+## Debugging Native application code
 
 ### Setting Detox up as a compiling dependency
 
@@ -122,7 +123,7 @@ parameters to disable various side effects and make life easier when debugging:
  }
 ```
 
-- Using a preconfigured `session` with an autostarting server removes the legwork of copying and
+- Using a preconfigured `session` with an auto-starting server removes the legwork of copying and
 pasting values to the instrumentation runner launch arguments dialog every time before any launch
 from the IDE. Otherwise, by default when the `session` object omitted, `server` and `sessionId`
 are randomly generated for every new test session.

docs/introduction/partials/_debugging-attach-chrome.mdx

@@ -0,0 +1,23 @@
+Edit your test file. Mark the test you wish to debug as `it.only`, and then add a `debugger` statement
+inside your test to mark a starting point, e.g.:
+
+```diff title="e2e/starter.test.js"
+ describe('Example', () => {
+   beforeAll(async () => {
+     await device.launchApp();
+   });
+
+-   it('should debug nicely :-)', async () => {
++   it.only('should debug nicely :-)', async () => {
+     await element(by.text('Filter')).tap();
++     debugger;
+
+   });
+```
+
+Open `Google Chrome` and go to `chrome://inspect` tab, where you'll see `./node_modules/.bin/jest` as a remote
+target waiting until you click `inspect` to attach to it.
+
+![](../../img/inspect-brk.png)
+
+Happy debugging!

docs/introduction/partials/_debugging-attach-vscode.mdx

@@ -0,0 +1 @@
+Coming soon!

docs/introduction/partials/_debugging-attach-webstorm.mdx

@@ -0,0 +1,31 @@
+Open your app project (where there tests are, among other things) on Webstorm.
+
+#### Set up a breakpoint
+
+Open the file where the test you wish to run is. Mark it as `it.only`, and set up a breakpoint in a line that suites your needs (note: This can actually be inside a helper function or a test-driver's function):
+
+![Webstorm breakpoint](../../img/webstorm/breakpoint.png)
+
+#### Create & Run a debug configuration
+
+Go to Webstorm's [`Edit Configurations`](https://www.jetbrains.com/help/webstorm/run-debug-configuration.html).
+
+Tap the `+` button and select the Node.js option:
+
+![Webstorm edit configurations](../../img/webstorm/new-configuration.png)
+
+Create a new debugging configuration using the default arguments (port 9229, etc.):
+
+![Webstorm node debugging configuration](../../img/webstorm/node-debug-configuration.png)
+
+Attach to Detox by "debugging" the new configuration (start it using the beetle button).
+
+![Webstorm start debug configuration](../../img/webstorm/run-debug-configuration.png)
+
+Let Detox run until stopped at the break-point:
+
+![Webstorm stopped at breakpoint](../../img/webstorm/stopped-at-breakpoint.png)
+
+Run step-by-step using Webstorm's debugging actions (step-over, step-into, etc.)
+
+Happy debugging!

docs/introduction/your-first-test.mdx

@@ -204,8 +204,8 @@ If you haven't changed the generated `e2e/starter.test.js`, you are likely to se
   …
 ```
 
-If you have created your own test, and it is failing, examine the error message, check out our [Investigating Failures](../guide/investigating-test-failure.mdx)
-and [Debugging](debugging.mdx) guides, and run your tests again after you fix the issue.
+If you have created your own test, and it is failing, examine the error message, check out our [Troubleshooting](../troubleshooting/running-tests.md)
+and [Debugging](debugging.mdx) guides..
 
 [matchers]: ../api/matchers.md
 [`by.id()`]: ../api/matchers.md#byidid