Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Exist v7 #285

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
strategy:
fail-fast: true
matrix:
jdk: ['8', '11']
jdk: ['17']
os: [ubuntu-latest]
runs-on: ${{ matrix.os }}
steps:
Expand All @@ -27,4 +27,4 @@ jobs:
- name: Test
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
run: mvn verify
run: mvn verify -P cypress-without-record
157 changes: 120 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,68 +7,82 @@
An application for monitoring, profiling and inspecting a running eXist-db instance.

## Features

The app includes:

* **Monitoring dashboard**: shows memory usage, running queries, locked threads, cache usage and more
* **Query profiling** page: essential for tuning queries and indexes
* **Index browser**: inspect existing indexes
* **Remote console**: send log messages from any query in eXist to the remote console.
- **Monitoring dashboard**: shows memory usage, running queries, locked threads, cache usage and more
- **Query profiling** page: essential for tuning queries and indexes
- **Index browser**: inspect existing indexes
- **Remote console**: send log messages from any query in eXist to the remote console.
Uses web sockets for real-time updates.
* **Data visualizer**: get a quick overview of the frequency of elements in a collection.
* **Remote Monitoring**: monitor multiple remote eXistdb instances. Provides timelines for long term monitoring.
- **Data visualizer**: get a quick overview of the frequency of elements in a collection.
- **Remote Monitoring**: monitor multiple remote eXistdb instances. Provides timelines for long term monitoring.

## Configure Monex

### Enable Remote Monitoring

#### Preconditions
Monex remote monitoring requires the eXistdb scheduler module to be enabled. Make sure it is enabled in *$eXistdb_home/extensions/build.properties*

# Scheduler module
include.module.scheduler = true
Monex remote monitoring requires the eXistdb scheduler module to be enabled. Make sure it is enabled in `$eXistdb_home/extensions/build.properties`

and in *$eXistdb_home/conf.xml* make sure the Scheduler module is not commented out:
```txt
# Scheduler module
include.module.scheduler = true
```

<module uri="http://exist-db.org/xquery/scheduler"
class="org.exist.xquery.modules.scheduler.SchedulerModule" />
and in `$eXistdb_home/conf.xml` make sure the Scheduler module is not commented out:

```xml
<module uri="http://exist-db.org/xquery/scheduler"
class="org.exist.xquery.modules.scheduler.SchedulerModule" />
```

##### Rebuilding eXistdb
This needs only to be done if *include.module.scheduler* in *extensions/build.properties* was set to *false*. Then eXistdb has to be rebuild to enable the scheduler module. Shutdown the database and in the root of the eXistdb project simply call

./build.sh
This needs only to be done if `include.module.scheduler` in `extensions/build.properties` was set to `false`. Then eXistdb has to be rebuild to enable the scheduler module. Shutdown the database and in the root of the eXistdb project simply call

```shell
./build.sh
```

After starting the database again, the remote monitoring tab should show no more error warnings.

#### Adding an eXistdb instance to monitor
For each eXistdb instance to monitor its url and its unique token is needed. The token can be found in the data directory on the filesystem . The file is called *jmxservlet.token*. The path to the data directory can be found in *$existdb_home/conf.xml*

<db-connection files="path-to-your-data-dir" ... />
For each eXistdb instance to monitor its url and its unique token is needed. The token can be found in the data directory on the filesystem . The file is called `jmxservlet.token`. The path to the data directory can be found in `$existdb_home/conf.xml`

Each eXistdb installation to monitor is added as an instance entry at */db/apps/monex/instances.xml*.

**Sample Monex Instances**
```xml
<db-connection files="path-to-your-data-dir" ... />
```

<instance name="localhost"
url="http://localhost:8080/exist"
token="3268b570-392e-56ea-9550-117012413e15" cron="0 * * * * ?">
<poll cron="0/30 * * * * ?" store="yes">
<alert name="More than 30 threads waiting for locks to be released"
condition="count($jmx//LockManager/WaitingThreads/row) > 30"/>
<alert name="More than 40 brokers active"
condition="$jmx//Database/ActiveBrokers &gt; 10"/>
<alert name="Process CPU load &gt; 1.0"
condition="$jmx//UnixOperatingSystem/ProcessCpuLoad &gt; 0.5"/>
</poll>
</instance>
Each eXistdb installation to monitor is added as an instance entry at `/db/apps/monex/instances.xml`.

#### Sample Monex Instances

```xml
<instance name="localhost"
url="http://localhost:8080/exist"
token="3268b570-392e-56ea-9550-117012413e15" cron="0 * * * * ?">
<poll cron="0/30 * * * * ?" store="yes">
<alert name="More than 30 threads waiting for locks to be released"
condition="count($jmx//LockManager/WaitingThreads/row) > 30"/>
<alert name="More than 40 brokers active"
condition="$jmx//Database/ActiveBrokers &gt; 10"/>
<alert name="Process CPU load &gt; 1.0"
condition="$jmx//UnixOperatingSystem/ProcessCpuLoad &gt; 0.5"/>
</poll>
</instance>
```

In the Monex Remote Monitoring tab click "Run" to start all remote monitoring jobs. You should now see an entry "localhost" beneath "Remote Monitoring" and beneath that an entry "Timelines".


## Building

If you wish to build Monex from source code you should follow these steps:

1. Ensure you have Git, Apache Maven 3.3+, and Java JDK 8 installed and available:
1. Ensure you have Git, Apache Maven 3.3+, and Java JDK 8 installed and available:

```bash
$ git --version
git version 2.20.0
Expand All @@ -82,11 +96,80 @@ Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.1", arch: "x86_64", family: "mac"
```

3. Clone and build an EXPath package by running:
1. Clone and build an EXPath package by running:

```bash
$ git clone https://github.com/eXist-db/monex.git
$ cd monex
$ mvn package
git clone https://github.com/eXist-db/monex.git
cd monex
mvn package
```

The resulting XAR will be in the `target/` folder.

## Release Procedure

This project is configured to use the [Maven Release Plugin](https://maven.apache.org/maven-release/maven-release-plugin/) to make creating and subsequently publishing a release easy.

The release plugin will take care of:

1. Testing the project (all tests must pass)
2. Verifying all rules, e.g. license declarations present, etc.
3. Creating a Git Tag and pushing the Tag to GitHub
4. Building and signing the artifacts (e.g. EXPath Pkg `.xar` file).

Before performing the release, in addition to the Build requirements you need an installed and functioning copy of GPG or [GnuPG](https://gnupg.org/) with your private key setup correctly for signing.

Only users with push access to the GitHub repo can act as release manager.

### Preparation

Before creating the release, check if:

- [ ] the changelog is up-to-date for the planned release?
- [ ] the target versions for eXistdb are correctly declared in `pom.xml`?

```xml
<exist.java-api.version>5.4.0</exist.java-api.version>
<exist.processor.version>7.0.0-SNAPSHOT</exist.processor.version>
```


#### Changelog

To edit release notes for the planned release within `xar-assembly.xml` located in the root of this repo, e.g.:

```xml
<changelog>
<change version="4.0.0">
<ul xmlns="http://www.w3.org/1999/xhtml">
<li>Breaking: Due to internal API changes this version of monex requires eXist-db version 6.1.0 or later - <a href="https://github.com/eXist-db/monex/pull/210">#210</a>, <a href="https://github.com/eXist-db/monex/pull/223">#223</a></li>
<li>Fixed: Added missing release notes for 3.0.5 release - <a href="https://github.com/eXist-db/monex/issues/217">#217</a></li>
</ul>
</change>
</changelog>
```

### Release

To perform the release, from within your local Git cloned repository run:

```bash
mvn release:prepare && mvn release:perform
```

You will be prompted for the answers to a few questions along the way. The default response will be provided for you, and you can simply press "Enter" (or "Return") to accept it. Alternatively you may enter your own value and press "Enter" (or "Return").

```bash
What is the release version for "monex"? (org.exist-db.apps:monex) 2.3.1: : 2.4.0
What is SCM release tag or label for "monex"? (org.exist-db.apps:monex) 2.4.0: :
What is the new development version for "monex"? (org.exist-db.apps:monex) 2.4.1-SNAPSHOT: :
```

- For the `release version`, please sensibly consider using the next appropriate [SemVer 2.0.0](https://semver.org/) version number.
- For the `SCM release tag`, please use the same value as the `release version`.
- For the `new development version`, the default value should always suffice.

Once the release process completes, there will be a `.xar` file in the `target/` sub-folder. This file may be published to:

1. [GitHub Releases](https://github.com/eXist-db/monex/releases)
2. [The eXist-db Public EXPath Repository](https://exist-db.org/exist/apps/public-repo/admin)
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "monex",
"version": "4.2.1-SNAPSHOT",
"version": "5.0.0-SNAPSHOT",
"description": "Monitoring Application for eXist-db",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
Expand Down
19 changes: 9 additions & 10 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

<groupId>org.exist-db.apps</groupId>
<artifactId>monex</artifactId>
<version>4.2.1-SNAPSHOT</version>
<version>5.0.0-SNAPSHOT</version>

<name>Monex</name>
<description>An application for monitoring, profiling and inspecting a running eXist-db instance.</description>
Expand Down Expand Up @@ -45,11 +45,11 @@

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.build.source>1.8</project.build.source>
<project.build.target>1.8</project.build.target>
<project.build.source>17</project.build.source>
<project.build.target>17</project.build.target>

<exist.java-api.version>5.4.0</exist.java-api.version> <!-- The eXist-db XQuery Java Module API version -->
<exist.processor.version>6.1.0</exist.processor.version> <!-- The version of eXist-db needed by this EXPath Module -->
<exist.java-api.version>7.0.0-SNAPSHOT</exist.java-api.version> <!-- The eXist-db XQuery Java Module API version -->
<exist.processor.version>7.0.0-SNAPSHOT</exist.processor.version> <!-- The version of eXist-db needed by this EXPath Module -->

<jackson.version>2.18.2</jackson.version>
<websocket.api.version>1.1</websocket.api.version>
Expand Down Expand Up @@ -408,11 +408,10 @@
<goal>exec</goal>
</goals>
<configuration>
<executable>npm</executable>
<executable>npx</executable>
<arguments>
<argument>run</argument>
<argument>cypress</argument>
<argument>--</argument>
<argument>run</argument>
</arguments>
</configuration>
</execution>
Expand Down Expand Up @@ -441,10 +440,10 @@
<goal>exec</goal>
</goals>
<configuration>
<executable>npm</executable>
<executable>npx</executable>
<arguments>
<argument>run</argument>
<argument>cypress</argument>
<argument>run</argument>
<argument>--</argument>
<argument>--record</argument>
</arguments>
Expand Down
12 changes: 6 additions & 6 deletions src/main/java/org/exist/console/xquery/ConsoleModule.java
Original file line number Diff line number Diff line change
Expand Up @@ -46,27 +46,27 @@ public class ConsoleModule extends AbstractInternalModule {

private static ConsoleAdapter adapter = null;

public static void log(String channel, String message) {
public static void log(final String channel, final String message) {
log(channel, false, message);
}

public static void log(String channel, boolean json, String message) {
public static void log(final String channel, final boolean json, final String message) {
if (adapter != null) {
adapter.log(channel, json, message);
}
}

public static void log(String channel, String source, int line, int column, String message) {
public static void log(final String channel, final String source, final int line, final int column, final String message) {
log(channel, source, line, column, false, message);
}

public static void log(String channel, String source, int line, int column, boolean json, String message) {
public static void log(final String channel, final String source, final int line, final int column, final boolean json, final String message) {
if (adapter != null) {
adapter.log(channel, source, line, column, json, message);
}
}

public static void send(String channel, String json) {
public static void send(final String channel, final String json) {
if (adapter != null) {
adapter.send(channel, json);
}
Expand All @@ -80,7 +80,7 @@ public static void setAdapter(final ConsoleAdapter consoleAdapter) {
adapter = consoleAdapter;
}

public ConsoleModule(Map<String, List<? extends Object>> parameters) {
public ConsoleModule(final Map<String, List<?>> parameters) {
super(functions, parameters, false);
}

Expand Down
6 changes: 3 additions & 3 deletions src/main/java/org/exist/console/xquery/JMXToken.java
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ public class JMXToken extends BasicFunction {
new FunctionReturnSequenceType(Type.STRING, Cardinality.ZERO_OR_ONE, "The authentication token")
);

public JMXToken(XQueryContext context) {
public JMXToken(final XQueryContext context) {
super(context, signature);
}

@Override
public Sequence eval(Sequence[] sequences, Sequence sequence) throws XPathException {
public Sequence eval(final Sequence[] sequences, final Sequence sequence) throws XPathException {
if (!context.getEffectiveUser().hasDbaRole()) {
throw new XPathException(this, "Only a dba user is allowed to retrieve the JMX access token.");
}
Expand All @@ -74,7 +74,7 @@ public Sequence eval(Sequence[] sequences, Sequence sequence) throws XPathExcept
if (key != null) {
return new StringValue(key);
}
} catch (IOException ex) {
} catch (final IOException ex) {
throw new XPathException(this, "Exception while reading token file: " + ex.getMessage(), ex);
}
}
Expand Down
Loading
Loading