Merge pull request amazon-archives#22 from zhongnansu/test

Update doc and naming

Author: abbashus

README.md

@@ -1,14 +1,15 @@
 # Open Distro Elasticsearch SQL CLI
 
+ODFE: Open Distro for Elasticsearch 
 
-SQL CLI is a stand alone Python application and can be launched by a wake word `escli`. It serves as a support only for 
-[Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/). You can move 
-it around to other machines without having to install Elasticsearch on them.
+ODFE SQL CLI is a stand alone Python application and can be launched by a wake word `odfesql`. It serves as a support only for 
+[Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/). You can run 
+it on any OS we support, and connect to any valid remote endpoint without installing Elasticsearch.
 
 
 ## Installation
-- `pip install escli` 
-- escli is compatible with Python 3, and python 2 is going away soon https://pythonclock.org/ 
+- `pip install odfesql` 
+- odfe sql cli is compatible with Python 3, because Python 2 is no longer maintained since 01/01/2020 https://pythonclock.org/ 
 
 
 ## Configuration
@@ -25,10 +26,11 @@ See the file itself for a description of all available options.
     - Enable horizontal display (by default) and vertical display when output is too wide
     - Pagination for long output
 - Syntax highlighting
-- Connect to Elasticsearch node/cluster with authentication on either ES localhost, Open Distro ES, or AES,
+- Connect to Elasticsearch node/cluster with/without security on either **ES localhost, Open Distro ES, or AWS Elasticsearch Domain**.
+Refer to [test plan](./tests/test_plan.md) on how to connect to different instance with/without security
 - Load Config file
 - Run single query from Command Line with parameters
-    - *endpoint:* no need to specify a parameter, anything follow by wake word `escli` should be the endpoint. 
+    - *endpoint:* no need to specify a parameter, anything follow by wake word `odfesql` should be the endpoint. 
     By default, it’s http://localhost:9200
     - *--help:* help page for options and params
     - *-q:* follow by a single query user wants to run.
@@ -45,7 +47,8 @@ See the file itself for a description of all available options.
 
 
 ## Basic Usage
-![](./screenshots/usage.gif)
+- The CLI supports all types of query that ODFE SQL supports. See [ODFE SQL basic usage](https://github.com/opendistro-for-elasticsearch/sql#basic-usage)
+- ![](./screenshots/usage.gif)
 
 
 
@@ -69,30 +72,3 @@ See the [LICENSE](./LICENSE.TXT) file for our project's licensing. We will ask y
 ## Copyright
 
 Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
-
-
-
-## Development Guide
-### Build the application in development
-- `pip install virtualenv`
-- `virtualenv venv` to create virtual environment for **Python 3**
-- `source ./venv/bin/activate` activate virtual env.
-- `cd` into project root folder.
-- `pip install --editable .` will install all dependencies from `setup.py`.
-
-### Run CLI
-- Start an Elasticsearch instance from either local, Docker with Open Distro SQL plugin, or AWS Elasticsearch
-- To launch the cli, use wake word `escli` followed by endpoint of your running ES instance. If not specifying 
-any endpoint, it uses http://localhost:9200 by default. 
-
-### Testing Procedure
-- Prerequisites
-    - Build the application
-    - Start a local Elasticsearch instance with 
-    [Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/) installed
-    and listening at http://localhost:9200.
-- `pip install -r requirements-dev.txt` Install test frameworks including Pytest and mock.
-- `cd` into `tests` and run `pytest`
-
-### Style
-- Use [black](https://github.com/psf/black) to format code, with option of `--line-length 120`

development_guide.md

@@ -0,0 +1,59 @@
+## Development Guide
+### Development Environment Set Up
+- `pip install virtualenv`
+- `virtualenv venv` to create virtual environment for **Python 3**
+- `source ./venv/bin/activate` activate virtual env.
+- `cd` into project root folder.
+- `pip install --editable .` will install all dependencies from `setup.py`.
+
+### Run CLI
+- Start an Elasticsearch instance from either local, Docker with Open Distro SQL plugin, or AWS Elasticsearch
+- To launch the cli, use wake word `odfesql` followed by endpoint of your running ES instance. If not specifying 
+any endpoint, it uses http://localhost:9200 by default. 
+
+### Testing
+- Prerequisites
+    - Build the application
+    - Start a local Elasticsearch instance (OSS) with 
+    [Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/) installed
+    and listening at http://localhost:9200.
+- Pytest
+    - `pip install -r requirements-dev.txt` Install test frameworks including Pytest and mock.
+    - `cd` into `tests` and run `pytest`
+- Refer to [test_plan](./tests/test_plan.md) for manual test guidance.
+
+### Style
+- Use [black](https://github.com/psf/black) to format code, with option of `--line-length 120`
+
+## Release guide
+
+- Package Manager: pip
+- Repository of software for Python: PyPI
+
+### Workflow
+
+1. Update version number
+    1. Modify the version number in `_init_.py` under `escli` package. It will be used by `setup.py` for release.
+2. Create/Update `setup.py` (if needed)
+    1. For more details refer to https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py 
+3. Update README.md, Legal and copyright files(if needed)
+    1. Update README.md when there is a critical feature added.
+    2. Update `THIRD-PARTY` files if there is a new dependency added.
+4. Generate distribution archives
+    1. Make sure you have the latest versions of `setuptools` and `wheel` installed:  `pip install --user --upgrade setuptools wheel`
+    2. Run this command from the same directory where `setup.py` is located: `python3 setup.py sdist bdist_wheel`
+5. Upload the distribution archives to TestPyPI
+    1. Register an account on [testPyPI](https://test.pypi.org/)
+    2. `pip install --user —upgrade twine `
+    3. `python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*`
+6. Install your package from TestPyPI and do manual test
+    1. `pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple odfe-sql`
+7. Upload to PyPI
+    1. Register an account on [PyPI](https://pypi.org/), note that these are two separate servers and the credentials from the test server are not shared with the main server.
+    2. Use `twine upload dist/*` to upload your package and enter your credentials for the account you registered on PyPI.
+8. Install your package from PyPI using `pip install [your-package-name]`
+
+### Reference
+- https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56
+- https://packaging.python.org/tutorials/packaging-projects/
+- https://packaging.python.org/guides/using-testpypi/

escli/__init__.py

@@ -12,4 +12,4 @@
 See the License for the specific language governing permissions and
 limitations under the License.
 """
-__version__ = "1.0.7"
+__version__ = "1.0"

escli/essqlcli.py

@@ -45,7 +45,7 @@
 
 
 class ESSqlCli:
-    """ESSqlCli instance is used to build and run the ES SQL CLI."""
+    """ESSqlCli instance is used to build and run the ODFE SQL CLI."""
 
     def __init__(self, esclirc_file=None, always_use_pager=False):
         # Load conf file
@@ -123,7 +123,7 @@ def run_cli(self):
 
         while True:
             try:
-                text = self.prompt_app.prompt(message="escli> ")
+                text = self.prompt_app.prompt(message="odfesql> ")
             except KeyboardInterrupt:
                 continue  # Control-C pressed. Try again.
             except EOFError:

setup.py

@@ -41,7 +41,7 @@
     long_description = fh.read()
 
 setup(
-    name="escli",
+    name="odfesql",
     author="Zhongnan",
     author_email="[email protected]",
     version=version,
@@ -53,7 +53,7 @@
     long_description=long_description,
     long_description_content_type="text/markdown",
     install_requires=install_requirements,
-    entry_points={"console_scripts": ["escli=escli.main:cli"]},
+    entry_points={"console_scripts": ["odfesql=escli.main:cli"]},
     classifiers=[
         "Intended Audience :: Developers",
         "License :: OSI Approved :: Apache Software License",

tests/test_plan.md

@@ -0,0 +1,36 @@
+# Test Plan
+ The purpose of this checklist is to guide you through the basic usage of ODFE SQL CLI, as well as a manual test process. 
+ 
+## Display
+ * [ ] Test pagination with different output length / width.
+ * [ ] Test table formatted output.
+ * [ ] Test successful conversion from horizontal to vertical display with confirmation.
+ * [ ] Test warning message when output > 200 rows of data. (Limited by ODFE SQL syntax)
+
+
+## Connection
+* [ ] Test connection to a local Elasticsearch instance
+  * [ ] Standard Elastic version, with/without authentication by [X-pack security](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/security-getting-started.html)
+  * [ ] OSS version, no authentication
+  * [ ] OSS version, install [ODFE Security plugin](https://opendistro.github.io/for-elasticsearch-docs/docs/install/plugins/) to enable authentication and SSL
+* [ ] Test connection to [AWS Elasticsearch domain](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) (with ODFE SQL plugin installed by default)
+* [ ] Test connection fail when connecting to invalid endpoint
+* [ ] Test reconnection when connection lost during execution
+
+
+## Execution
+* [ ] Test successful execution given a query
+* [ ] Test unsuccessful execution with an invalid SQL query
+* [ ] Test load config file
+
+
+## Nested commands with params
+* [ ] Test dump query result to a file
+* [ ] Test explain option `-e`
+* [ ] Test query and format option `-q`, `-f`
+* [ ] Test vertical option `-v`
+
+
+## OS and Python compatibility
+* [ ] Manually test on Linux(Ubuntu), Windows and mac-os.
+* [ ] Test against python 3.X versions