Real Storage asv perf POC (#2165)

#### Reference Issues/PRs
<!--Example: Fixes #1234. See also #3456.-->

Main idea of this set is to provide repeatable set of performance
measurments against persistent storage initially ASW S3 and later any
other storage that we support.

Different types of storage provide different type of performance and
characteristics. Thus having one and the same asv benchmark test to do
both is far from optimal

Persistent storages provide a way to setup once the data needed for the
tests (especially for read tests/ those that do not modify anything)

At the same time they should isolate that data from other other tests -
that do writes by providing a temporary storage for their use

The approach to build this kind of test setup is first to separate the
logic for setting up the libraries and symbols data from the actual asv.
In other words each test should use anothe class which is specialized
for dealing with setting up data, making some checks and wiping it out
if necessary.

Having a setup class that is not part of asv test provides way to
controlled setting up environment on demand and wiping it out, and all
other stuff. This class can fascilitate easily that logic for all types
of storages.

In this model the asv test becomes simple to write.
Most importantly the tests are now ASV _independent_, then can be
quickly moved to other tool etc if needed

It also gives opportunity to use parts of presetup logic for tests
outside of asv itself. In other words the persistent part can be shared
accross different types of tests.

Adding later new storage type (azure etc) is very easy (provided that it
is shared type of storage similar to aws):
  - extend logic in base class ...
  - inherit class of the primary implementation of logic AWS - that's it

#### What does this implement or fix?

#### Any other comments?

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Georgi Rusev <Georgi Rusev>

Author: grusev

.github/actions/set_persistent_storage_env_vars/action.yml

@@ -7,6 +7,7 @@ inputs:
   aws_access_key:    {required: true, type: string, description: The value for the AWS Access key}      
   aws_secret_key:    {required: true, type: string, description: The value for the AWS Secret key}
   strategy_branch:   {default: 'ignore', type: string, description: a unique combination of the parameters for the given job strategy branch, e.g. linux_cp36} 
+  shared_storage_prefix:   {default: 'none', type: string, description: a prefix string that will be used for persistent storage}
 runs:
   using: "composite"
   steps:
@@ -19,7 +20,11 @@ runs:
         echo "ARCTICDB_PERSISTENT_STORAGE_UNIQUE_ID=${{ github.ref_name }}_${{ github.run_id }}" >> $GITHUB_ENV
         echo "ARCTICDB_PERSISTENT_STORAGE_STRATEGY_BRANCH=${{ inputs.strategy_branch }}" >> $GITHUB_ENV
         # This is the top level path for all test, this is where to write data that should be shared between jobs (e.g. seed job)
-        echo "ARCTICDB_PERSISTENT_STORAGE_SHARED_PATH_PREFIX=ci_tests/${{ github.ref_name }}_${{ github.run_id }}" >> $GITHUB_ENV
+        if [ "${{ inputs.shared_storage_prefix }}" == "none" ]; then
+          echo "ARCTICDB_PERSISTENT_STORAGE_SHARED_PATH_PREFIX=ci_tests/${{ github.ref_name }}_${{ github.run_id }}" >> $GITHUB_ENV
+        else
+          echo "ARCTICDB_PERSISTENT_STORAGE_SHARED_PATH_PREFIX=ci_tests/${{ inputs.shared_storage_prefix }}" >> $GITHUB_ENV
+        fi
         # This is a path that should be used for specific job and its tests to avoid cross contamination and race conditions
         echo "ARCTICDB_PERSISTENT_STORAGE_UNIQUE_PATH_PREFIX=ci_tests/${{ github.ref_name }}_${{ github.run_id }}_${{ inputs.strategy_branch }}" >> $GITHUB_ENV
         # S3 Specific

.github/workflows/analysis_workflow.yml

@@ -9,6 +9,14 @@ on:
         description: Tag of the ArcticDB development image to use for benchmark and code coverage flows
         type: string
         default: latest
+      suite_to_run:
+          description: Run LMDB suite or REAL storage
+          type: choice
+          options:
+            - 'LMDB'
+            - 'REAL'
+          default: 'LMDB'
+  
 
   schedule: # Schedule the job to run at 12 a.m. daily
     - cron: '0 0 * * *'
@@ -50,6 +58,7 @@ jobs:
       run_all_benchmarks: ${{ inputs.run_all_benchmarks || false }}
       run_on_pr_head: ${{ github.event_name == 'pull_request_target' }}
       dev_image_tag: ${{ inputs.dev_image_tag || 'latest' }}
+      suite_to_run: ${{ inputs.suite_to_run || 'LMDB'}}
 
   publish_benchmark_results_to_gh_pages:  
     name: Publish benchmark results to gh-pages
@@ -106,6 +115,7 @@ jobs:
   
 
   run-asv-check-script:
+    name: Executes asv tests checks
     timeout-minutes: 120
     runs-on: ubuntu-latest
     container: ghcr.io/man-group/arcticdb-dev:latest
@@ -141,6 +151,13 @@ jobs:
           cmake -P cpp/CMake/CpuCount.cmake | sed 's/^-- //' | tee -a $GITHUB_ENV
         env:
           CMAKE_BUILD_PARALLEL_LEVEL: ${{vars.CMAKE_BUILD_PARALLEL_LEVEL}}
+
+      - name: Set persistent storage variables
+        uses: ./.github/actions/set_persistent_storage_env_vars
+        with:
+            bucket: "arcticdb-asv-real-storage"
+            aws_access_key: "${{ secrets.AWS_S3_ACCESS_KEY }}"
+            aws_secret_key: "${{ secrets.AWS_S3_SECRET_KEY }}"      
 
       - name: Install ASV
         shell: bash -el {0}
@@ -152,7 +169,7 @@ jobs:
       
       - name: Build project for ASV
         run: |
-          python -m pip install -ve .
+          python -m pip install -ve .[Testing]
           
       - name: Run ASV Tests Check script
         run: |

.github/workflows/benchmark_commits.yml

@@ -6,6 +6,7 @@ on:
       commit:             {required: true, type: string, description: commit hash that will be benchmarked}
       run_on_pr_head:     {required: false, default: false, type: boolean, description: Specifies if the benchmark should run on PR head branch}
       dev_image_tag:      {required: false, default: 'latest', type: string, description: Tag of the ArcticDB development image}
+      suite_to_run:       {required: true, type: string, default: 'LMDB', description: Default benchmark on 'LMDB' storage (or 'REAL' storage)}
 jobs:
   start_ec2_runner:
     uses: ./.github/workflows/ec2_runner_jobs.yml
@@ -62,6 +63,7 @@ jobs:
           aws_access_key: "${{ secrets.AWS_S3_ACCESS_KEY }}"
           aws_secret_key: "${{ secrets.AWS_S3_SECRET_KEY }}"
           strategy_branch: "${{ inputs.commit }}"
+          shared_storage_prefix: "_github_runner_"
 
       - name: Install ASV
         shell: bash -el {0}
@@ -76,7 +78,13 @@ jobs:
         shell: bash -el {0}
         run: | 
           git config --global --add safe.directory .
-          python -m asv run -v --show-stderr ${{ inputs.commit }}^!
+          if [ "${{ github.event.inputs.suite_to_run }}" == "REAL" ]; then
+            SUITE='^(real_).*'
+          else
+            SUITE='^(?!real_).*'
+          fi
+          echo "selection SUITE=$SUITE"
+          python -m asv run -v --show-stderr --bench $SUITE ${{ inputs.commit }}^!
         
       - name: Benchmark against master
         if: github.event_name == 'pull_request_target' && inputs.run_all_benchmarks == false