Sync master with v1.8.0 tag

Author: ashahba

Contribute.md

@@ -71,10 +71,10 @@ required:
    first step where you setup the directories for your model. In this
    function, add commands to install any third-party dependencies within
    an `if [ ${NOINSTALL} != "True" ]; then` conditional block. The
-   purpose of the `NOINSTALL` flag is to be able to skip the installs
-   for quicker iteration when running on bare metal or debugging. If
-   your model requires the `PYTHONPATH` environment variable to be setup
-   to find model code or dependencies, that should be done in the
+   purpose of the `--noinstall` flag or `NOINSTALL` env var is to be able 
+   to skip the installs for quicker iteration when running on bare metal 
+   or debugging. If your model requires the `PYTHONPATH` environment variable 
+   to be setup to find model code or dependencies, that should be done in the
    model's function. Next, setup the command that will be run. The
    standard launch script args are already added to the `CMD` variable,
    so your model function will only need to add on more args if you have

Jenkinsfile

@@ -15,11 +15,9 @@ node('skx') {
             # don't know OS, so trying both apt-get and yum install
             sudo apt-get clean || sudo yum update -y
             sudo apt-get update -y || sudo yum install -y epel-release
-            sudo apt-get install -y python-dev python-pip || sudo yum install -y python2-devel python2-pip
             sudo apt-get install -y python3-dev python3-pip || sudo yum install -y python36-devel python36-pip
 
             # virtualenv 16.3.0 is broken do not use it
-            python2 -m pip install --no-cache-dir --user --upgrade pip==19.0.3 virtualenv!=16.3.0 tox
             python3 -m pip install --no-cache-dir --user --upgrade pip==19.0.3 virtualenv!=16.3.0 tox
             """
         }
@@ -29,7 +27,7 @@ node('skx') {
             set -e
 
             cd intel-models
-            ~/.local/bin/tox -e py2.7-flake8 -e py3-flake8
+            ~/.local/bin/tox -e py3-flake8
             """
         }
         stage('Unit tests') {
@@ -38,7 +36,7 @@ node('skx') {
             set -e
 
             cd intel-models
-            ~/.local/bin/tox -e py2.7-py.test -e py3-py.test
+            ~/.local/bin/tox -e py3-py.test
             """
         }
         // put benchmarks here later

Makefile

@@ -37,8 +37,6 @@ venv: $(ACTIVATE)
 	@echo -n "Using "
 	@. $(ACTIVATE) && python --version
 
-venv2: venv
-
 venv3: PY_VERSION=3
 venv3: $(ACTIVATE)
 	@echo -n "Using "
@@ -49,33 +47,13 @@ tox:
 
 lint:
 	@echo "Running style check..."
-	tox -e py2.7-flake8 -e py3-flake8
-
-lint2:
-	@echo "Running style check..."
-	tox -e py2.7-flake8
-
-lint3:
-	@echo "Running style check python 3 "
 	tox -e py3-flake8
 
 unit_test:
 	@echo "Running unit tests..."
-	tox -e py2.7-py.test -e py3-py.test
-
-unit_test2:
-	@echo "Running unit tests python 2..."
-	tox -e py2.7-py.test
-
-unit_test3:
-	@echo "Running unit tests python 3..."
 	tox -e py3-py.test
 
 test: lint unit_test
 
-test2: lint2 unit_test2
-
-test3: lint3 unit_test3
-
 clean:
 	rm -rf .venv .venv3 .tox

README.md

@@ -34,7 +34,5 @@ We hope this structure is intuitive and helps you find what you are looking for;
 
 ![Repo Structure](repo_structure.png)
 
-*Note: For model quantization and optimization tools, see [https://github.com/IntelAI/tools](https://github.com/IntelAI/tools)*.
-
 ## How to Contribute
 If you would like to add a new benchmarking script, please use [this guide](/Contribute.md).

benchmarks/README.md

@@ -25,16 +25,27 @@ dependencies to be installed:
 | Image Recognition      | TensorFlow    | [ResNet 50v1.5](https://github.com/tensorflow/models/tree/master/official/resnet) | Training | [FP32](image_recognition/tensorflow/resnet50v1_5/README.md#fp32-training-instructions) [BFloat16**](image_recognition/tensorflow/resnet50v1_5/README.md#bfloat16-training-instructions)|
 | Language Modeling      | TensorFlow    | [BERT](https://arxiv.org/pdf/1810.04805.pdf)                | Inference | [FP32](language_modeling/tensorflow/bert_large/README.md#fp32-inference-instructions) [BFloat16**](language_modeling/tensorflow/bert_large/README.md#bfloat16-inference-instructions) |
 | Language Modeling      | TensorFlow    | [BERT](https://arxiv.org/pdf/1810.04805.pdf)                | Training  | [FP32](language_modeling/tensorflow/bert_large/README.md#fp32-training-instructions) [BFloat16**](language_modeling/tensorflow/bert_large/README.md#bfloat16-training-instructions) |
+| Language Translation   | TensorFlow    | [BERT](https://arxiv.org/pdf/1810.04805.pdf)                | Inference | [FP32](language_translation/tensorflow/bert/README.md#fp32-inference-instructions) |
 | Language Translation   | TensorFlow    | [GNMT*](https://arxiv.org/pdf/1609.08144.pdf)                | Inference | [FP32](language_translation/tensorflow/mlperf_gnmt/README.md#fp32-inference-instructions) |
-| Reinforcement          | TensorFlow    | [MiniGo](https://arxiv.org/abs/1712.01815.pdf)              | Training  | [FP32](reinforcement/tensorflow/minigo/README.md#fp32-training-instructions)|
 | Language Translation   | TensorFlow    | [Transformer_LT_Official ](https://arxiv.org/pdf/1706.03762.pdf)| Inference | [FP32](language_translation/tensorflow/transformer_lt_official/README.md#fp32-inference-instructions) |
 | Language Translation   | TensorFlow    | [Transformer_LT_mlperf ](https://arxiv.org/pdf/1706.03762.pdf)| Training | [FP32](language_translation/tensorflow/transformer_mlperf/README.md#fp32-training-instructions) [BFloat16**](language_translation/tensorflow/transformer_mlperf/README.md#bfloat16-training-instructions) |
 | Object Detection       | TensorFlow    | [R-FCN](https://arxiv.org/pdf/1605.06409.pdf)               | Inference | [Int8](object_detection/tensorflow/rfcn/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/rfcn/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [SSD-MobileNet*](https://arxiv.org/pdf/1704.04861.pdf)       | Inference | [Int8](object_detection/tensorflow/ssd-mobilenet/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-mobilenet/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [SSD-ResNet34*](https://arxiv.org/pdf/1512.02325.pdf)        | Inference | [Int8](object_detection/tensorflow/ssd-resnet34/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [SSD-ResNet34](https://arxiv.org/pdf/1512.02325.pdf)        | Training  | [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-training-instructions) [BFloat16**](object_detection/tensorflow/ssd-resnet34/README.md#bf16-training-instructions) |
 | Recommendation         | TensorFlow    | [Wide & Deep Large Dataset](https://arxiv.org/pdf/1606.07792.pdf)	| Inference | [Int8](recommendation/tensorflow/wide_deep_large_ds/README.md#int8-inference-instructions) [FP32](recommendation/tensorflow/wide_deep_large_ds/README.md#fp32-inference-instructions) |
+| Recommendation         | TensorFlow    | [Wide & Deep Large Dataset](https://arxiv.org/pdf/1606.07792.pdf)	| Training | [FP32](recommendation/tensorflow/wide_deep_large_ds/README.md#fp32-training-instructions) |
 | Recommendation         | TensorFlow    | [Wide & Deep](https://arxiv.org/pdf/1606.07792.pdf)         | Inference | [FP32](recommendation/tensorflow/wide_deep/README.md#fp32-inference-instructions) |
+| Reinforcement          | TensorFlow    | [MiniGo](https://arxiv.org/abs/1712.01815.pdf)              | Training  | [FP32](reinforcement/tensorflow/minigo/README.md#fp32-training-instructions)|
+
+## TensorFlow Serving Use Cases
+
+| Use Case               | Framework     | Model               | Mode      | Instructions    |
+| -----------------------| --------------| ------------------- | --------- |------------------------------|
+| Image Recognition      | TensorFlow Serving | [Inception V3](https://arxiv.org/pdf/1512.00567.pdf)        | Inference | [FP32](image_recognition/tensorflow_serving/inceptionv3/README.md#fp32-inference-instructions) |
+| Image Recognition      | TensorFlow Serving | [ResNet 50v1.5](https://github.com/tensorflow/models/tree/master/official/resnet) | Inference | [FP32](image_recognition/tensorflow_serving/resnet50v1_5/README.md#fp32-inference-instructions) |
+| Language Translation   | TensorFlow Serving | [Transformer_LT_Official](https://arxiv.org/pdf/1706.03762.pdf) | Inference | [FP32](language_translation/tensorflow_serving/transformer_lt_official/README.md#fp32-inference-instructions) |
+| Object Detection       | TensorFlow Serving | [SSD-MobileNet](https://arxiv.org/pdf/1704.04861.pdf)       | Inference | [FP32](object_detection/tensorflow_serving/ssd-mobilenet/README.md#fp32-inference-instructions) |
 
 *Means the model belongs to [MLPerf](https://mlperf.org/) models and will be supported long-term.
 

benchmarks/common/base_benchmark_util.py

@@ -97,6 +97,11 @@ def _define_args(self):
             help="Specify how many MPI processes to launch per socket",
             dest="num_mpi", default=1)
 
+        self._common_arg_parser.add_argument(
+            "--mpi_hostnames",
+            help="Specify MPI hostnames string of the form --mpi_hostnames host1,host2,host3",
+            dest="mpi_hostnames", default=None)
+
         self._common_arg_parser.add_argument(
             "-d", "--data-location",
             help="Specify the location of the data. If this parameter is not "
@@ -188,6 +193,18 @@ def _define_args(self):
                  "with --accuracy-only and --mode=inference.",
             dest="output_results", action="store_true")
 
+        self._common_arg_parser.add_argument(
+            "--optimized-softmax",
+            help="Use tf.nn.softmax as opposed to basic math ops",
+            dest="optimized_softmax", choices=["True", "False"],
+            default=True)
+
+        self._common_arg_parser.add_argument(
+            "--experimental-gelu",
+            help="use tf.nn.gelu as opposed to basic math ops",
+            dest="experimental_gelu", choices=["True", "False"],
+            default=False)
+
         # Note this can't be a normal boolean flag, because we need to know when the user
         # does not explicitly set the arg value so that we can apply the appropriate
         # default value, depending on the the precision.
@@ -222,6 +239,12 @@ def _define_args(self):
                  "'models/benchmarks/common/tensorflow/logs' if no path is specified.",
             default="/models/benchmarks/common/tensorflow/logs")
 
+        self._common_arg_parser.add_argument(
+            "--tf-serving-version",
+            help="TF serving version to run the script with"
+                 "'master' if no value is specified.",
+            default="master")
+
         # Allow for additional command line args after --
         self._common_arg_parser.add_argument(
             "model_args", nargs="*",

benchmarks/common/base_model_init.py

@@ -57,8 +57,28 @@ def __init__(self, args, custom_args=[], platform_util=None):
         if not platform_util:
             raise ValueError("Did not find any platform info.")
 
-        # Invoke mpirun if mpi_num_processes env is not None
-        if os.environ["MPI_NUM_PROCESSES"] != "None":
+        # use case: bare-metal with openmpi, horovod and multi-node
+        if os.environ["MPI_HOSTNAMES"] != "None" and ("DOCKER" not in os.environ or os.environ["DOCKER"] == "False"):
+            if os.environ["MPI_NUM_PROCESSES"] != "None":
+                try:
+                    # slots per host calculation using MPI_NUM_PROCESSES and number of hosts
+                    host_names = os.environ["MPI_HOSTNAMES"]
+                    number_of_hosts = len(host_names.split(','))
+                    slots_per_host = int(int(os.environ["MPI_NUM_PROCESSES"]) / number_of_hosts)
+                    host_names = ",".join([host + ":" + str(slots_per_host) for host in host_names.split(',')])
+                    # see the [examples](https://horovod.readthedocs.io/en/latest/mpirun.html) for the mca flags
+                    self.python_exe = "mpirun " + " -x LD_LIBRARY_PATH " + " -x PYTHONPATH " \
+                        + " --allow-run-as-root -n " + os.environ["MPI_NUM_PROCESSES"] + " -H " + host_names \
+                        + " -mca pml ob1 -mca btl ^openib -mca btl_tcp_if_exclude " \
+                          "lo,docker0 --bind-to none --map-by slot " \
+                        + self.python_exe
+                except Exception as exception:
+                    raise ValueError("Caught exception calculating slots per host {}".format(str(exception)))
+            else:
+                raise ValueError("MPI_NUM_PROCESSES is required for MPI_HOSTNAMES and will be split evenly across the "
+                                 "hosts.")
+        # use case: docker with openmpi, single-node, multi-instance
+        elif os.environ["MPI_NUM_PROCESSES"] != "None":
             if os.environ["MPI_NUM_PROCESSES_PER_SOCKET"] == "1":
                 # Map by socket using OpenMPI by default (PPS=1).
                 self.python_exe = "mpirun --allow-run-as-root -n " + os.environ["MPI_NUM_PROCESSES"] \

benchmarks/common/tensorflow/container_init.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) 2020 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# This file includes runtime installs for model containers
+
+apt-get install numactl -y
+