Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update converter build instructions #487

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions docs/compute-engine/build/converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,18 @@

To build the pip package with the converter, Bazel needs to know for which
Python version to build the package.
If you are using a Python virtual environment, please run bazel from that
environment, as Bazel will automatically detect the Python version of the
virtual environment.

Bazel will detect the Python version the first time you try to build anything
that requires Python. If you have run Bazel outside of your virtual
environment and now want to run it in the environment, you have to make Bazel
re-detect the Python version. This requires a full clean, a normal `clean` is
not enough to trigger the redetection.:
You can set this through the environment variable `TF_PYTHON_VERSION`, for example

```bash
bazel clean --expunge
export TF_PYTHON_VERSION="3.12"
```

Install the TensorFlow pip package to ensure that all required dependencies
are available:
If you have run Bazel without setting the python version, or you want to change
the version, you have to make Bazel re-detect the Python version. This requires
a full clean, a normal `clean` is not enough to trigger the redetection:

```bash
pip install tensorflow
bazel clean --expunge
```

To build the pip package, now simply run
Expand All @@ -30,8 +23,15 @@ bazel build :build_pip_pkg
bazel-bin/build_pip_pkg artifacts
```

The script stores the wheel file in the `artifacts/` directory located in the
LCE root directory. To install the PIP package:
If you get an error about a `@pypi_lce_XXXXX` package, then it can help to
re-generate the `requirements.txt` file that Bazel uses. To do so, run

```bash
pip-compile --allow-unsafe --no-emit-index-url --strip-extras larq_compute_engine/requirements.in
```

After the bazel build is finished, the script stores the wheel file in the
`artifacts/` directory located in the LCE root directory. To install the PIP package:

```bash
pip install artifacts/*.whl
Expand Down
11 changes: 6 additions & 5 deletions docs/compute-engine/build/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ container, and we recommend first following
to try and build LCE outside of a Docker container.

There are scenarios in which building in the Docker container is preferred:

- To build a [`manylinux2010`](https://www.python.org/dev/peps/pep-0571/)-compatible pip package
- To build a benchmark binary for Android
- When the build outside of the container fails and can't be fixed.
Expand All @@ -17,13 +18,13 @@ to migrate the build targets in-between the host machine and the container.
To be able to build the LCE converter's
[`manylinux2010`](https://www.python.org/dev/peps/pep-0571/) compatible PIP
package, we need to use the
[`tensorflow:custom-op-ubuntu16`](https://hub.docker.com/r/tensorflow/tensorflow)
image.
[`tensorflow/build:2.16-python3.12`](https://hub.docker.com/r/tensorflow/tensorflow)
image (replace the Python version number with the desired version).

First, download the Docker image:

```bash
docker pull tensorflow/tensorflow:custom-op-ubuntu16
docker pull tensorflow/build:2.16-python3.12
```

Clone the LCE repository on the host machine:
Expand All @@ -38,7 +39,7 @@ directory inside the container:

```bash
docker run -it -v $PWD/lce-volume:/tmp/lce-volume \
-w /tmp/lce-volume tensorflow/tensorflow:custom-op-ubuntu16 /bin/bash
-w /tmp/lce-volume tensorflow/build:2.16-python3.12 /bin/bash
```

Now, you will be able to build bazel targets inside the container, by following
Expand All @@ -58,7 +59,7 @@ change the Python version to the desired version:

```bash
cd /tmp/lce-volume
export PYTHON_VERSION="3.8"
export TF_PYTHON_VERSION="3.12"
.github/tools/release_linux.sh
```

Expand Down