Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix outdated RAPIDS versions #501

Merged
merged 1 commit into from
Jan 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,10 @@ The RAPIDS versions for things like container images and install instructions ar
```python
versions = {
"stable": {
"rapids_container": "nvcr.io/nvidia/rapidsai/base:24.06-cuda11.8-py3.10",
"rapids_container": "nvcr.io/nvidia/rapidsai/base:24.12-cuda12.5-py3.12",
},
"nightly": {
"rapids_container": "rapidsai/base:24.08a-cuda12.5-py3.11",
"rapids_container": "rapidsai/base:25.02a-cuda12.5-py3.12",
},
}
```
Expand Down
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec:
spec:
initContainers:
- name: prepull-rapids
image: us-central1-docker.pkg.dev/nv-ai-infra/rapidsai/rapidsai/base:23.08-cuda12.0-py3.10
image: us-central1-docker.pkg.dev/nv-ai-infra/rapidsai/rapidsai/base:example
command: ["sh", "-c", "'true'"]
containers:
- name: pause
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -222,9 +222,9 @@
"In order to steam the container image to the GKE nodes our image needs to be stored in [Google Cloud Artifact Registry](https://cloud.google.com/artifact-registry/) in the same region as our cluster.\n",
"\n",
"```console\n",
"$ docker pull rapidsai/base:24.04-cuda11.8-py3.10\n",
"$ docker pull {{ rapids_container }}\n",
"\n",
"$ docker tag rapidsai/base:24.04-cuda11.8-py3.10 REGION-docker.pkg.dev/PROJECT/REPO/IMAGE:TAG\n",
"$ docker tag {{ rapids_container }} REGION-docker.pkg.dev/PROJECT/REPO/IMAGE:TAG\n",
"\n",
"$ docker push REGION-docker.pkg.dev/PROJECT/REPO/IMAGE:TAG\n",
"```\n",
Expand All @@ -243,7 +243,7 @@
},
{
"cell_type": "code",
"execution_count": 39,
"execution_count": 3,
"metadata": {},
"outputs": [
{
Expand All @@ -266,7 +266,7 @@
" spec:\n",
" initContainers:\n",
" - name: prepull-rapids\n",
" image: rapidsai/rapidsai-core:23.02-cuda11.8-runtime-ubuntu22.04-py3.10\n",
" image: us-central1-docker.pkg.dev/nv-ai-infra/rapidsai/rapidsai/base:example\n",
" command: [\"sh\", \"-c\", \"'true'\"]\n",
" containers:\n",
" - name: pause\n",
Expand All @@ -275,7 +275,7 @@
}
],
"source": [
"! cat image-prepuller.yaml"
"! cat ./image-prepuller.yaml"
]
},
{
Expand Down Expand Up @@ -575,7 +575,7 @@
"\n",
"cluster = KubeCluster(\n",
" name=\"rapids-dask-1\",\n",
" image=\"rapidsai/rapidsai-core:23.02-cuda11.8-runtime-ubuntu22.04-py3.10\", # Replace me with your cached image\n",
" image=\"{{ rapids_container }}\", # Replace me with your cached image\n",
" n_workers=4,\n",
" resources={\"limits\": {\"nvidia.com/gpu\": \"1\"}},\n",
" env={\"EXTRA_PIP_PACKAGES\": \"gcsfs\"},\n",
Expand Down Expand Up @@ -1117,7 +1117,7 @@
"\n",
" with KubeCluster(\n",
" name=\"rapids-dask-\" + uuid.uuid4().hex[:5],\n",
" image=\"rapidsai/rapidsai-core:23.02-cuda11.8-runtime-ubuntu22.04-py3.10\", # Replace me with your cached image\n",
" image=\"{{ rapids_container }}\", # Replace me with your cached image\n",
" n_workers=2,\n",
" resources={\"limits\": {\"nvidia.com/gpu\": \"1\"}},\n",
" env={\"EXTRA_PIP_PACKAGES\": \"gcsfs\"},\n",
Expand Down Expand Up @@ -1305,7 +1305,7 @@
"\n",
"cluster_spec = make_cluster_spec(\n",
" name=\"mock-jupyter-cluster\",\n",
" image=\"rapidsai/rapidsai-core:23.02-cuda11.8-runtime-ubuntu22.04-py3.10\", # Replace me with your cached image\n",
" image=\"{{ rapids_container }}\", # Replace me with your cached image\n",
" n_workers=1,\n",
" resources={\"limits\": {\"nvidia.com/gpu\": \"1\"}, \"requests\": {\"cpu\": \"50m\"}},\n",
" env={\"EXTRA_PIP_PACKAGES\": \"gcsfs dask-kubernetes\"},\n",
Expand Down Expand Up @@ -2234,7 +2234,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
"version": "3.12.8"
},
"vscode": {
"interpreter": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ spec:
fsGroup: 0
containers:
- name: rapids-notebook
image: us-central1-docker.pkg.dev/nv-ai-infra/rapidsai/rapidsai/base:23.08-cuda12.0-py3.10
image: us-central1-docker.pkg.dev/nv-ai-infra/rapidsai/rapidsai/base:example
resources:
limits:
nvidia.com/gpu: 1
Expand Down
54 changes: 7 additions & 47 deletions source/examples/rapids-sagemaker-higgs/notebook.ipynb

Large diffs are not rendered by default.

373 changes: 29 additions & 344 deletions source/examples/rapids-sagemaker-hpo/notebook.ipynb

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -6481,7 +6481,8 @@
"outputs": [],
"source": [
"# Choose the same RAPIDS image you used for launching the notebook session\n",
"rapids_image = \"rapidsai/notebooks:23.10a-cuda12.0-py3.10\"\n",
"rapids_image = \"{{ rapids_notebooks_container }}\"\n",
"\n",
"# Use the number of worker nodes in your Kubernetes cluster.\n",
"n_workers = 2\n",
"# Bucket that contains the processed data pickles\n",
Expand Down Expand Up @@ -8009,7 +8010,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
"version": "3.12.8"
}
},
"nbformat": 4,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ runcmd:
- systemctl restart docker

# Attempt to run a RAPIDS container to download the container layers and decompress them
- 'docker run --net=host --gpus=all --shm-size=256m rapidsai/rapidsai:cuda11.2-runtime-ubuntu18.04-py3.8 dask-scheduler --version'
- 'docker run --net=host --gpus=all --shm-size=256m rapidsai/base:latest --version'
106 changes: 20 additions & 86 deletions source/examples/xgboost-azure-mnmg-daskcloudprovider/notebook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -322,7 +323,6 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -609,7 +609,6 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -638,50 +637,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating scheduler instance\n",
"Assigned public IP\n",
"Network interface ready\n",
"Using Marketplace VM image with a Plan\n",
"Creating VM\n",
"Created VM dask-92c5978e-scheduler\n",
"Waiting for scheduler to run at 4.155.2.188:8786\n",
"Scheduler is running\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/skirui/anaconda3/envs/rapids-23.08/lib/python3.10/contextlib.py:142: UserWarning: Creating your cluster is taking a surprisingly long time. This is likely due to pending resources. Hang tight! \n",
" next(self.gen)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Creating worker instance\n",
"Creating worker instance\n",
"Network interface ready\n",
"Using Marketplace VM image with a Plan\n",
"Creating VM\n",
"Network interface ready\n",
"Using Marketplace VM image with a Plan\n",
"Creating VM\n",
"Created VM dask-92c5978e-worker-54f8d057\n",
"Created VM dask-92c5978e-worker-9f9a9c9b\n",
"CPU times: user 1.22 s, sys: 189 ms, total: 1.41 s\n",
"Wall time: 6min 58s\n"
]
}
],
"outputs": [],
"source": [
"%%time\n",
"\n",
Expand Down Expand Up @@ -1398,6 +1356,7 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -1438,7 +1397,9 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"### Step 3.b: Data Cleanup, Enhancement and Persisting Scripts"
]
Expand Down Expand Up @@ -1711,6 +1672,7 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand All @@ -1726,25 +1688,9 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/skirui/anaconda3/envs/rapids-23.08/lib/python3.10/site-packages/dask/dataframe/io/parquet/core.py:411: FutureWarning: The `chunksize` argument is deprecated, and will be removed in a future release. Setting the `blocksize` argument instead. Please see documentation on the `blocksize` argument for more information.\n",
" warnings.warn(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall clock time taken for ETL and persisting : 83.1002215759363 s\n"
]
}
],
"outputs": [],
"source": [
"tic = timer()\n",
"X_train, y_train, X_infer, y_infer = taxi_data_loader(\n",
Expand Down Expand Up @@ -2101,7 +2047,9 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"### Step 4.a: Set training Parameters\n",
"\n",
Expand Down Expand Up @@ -2137,6 +2085,7 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -2177,6 +2126,7 @@
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
Expand Down Expand Up @@ -2427,25 +2377,9 @@
},
{
"cell_type": "code",
"execution_count": 34,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/skirui/anaconda3/envs/rapids-23.08/lib/python3.10/site-packages/dask/dataframe/core.py:7047: FutureWarning: Meta is not valid, `map_partitions` and `map_overlap` expects output to be a pandas object. Try passing a pandas object as meta or a dict or tuple representing the (name, dtype) of the columns. In the future the meta you passed will not work.\n",
" warnings.warn(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall clock time taken for this cell : 5.638823717948981 s\n"
]
}
],
"outputs": [],
"source": [
"tic = timer()\n",
"predictions = X_infer.map_partitions(\n",
Expand Down Expand Up @@ -2537,9 +2471,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "rapids-23.10",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "rapids-23.10"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -2551,7 +2485,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
"version": "3.12.8"
}
},
"nbformat": 4,
Expand Down
Loading