Rapids on snowflake deployment

[ncclementi]

Closes the deployment part of #419

For the "Show reading data from Snowflake into a cudf dataframe" , I created this issue for tracking purposes #494

This is WIP but it's almost there for review, need a few more sections with screenshots.

TODO:

• Figure out failures in CI. Not sure why I'm getting the failures, I can build locally although I get a warning that says< I assume we fail on warning sin CI. It seems it doesn't like the $. Any suggestions @jacobtomlinson ?

[ncclementi]

Can't seem to find a fix for the failure:

Looks like it is sort of a known issue sphinx-doc/sphinx#3175 but the suggested fix, I found relies on this sphinx syntax and when trying this, it doesn't render as expected. Any ideas?

.. code-block:: sql :force: 
   USE ROLE CONTAINER_USER_ROLE;
   CALL SYSTEM$REGISTRY_LIST_IMAGES('/CONTAINER_HOL_DB/PUBLIC/IMAGE_REPO');

[ncclementi]

@jacobtomlinson I think this is ready for a first review. That being said while checking if the volume mount instructions where working correctly I realized that the mount won't work unless we have workspace as a directory created.

We have two options here.

1. Just mount /home/rapids so all the changes made to the notebooks in there would remain, as well as things added.
2. Modify the Dockerfilesuch that we create this workspace. I think it would be something like this?
   Any strong preferences here?

FROM rapidsai/notebooks:25.02a-cuda11.8-py3.11-amd64

RUN pip install "snowflake-snowpark-python[pandas]" snowflake-connector-python

USER rapids
WORKDIR /home/rapids/workspace

TODO

• Add note on volume mount and how to use after making decision.
• Update Dockerfile image?, currently using nightly.

[jacobtomlinson]

I struggled with some permissions error when following these instructions. Could you take a look and see wha I'm doing wrong?

Also to answer your question I wouldn't set the working directory I would just create an empty directory.

We don't need to creater the directory ahead of time. Setting mountPath: /home/rapids/notebooks/workspace works for me.

[jacobtomlinson]

This will go stale. We usually use templating to ensure it always shows the correct RAPIDS image. The requirements to use Python 3.11 may add a little complexity here.

[ncclementi]

Yes, the requirement of python 3.11 and explicit amd64 platform will make the templating a bit more complicated. I'll take a look into it

[jacobtomlinson]

I managed to test the instructions end-to-end and I managed to get a working notebook instance and ran some cudf code 🎉.

I found a few more things that need tweaking though and left a few more comments.

[jacobtomlinson]

This line didn't work for me, it just said the service didn't exist, but the rest of the commands did so I think I cleaned up successfully.

[ncclementi]

Interesting, this is as suggested by snowflake. Did your service had a different name or something? I'll give this another try after again soon and check.

[jacobtomlinson]

I am still seeing this

[ncclementi]

I can replicate it,I guess this is a bug on their snowflake quickstart.

[ncclementi]

I had a good amount of trouble launching things again, I think there was some cleanup and role mix up instructions that didn't remove everything. But hopefully that's sorted.

Yet TODO:

• resolve templating of rapids image in the doc to point to latest notebook but on py 3.11 and amd platform
• Try to add a file to the workspace, I modify the spec where I change the uid: 1001.
• add a paragraph about the volume mount.

[ncclementi]

This should be ready now for final review, no TODOs left. Looks like I mess up something when rebasing, I think I fix it and it should be fine.

Do we have squashing here when merging or do I need to cleanup the commits?

[jacobtomlinson]

I ran through end-to-end and everything works great for me. I added a couple of comments but generally this is great.

[jacobtomlinson]

I am still seeing this

[jacobtomlinson]

This looks great. I think we are at a point where we can merge this and follow up in additional PRs.