Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global CMake for tutorials #91

Open
wants to merge 51 commits into
base: main
Choose a base branch
from

Conversation

cedricchevalier19
Copy link
Member

@cedricchevalier19 cedricchevalier19 commented May 13, 2024

The goal is to provide a joint built for all exercises (and their solutions).

This way, compilation is straightforward for IDE users.

  • Currently, most of the exercises are built using the top-level CMake.

  • However, not all builds make sense, as CUDA builds are wrong for the first exercises. I have put configuration warnings when Kokkos is not configured correctly, as explaining why it is not working is still valuable.

  • I have also refactored the fetch_content implementation: it is now called in an overloaded find_package to keep CMakeLists.txt as standard as possible.

The user can set three CMake variables to control the build system:

  • CMAKE_DISABLE_FIND_PACKAGE_Kokkos: to force a build of a custom Kokkos and ignore any installed Kokkos
  • CMAKE_REQUIRE_FIND_PACKAGE_Kokkos: to force finding an installed Kokkos and to fail if not found
  • KokkosTutorials_KOKKOS_SOURCE_PATH: to specify a Kokkos source directory and by-pass fetch_content when building a custom Kokkos.

I can improve the global CMake by using external_project for the first exercises to "sandbox" their builds and allow a custom Kokkos.

@cedricchevalier19 cedricchevalier19 mentioned this pull request Jun 5, 2024
4 tasks
@cedricchevalier19
Copy link
Member Author

One difficulty is that the first 3-4 exercises are not perfect Kokkos code and can have issue when a Kokkos device backend is enabled.

@cedricchevalier19 cedricchevalier19 self-assigned this Jun 5, 2024
@cedricchevalier19 cedricchevalier19 marked this pull request as ready for review June 10, 2024 14:18
@cedricchevalier19
Copy link
Member Author

I will add Kokkos-Kernels tutorials before merge, but I want to know other thoughts before continuing.

A lot of files are touched, but their modifications are pretty similar.

KokkosTutorials_KOKKOS_SOURCE_DIR can be used to point to Kokkos source and avoid downloading.
@cedricchevalier19 cedricchevalier19 mentioned this pull request Nov 15, 2024
6 tasks
@pzehner
Copy link

pzehner commented Feb 25, 2025

I think this was a good improvement for the maintenance of the tutorials. It's sad to see this PR stalled. Could the current improvements be incorporated, and the moot points be tackled down by another PR?

By the way, we used a very similar approach for the CExA Kokkos tutorials and it serves our needs just right.

@dalg24
Copy link
Member

dalg24 commented Feb 25, 2025

The conflicts would need to be resolved.
I would want to double check that when building individual exercises we do not re-download the source as it was the case last summer.

@pzehner
Copy link

pzehner commented Feb 25, 2025

The current common.cmake forces FetchContent_Declare to use a source directory, so that it would not re-download it (this is actually an option passed to ExternalProject_Add). And it uses a directory in the source tree.

The new FindKokkos.cmake with the same option uses a directory in the build tree instead, which makes it always downloaded.

Note that the download directory can be set instead.

To re-use the already downloaded sources, we can either use a directory somewhere in the source tree that makes sense (and would be in .gitignore, this is kinda what we use in the CExA tutorial), or in the system temporary directory (by instance $ENV{TMP}/kokkos).

endif ()

# Where to find Kokkos' source code. This might be set by the user.
set(KokkosTutorials_KOKKOS_SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}/dep/kokkos" CACHE PATH "Description for KokkosTutorials_KOKKOS_SOURCE_DIR")
Copy link

@pzehner pzehner Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line should be changed to use a unique directory not in the build tree.

@cedricchevalier19
Copy link
Member Author

I will update against the current main branch and check how we download Kokkos.

…r19/cmake

# Conflicts:
#	Exercises/common.cmake
#	Exercises/fortran-kokkosinterface/Begin/CMakeLists.txt
#	Exercises/fortran-kokkosinterface/Solution/CMakeLists.txt
#	Exercises/instances/CMakeLists.txt
#	Exercises/scatter_view/Begin/CMakeLists.txt
#	Exercises/subview/Begin/exercise_subview_begin.cpp
#	Exercises/subview/Solution/exercise_subview_solution.cpp
#	Exercises/virtualfunction/Begin/Makefile
#	Exercises/virtualfunction/Solution/Makefile
Before this commit, we had a fake "find_package(Kokkos)" to make tutorial CMakeLists.txt minimal.

However, using builtin CMake features like `CMAKE_DISABLE_FIND_PACKAGE_<>` or `CMAKE_REQUIRE_FIND_PACKAGE_<>` was not working properly.

This commit use a simple `include(../cmake/SetUpKokkos.cmake)`, and it
is this file that can be a fine example of how to integrate Kokkos in a project.
This way we can promote the use of SHA256
@cedricchevalier19
Copy link
Member Author

@JBludau and @dalg24, I finally decided to follow your advice and I've factorized the code checking for a gpu. Furthermore, to apply the good practice described in the documentation to use either find_package and fetch_content, I have reverted to hide the magic behind a custom FindKokkos.cmake.

I can update the CI to use the new global CMake but I think it is better to leave it to a separate PR, this one being already too large.

I do not understand the issue with Windows but it looks similar to the one of #107.

else ()
if (EXISTS ${KokkosTutorials_KOKKOS_SOURCE_DIR})
add_subdirectory(${KokkosTutorials_KOKKOS_SOURCE_DIR} Kokkos)
else ()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave early here too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking but do you know how does that play with multi-config generators?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will test and report it.

@pzehner
Copy link

pzehner commented Feb 26, 2025

So, creating a custom FindKokkos.cmake is not a good idea?

else ()
if (EXISTS ${KokkosTutorials_KOKKOS_SOURCE_DIR})
add_subdirectory(${KokkosTutorials_KOKKOS_SOURCE_DIR} Kokkos)
else ()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking but do you know how does that play with multi-config generators?

cedricchevalier19 and others added 7 commits February 26, 2025 16:50
Co-authored-by: Paul Zehner <[email protected]>
Co-authored-by: Paul Zehner <[email protected]>
Co-authored-by: Paul Zehner <[email protected]>
…r19/cmake

# Conflicts:
#	Exercises/tasking/Begin/CMakeLists.txt
#	Exercises/tasking/Solution/CMakeLists.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants