SNOW-1799654: Improve MANIFEST.in File for Better Packaging Consistency

[shashankrushiya]

What is the current behavior?

Currently, the MANIFEST.in file for the snowflake-snowpark-python package does not fully follow best practices for Python packaging. Essential files like README.md, LICENSE.txt, and source files with typing information are already included, but there are a few inconsistencies in the structure, and some unnecessary files are excluded in a less organized way.

What is the desired behavior?

The desired behavior is to have a streamlined and well-structured MANIFEST.in file that includes all essential documentation and source files while excluding files only needed for development (e.g., .pre-commit-config.yaml, CONTRIBUTING.md, license_header.txt, etc.). This includes adding support for *.pyi files within src/snowflake/snowpark, which helps ensure compatibility and type safety for developers using type hints.

How would this improve snowflake-snowpark-python?

This update would improve the package by:

1. Ensuring that essential files and source code are consistently included, allowing for a more predictable and complete package for
   users.
2. Simplifying the distribution by excluding unnecessary files, which reduces the package size and avoids potential confusion with
   non-essential files.
3. Enhancing the development experience with better-organized files, leading to a cleaner, more manageable project structure.

References, Other Background

This request is motivated by standard best practices in Python packaging and aims to improve both the user and developer experience by refining the file inclusion and exclusion rules in the MANIFEST.in.

These changes are intended to align the snowflake-snowpark-python package with industry standards for efficient, minimal, and complete packaging.