Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolving objectives issue introduced with introduction of pass search #1585

Merged
merged 1 commit into from
Feb 1, 2025

Resolving objectives issue introduced with introduction of pass search

e641d84
Select commit
Loading
Failed to load commit list.
Merged

Resolving objectives issue introduced with introduction of pass search #1585

Resolving objectives issue introduced with introduction of pass search
e641d84
Select commit
Loading
Failed to load commit list.
Azure Pipelines / Olive CI succeeded Feb 1, 2025 in 27m 56s

Build #20250131.6 had test failures

Details

Tests

  • Failed: 3 (0.09%)
  • Passed: 3,034 (92.67%)
  • Other: 237 (7.24%)
  • Total: 3,274
Code coverage

  • 8537 of 37230 lines covered (22.93%)

Annotations

Check failure on line 1 in test_bert[bert_ptq_cpu.json-local_system-joint-tpe]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_bert[bert_ptq_cpu.json-local_system-joint-tpe]

AssertionError: footprints is empty. The search must have failed for all accelerator specs.
Raw output
sampler = 'tpe', execution_order = 'joint', system = 'local_system'
olive_json = 'bert_ptq_cpu.json'

    @pytest.mark.parametrize("sampler", ["tpe"])
    @pytest.mark.parametrize("execution_order", ["joint"])
    @pytest.mark.parametrize("system", ["local_system"])
    @pytest.mark.parametrize("olive_json", ["bert_ptq_cpu.json"])
    def test_bert(sampler, execution_order, system, olive_json):
        from olive.workflows import run as olive_run
    
        olive_config = patch_config(olive_json, sampler, execution_order, system)
    
        footprint = olive_run(olive_config, tempdir=os.environ.get("OLIVE_TEMPDIR", None))
>       check_output(footprint)

D:\a\_work\1\s\examples\test\local\test_bert_ptq_cpu.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
D:\a\_work\1\s\examples\test\utils.py:20: in check_output
    assert_nodes(footprints)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

footprints = {}

    def assert_nodes(footprints):
>       assert footprints, "footprints is empty. The search must have failed for all accelerator specs."
E       AssertionError: footprints is empty. The search must have failed for all accelerator specs.

D:\a\_work\1\s\examples\test\utils.py:25: AssertionError

Check failure on line 1 in test_optimum_conversion_pass[extra_args1]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_optimum_conversion_pass[extra_args1]

huggingface_hub.errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/hf-internal-testing/tiny-random-gptj/tree/main?recursive=True&expand=False
Raw output
response = <Response [504]>, endpoint_name = None

    def hf_raise_for_status(response: Response, endpoint_name: Optional[str] = None) -> None:
        """
        Internal version of `response.raise_for_status()` that will refine a
        potential HTTPError. Raised exception will be an instance of `HfHubHTTPError`.
    
        This helper is meant to be the unique method to raise_for_status when making a call
        to the Hugging Face Hub.
    
    
        Example:
        ```py
            import requests
            from huggingface_hub.utils import get_session, hf_raise_for_status, HfHubHTTPError
    
            response = get_session().post(...)
            try:
                hf_raise_for_status(response)
            except HfHubHTTPError as e:
                print(str(e)) # formatted message
                e.request_id, e.server_message # details returned by server
    
                # Complete the error message with additional information once it's raised
                e.append_to_message("\n`create_commit` expects the repository to exist.")
                raise
        ```
    
        Args:
            response (`Response`):
                Response from the server.
            endpoint_name (`str`, *optional*):
                Name of the endpoint that has been called. If provided, the error message
                will be more complete.
    
        <Tip warning={true}>
    
        Raises when the request has failed:
    
            - [`~utils.RepositoryNotFoundError`]
                If the repository to download from cannot be found. This may be because it
                doesn't exist, because `repo_type` is not set correctly, or because the repo
                is `private` and you do not have access.
            - [`~utils.GatedRepoError`]
                If the repository exists but is gated and the user is not on the authorized
                list.
            - [`~utils.RevisionNotFoundError`]
                If the repository exists but the revision couldn't be find.
            - [`~utils.EntryNotFoundError`]
                If the repository exists but the entry (e.g. the requested file) couldn't be
                find.
            - [`~utils.BadRequestError`]
                If request failed with a HTTP 400 BadRequest error.
            - [`~utils.HfHubHTTPError`]
                If request failed for a reason not listed above.
    
        </Tip>
        """
        try:
>           response.raise_for_status()

C:\hostedtoolcache\windows\Python\3.10.16\x64\lib\site-packages\huggingface_hub\utils\_http.py:406: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Response [504]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/hf-internal-testing/tiny-random-gptj/tree/main?recursive=True&expand=False

C:\hostedtoolcache\windows\Pyth

Check failure on line 1 in test_optimum_conversion_pass_with_components[None-None-None]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_optimum_conversion_pass_with_components[None-None-None]

requests.exceptions.ReadTimeout: (ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 1bd08e17-7f26-477d-94ac-b6668b46949e)')
Raw output
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x0000026A40443190>
conn = <urllib3.connection.HTTPSConnection object at 0x0000026A40441510>
method = 'GET'
url = '/hf-internal-testing/tiny-random-OPTForCausalLM/resolve/main/config.json'
body = None
headers = {'user-agent': 'unknown/None; hf_hub/0.28.1; python/3.10.14; torch/2.6.0; transformers/4.44.2; session_id/d4017ed8c243...gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'X-Amzn-Trace-Id': '1bd08e17-7f26-477d-94ac-b6668b46949e'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
timeout = Timeout(connect=10, read=10, total=None), chunked = False
response_conn = <urllib3.connection.HTTPSConnection object at 0x0000026A40441510>
preload_content = False, decode_content = False, enforce_content_length = True

    def _make_request(
        self,
        conn: BaseHTTPConnection,
        method: str,
        url: str,
        body: _TYPE_BODY | None = None,
        headers: typing.Mapping[str, str] | None = None,
        retries: Retry | None = None,
        timeout: _TYPE_TIMEOUT = _DEFAULT_TIMEOUT,
        chunked: bool = False,
        response_conn: BaseHTTPConnection | None = None,
        preload_content: bool = True,
        decode_content: bool = True,
        enforce_content_length: bool = True,
    ) -> BaseHTTPResponse:
        """
        Perform a request on a given urllib connection object taken from our
        pool.
    
        :param conn:
            a connection from one of our connection pools
    
        :param method:
            HTTP request method (such as GET, POST, PUT, etc.)
    
        :param url:
            The URL to perform the request on.
    
        :param body:
            Data to send in the request body, either :class:`str`, :class:`bytes`,
            an iterable of :class:`str`/:class:`bytes`, or a file-like object.
    
        :param headers:
            Dictionary of custom headers to send, such as User-Agent,
            If-None-Match, etc. If None, pool headers are used. If provided,
            these headers completely replace any pool-specific headers.
    
        :param retries:
            Configure the number of retries to allow before raising a
            :class:`~urllib3.exceptions.MaxRetryError` exception.
    
            Pass ``None`` to retry until you receive a response. Pass a
            :class:`~urllib3.util.retry.Retry` object for fine-grained control
            over different types of retries.
            Pass an integer number to retry connection errors that many times,
            but no other types of errors. Pass zero to never retry.
    
            If ``False``, then retries are disabled and any exception is raised
            immediately. Also, instead of raising a MaxRetryError on redirects,
            the redirect response will be returned.
    
        :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
    
        :param timeout:
            If specified, overrides the default timeout for this one
            request. It may be a float (in seconds) or an instance of
            :class:`urllib3.util.Timeout`.
    
        :param chunked:
            If True, urllib3 will send the body using chunked transfer
            encoding. Otherwise, urllib3 will send the body using the standard
            content-length form. Defaults to False.
    
        :param response_conn:
            Set this to ``None`` if you will handle releasing the connection or
            set the connection to have the response release it.
    
        :param preload_content:
          If True, the response's body will be preloaded during construction.
    
        :param decode_content:
            If True, will attempt to decode the body based on the
            'content-encoding' header.
    
        :param enforce_content_length:
            Enforce content length checking. Body returned by server must match
            value of Content-Length header, if p