V2 Performance Signal Detected by TorchBench CI on '2.5.0.dev20240828+cu124'

[xuzhao9]

TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240606+cu121

Base PyTorch commit: ffaea656b5d8ff6518669494cc8f664b94f8e8b1

Affected PyTorch version: 2.5.0.dev20240828+cu124

Affected PyTorch commit: 55236d0cb7d183f2b64982d8718e9012ae8a40d8

Affected Tests:

• test_train[Super_SloMo-cuda-eager]: +14.63055%
• test_eval[Super_SloMo-cuda-eager]: +11.06725%
• test_train[Super_SloMo-cuda-jit]: +14.74649%
• test_eval[Super_SloMo-cuda-jit]: +11.21490%
• test_eval[alexnet-cuda-eager]: +10.52737%
• test_eval[alexnet-cuda-jit]: +13.68942%
• test_train[dcgan-cuda-eager]: +63.67888%
• test_train[dcgan-cuda-jit]: +63.53697%
• test_eval[pyhpc_equation_of_state-cpu-jit]: +12.71203%
• test_eval[pytorch_stargan-cpu-jit]: -17.93173%
• test_eval[resnext50_32x4d-cpu-jit]: +22.56125%
• test_eval[shufflenet_v2_x1_0-cpu-eager]: +15.75789%
• test_eval[shufflenet_v2_x1_0-cuda-jit]: -27.53080%
• test_eval[timm_regnet-cuda-eager]: +170.03271%
• test_eval[timm_regnet-cuda-jit]: +355.30840%
• test_train[yolov3-cuda-eager]: -9.82974%

cc @xuzhao9

Result json:

{
  "start": "ffaea656b5d8ff6518669494cc8f664b94f8e8b1",
  "end": "55236d0cb7d183f2b64982d8718e9012ae8a40d8",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "91df66ee740",
      "commit1_time": "2024-08-06 04:35:06 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.10881321523338557
      },
      "commit2": "728374d7f7f",
      "commit2_time": "2024-08-06 04:37:15 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07583622010424733
      }
    },
    {
      "commit1": "345bea01dc6",
      "commit1_time": "2024-08-06 02:35:45 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0767341036349535
      },
      "commit2": "39c9b75a680",
      "commit2_time": "2024-08-06 02:47:35 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08687919285148382
      }
    },
    {
      "commit1": "4e610924d4d",
      "commit1_time": "2024-08-06 03:47:58 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08636773796752095
      },
      "commit2": "4260f365baa",
      "commit2_time": "2024-08-06 03:58:43 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07451506610959768
      }
    },
    {
      "commit1": "4260f365baa",
      "commit1_time": "2024-08-06 03:58:43 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07451506610959768
      },
      "commit2": "91df66ee740",
      "commit2_time": "2024-08-06 04:35:06 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.10881321523338557
      }
    },
    {
      "commit1": "7045bc5a77b",
      "commit1_time": "2024-08-06 16:59:53 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08418234270066023
      },
      "commit2": "ffdf48e63b9",
      "commit2_time": "2024-08-06 17:03:17 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06646580919623375
      }
    },
    {
      "commit1": "ffdf48e63b9",
      "commit1_time": "2024-08-06 17:03:17 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06646580919623375
      },
      "commit2": "a94c441e48e",
      "commit2_time": "2024-08-06 17:03:39 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.10409250548109412
      }
    },
    {
      "commit1": "a94c441e48e",
      "commit1_time": "2024-08-06 17:03:39 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.10409250548109412
      },
      "commit2": "406b50835b6",
      "commit2_time": "2024-08-06 17:03:39 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07851395206525921
      }
    },
    {
      "commit1": "406b50835b6",
      "commit1_time": "2024-08-06 17:03:39 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07851395206525921
      },
      "commit2": "f50621989b8",
      "commit2_time": "2024-08-06 17:03:39 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.067367684841156
      }
    }
  ]
}

Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/10601302246