Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: cluster_creation failed #143042

Open
cockroach-teamcity opened this issue Mar 18, 2025 · 112 comments
Open

roachtest: cluster_creation failed #143042

cockroach-teamcity opened this issue Mar 18, 2025 · 112 comments
Labels
branch-release-25.1.3-rc O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 18, 2025 •
edited by cockroach-jira-scripts
Loading

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test kv0/enc=false/nodes=3/cpu=32/size=4kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19129120-1742276174-19-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T05:57:57Z},{Key=Name,Value=teamcity-19129120-1742276174-19-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19129120-1742276174-19-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T05:57:57Z},{Key=Name,Value=teamcity-19129120-1742276174-19-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3635617081 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1731082694 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-48626
@cockroach-teamcity cockroach-teamcity added O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue branch-release-25.1.3-rc labels Mar 18, 2025
This was referenced Mar 18, 2025
Open
Open
Open
@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test kv95/enc=false/nodes=3/cpu=32/seq failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-31-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:18:34Z},{Key=Name,Value=teamcity-19129120-1742276174-31-n4cpu32-0004},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-31-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:18:34Z},{Key=Name,Value=teamcity-19129120-1742276174-31-n4cpu32-0004},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script2409482204 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping646159607 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m7gd.xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m7gd.xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test kv95/enc=false/nodes=3/cpu=32/size=4kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-34-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:20:46Z},{Key=Name,Value=teamcity-19129120-1742276174-34-n4cpu32-0004}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-34-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:20:46Z},{Key=Name,Value=teamcity-19129120-1742276174-34-n4cpu32-0004}] --user-data file:///tmp/aws-startup-script2424026682 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3000946059 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m7gd.xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m7gd.xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 18, 2025
Open
@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:34:02Z},{Key=Name,Value=teamcity-19129120-1742276174-45-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-45-n4cpu96}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:34:02Z},{Key=Name,Value=teamcity-19129120-1742276174-45-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-45-n4cpu96}] --user-data file:///tmp/aws-startup-script203937029 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1693560233 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6a.24xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6a.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/A/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:39:55Z},{Key=Name,Value=teamcity-19129120-1742276174-51-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-51-n4cpu32},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:39:55Z},{Key=Name,Value=teamcity-19129120-1742276174-51-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-51-n4cpu32},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script611885292 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1085365416 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/C/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:45:49Z},{Key=Name,Value=teamcity-19129120-1742276174-57-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-57-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:45:49Z},{Key=Name,Value=teamcity-19129120-1742276174-57-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-57-n4cpu32}] --user-data file:///tmp/aws-startup-script120439213 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2977409188 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/D/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-60-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:47:48Z},{Key=Name,Value=teamcity-19129120-1742276174-60-n4cpu32-0001},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-60-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:47:48Z},{Key=Name,Value=teamcity-19129120-1742276174-60-n4cpu32-0001},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script1160818605 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1149799629 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/D/nodes=3/cpu=32/uniform failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-63-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:49:51Z},{Key=Name,Value=teamcity-19129120-1742276174-63-n4cpu32-0002},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-63-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-18T06:49:51Z},{Key=Name,Value=teamcity-19129120-1742276174-63-n4cpu32-0002},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script4046855917 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2338131539 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/E/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:53:16Z},{Key=Name,Value=teamcity-19129120-1742276174-67-n4cpu32-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-67-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-18T06:53:16Z},{Key=Name,Value=teamcity-19129120-1742276174-67-n4cpu32-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-67-n4cpu32}] --user-data file:///tmp/aws-startup-script1226173740 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1091053055 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test ycsb/F/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kcMgxL7aujRU2tkT4J5v_pBNccI= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-18T06:55:15Z},{Key=Name,Value=teamcity-19129120-1742276174-70-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-70-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-18T06:55:15Z},{Key=Name,Value=teamcity-19129120-1742276174-70-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19129120-1742276174-70-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script2508000251 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3294723823 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 89ce402f3e337ec547bac75d63438201b5f6ba19:

test costfuzz/rand-multi-region failed: (test_runner.go:847).func4: in provider: gce: T2A instances are not supported outside of [asia-southeast1-b,asia-southeast1-c,europe-west4-a,europe-west4-b,europe-west4-c,us-central1-a,us-central1-b,us-central1-f] [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=4
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test kv0/enc=false/nodes=3/lease=expiration failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kqa-XM3kEP6xv4JlsY-c_wR0f1Y= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T05:59:38Z},{Key=Name,Value=teamcity-19141871-1742362366-25-n4cpu8-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19141871-1742362366-25-n4cpu8}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T05:59:38Z},{Key=Name,Value=teamcity-19141871-1742362366-25-n4cpu8-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19141871-1742362366-25-n4cpu8}] --user-data file:///tmp/aws-startup-script3319378340 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3691289333 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.2xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.2xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test tpccbench/nodes=3/cpu=16/mt-shared-process failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.4xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kqa-XM3kEP6xv4JlsY-c_wR0f1Y= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19141871-1742362366-42-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:27:30Z},{Key=Name,Value=teamcity-19141871-1742362366-42-n4cpu16-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19141871-1742362366-42-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:27:30Z},{Key=Name,Value=teamcity-19141871-1742362366-42-n4cpu16-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script4200315744 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2140391271 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test ycsb/A/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kqa-XM3kEP6xv4JlsY-c_wR0f1Y= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19141871-1742362366-45-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:29:12Z},{Key=Name,Value=teamcity-19141871-1742362366-45-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19141871-1742362366-45-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:29:12Z},{Key=Name,Value=teamcity-19141871-1742362366-45-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3197677820 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3297002184 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test ycsb/B/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-kqa-XM3kEP6xv4JlsY-c_wR0f1Y= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19141871-1742362366-48-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:30:55Z},{Key=Name,Value=teamcity-19141871-1742362366-48-n4cpu32-0001}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19141871-1742362366-48-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-19T06:30:55Z},{Key=Name,Value=teamcity-19141871-1742362366-48-n4cpu32-0001}] --user-data file:///tmp/aws-startup-script2925285959 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2574081285 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 19, 2025
Open
@cockroach-teamcity
Copy link
Member Author

Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test follower-reads/survival=region/locality=regional/reads=bounded-staleness/insufficient-quorum failed: (test_runner.go:847).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --format json --labels spot=true,usage=roachtest,roachprod=true,cluster=teamcity-19141870-1742362377-78-n6cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2025-03-19t09_32_09z --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account [email protected] --provisioning-model SPOT --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1923555498 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-19141870-1742362377-78-n6cpu4-geo-0004 teamcity-19141870-1742362377-78-n6cpu4-geo-0005]: failed to run: gcloud compute instances create --subnet default --format json --labels spot=true,usage=roachtest,roachprod=true,cluster=teamcity-19141870-1742362377-78-n6cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2025-03-19t09_32_09z --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account [email protected] --provisioning-model SPOT --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script1923555498 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-19141870-1742362377-78-n6cpu4-geo-0004 teamcity-19141870-1742362377-78-n6cpu4-geo-0005
stdout: 
stderr: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-19141870-1742362377-78-n6cpu4-geo-0005].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240319' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20250312'.
 - You are creating a global DNS VM. VM instances using global DNS are vulnerable to cross-regional outages. To reduce the risk of widespread service disruption, use zonal DNS instead. Learn more at https://cloud.google.com/compute/docs/networking/zonal-dns
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
	metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
	limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
	limit = 30000.0
	dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=4
  • runtimeAssertionsBuild=true
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 2311ca6124584e6f972eb86993aa98e85337aaa2:

test follower-reads/survival=zone/locality=regional/reads=bounded-staleness failed: (test_runner.go:847).func4: in provider: gce: Command: gcloud [compute instances create --subnet default --format json --labels spot=true,usage=roachtest,cluster=teamcity-19141870-1742362377-79-n6cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2025-03-19t09_32_42z,roachprod=true --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account [email protected] --provisioning-model SPOT --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script3267055417 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-19141870-1742362377-79-n6cpu4-geo-0004 teamcity-19141870-1742362377-79-n6cpu4-geo-0005]: failed to run: gcloud compute instances create --subnet default --format json --labels spot=true,usage=roachtest,cluster=teamcity-19141870-1742362377-79-n6cpu4-geo,lifetime=12h0m0s,arch=amd64,created=2025-03-19t09_32_42z,roachprod=true --scopes cloud-platform --image ubuntu-2204-jammy-v20240319 --image-project ubuntu-os-cloud --boot-disk-type pd-ssd --service-account [email protected] --provisioning-model SPOT --local-ssd interface=NVME --machine-type n2-standard-4 --min-cpu-platform Intel Ice Lake --metadata-from-file startup-script=/tmp/gce-startup-script3267055417 --project cockroach-ephemeral --boot-disk-size=32GB --zone us-west1-b teamcity-19141870-1742362377-79-n6cpu4-geo-0004 teamcity-19141870-1742362377-79-n6cpu4-geo-0005
stdout: 
stderr: Created [https://www.googleapis.com/compute/v1/projects/cockroach-ephemeral/zones/us-west1-b/instances/teamcity-19141870-1742362377-79-n6cpu4-geo-0004].
WARNING: Some requests generated warnings:
 - Disk size: '32 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.
 - The resource 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20240319' is deprecated. A suggested replacement is 'projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20250312'.
 - You are creating a global DNS VM. VM instances using global DNS are vulnerable to cross-regional outages. To reduce the risk of widespread service disruption, use zonal DNS instead. Learn more at https://cloud.google.com/compute/docs/networking/zonal-dns
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY' exceeded.  Limit: 30000.0 in region us-west1.
	metric name = compute.googleapis.com/local_ssd_total_storage_per_vm_family
	limit name = LOCAL-SSD-TOTAL-GB-PER-VM-FAMILY-per-project-region
	limit = 30000.0
	dimensions = region: us-west1
vm_family: N2
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.: exit status 1 [owner=test-eng]

Parameters:

  • cloud=gce
  • coverageBuild=false
  • cpu=4
  • runtimeAssertionsBuild=true
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ e5022b9fa6fd707e20daa4ae7e0a842bd8ce35cf:

test kv0/enc=false/nodes=3/cpu=32/mt-shared-process failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-rIeqEx7O2QAKk3D5k2fzN8htZNs= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-20T06:03:05Z},{Key=Name,Value=teamcity-19154713-1742448939-24-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-20T06:03:05Z},{Key=Name,Value=teamcity-19154713-1742448939-24-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script2729968624 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping500523051 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6id.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6id.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ e5022b9fa6fd707e20daa4ae7e0a842bd8ce35cf:

test kv0/enc=false/nodes=3/size=4kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-rIeqEx7O2QAKk3D5k2fzN8htZNs= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-28-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:06:20Z},{Key=Name,Value=teamcity-19154713-1742448939-28-n4cpu8-0001},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-28-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:06:20Z},{Key=Name,Value=teamcity-19154713-1742448939-28-n4cpu8-0001},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script815992966 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3756997670 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ e5022b9fa6fd707e20daa4ae7e0a842bd8ce35cf:

test kv95/enc=false/nodes=3/lease=expiration failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-rIeqEx7O2QAKk3D5k2fzN8htZNs= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-34-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:12:19Z},{Key=Name,Value=teamcity-19154713-1742448939-34-n4cpu8-0002},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19154713-1742448939-34-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:12:19Z},{Key=Name,Value=teamcity-19154713-1742448939-34-n4cpu8-0002},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script3917370731 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping140833072 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.2xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.2xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ e5022b9fa6fd707e20daa4ae7e0a842bd8ce35cf:

test kv95/enc=false/nodes=3/size=64kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-rIeqEx7O2QAKk3D5k2fzN8htZNs= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19154713-1742448939-37-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:14:14Z},{Key=Name,Value=teamcity-19154713-1742448939-37-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19154713-1742448939-37-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-20T06:14:14Z},{Key=Name,Value=teamcity-19154713-1742448939-37-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script2219492507 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2096355088 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/F/nodes=3/cpu=32/uniform failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-1dRObmw8UPuSEnj0Dc6l5RpyDb8= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19229915-1743053643-81-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-27T07:12:19Z},{Key=Name,Value=teamcity-19229915-1743053643-81-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19229915-1743053643-81-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-27T07:12:19Z},{Key=Name,Value=teamcity-19229915-1743053643-81-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3851165229 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3947938393 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test cdc/tpcc-1000/sink=kafka failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.4xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-12-n4cpu16-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-12-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T05:44:45Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-12-n4cpu16-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-12-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T05:44:45Z}] --user-data file:///tmp/aws-startup-script2307741167 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping335637846 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv0/enc=false/nodes=3/cpu=96 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19243234-1743139958-20-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T05:52:38Z},{Key=Name,Value=teamcity-19243234-1743139958-20-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19243234-1743139958-20-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T05:52:38Z},{Key=Name,Value=teamcity-19243234-1743139958-20-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script1534656598 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping588750036 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6a.24xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6a.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv95/enc=false/nodes=3/tracing failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19243234-1743139958-30-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:04:55Z},{Key=Name,Value=teamcity-19243234-1743139958-30-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19243234-1743139958-30-n4cpu8},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:04:55Z},{Key=Name,Value=teamcity-19243234-1743139958-30-n4cpu8-0004},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script89133200 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3743828984 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=8
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv95limited-spans/enc=false/nodes=1/cpu=32/seq/no-load-splitting failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-34-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:08:11Z},{Key=Name,Value=teamcity-19243234-1743139958-34-n2cpu32-0002},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-34-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:08:11Z},{Key=Name,Value=teamcity-19243234-1743139958-34-n2cpu32-0002},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script502267681 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping17990946 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/A/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-43-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-43-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:17:30Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-43-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-43-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:17:30Z}] --user-data file:///tmp/aws-startup-script2276485694 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3245410410 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/A/nodes=3/cpu=32/uniform failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-46-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:19:11Z},{Key=Name,Value=teamcity-19243234-1743139958-46-n4cpu32-0002},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-46-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:19:11Z},{Key=Name,Value=teamcity-19243234-1743139958-46-n4cpu32-0002},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script130622770 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3170456535 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/B/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:22:15Z},{Key=Name,Value=teamcity-19243234-1743139958-50-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-50-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:22:15Z},{Key=Name,Value=teamcity-19243234-1743139958-50-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-50-n4cpu32}] --user-data file:///tmp/aws-startup-script608088012 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3405922344 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/D/nodes=3/cpu=32 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-55-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-55-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:26:30Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-55-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-55-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-28T06:26:30Z}] --user-data file:///tmp/aws-startup-script1234348020 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2059741622 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c7gd.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c7gd.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/F/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-64-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-64-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:50:22Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-64-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-64-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:50:22Z}] --user-data file:///tmp/aws-startup-script3470601792 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2371012646 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/F/nodes=3/cpu=32/uniform failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-LE2qHnT9sEwwAQDr_T30x8OXbSU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-67-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-67-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:52:08Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19243234-1743139958-67-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19243234-1743139958-67-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-28T06:52:08Z}] --user-data file:///tmp/aws-startup-script4276255540 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4190493475 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv0/enc=false/nodes=3/cpu=32/size=4kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19255434-1743226435-24-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:01:24Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19255434-1743226435-24-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-24-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:01:24Z}] --user-data file:///tmp/aws-startup-script1822524357 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1906401364 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c7gd.8xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c7gd.8xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv0/enc=false/nodes=3/cpu=96 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19255434-1743226435-27-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-27-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-29T06:03:04Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Name,Value=teamcity-19255434-1743226435-27-n4cpu96-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-27-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-29T06:03:04Z}] --user-data file:///tmp/aws-startup-script1981157716 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3211278800 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6a.24xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6a.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv95/enc=false/nodes=1/cpu=32/mt-shared-process failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-32-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:07:37Z},{Key=Name,Value=teamcity-19255434-1743226435-32-n2cpu32-0001},{Key=Spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-32-n2cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:07:37Z},{Key=Name,Value=teamcity-19255434-1743226435-32-n2cpu32-0001},{Key=Spot,Value=true}] --user-data file:///tmp/aws-startup-script3571705349 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4191813631 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv95/enc=false/nodes=3/cpu=32/size=4kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-36-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:10:49Z},{Key=Name,Value=teamcity-19255434-1743226435-36-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-36-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:10:49Z},{Key=Name,Value=teamcity-19255434-1743226435-36-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script1472921336 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2033553501 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test restore/tpce/400GB/aws/nodes=4/cpus=16 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-44-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:25:23Z},{Key=Name,Value=teamcity-19255434-1743226435-44-n4cpu16-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-44-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:25:23Z},{Key=Name,Value=teamcity-19255434-1743226435-44-n4cpu16-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script1524375420 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2701609659 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m7g.4xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m7g.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/A/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-51-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:32:30Z},{Key=Name,Value=teamcity-19255434-1743226435-51-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19255434-1743226435-51-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:32:30Z},{Key=Name,Value=teamcity-19255434-1743226435-51-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script2249123168 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3243531847 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/C/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-a1-KYa3d19jS4w_WOaGI7Ch6j7U= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:37:24Z},{Key=Name,Value=teamcity-19255434-1743226435-56-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-56-n4cpu32},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-29T06:37:24Z},{Key=Name,Value=teamcity-19255434-1743226435-56-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19255434-1743226435-56-n4cpu32},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script1603008453 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3308491245 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 29, 2025
Open
@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv0/enc=false/nodes=3/cpu=96 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-23-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:01:15Z},{Key=Name,Value=teamcity-19262015-1743312865-23-n4cpu96-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-23-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:01:15Z},{Key=Name,Value=teamcity-19262015-1743312865-23-n4cpu96-0002}] --user-data file:///tmp/aws-startup-script1212234609 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping465997899 --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c6a.24xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c6a.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test tpcc/headroom/n4cpu16 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-30T06:35:15Z},{Key=Name,Value=teamcity-19262015-1743312865-41-n4cpu16-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-41-n4cpu16},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-30T06:35:15Z},{Key=Name,Value=teamcity-19262015-1743312865-41-n4cpu16-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-41-n4cpu16},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script2955888552 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1914212473 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test tpccbench/nodes=3/cpu=16/lease=expiration failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-45-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-30T06:38:24Z},{Key=Name,Value=teamcity-19262015-1743312865-45-n4cpu16-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-45-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-30T06:38:24Z},{Key=Name,Value=teamcity-19262015-1743312865-45-n4cpu16-0002}] --user-data file:///tmp/aws-startup-script783704072 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2492570115 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=16
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/A/nodes=3/cpu=32/uniform failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-30T06:40:09Z},{Key=Name,Value=teamcity-19262015-1743312865-48-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-48-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Created,Value=2025-03-30T06:40:09Z},{Key=Name,Value=teamcity-19262015-1743312865-48-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-48-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script3131829604 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping230665984 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/B/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-51-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:41:59Z},{Key=Name,Value=teamcity-19262015-1743312865-51-n4cpu32-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-51-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:41:59Z},{Key=Name,Value=teamcity-19262015-1743312865-51-n4cpu32-0002}] --user-data file:///tmp/aws-startup-script1139395233 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping1085946326 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/C/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-sktswaJ7kH8scZM-24QFgz8SiHg= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:46:19Z},{Key=Name,Value=teamcity-19262015-1743312865-56-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-56-n4cpu32}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-30T06:46:19Z},{Key=Name,Value=teamcity-19262015-1743312865-56-n4cpu32-0002},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19262015-1743312865-56-n4cpu32}] --user-data file:///tmp/aws-startup-script1964964729 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping448262423 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv0/enc=false/nodes=3/cpu=32/size=64kb failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6id.8xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-tpfUkXafokak16NHxporPCNSQMU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19267579-1743399211-23-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-31T05:59:50Z},{Key=Name,Value=teamcity-19267579-1743399211-23-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19267579-1743399211-23-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-31T05:59:50Z},{Key=Name,Value=teamcity-19267579-1743399211-23-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3145913413 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2678328690 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test kv95/enc=false/nodes=3/cpu=96 failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c6a.24xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-tpfUkXafokak16NHxporPCNSQMU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-31T06:13:59Z},{Key=Name,Value=teamcity-19267579-1743399211-34-n4cpu96-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19267579-1743399211-34-n4cpu96}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=Created,Value=2025-03-31T06:13:59Z},{Key=Name,Value=teamcity-19267579-1743399211-34-n4cpu96-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19267579-1743399211-34-n4cpu96}] --user-data file:///tmp/aws-startup-script489938815 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2658428449 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=96
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/A/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-tpfUkXafokak16NHxporPCNSQMU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-31T06:34:46Z},{Key=Name,Value=teamcity-19267579-1743399211-52-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19267579-1743399211-52-n4cpu32},{Key=Lifetime,Value=12h0m0s}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-31T06:34:46Z},{Key=Name,Value=teamcity-19267579-1743399211-52-n4cpu32-0003},{Key=Spot,Value=true},{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-19267579-1743399211-52-n4cpu32},{Key=Lifetime,Value=12h0m0s}] --user-data file:///tmp/aws-startup-script906536936 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2417732877 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/E/nodes=3/cpu=32/isolation-level=read-committed failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c7gd.8xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-tpfUkXafokak16NHxporPCNSQMU= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19267579-1743399211-61-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-31T07:04:26Z},{Key=Name,Value=teamcity-19267579-1743399211-61-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=Cluster,Value=teamcity-19267579-1743399211-61-n4cpu32},{Key=Lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=Created,Value=2025-03-31T07:04:26Z},{Key=Name,Value=teamcity-19267579-1743399211-61-n4cpu32-0001},{Key=Spot,Value=true},{Key=Roachprod,Value=true}] --user-data file:///tmp/aws-startup-script3033693056 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping2306162746 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): There is no Spot capacity available that matches your request.: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cluster_creation failed with artifacts on release-25.1.3-rc @ 70ba2ee1e23ea1c6a8b6c3983164857f5cc97960:

test ycsb/E/nodes=3/cpu=32/mvcc-range-keys=global failed: (test_runner.go:847).func4: in provider: aws: failed to run: aws ec2 describe-instances --region us-east-2 --instance-ids i-04ca4d13ee42e4a74 --output json: stderr: 
An error occurred (InvalidInstanceID.NotFound) when calling the DescribeInstances operation: The instance ID 'i-04ca4d13ee42e4a74' does not exist: exit status 254 [owner=test-eng]

Parameters:

  • cloud=aws
  • coverageBuild=false
  • cpu=32
  • runtimeAssertionsBuild=false
  • ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-release-25.1.3-rc O-roachtest O-robot Originated from a bot. T-testeng TestEng Team X-infra-flake the automatically generated issue was closed due to an infrastructure problem not a product issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cockroach-teamcity