Crash error in TiDB

[curry-oss]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

USE test;
DROP TABLE IF EXISTS t0;
DROP TABLE IF EXISTS t1;
CREATE TABLE t0(c0 BIT(29), c1 NUMERIC PRIMARY KEY);
CREATE TABLE t1 LIKE t0;
INSERT IGNORE INTO t1(c0, c1) VALUES (100010001110010001111, -1999727919);
SELECT t0.c0, t0.c1, t1.c0, t1.c1 FROM t0 STRAIGHT_JOIN t1 ON t0.c1 WHERE ((t1.c0)LIKE(t1.c0));

global:
  user: "tidb"
  ssh_port: 22
  deploy_dir: "/tidb-deploy"
  data_dir: "/tidb-data"
server_configs: {}
pd_servers:
  - host: {MyIP}
tikv_servers:
  - host: {MyIP}
tiflash_servers:
  - host: {MyIP}
    deploy_dir: /tiflash/deploy1
    data_dir: /tiflash/data1
tidb_servers:
  - host: {MyIP}
monitoring_servers:
  - host: {MyIP}
grafana_servers:
  - host: {MyIP}
alertmanager_servers:
  - host: {MyIP}

2. What did you expect to see? (Required)

see the following case:

3. What did you see instead (Required)

I found that when executing this query, TiKV consistently crashes and returns “Region is unavailable.” Meanwhile, by checking the TiKV logs, I identified a potential issue caused by an array out-of-bounds error: "range end index 5 out of range for slice of length 4".

-- After several miniutes: TiDB throws error
mysql> SELECT t0.c0, t0.c1, t1.c0, t1.c1 FROM  t0 STRAIGHT_JOIN t1 ON t0.c1 WHERE ((t1.c0)LIKE(t1.c0));
ERROR 9005 (HY000): Region is unavailable

-- TIKV LOG: [FATAL] [lib.rs:479] ["range end index 5 out of range for slice of length 4"] [backtrace="   0: tikv_util::set_panic_hook::{{closure}}\n             at /workspace/source/tikv/components/tikv_util/src/lib.rs:478:18\n   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2029:9\n      std::panicking::rust_panic_with_hook\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:783:13\n   2: std::panicking::begin_panic_handler::{{closure}}\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:657:13\n   3: std::sys_common::backtrace::__rust_end_short_backtrace\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:171:18\n   4: rust_begin_unwind\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:645:5\n   5: core::panicking::panic_fmt\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:72:14\n   6: core::slice::index::slice_end_index_len_fail_rt\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/slice/index.rs:76:5\n      core::slice::index::slice_end_index_len_fail\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/slice/index.rs:68:9\n   7: <core::ops::range::Range<usize> as core::slice::index::SliceIndex<[T]>>::index\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/slice/index.rs:394:13\n      core::slice::index::<impl core::ops::index::Index<I> for [T]>::index\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/slice/index.rs:18:9\n      tidb_query_expr::impl_like::like\n             at /workspace/source/tikv/components/tidb_query_expr/src/impl_like.rs:50:48\n   8: <tidb_query_expr::types::function::Arg<Arg0_,tidb_query_expr::types::function::Arg<Arg1_,tidb_query_expr::types::function::Arg<Arg2_,tidb_query_expr::types::function::Null>>> as tidb_query_expr::impl_like::Like_Fn<C,CS>>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/impl_like.rs:7:1\n      <tidb_query_expr::impl_like::Like_Evaluator<C,CS> as tidb_query_expr::types::function::Evaluator>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/impl_like.rs:7:1\n      <tidb_query_expr::types::function::ArgConstructor<A,E> as tidb_query_expr::types::function::Evaluator>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/types/function.rs:261:17\n      <tidb_query_expr::types::function::ArgConstructor<A,E> as tidb_query_expr::types::function::Evaluator>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/types/function.rs:261:17\n      <tidb_query_expr::types::function::ArgConstructor<A,E> as tidb_query_expr::types::function::Evaluator>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/types/function.rs:245:17\n   9: tidb_query_expr::impl_like::like_fn_meta::run\n             at /workspace/source/tikv/components/tidb_query_expr/src/impl_like.rs:7:1\n  10: tidb_query_expr::types::expr_eval::<impl tidb_query_expr::types::expr::RpnExpression>::eval_decoded\n             at /workspace/source/tikv/components/tidb_query_expr/src/types/expr_eval.rs:308:31\n  11: tidb_query_expr::types::expr_eval::<impl tidb_query_expr::types::expr::RpnExpression>::eval\n             at /workspace/source/tikv/components/tidb_query_expr/src/types/expr_eval.rs:218:9\n      tidb_query_executors::selection_executor::BatchSelectionExecutor<Src>::handle_src_result\n             at /workspace/source/tikv/components/tidb_query_executors/src/selection_executor.rs:82:19\n      <tidb_query_executors::selection_executor::BatchSelectionExecutor<Src> as tidb_query_executors::interface::BatchExecutor>::next_batch::{{closure}}\n             at /workspace/source/tikv/components/tidb_query_executors/src/selection_executor.rs:182:25\n  12: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9\n      <tidb_query_common::execute_stats::WithSummaryCollector<C,T> as tidb_query_executors::interface::BatchExecutor>::next_batch::{{closure}}\n             at /workspace/source/tikv/components/tidb_query_executors/src/interface.rs:114:55\n  13: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9\n      <alloc::boxed::Box<T> as tidb_query_executors::interface::BatchExecutor>::next_batch::{{closure}}\n             at /workspace/source/tikv/components/tidb_query_executors/src/interface.rs:82:40\n  14: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9\n      tidb_query_executors::runner::BatchExecutorsRunner<SS>::internal_handle_request::{{closure}}\n             at /workspace/source/tikv/components/tidb_query_executors/src/runner.rs:655:72\n  15: <tikv_util::quota_limiter::CpuObserveFuture<F> as core::future::future::Future>::poll\n             at /workspace/source/tikv/components/tikv_util/src/quota_limiter.rs:177:19\n      tidb_query_executors::runner::BatchExecutorsRunner<SS>::handle_request::{{closure}}\n             at /workspace/source/tikv/components/tidb_query_executors/src/runner.rs:519:22\n      <tikv::coprocessor::dag::BatchDagHandler as tikv::coprocessor::RequestHandler>::handle_request::{{closure}}\n             at /workspace/source/tikv/src/coprocessor/dag/mod.rs:124:51\n  16: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9\n      <tikv::coprocessor::interceptors::deadline::DeadlineChecker<F> as core::future::future::Future>::poll\n             at /workspace/source/tikv/src/coprocessor/interceptors/deadline.rs:37:9\n      <tikv::coprocessor::interceptors::tracker::Tracker<F,E> as core::future::future::Future>::poll\n             at /workspace/source/tikv/src/coprocessor/interceptors/tracker.rs:54:19\n  17: <tikv::coprocessor::interceptors::concurrency_limiter::ConcurrencyLimiter<PF,F> as core::future::future::Future>::poll\n             at /workspace/source/tikv/src/coprocessor/interceptors/concurrency_limiter.rs:112:15\n      tikv::coprocessor::endpoint::Endpoint<E>::handle_unary_request_impl::{{closure}}\n             at /workspace/source/tikv/src/coprocessor/endpoint.rs:483:87\n      <resource_metering::InTags<T> as core::future::future::Future>::poll\n             at /workspace/source/tikv/components/resource_metering/src/lib.rs:274:9\n      <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/future/future/map.rs:55:37\n      <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/lib.rs:86:13\n      <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/future/future/map.rs:55:37\n      <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/lib.rs:86:13\n  18: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/future/future/map.rs:55:37\n      <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll\n             at /workspace/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/futures-util-0.3.31/src/lib.rs:86:13\n      <resource_control::future::ControlledFuture<F> as core::future::future::Future>::poll\n             at /workspace/source/tikv/components/resource_control/src/future.rs:48:19\n  19: resource_control::future::with_resource_limiter::{{closure}}\n             at /workspace/source/tikv/components/resource_control/src/future.rs:240:11\n      <tracker::tls::TrackedFuture<F> as core::future::future::Future>::poll::{{closure}}\n             at /workspace/source/tikv/components/tracker/src/tls.rs:64:23\n      std::thread::local::LocalKey<T>::try_with\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:270:16\n      std::thread::local::LocalKey<T>::with\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:246:9\n      <tracker::tls::TrackedFuture<F> as core::future::future::Future>::poll\n             at /workspace/source/tikv/components/tracker/src/tls.rs:62:27\n      yatp::task::future::RawTask<F>::poll\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:59:9\n  20: yatp::task::future::TaskCell::poll\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:103:9\n      <yatp::task::future::Runner as yatp::pool::runner::Runner>::handle\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:387:20\n  21: <tikv_util::yatp_pool::YatpPoolRunner<T> as yatp::pool::runner::Runner>::handle\n             at /workspace/source/tikv/components/tikv_util/src/yatp_pool/mod.rs:199:24\n      <yatp::queue::multilevel::TrackedRunner<R> as yatp::pool::runner::Runner>::handle\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/queue/multilevel.rs:285:19\n  22: yatp::pool::worker::WorkerThread<T,R>::run\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/worker.rs:48:13\n      yatp::pool::builder::LazyBuilder<T>::build::{{closure}}\n             at /workspace/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/builder.rs:114:25\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:155:18\n  23: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:529:17\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9\n      std::panicking::try::do_call\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552:40\n      std::panicking::try\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:516:19\n      std::panic::catch_unwind\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:142:14\n      std::thread::Builder::spawn_unchecked_::{{closure}}\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:528:30\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5\n  24: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9\n      std::sys::unix::thread::Thread::new::thread_start\n             at /root/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:108:17\n  25: <unknown>\n  26: <unknown>\n"] [location=components/tidb_query_expr/src/impl_like.rs:50] [thread_name=unified-read-pool-2] [thread_id=35]

4. What is your TiDB version? (Required)

mysql> select version();
+--------------------+
| version() |
+--------------------+
| 8.0.11-TiDB-v8.5.0 |
+--------------------+
1 row in set (0.04 sec)

[curry-oss]

/label affects-8.5

[curry-oss]

Hi, has there been any progress?

[zanmato1984]

TiKV crashed by the fatal error. It seems like something wrong in the tikv coprocessor.