rocksdb 8.9.1 unit test failed

[adv0518]

hello.

system: centos-8-x64, it's docker image, in cetnos7.4.
gcc version: 8.3.1 20191121(red hat 8.3.1-5)

1. db/compaction/compaction_job_stats_test.cc:456 Failure actual:63586 vs 63196.8
2. db/compaction/compaction_job_stats_test.cc:456 Failure actual:63586 vs 63196.8
3. db/db_wal_test.cc:2469 Failure actual:18427904 vs 18427904
4. cache/tiered_secondary_cache_test.cc:524 which is 0 3u which is 3
5. cache/tiered_secondary_cache_test.cc:382 which is 0 3u which is 3
6. cache/tiered_secondary_cache_test.cc:267 which is 0 3u which is 1
7. cache/tiered_secondary_cache_test.cc:623 which is 0 3u which is 8

---

thanks.

[adamretter]

system: centos-8-x64, it's docker image, in cetnos7.4.

I am a bit confused about which of the above versions is the system version or the Docker version? Are you running the tests inside Docker? If so, what is the exact Docker image that is used please?

[adv0518]

do command: make check
failed info:

Makefile:185: Warning: Compiling in debug mode. Don't use the resulting binary in production
Generating parallel test scripts for blob_counting_iterator_test
Generating parallel test scripts for blob_file_builder_test
Generating parallel test scripts for blob_file_addition_test
Generating parallel test scripts for cache_reservation_manager_test
Generating parallel test scripts for cache_test
Generating parallel test scripts for tiered_secondary_cache_test
Generating parallel test scripts for lru_cache_test
Generating parallel test scripts for compressed_secondary_cache_test
Generating parallel test scripts for blob_file_cache_test
Generating parallel test scripts for blob_file_garbage_test
Generating parallel test scripts for blob_file_reader_test
Generating parallel test scripts for blob_garbage_meter_test
Generating parallel test scripts for blob_source_test
Generating parallel test scripts for db_blob_basic_test
Generating parallel test scripts for db_blob_compaction_test
Generating parallel test scripts for db_blob_corruption_test
Generating parallel test scripts for db_blob_index_test
Generating parallel test scripts for column_family_test
Generating parallel test scripts for compact_files_test
Generating parallel test scripts for clipping_iterator_test
Generating parallel test scripts for compaction_iterator_test
Generating parallel test scripts for compaction_job_test
Generating parallel test scripts for compaction_job_stats_test
Generating parallel test scripts for compaction_picker_test
Generating parallel test scripts for compaction_service_test
Generating parallel test scripts for tiered_compaction_test
Generating parallel test scripts for comparator_db_test
Generating parallel test scripts for corruption_test
Generating parallel test scripts for cuckoo_table_db_test
Generating parallel test scripts for db_basic_test
Generating parallel test scripts for db_compaction_filter_test
Generating parallel test scripts for db_block_cache_test
Generating parallel test scripts for db_compaction_test
Generating parallel test scripts for db_clip_test
Generating parallel test scripts for db_dynamic_level_test
Generating parallel test scripts for db_encryption_test
Generating parallel test scripts for db_flush_test
Generating parallel test scripts for db_readonly_with_timestamp_test
Generating parallel test scripts for import_column_family_test
Generating parallel test scripts for db_with_timestamp_basic_test
Generating parallel test scripts for db_inplace_update_test
Generating parallel test scripts for db_io_failure_test
Generating parallel test scripts for db_iter_test
Generating parallel test scripts for db_iter_stress_test
Generating parallel test scripts for db_iterator_test
Generating parallel test scripts for db_log_iter_test
Generating parallel test scripts for db_kv_checksum_test
Generating parallel test scripts for db_memtable_test
Generating parallel test scripts for db_merge_operator_test
Generating parallel test scripts for db_merge_operand_test
Generating parallel test scripts for db_options_test
Generating parallel test scripts for db_properties_test
Generating parallel test scripts for db_range_del_test
Generating parallel test scripts for db_rate_limiter_test
Generating parallel test scripts for db_secondary_test
Generating parallel test scripts for db_sst_test
Generating parallel test scripts for db_statistics_test
Generating parallel test scripts for db_table_properties_test
Generating parallel test scripts for db_tailing_iter_test
Generating parallel test scripts for db_test
Generating parallel test scripts for db_test2
Generating parallel test scripts for db_logical_block_size_cache_test
Generating parallel test scripts for db_universal_compaction_test
Generating parallel test scripts for db_with_timestamp_compaction_test
Generating parallel test scripts for db_wal_test
Generating parallel test scripts for db_write_buffer_manager_test
Generating parallel test scripts for db_write_test
Generating parallel test scripts for dbformat_test
Generating parallel test scripts for error_handler_fs_test
Generating parallel test scripts for external_sst_file_basic_test
Generating parallel test scripts for external_sst_file_test
Generating parallel test scripts for fault_injection_test
Generating parallel test scripts for file_indexer_test
Generating parallel test scripts for filename_test
Generating parallel test scripts for flush_job_test
Generating parallel test scripts for listener_test
Generating parallel test scripts for manual_compaction_test
Generating parallel test scripts for log_test
Generating parallel test scripts for memtable_list_test
Generating parallel test scripts for merge_helper_test
Generating parallel test scripts for merge_test
Generating parallel test scripts for obsolete_files_test
Generating parallel test scripts for options_file_test
Generating parallel test scripts for perf_context_test
Generating parallel test scripts for periodic_task_scheduler_test
Generating parallel test scripts for plain_table_db_test
Please install gflags to run this test... Skipping...
Generating parallel test scripts for prefix_test
Generating parallel test scripts for repair_test
Generating parallel test scripts for range_del_aggregator_test
Generating parallel test scripts for range_tombstone_fragmenter_test
Generating parallel test scripts for seqno_time_test
Generating parallel test scripts for table_properties_collector_test
Generating parallel test scripts for version_builder_test
Generating parallel test scripts for version_edit_test
Generating parallel test scripts for version_set_test
Generating parallel test scripts for wal_manager_test
Generating parallel test scripts for db_wide_basic_test
Generating parallel test scripts for wide_column_serialization_test
Generating parallel test scripts for wide_columns_helper_test
Generating parallel test scripts for write_batch_test
Generating parallel test scripts for write_controller_test
Generating parallel test scripts for write_callback_test
Generating parallel test scripts for env_basic_test
Generating parallel test scripts for io_posix_test
Generating parallel test scripts for mock_env_test
Generating parallel test scripts for delete_scheduler_test
Generating parallel test scripts for prefetch_test
Generating parallel test scripts for random_access_file_reader_test
Generating parallel test scripts for auto_roll_logger_test
Generating parallel test scripts for env_logger_test
Generating parallel test scripts for event_logger_test
Generating parallel test scripts for arena_test
Generating parallel test scripts for memory_allocator_test
Generating parallel test scripts for skiplist_test
Generating parallel test scripts for inlineskiplist_test
Generating parallel test scripts for write_buffer_manager_test
Generating parallel test scripts for histogram_test
Generating parallel test scripts for iostats_context_test
Generating parallel test scripts for statistics_test
Generating parallel test scripts for stats_history_test
Generating parallel test scripts for configurable_test
Generating parallel test scripts for options_settable_test
Generating parallel test scripts for customizable_test
Generating parallel test scripts for options_test
Generating parallel test scripts for block_test
Generating parallel test scripts for block_based_table_reader_test
Generating parallel test scripts for data_block_hash_index_test
Generating parallel test scripts for full_filter_block_test
Generating parallel test scripts for partitioned_filter_block_test
Generating parallel test scripts for cleanable_test
Generating parallel test scripts for cuckoo_table_builder_test
Please install gflags to run this test... Skipping...
Generating parallel test scripts for cuckoo_table_reader_test
Generating parallel test scripts for merger_test
Generating parallel test scripts for sst_file_reader_test
Generating parallel test scripts for block_fetcher_test
Generating parallel test scripts for table_test
Generating parallel test scripts for testutil_test
Please install gflags to run block_cache_trace_analyzer_test
Generating parallel test scripts for block_cache_trace_analyzer_test
Please install gflags to run io_tracer_parser_test
Generating parallel test scripts for io_tracer_parser_test
Generating parallel test scripts for ldb_cmd_test
Generating parallel test scripts for reduce_levels_test
Generating parallel test scripts for sst_dump_test
Please install gflags to run trace_analyzer test
Generating parallel test scripts for trace_analyzer_test
Generating parallel test scripts for block_cache_tracer_test
Generating parallel test scripts for io_tracer_test
Generating parallel test scripts for autovector_test
Please install gflags to run this test... Skipping...
Generating parallel test scripts for bloom_test
Generating parallel test scripts for coding_test
Generating parallel test scripts for crc32c_test
Generating parallel test scripts for defer_test
Please install gflags to run this test... Skipping...
Generating parallel test scripts for dynamic_bloom_test
Generating parallel test scripts for filelock_test
Generating parallel test scripts for file_reader_writer_test
NPHash64 id: 251012e7
Generating parallel test scripts for hash_test
Generating parallel test scripts for heap_test
Generating parallel test scripts for random_test
Generating parallel test scripts for rate_limiter_test
Generating parallel test scripts for repeatable_thread_test
Generating parallel test scripts for ribbon_test
Generating parallel test scripts for slice_test
Generating parallel test scripts for slice_transform_test
Generating parallel test scripts for timer_queue_test
Generating parallel test scripts for timer_test
Generating parallel test scripts for thread_list_test
Generating parallel test scripts for thread_local_test
Generating parallel test scripts for udt_util_test
Generating parallel test scripts for work_queue_test
Generating parallel test scripts for agg_merge_test
Generating parallel test scripts for blob_db_test
Generating parallel test scripts for backup_engine_test
Generating parallel test scripts for cassandra_format_test
Generating parallel test scripts for cassandra_functional_test
Generating parallel test scripts for cassandra_row_merge_test
Generating parallel test scripts for cassandra_serialize_test
Generating parallel test scripts for checkpoint_test
Generating parallel test scripts for memory_test
Generating parallel test scripts for env_timed_test
Generating parallel test scripts for stringappend_test
Generating parallel test scripts for object_registry_test
Generating parallel test scripts for option_change_migration_test
Generating parallel test scripts for options_util_test
Generating parallel test scripts for hash_table_test
Generating parallel test scripts for persistent_cache_test
Generating parallel test scripts for cache_simulator_test
Generating parallel test scripts for sim_cache_test
Generating parallel test scripts for compact_on_deletion_collector_test
Generating parallel test scripts for optimistic_transaction_test
Generating parallel test scripts for range_locking_test
Generating parallel test scripts for point_lock_manager_test
Generating parallel test scripts for transaction_test
Generating parallel test scripts for write_prepared_transaction_test
Generating parallel test scripts for write_unprepared_transaction_test
Generating parallel test scripts for write_committed_transaction_ts_test
Generating parallel test scripts for timestamped_snapshot_test
Generating parallel test scripts for ttl_test
Generating parallel test scripts for util_merge_operators_test
Generating parallel test scripts for write_batch_with_index_test
make[2]: Leaving directory '/root/rocksdb-8.9.1'
make[1]: Leaving directory '/root/rocksdb-8.9.1'
GEN check
make[1]: Entering directory '/root/rocksdb-8.9.1'
$DEBUG_LEVEL is 1, $LIB_MODE is shared
Makefile:185: Warning: Compiling in debug mode. Don't use the resulting binary in production

To monitor subtest <duration,pass/fail,name>,
run "make watch-log" in a separate window

Computers / CPU cores / Max jobs to run
1:local / 32 / 32

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 133s Left: 10881 AVG: 0.01s local:32/4322/100%/0.0s Note: Google Test filter = CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from CompactionJobStatsTest/CompactionJobStatsTest
[ RUN ] CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0
db/compaction/compaction_job_stats_test.cc:456: Failure
Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): db/compaction/compaction_job_stats_test.cc:456: Failure
Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7f9fec5b270f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7f9fec59cb25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7f9fecf6706b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7f9fecf6d50c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7f9fecf6d567] ?? ??:0
#5 /lib64/libstdc++.so.6(+0x967c8) [0x7f9fecf6d7c8] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(+0x5e40a) [0x7f9feece540a] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZNK7testing8internal12AssertHelperaSERKNS_7MessageE+0x6b) [0x7f9feed4cd6b] ?? ??:0
#8 ./compaction_job_stats_test() [0x41ebb3] rocksdb::CompactionJobStatsChecker::Verify(rocksdb::CompactionJobStats const&, rocksdb::CompactionJobStats const&) /root/rocksdb-8.9.1/db/compaction/compaction_job_stats_test.cc:455 (discriminator 7)
#9 ./compaction_job_stats_test() [0x41c9fd] rocksdb::CompactionJobStatsChecker::OnCompactionCompleted(rocksdb::DB*, rocksdb::CompactionJobInfo const&) /usr/include/c++/8/bits/stl_deque.h:1592
#10 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl27NotifyOnCompactionCompletedEPNS_16ColumnFamilyDataEPNS_10CompactionERKNS_6StatusERKNS_18CompactionJobStatsEi+0x240) [0x7f9fee3a4a00] ?? ??:0
#11 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl20BackgroundCompactionEPbPNS_10JobContextEPNS_9LogBufferEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x10ad) [0x7f9fee3ac72d] ?? ??:0
#12 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl24BackgroundCallCompactionEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x126) [0x7f9fee3ae546] ?? ??:0
#13 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl16BGWorkCompactionEPv+0x78) [0x7f9fee3aecf8] ?? ??:0
#14 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl8BGThreadEm+0x39d) [0x7f9fee71a97d] ?? ??:0
#15 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl15BGThreadWrapperEPv+0x5e) [0x7f9fee71acfe] ?? ??:0
#16 /lib64/libstdc++.so.6(+0xc2b73) [0x7f9fecf99b73] ?? ??:0
#17 /lib64/libpthread.so.0(+0x82de) [0x7f9fedb472de] ?? ??:0
#18 /lib64/libc.so.6(clone+0x43) [0x7f9fec676e83] ?? ??:0
t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0: line 4: 1733842 Aborted (core dumped) TEST_TMPDIR=$d ./compaction_job_stats_test --gtest_filter=CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0
Note: Google Test filter = CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from CompactionJobStatsTest/CompactionJobStatsTest
[ RUN ] CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1
db/compaction/compaction_job_stats_test.cc:456: Failure
Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): db/compaction/compaction_job_stats_test.cc:456: Failure
Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7fe80d31870f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7fe80d302b25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7fe80dccd06b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7fe80dcd350c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7fe80dcd3567] ?? ??:0
#5 /lib64/libstdc++.so.6(+0x967c8) [0x7fe80dcd37c8] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(+0x5e40a) [0x7fe80fa4b40a] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZNK7testing8internal12AssertHelperaSERKNS_7MessageE+0x6b) [0x7fe80fab2d6b] ?? ??:0
#8 ./compaction_job_stats_test() [0x41ebb3] rocksdb::CompactionJobStatsChecker::Verify(rocksdb::CompactionJobStats const&, rocksdb::CompactionJobStats const&) /root/rocksdb-8.9.1/db/compaction/compaction_job_stats_test.cc:455 (discriminator 7)
#9 ./compaction_job_stats_test() [0x41c9fd] rocksdb::CompactionJobStatsChecker::OnCompactionCompleted(rocksdb::DB*, rocksdb::CompactionJobInfo const&) /usr/include/c++/8/bits/stl_deque.h:1592
#10 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl27NotifyOnCompactionCompletedEPNS_16ColumnFamilyDataEPNS_10CompactionERKNS_6StatusERKNS_18CompactionJobStatsEi+0x240) [0x7fe80f10aa00] ?? ??:0
#11 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl20BackgroundCompactionEPbPNS_10JobContextEPNS_9LogBufferEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x10ad) [0x7fe80f11272d] ?? ??:0
#12 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl24BackgroundCallCompactionEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x126) [0x7fe80f114546] ?? ??:0
#13 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl16BGWorkCompactionEPv+0x78) [0x7fe80f114cf8] ?? ??:0
#14 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl8BGThreadEm+0x39d) [0x7fe80f48097d] ?? ??:0
#15 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl15BGThreadWrapperEPv+0x5e) [0x7fe80f480cfe] ?? ??:0
#16 /lib64/libstdc++.so.6(+0xc2b73) [0x7fe80dcffb73] ?? ??:0
#17 /lib64/libpthread.so.0(+0x82de) [0x7fe80e8ad2de] ?? ??:0
#18 /lib64/libc.so.6(clone+0x43) [0x7fe80d3dce83] ?? ??:0
t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1: line 4: 1733847 Aborted (core dumped) TEST_TMPDIR=$d ./compaction_job_stats_test --gtest_filter=CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1
ETA: 145s Left: 8157 AVG: 0.02s local:32/7046/100%/0.0s Note: Google Test filter = DBWALTest.TruncateLastLogAfterRecoverWALEmpty
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBWALTest
[ RUN ] DBWALTest.TruncateLastLogAfterRecoverWALEmpty
db/db_wal_test.cc:2472: Failure
Expected: (GetAllocatedFileSize(last_log)) < (preallocated_size), actual: 18427904 vs 18427904
terminate called without an active exception
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7f38f85a370f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7f38f858db25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7f38f8f5806b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7f38f8f5e50c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7f38f8f5e567] ?? ??:0
#5 ./db_wal_test() [0x454dcf] rocksdb::DBWALTest_TruncateLastLogAfterRecoverWALEmpty_Test::TestBody() /usr/include/c++/8/bits/basic_string.h:312
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x3e) [0x7f38fad48d1e] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing4Test3RunEv+0xd2) [0x7f38fad3f4a2] ?? ??:0
#8 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8TestInfo3RunEv+0x118) [0x7f38fad3f5e8] ?? ??:0
#9 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8TestCase3RunEv+0xb4) [0x7f38fad3f6c4] ?? ??:0
#10 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal12UnitTestImpl11RunAllTestsEv+0x474) [0x7f38fad3fbc4] ?? ??:0
#11 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0x3e) [0x7f38fad4925e] ?? ??:0
#12 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f38fad3fcdb] ?? ??:0
#13 ./db_wal_test() [0x42672d] main /root/rocksdb-8.9.1/db/db_wal_test.cc:2680
#14 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f38f858f6a3] ?? ??:0
#15 ./db_wal_test() [0x42911e] _start ??:?
t/run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty: line 4: 2523881 Aborted (core dumped) TEST_TMPDIR=$d ./db_wal_test --gtest_filter=DBWALTest.TruncateLastLogAfterRecoverWALEmpty
ETA: 20s Left: 1510 AVG: 0.01s local:32/13693/100%/0.0s Note: Google Test filter = DBTieredSecondaryCacheTest.BasicMultiGetTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTieredSecondaryCacheTest
[ RUN ] DBTieredSecondaryCacheTest.BasicMultiGetTest
cache/tiered_secondary_cache_test.cc:382: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
3u
Which is: 3
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): cache/tiered_secondary_cache_test.cc:382: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
3u
Which is: 3
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7fb70a68a70f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7fb70a674b25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7fb70b03f06b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7fb70b04550c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7fb70b045567] ?? ??:0
#5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7fb70b04581d] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7fb70ce30305] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7fb70ce26cdb] ?? ??:0
#8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711
#9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fb70a6766a3] ?? ??:0
#10 ./tiered_secondary_cache_test() [0x40881e] _start ??:?
t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest: line 4: 2877125 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.BasicMultiGetTest
Note: Google Test filter = DBTieredSecondaryCacheTest.BasicTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTieredSecondaryCacheTest
[ RUN ] DBTieredSecondaryCacheTest.BasicTest
cache/tiered_secondary_cache_test.cc:267: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
1u
Which is: 1
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): cache/tiered_secondary_cache_test.cc:267: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
1u
Which is: 1
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7f3326a4570f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7f3326a2fb25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7f33273fa06b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7f332740050c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7f3327400567] ?? ??:0
#5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7f332740081d] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7f33291eb305] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f33291e1cdb] ?? ??:0
#8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711
#9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f3326a316a3] ?? ??:0
#10 ./tiered_secondary_cache_test() [0x40881e] _start ??:?
t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest: line 4: 2877202 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.BasicTest
Note: Google Test filter = DBTieredSecondaryCacheTest.WaitAllTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTieredSecondaryCacheTest
[ RUN ] DBTieredSecondaryCacheTest.WaitAllTest
cache/tiered_secondary_cache_test.cc:524: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
3u
Which is: 3
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): cache/tiered_secondary_cache_test.cc:524: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
3u
Which is: 3
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7efd1f15270f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7efd1f13cb25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7efd1fb0706b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7efd1fb0d50c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7efd1fb0d567] ?? ??:0
#5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7efd1fb0d81d] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7efd218f8305] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7efd218eecdb] ?? ??:0
#8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711
#9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7efd1f13e6a3] ?? ??:0
#10 ./tiered_secondary_cache_test() [0x40881e] _start ??:?
t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest: line 4: 2877258 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.WaitAllTest
Note: Google Test filter = DBTieredSecondaryCacheTest.IterateTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTieredSecondaryCacheTest
[ RUN ] DBTieredSecondaryCacheTest.IterateTest
cache/tiered_secondary_cache_test.cc:623: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
8u
Which is: 8
terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException'
what(): cache/tiered_secondary_cache_test.cc:623: Failure
Expected equality of these values:
nvm_sec_cache()->num_insert_saved()
Which is: 0
8u
Which is: 8
Received signal 6 (Aborted)
Invoking GDB for stack trace...
GDB failed; falling back on backtrace+addr2line...
#0 /lib64/libc.so.6(gsignal+0x10f) [0x7f512e5bf70f] ?? ??:0
#1 /lib64/libc.so.6(abort+0x127) [0x7f512e5a9b25] ?? ??:0
#2 /lib64/libstdc++.so.6(+0x9006b) [0x7f512ef7406b] ?? ??:0
#3 /lib64/libstdc++.so.6(+0x9650c) [0x7f512ef7a50c] ?? ??:0
#4 /lib64/libstdc++.so.6(+0x96567) [0x7f512ef7a567] ?? ??:0
#5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7f512ef7a81d] ?? ??:0
#6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7f5130d65305] ?? ??:0
#7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f5130d5bcdb] ?? ??:0
#8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711
#9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f512e5ab6a3] ?? ??:0
#10 ./tiered_secondary_cache_test() [0x40881e] _start ??:?
t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest: line 4: 2877201 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.IterateTest
ETA: 0s Left: 2 AVG: 0.01s local:2/15201/100%/0.0s ETA: 0s Left: 0 AVG: 0.01s local:0/15203/100%/0.0s
Seq Host Starttime JobRuntime Send Receive Exitval Signal Command
3522 : 1705129126.818 7.490 0 0 134 0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0 >& t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0 || bash -c "cat t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0; exit $?"
3523 : 1705129126.824 7.492 0 0 134 0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1 >& t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1 || bash -c "cat t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1; exit $?"
7008 : 1705129205.799 6.342 0 0 134 0 t/run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty >& t/log-run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty || bash -c "cat t/log-run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty; exit $?"
13516 : 1705129266.253 4.743 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest; exit $?"
13517 : 1705129266.286 4.730 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest; exit $?"
13519 : 1705129266.336 4.687 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest; exit $?"
13518 : 1705129266.328 4.820 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest; exit $?"
make[1]: *** [Makefile:997: check_0] Error 1
make[1]: Leaving directory '/root/rocksdb-8.9.1'
make: *** [Makefile:1053: check] Error 2
[root@ff6189696b45 rocksdb-8.9.1]#

[adv0518]

gcc version:

[root@ff6189696b45 rocksdb-8.9.1]# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)

[adv0518]

operate system info(it's a docker):

[root@ff6189696b45 rocksdb-8.9.1]# more /etc/redhat-release
CentOS Linux release 8.5.2111

[rhubner]

Hello @adv0518 ,

I tested compilation on old unmaintained Centos 8 and I didn't found any problem. I tested two version, latest from main branch and the version you mentioned. I compiled everything with make : make all and then run the test you mentioned : compaction_job_stats_test, db_wal_test, tiered_secondary_cache_test.

Can you please confirm if this problem still persist or if you can provide more info how are you compiling and how we can replicate the issue.

$ cat /etc/redhat-release
CentOS Linux release 8.5.2111

Radek