tvl-depot/absl/container/internal/hashtablez_sampler_test.cc

356 lines
11 KiB
C++
Raw Normal View History

Export of internal Abseil changes. -- 7fa1107161a03dac53fb84c2b06d8092616c7b13 by Abseil Team <absl-team@google.com>: Harden the generic stacktrace implementation for use during early program execution PiperOrigin-RevId: 226375950 -- 079f9969329f5eb66f647dd3c44b17541b1bf217 by Matt Kulukundis <kfm@google.com>: Workaround platforms that have over-aggressive warnings on -Wexit-time-destructors PiperOrigin-RevId: 226362948 -- 1447943f509be681ca5495add0162c750ef237f1 by Matt Kulukundis <kfm@google.com>: Switch from 64 to size_t atomics so they work on embedded platforms that do not have 64 bit atomics. PiperOrigin-RevId: 226210704 -- d14d49837ae2bcde74051e0c79c18ee0f43866b9 by Tom Manshreck <shreck@google.com>: Develop initial documentation for API breaking changes process: PiperOrigin-RevId: 226210021 -- 7ea3d7fe0e86979dab83a5fc9cc3bf1d6cb3bd53 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 226195522 -- 7de873e880d7f016a4fa1e08d626f0535cc470af by Abseil Team <absl-team@google.com>: Make Abseil LICENSE files newline terminated, with a single trailing blank line. Also remove line-ending whitespace. PiperOrigin-RevId: 226182949 -- 7d00643fadfad7f0d992c68bd9d2ed2e5bc960b0 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226045282 -- c4a0a11c0ce2875271191e477f3d36eaaeca4613 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226038273 -- 8ee4ebbb1ae5cda119e436e5ff7e3aa966608c10 by Matt Kulukundis <kfm@google.com>: Adds a global sampler which tracks a fraction of live tables for collecting telemetry data. PiperOrigin-RevId: 226032080 -- d576446f050518cd1b0ae447d682d8552f0e7e30 by Mark Barolak <mbar@google.com>: Replace an internal CaseEqual function with calls to the identical absl::EqualsIgnoreCase. This closes out a rather old TODO. PiperOrigin-RevId: 226024779 -- 6b23f1ee028a5ffa608c920424f1220a117a8f3d by Abseil Team <absl-team@google.com>: Add December 2018 LTS branch to list of LTS branches. PiperOrigin-RevId: 226011333 -- bb0833a43bdaef4c8c059b17bcd27ba9a085a114 by Mark Barolak <mbar@google.com>: Explicitly state that when the SimpleAtoi family of functions encounter an error, the value of their output parameter is unspecified. Also standardize the name of the output parameter to be `out`. PiperOrigin-RevId: 225997035 -- 46c1876b1a248eabda7545daa61a74a4cdfe9077 by Abseil Team <absl-team@google.com>: Remove deprecated CMake function absl_test, absl_library and absl_header_library PiperOrigin-RevId: 225950041 GitOrigin-RevId: 7fa1107161a03dac53fb84c2b06d8092616c7b13 Change-Id: I2ca9d3aada9292614527d1339a7557494139b806
2018-12-20 21:29:59 +01:00
// Copyright 2018 The Abseil Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
Export of internal Abseil changes. -- 7fa1107161a03dac53fb84c2b06d8092616c7b13 by Abseil Team <absl-team@google.com>: Harden the generic stacktrace implementation for use during early program execution PiperOrigin-RevId: 226375950 -- 079f9969329f5eb66f647dd3c44b17541b1bf217 by Matt Kulukundis <kfm@google.com>: Workaround platforms that have over-aggressive warnings on -Wexit-time-destructors PiperOrigin-RevId: 226362948 -- 1447943f509be681ca5495add0162c750ef237f1 by Matt Kulukundis <kfm@google.com>: Switch from 64 to size_t atomics so they work on embedded platforms that do not have 64 bit atomics. PiperOrigin-RevId: 226210704 -- d14d49837ae2bcde74051e0c79c18ee0f43866b9 by Tom Manshreck <shreck@google.com>: Develop initial documentation for API breaking changes process: PiperOrigin-RevId: 226210021 -- 7ea3d7fe0e86979dab83a5fc9cc3bf1d6cb3bd53 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 226195522 -- 7de873e880d7f016a4fa1e08d626f0535cc470af by Abseil Team <absl-team@google.com>: Make Abseil LICENSE files newline terminated, with a single trailing blank line. Also remove line-ending whitespace. PiperOrigin-RevId: 226182949 -- 7d00643fadfad7f0d992c68bd9d2ed2e5bc960b0 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226045282 -- c4a0a11c0ce2875271191e477f3d36eaaeca4613 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226038273 -- 8ee4ebbb1ae5cda119e436e5ff7e3aa966608c10 by Matt Kulukundis <kfm@google.com>: Adds a global sampler which tracks a fraction of live tables for collecting telemetry data. PiperOrigin-RevId: 226032080 -- d576446f050518cd1b0ae447d682d8552f0e7e30 by Mark Barolak <mbar@google.com>: Replace an internal CaseEqual function with calls to the identical absl::EqualsIgnoreCase. This closes out a rather old TODO. PiperOrigin-RevId: 226024779 -- 6b23f1ee028a5ffa608c920424f1220a117a8f3d by Abseil Team <absl-team@google.com>: Add December 2018 LTS branch to list of LTS branches. PiperOrigin-RevId: 226011333 -- bb0833a43bdaef4c8c059b17bcd27ba9a085a114 by Mark Barolak <mbar@google.com>: Explicitly state that when the SimpleAtoi family of functions encounter an error, the value of their output parameter is unspecified. Also standardize the name of the output parameter to be `out`. PiperOrigin-RevId: 225997035 -- 46c1876b1a248eabda7545daa61a74a4cdfe9077 by Abseil Team <absl-team@google.com>: Remove deprecated CMake function absl_test, absl_library and absl_header_library PiperOrigin-RevId: 225950041 GitOrigin-RevId: 7fa1107161a03dac53fb84c2b06d8092616c7b13 Change-Id: I2ca9d3aada9292614527d1339a7557494139b806
2018-12-20 21:29:59 +01:00
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "absl/container/internal/hashtablez_sampler.h"
#include <atomic>
#include <limits>
#include <random>
#include "gmock/gmock.h"
#include "gtest/gtest.h"
#include "absl/base/attributes.h"
#include "absl/container/internal/have_sse.h"
#include "absl/synchronization/blocking_counter.h"
#include "absl/synchronization/internal/thread_pool.h"
#include "absl/synchronization/mutex.h"
#include "absl/synchronization/notification.h"
#include "absl/time/clock.h"
#include "absl/time/time.h"
#if SWISSTABLE_HAVE_SSE2
constexpr int kProbeLength = 16;
#else
constexpr int kProbeLength = 8;
#endif
namespace absl {
namespace container_internal {
class HashtablezInfoHandlePeer {
public:
static bool IsSampled(const HashtablezInfoHandle& h) {
return h.info_ != nullptr;
}
static HashtablezInfo* GetInfo(HashtablezInfoHandle* h) { return h->info_; }
};
namespace {
using ::absl::synchronization_internal::ThreadPool;
using ::testing::IsEmpty;
using ::testing::UnorderedElementsAre;
std::vector<size_t> GetSizes(HashtablezSampler* s) {
std::vector<size_t> res;
s->Iterate([&](const HashtablezInfo& info) {
res.push_back(info.size.load(std::memory_order_acquire));
});
return res;
}
HashtablezInfo* Register(HashtablezSampler* s, size_t size) {
auto* info = s->Register();
assert(info != nullptr);
info->size.store(size);
return info;
}
TEST(HashtablezInfoTest, PrepareForSampling) {
absl::Time test_start = absl::Now();
HashtablezInfo info;
absl::MutexLock l(&info.init_mu);
info.PrepareForSampling();
EXPECT_EQ(info.capacity.load(), 0);
EXPECT_EQ(info.size.load(), 0);
EXPECT_EQ(info.num_erases.load(), 0);
EXPECT_EQ(info.max_probe_length.load(), 0);
EXPECT_EQ(info.total_probe_length.load(), 0);
EXPECT_EQ(info.hashes_bitwise_or.load(), 0);
EXPECT_EQ(info.hashes_bitwise_and.load(), ~size_t{});
EXPECT_GE(info.create_time, test_start);
info.capacity.store(1, std::memory_order_relaxed);
info.size.store(1, std::memory_order_relaxed);
info.num_erases.store(1, std::memory_order_relaxed);
info.max_probe_length.store(1, std::memory_order_relaxed);
info.total_probe_length.store(1, std::memory_order_relaxed);
info.hashes_bitwise_or.store(1, std::memory_order_relaxed);
info.hashes_bitwise_and.store(1, std::memory_order_relaxed);
info.create_time = test_start - absl::Hours(20);
info.PrepareForSampling();
EXPECT_EQ(info.capacity.load(), 0);
EXPECT_EQ(info.size.load(), 0);
EXPECT_EQ(info.num_erases.load(), 0);
EXPECT_EQ(info.max_probe_length.load(), 0);
EXPECT_EQ(info.total_probe_length.load(), 0);
EXPECT_EQ(info.hashes_bitwise_or.load(), 0);
EXPECT_EQ(info.hashes_bitwise_and.load(), ~size_t{});
EXPECT_GE(info.create_time, test_start);
}
TEST(HashtablezInfoTest, RecordStorageChanged) {
HashtablezInfo info;
absl::MutexLock l(&info.init_mu);
info.PrepareForSampling();
RecordStorageChangedSlow(&info, 17, 47);
EXPECT_EQ(info.size.load(), 17);
EXPECT_EQ(info.capacity.load(), 47);
RecordStorageChangedSlow(&info, 20, 20);
EXPECT_EQ(info.size.load(), 20);
EXPECT_EQ(info.capacity.load(), 20);
}
TEST(HashtablezInfoTest, RecordInsert) {
HashtablezInfo info;
absl::MutexLock l(&info.init_mu);
info.PrepareForSampling();
EXPECT_EQ(info.max_probe_length.load(), 0);
RecordInsertSlow(&info, 0x0000FF00, 6 * kProbeLength);
EXPECT_EQ(info.max_probe_length.load(), 6);
EXPECT_EQ(info.hashes_bitwise_and.load(), 0x0000FF00);
EXPECT_EQ(info.hashes_bitwise_or.load(), 0x0000FF00);
RecordInsertSlow(&info, 0x000FF000, 4 * kProbeLength);
EXPECT_EQ(info.max_probe_length.load(), 6);
EXPECT_EQ(info.hashes_bitwise_and.load(), 0x0000F000);
EXPECT_EQ(info.hashes_bitwise_or.load(), 0x000FFF00);
RecordInsertSlow(&info, 0x00FF0000, 12 * kProbeLength);
EXPECT_EQ(info.max_probe_length.load(), 12);
EXPECT_EQ(info.hashes_bitwise_and.load(), 0x00000000);
EXPECT_EQ(info.hashes_bitwise_or.load(), 0x00FFFF00);
}
TEST(HashtablezInfoTest, RecordErase) {
HashtablezInfo info;
absl::MutexLock l(&info.init_mu);
info.PrepareForSampling();
EXPECT_EQ(info.num_erases.load(), 0);
EXPECT_EQ(info.size.load(), 0);
RecordInsertSlow(&info, 0x0000FF00, 6 * kProbeLength);
EXPECT_EQ(info.size.load(), 1);
RecordEraseSlow(&info);
EXPECT_EQ(info.size.load(), 0);
EXPECT_EQ(info.num_erases.load(), 1);
}
Export of internal Abseil changes. -- 3d20ce6cd6541579abecaba169d4b8716d511272 by Jon Cohen <cohenjon@google.com>: Only use LSAN for clang version >= 3.5. This should fix https://github.com/abseil/abseil-cpp/issues/244 PiperOrigin-RevId: 234675129 -- e15bd4ec7a81aa95cc3d09fa1e0e81d58ae478fb by Conrad Parker <conradparker@google.com>: Fix errors in apply() sample code The following changes are made: * Make the example method public. * Give the two user functions different names to avoid confusion about whether apply() can select the correct overload of a function based on its tuple argument (it can't). * Pass tuple2 to the second example apply() invocation, instead of passing its contents individually. * Fix a s/tuple/tuple3/ typo in the third example apply() invocation. PiperOrigin-RevId: 234223407 -- de0ed71e21bc76ddf9fe715fdbaef74cd0df95c7 by Abseil Team <absl-team@google.com>: First test if a macro is defined to avoid -Wundef. ABSL clients may need to compile their code with the -Wundef warning flag. It will be helpful if ABSL header files can be compiled without the -Wundef warning. How to avoid the -Wundef warning: If a macro may be undefined, we need to first test whether the macro is defined before testing its value. We can't rely on the C preprocessor rule that an undefined macro has the value 0L. PiperOrigin-RevId: 234201123 -- fa484ad7dae0cac21140a96662809ecb0ec8eb5d by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 234185697 -- d69b1baef681e27954b065375ecf9c2320463b2b by Samuel Benzaquen <sbenza@google.com>: Mix pointers more thoroughly. Some pointer alignments interact badly with the mixing constant. By mixing twice we reduce this problem. PiperOrigin-RevId: 234178401 -- 1041d0e474610f3a8fea0db90958857327d6da1c by Samuel Benzaquen <sbenza@google.com>: Record rehashes in the hashtablez struct. Only recording the probe length on insertion causes a huge overestimation of the total probe length at any given time. With natural growth, elements are inserted when the load factor is between (max load/2, max load). However, after a rehash the majority of elements are actually inserted when the load factor is less than max/2 and have a much lower average probe length. Also reset some values when the table is cleared. PiperOrigin-RevId: 234013580 -- 299205caf3c89c47339f7409bc831746602cea84 by Mark Barolak <mbar@google.com>: Fix a sample code snippet that assumes `absl::string_view::const_iterator` is `const char*`. This is generally true, however in C++17 builds, absl::string_view is an alias for std::string_view and on MSVC, the std::string_view::const_iterator is an object instead of just a pointer. PiperOrigin-RevId: 233844595 -- af6c6370cf51a1e6c1469c79dfb2a486a4009136 by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 233773470 -- 6e59e4b8e2bb6101b448f0f32b0bea81fe399ccf by Abseil Team <absl-team@google.com>: fix typo in {Starts|Ends}WithIgnoreCase comment in match.h PiperOrigin-RevId: 233662951 GitOrigin-RevId: 3d20ce6cd6541579abecaba169d4b8716d511272 Change-Id: Ib9a29b1c38c6aedf5d9f3f7f00596e8d30e864dd
2019-02-19 23:29:09 +01:00
TEST(HashtablezInfoTest, RecordRehash) {
HashtablezInfo info;
absl::MutexLock l(&info.init_mu);
info.PrepareForSampling();
RecordInsertSlow(&info, 0x1, 0);
RecordInsertSlow(&info, 0x2, kProbeLength);
RecordInsertSlow(&info, 0x4, kProbeLength);
RecordInsertSlow(&info, 0x8, 2 * kProbeLength);
EXPECT_EQ(info.size.load(), 4);
EXPECT_EQ(info.total_probe_length.load(), 4);
RecordEraseSlow(&info);
RecordEraseSlow(&info);
EXPECT_EQ(info.size.load(), 2);
EXPECT_EQ(info.total_probe_length.load(), 4);
EXPECT_EQ(info.num_erases.load(), 2);
RecordRehashSlow(&info, 3 * kProbeLength);
EXPECT_EQ(info.size.load(), 2);
EXPECT_EQ(info.total_probe_length.load(), 3);
EXPECT_EQ(info.num_erases.load(), 0);
}
Export of internal Abseil changes. -- 7fa1107161a03dac53fb84c2b06d8092616c7b13 by Abseil Team <absl-team@google.com>: Harden the generic stacktrace implementation for use during early program execution PiperOrigin-RevId: 226375950 -- 079f9969329f5eb66f647dd3c44b17541b1bf217 by Matt Kulukundis <kfm@google.com>: Workaround platforms that have over-aggressive warnings on -Wexit-time-destructors PiperOrigin-RevId: 226362948 -- 1447943f509be681ca5495add0162c750ef237f1 by Matt Kulukundis <kfm@google.com>: Switch from 64 to size_t atomics so they work on embedded platforms that do not have 64 bit atomics. PiperOrigin-RevId: 226210704 -- d14d49837ae2bcde74051e0c79c18ee0f43866b9 by Tom Manshreck <shreck@google.com>: Develop initial documentation for API breaking changes process: PiperOrigin-RevId: 226210021 -- 7ea3d7fe0e86979dab83a5fc9cc3bf1d6cb3bd53 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 226195522 -- 7de873e880d7f016a4fa1e08d626f0535cc470af by Abseil Team <absl-team@google.com>: Make Abseil LICENSE files newline terminated, with a single trailing blank line. Also remove line-ending whitespace. PiperOrigin-RevId: 226182949 -- 7d00643fadfad7f0d992c68bd9d2ed2e5bc960b0 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226045282 -- c4a0a11c0ce2875271191e477f3d36eaaeca4613 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226038273 -- 8ee4ebbb1ae5cda119e436e5ff7e3aa966608c10 by Matt Kulukundis <kfm@google.com>: Adds a global sampler which tracks a fraction of live tables for collecting telemetry data. PiperOrigin-RevId: 226032080 -- d576446f050518cd1b0ae447d682d8552f0e7e30 by Mark Barolak <mbar@google.com>: Replace an internal CaseEqual function with calls to the identical absl::EqualsIgnoreCase. This closes out a rather old TODO. PiperOrigin-RevId: 226024779 -- 6b23f1ee028a5ffa608c920424f1220a117a8f3d by Abseil Team <absl-team@google.com>: Add December 2018 LTS branch to list of LTS branches. PiperOrigin-RevId: 226011333 -- bb0833a43bdaef4c8c059b17bcd27ba9a085a114 by Mark Barolak <mbar@google.com>: Explicitly state that when the SimpleAtoi family of functions encounter an error, the value of their output parameter is unspecified. Also standardize the name of the output parameter to be `out`. PiperOrigin-RevId: 225997035 -- 46c1876b1a248eabda7545daa61a74a4cdfe9077 by Abseil Team <absl-team@google.com>: Remove deprecated CMake function absl_test, absl_library and absl_header_library PiperOrigin-RevId: 225950041 GitOrigin-RevId: 7fa1107161a03dac53fb84c2b06d8092616c7b13 Change-Id: I2ca9d3aada9292614527d1339a7557494139b806
2018-12-20 21:29:59 +01:00
TEST(HashtablezSamplerTest, SmallSampleParameter) {
SetHashtablezEnabled(true);
SetHashtablezSampleParameter(100);
for (int i = 0; i < 1000; ++i) {
int64_t next_sample = 0;
HashtablezInfo* sample = SampleSlow(&next_sample);
EXPECT_GT(next_sample, 0);
EXPECT_NE(sample, nullptr);
UnsampleSlow(sample);
}
}
TEST(HashtablezSamplerTest, LargeSampleParameter) {
SetHashtablezEnabled(true);
SetHashtablezSampleParameter(std::numeric_limits<int32_t>::max());
for (int i = 0; i < 1000; ++i) {
int64_t next_sample = 0;
HashtablezInfo* sample = SampleSlow(&next_sample);
EXPECT_GT(next_sample, 0);
EXPECT_NE(sample, nullptr);
UnsampleSlow(sample);
}
}
TEST(HashtablezSamplerTest, Sample) {
SetHashtablezEnabled(true);
SetHashtablezSampleParameter(100);
int64_t num_sampled = 0;
int64_t total = 0;
Export of internal Abseil changes. -- 0925eb11f7730d5a1e538b9e6c2d1f8239f5fdc0 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 250694613 -- 4e1690e492a8399da1b1450ff5f21adf435fcef5 by Greg Falcon <gfalcon@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 250684222 -- c7281b44eb005c21d45bdc0b437e47c7787275bc by Derek Mauro <dmauro@google.com>: Fix warnings on Xcode/clang. PiperOrigin-RevId: 250552120 -- 5ea3b1cfa759a7e6c70e7558f27c762baf818f7d by CJ Johnson <johnsoncj@google.com>: Minor edits to InlinedVector benchmarks: Adds DoNotOptimize to the member functions of NontrivialType, removes unused template parameter for InlVec typeder PiperOrigin-RevId: 250505482 -- 7fe9c02b49121936c5b47034f20272a916111174 by Matt Kulukundis <kfm@google.com>: Internal change. PiperOrigin-RevId: 250376825 -- ad348c9c0eb37449874648e8544c56343c1dfaef by CJ Johnson <johnsoncj@google.com>: Minor edits to InlinedVector benchmark PiperOrigin-RevId: 250361830 -- 55e8b411431e982059af73b160fa6bcef90e87f7 by CJ Johnson <johnsoncj@google.com>: Switches use of allocator_traits to the Abseil backport to leverage existing bug fixes and workarounds PiperOrigin-RevId: 250359321 -- b0edbe3218940128523e36388a31ff90df01a364 by CJ Johnson <johnsoncj@google.com>: Updates exception safety test for InlinedVector to use TYPED_TEST (with fixtures) which increases the coverage without complicating the code. PiperOrigin-RevId: 250359198 -- 8ab55c9a8c191aabcb562cf1789f360eba0b1a81 by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 250281509 -- dd8a67f4f9e5e8930457203c18205183b8306b5a by Abseil Team <absl-team@google.com>: Change the suggestions for the non-strict Next/PrevWeekday() calls. Previously we suggested `PrevWeekday(d, Weekday::thursday) + 7` to get the _following_ Thursday if d is not already a Thursday, but `NextWeekday(d - 1, Weekday::thursday)` is more intuitive, and probably even a little faster. Similarly for the _previous_ Thursday if d is not already a Thursday, suggest `PrevWeekday(d + 1, Weekday::thursday)` instead of `NextWeekday(d, Weekday::thursday) - 7`. PiperOrigin-RevId: 249840844 -- 4775dce26cdb0560011a5d1ecdc8e0c20f856911 by Abseil Team <absl-team@google.com>: Change the DbgHelp lib pragma to use lowercase and an extension. This matches the conventions used in the Windows SDK. PiperOrigin-RevId: 249826229 -- cbc49e865f3715ebe1983307d4f02e50998b2958 by Abseil Team <absl-team@google.com>: Introduce Abseil prefixed thread annotation macros. PiperOrigin-RevId: 249825200 -- 8e97c8dfbadb78743c2421b494398be25f632cb1 by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 249737936 -- 0d720538bc6af3e77ac19def27a4a0514c2800d2 by Abseil Team <absl-team@google.com>: Tell CMake Abseil is a C++ project. PiperOrigin-RevId: 249726509 -- 20651845edf236757b3ffb3eaeff330af81be40a by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 249726377 GitOrigin-RevId: 0925eb11f7730d5a1e538b9e6c2d1f8239f5fdc0 Change-Id: I05d18d30ad4e9ace6b60a17d2dc6fd699643fe30
2019-05-30 17:42:34 +02:00
double sample_rate = 0.0;
Export of internal Abseil changes. -- 7fa1107161a03dac53fb84c2b06d8092616c7b13 by Abseil Team <absl-team@google.com>: Harden the generic stacktrace implementation for use during early program execution PiperOrigin-RevId: 226375950 -- 079f9969329f5eb66f647dd3c44b17541b1bf217 by Matt Kulukundis <kfm@google.com>: Workaround platforms that have over-aggressive warnings on -Wexit-time-destructors PiperOrigin-RevId: 226362948 -- 1447943f509be681ca5495add0162c750ef237f1 by Matt Kulukundis <kfm@google.com>: Switch from 64 to size_t atomics so they work on embedded platforms that do not have 64 bit atomics. PiperOrigin-RevId: 226210704 -- d14d49837ae2bcde74051e0c79c18ee0f43866b9 by Tom Manshreck <shreck@google.com>: Develop initial documentation for API breaking changes process: PiperOrigin-RevId: 226210021 -- 7ea3d7fe0e86979dab83a5fc9cc3bf1d6cb3bd53 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 226195522 -- 7de873e880d7f016a4fa1e08d626f0535cc470af by Abseil Team <absl-team@google.com>: Make Abseil LICENSE files newline terminated, with a single trailing blank line. Also remove line-ending whitespace. PiperOrigin-RevId: 226182949 -- 7d00643fadfad7f0d992c68bd9d2ed2e5bc960b0 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226045282 -- c4a0a11c0ce2875271191e477f3d36eaaeca4613 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226038273 -- 8ee4ebbb1ae5cda119e436e5ff7e3aa966608c10 by Matt Kulukundis <kfm@google.com>: Adds a global sampler which tracks a fraction of live tables for collecting telemetry data. PiperOrigin-RevId: 226032080 -- d576446f050518cd1b0ae447d682d8552f0e7e30 by Mark Barolak <mbar@google.com>: Replace an internal CaseEqual function with calls to the identical absl::EqualsIgnoreCase. This closes out a rather old TODO. PiperOrigin-RevId: 226024779 -- 6b23f1ee028a5ffa608c920424f1220a117a8f3d by Abseil Team <absl-team@google.com>: Add December 2018 LTS branch to list of LTS branches. PiperOrigin-RevId: 226011333 -- bb0833a43bdaef4c8c059b17bcd27ba9a085a114 by Mark Barolak <mbar@google.com>: Explicitly state that when the SimpleAtoi family of functions encounter an error, the value of their output parameter is unspecified. Also standardize the name of the output parameter to be `out`. PiperOrigin-RevId: 225997035 -- 46c1876b1a248eabda7545daa61a74a4cdfe9077 by Abseil Team <absl-team@google.com>: Remove deprecated CMake function absl_test, absl_library and absl_header_library PiperOrigin-RevId: 225950041 GitOrigin-RevId: 7fa1107161a03dac53fb84c2b06d8092616c7b13 Change-Id: I2ca9d3aada9292614527d1339a7557494139b806
2018-12-20 21:29:59 +01:00
for (int i = 0; i < 1000000; ++i) {
HashtablezInfoHandle h = Sample();
++total;
if (HashtablezInfoHandlePeer::IsSampled(h)) {
++num_sampled;
}
sample_rate = static_cast<double>(num_sampled) / total;
if (0.005 < sample_rate && sample_rate < 0.015) break;
}
EXPECT_NEAR(sample_rate, 0.01, 0.005);
}
TEST(HashtablezSamplerTest, Handle) {
auto& sampler = HashtablezSampler::Global();
HashtablezInfoHandle h(sampler.Register());
auto* info = HashtablezInfoHandlePeer::GetInfo(&h);
info->hashes_bitwise_and.store(0x12345678, std::memory_order_relaxed);
bool found = false;
sampler.Iterate([&](const HashtablezInfo& h) {
if (&h == info) {
EXPECT_EQ(h.hashes_bitwise_and.load(), 0x12345678);
found = true;
}
});
EXPECT_TRUE(found);
h = HashtablezInfoHandle();
found = false;
sampler.Iterate([&](const HashtablezInfo& h) {
if (&h == info) {
// this will only happen if some other thread has resurrected the info
// the old handle was using.
if (h.hashes_bitwise_and.load() == 0x12345678) {
found = true;
}
}
});
EXPECT_FALSE(found);
}
TEST(HashtablezSamplerTest, Registration) {
HashtablezSampler sampler;
auto* info1 = Register(&sampler, 1);
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(1));
auto* info2 = Register(&sampler, 2);
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(1, 2));
info1->size.store(3);
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(3, 2));
sampler.Unregister(info1);
sampler.Unregister(info2);
}
TEST(HashtablezSamplerTest, Unregistration) {
HashtablezSampler sampler;
std::vector<HashtablezInfo*> infos;
for (size_t i = 0; i < 3; ++i) {
infos.push_back(Register(&sampler, i));
}
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(0, 1, 2));
sampler.Unregister(infos[1]);
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(0, 2));
infos.push_back(Register(&sampler, 3));
infos.push_back(Register(&sampler, 4));
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(0, 2, 3, 4));
sampler.Unregister(infos[3]);
EXPECT_THAT(GetSizes(&sampler), UnorderedElementsAre(0, 2, 4));
sampler.Unregister(infos[0]);
sampler.Unregister(infos[2]);
sampler.Unregister(infos[4]);
EXPECT_THAT(GetSizes(&sampler), IsEmpty());
}
TEST(HashtablezSamplerTest, MultiThreaded) {
HashtablezSampler sampler;
Notification stop;
ThreadPool pool(10);
for (int i = 0; i < 10; ++i) {
pool.Schedule([&sampler, &stop]() {
std::random_device rd;
std::mt19937 gen(rd());
std::vector<HashtablezInfo*> infoz;
while (!stop.HasBeenNotified()) {
if (infoz.empty()) {
infoz.push_back(sampler.Register());
}
switch (std::uniform_int_distribution<>(0, 2)(gen)) {
case 0: {
infoz.push_back(sampler.Register());
break;
}
case 1: {
size_t p =
std::uniform_int_distribution<>(0, infoz.size() - 1)(gen);
HashtablezInfo* info = infoz[p];
infoz[p] = infoz.back();
infoz.pop_back();
sampler.Unregister(info);
break;
}
case 2: {
absl::Duration oldest = absl::ZeroDuration();
sampler.Iterate([&](const HashtablezInfo& info) {
oldest = std::max(oldest, absl::Now() - info.create_time);
});
ASSERT_GE(oldest, absl::ZeroDuration());
break;
}
}
}
});
}
// The threads will hammer away. Give it a little bit of time for tsan to
// spot errors.
absl::SleepFor(absl::Seconds(3));
stop.Notify();
}
Export of internal Abseil changes. -- 5804cc13b413412988248835459b90cd15ec43d9 by Abseil Team <absl-team@google.com>: Mark raw_hash_set::clear() with the ABSL_ATTRIBUTE_REINITIALIZES attribute. This prevents false positives in the clang-tidy check bugprone-use-after-move; it allows reset() to be called on a moved-from raw_hash_set without any warnings, and the raw_hash_set will thereafter be regarded as initialized again. PiperOrigin-RevId: 230717196 -- ff5961a5600ae19b69a9cba6912126cdf2858f38 by CJ Johnson <johnsoncj@google.com>: Swaps DisableIfIntegral<> for EnableIfInputIterator<> for Iterator member functions of InlinedVector PiperOrigin-RevId: 230559521 -- 3f9754ccbeecbd40f235c6f2465279e045ff51d9 by Derek Mauro <dmauro@google.com>: Import GitHub PR 254 https://github.com/abseil/abseil-cpp/pull/254 Fixes warnings from -Wclass-memaccess (base_internal::ThreadIdentity? with no trivial copy-assignment). PiperOrigin-RevId: 230536048 -- 8af03a654ce9a4a7f55384bc7eb1ed64878ac2ec by Chris Kennelly <ckennelly@google.com>: absl: cap SpinLock backoff to 4ms The current backoff logic has 3 problems: 1. It can produce too high values (up to 256ms), which can negatively affect tail latency. The value was chosen long time ago and now it's a good idea to reconsider it. 2. It does not have low bound, so on any iteration it can produce a very small value that will lead to unnecessary cpu consumption. 3. It does not increase low bound with the number of iterations. So if the SpinLock is actually somehow locked for a very prolonged time, a waiter can still wake periodically. Rework the logic to solve these problems. Add lower bound of 128us, no code should rely on absence of episodic delays in this range as they can occur everywhere. Lower upper bound to 4ms. A thread sleeping for 4ms does not consume significant cpu time (see below). Grow lower bound with the number of iterations. This is cpu consumption of a process doing usleep(x) in a loop (sampled with ps): 64us -> 4.0% 128us -> 2.7% 256us -> 3.5% 512us -> 2.8% 1024us -> 1.6% 2048us -> 0.6% 4096us -> 0.3% 8192us -> 0.0% Few millisecond sleeps do not consume significant time. PiperOrigin-RevId: 230534015 -- 37ebba92289ca556cb2412cd8b3cb4c1ead3def7 by Samuel Benzaquen <sbenza@google.com>: Add override and dispose hooks to the hashtable sampler. PiperOrigin-RevId: 230353438 -- 89c8f90175233ce9964eb3412df04e8a3cff0c0f by Andy Getzendanner <durandal@google.com>: Fix a comment typo. PiperOrigin-RevId: 229986838 GitOrigin-RevId: 5804cc13b413412988248835459b90cd15ec43d9 Change-Id: Iedb5e2cc9c0b924635c1c87b537780ab6b5b899f
2019-01-24 16:23:40 +01:00
TEST(HashtablezSamplerTest, Callback) {
HashtablezSampler sampler;
auto* info1 = Register(&sampler, 1);
auto* info2 = Register(&sampler, 2);
static const HashtablezInfo* expected;
auto callback = [](const HashtablezInfo& info) {
// We can't use `info` outside of this callback because the object will be
// disposed as soon as we return from here.
EXPECT_EQ(&info, expected);
};
// Set the callback.
EXPECT_EQ(sampler.SetDisposeCallback(callback), nullptr);
expected = info1;
sampler.Unregister(info1);
// Unset the callback.
EXPECT_EQ(callback, sampler.SetDisposeCallback(nullptr));
expected = nullptr; // no more calls.
sampler.Unregister(info2);
}
Export of internal Abseil changes. -- 7fa1107161a03dac53fb84c2b06d8092616c7b13 by Abseil Team <absl-team@google.com>: Harden the generic stacktrace implementation for use during early program execution PiperOrigin-RevId: 226375950 -- 079f9969329f5eb66f647dd3c44b17541b1bf217 by Matt Kulukundis <kfm@google.com>: Workaround platforms that have over-aggressive warnings on -Wexit-time-destructors PiperOrigin-RevId: 226362948 -- 1447943f509be681ca5495add0162c750ef237f1 by Matt Kulukundis <kfm@google.com>: Switch from 64 to size_t atomics so they work on embedded platforms that do not have 64 bit atomics. PiperOrigin-RevId: 226210704 -- d14d49837ae2bcde74051e0c79c18ee0f43866b9 by Tom Manshreck <shreck@google.com>: Develop initial documentation for API breaking changes process: PiperOrigin-RevId: 226210021 -- 7ea3d7fe0e86979dab83a5fc9cc3bf1d6cb3bd53 by Abseil Team <absl-team@google.com>: Import of CCTZ from GitHub. PiperOrigin-RevId: 226195522 -- 7de873e880d7f016a4fa1e08d626f0535cc470af by Abseil Team <absl-team@google.com>: Make Abseil LICENSE files newline terminated, with a single trailing blank line. Also remove line-ending whitespace. PiperOrigin-RevId: 226182949 -- 7d00643fadfad7f0d992c68bd9d2ed2e5bc960b0 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226045282 -- c4a0a11c0ce2875271191e477f3d36eaaeca4613 by Matt Kulukundis <kfm@google.com>: Internal cleanup PiperOrigin-RevId: 226038273 -- 8ee4ebbb1ae5cda119e436e5ff7e3aa966608c10 by Matt Kulukundis <kfm@google.com>: Adds a global sampler which tracks a fraction of live tables for collecting telemetry data. PiperOrigin-RevId: 226032080 -- d576446f050518cd1b0ae447d682d8552f0e7e30 by Mark Barolak <mbar@google.com>: Replace an internal CaseEqual function with calls to the identical absl::EqualsIgnoreCase. This closes out a rather old TODO. PiperOrigin-RevId: 226024779 -- 6b23f1ee028a5ffa608c920424f1220a117a8f3d by Abseil Team <absl-team@google.com>: Add December 2018 LTS branch to list of LTS branches. PiperOrigin-RevId: 226011333 -- bb0833a43bdaef4c8c059b17bcd27ba9a085a114 by Mark Barolak <mbar@google.com>: Explicitly state that when the SimpleAtoi family of functions encounter an error, the value of their output parameter is unspecified. Also standardize the name of the output parameter to be `out`. PiperOrigin-RevId: 225997035 -- 46c1876b1a248eabda7545daa61a74a4cdfe9077 by Abseil Team <absl-team@google.com>: Remove deprecated CMake function absl_test, absl_library and absl_header_library PiperOrigin-RevId: 225950041 GitOrigin-RevId: 7fa1107161a03dac53fb84c2b06d8092616c7b13 Change-Id: I2ca9d3aada9292614527d1339a7557494139b806
2018-12-20 21:29:59 +01:00
} // namespace
} // namespace container_internal
} // namespace absl