New Cache Config and LRU GC #1308

wu-hui · 2023-05-08T16:49:56Z

First half of the implementation, Android implementation will be a separate PR which will get merged into this one before everything is merged into main.

dconeybe

I didn't do a thorough review of everything, but I see major issues in the design of the "include" files. Those files must be platform-independent and cannot reference stuff in the ios sdk because they are used on Android as well. Also, the static_cast is troublesome to me because in principle we should be able to use the class hierarchy to avoid casting. Otherwise, there isn't much point to having a common base class IMO.

firestore/integration_test_internal/src/firestore_test.cc

firestore/integration_test_internal/src/settings_test.cc

firestore/src/include/firebase/firestore/local_cache_settings.h

firestore/src/common/local_cache_settings.cc

firestore/src/common/settings.cc

firestore/src/main/firestore_main.cc

wu-hui

Thanks for the review!

firestore/integration_test_internal/src/firestore_test.cc

firestore/integration_test_internal/src/settings_test.cc

firestore/src/common/local_cache_settings.cc

firestore/src/include/firebase/firestore/local_cache_settings.h

firestore/src/main/firestore_main.cc

firestore/src/common/settings.cc

wu-hui · 2023-06-02T13:55:53Z

@dconeybe I took some nice stuff from your prototype and added to this PR, but I decided to keep the enum/static_cast around.

I did try to use absl::variant and Impl classes, and I do not quite like the result, mostly because in the end the approved api does not work well. I also think they add too much complexity for a small gain. enum/static_cast are very well understood and used in our code base in many places.

Please let me know your thoughts, we can discuss this further.

dconeybe · 2023-06-02T17:12:29Z

I took a quick look and left some minor comments. I'll do a more holistic review later today or early next week.

dconeybe

I'm still concerned about the "main" API bleeding into the public header file. I don't see how this will work on Android. Please correct me if I'm wrong about that. Other than that, I've just left a few comments. I'll do a deeper review once I feel that the API will work for both Android and main.

firestore/src/include/firebase/firestore/local_cache_settings.h

dconeybe · 2023-06-07T16:01:20Z

firestore/src/include/firebase/firestore/settings.h

+   * Returns a reference to the `LocalCacheSettings` instance
+   * used to configure this SDK.
+   */
+  const LocalCacheSettings& local_cache_settings();


Consider returning by value (i.e. returning a copy) from local_cache_settings() since the lifetime of the returned reference is... complicated. Specifically, the returned reference becomes invalid if there is a subsequent call to set_local_cache_settings() or set_persistence_enabled() or a few other functions. IMO in this particular instance the clarity and memory safety of returning by value far outweighs any performance benefit of returning by reference.

I agree with what you said, but I don't think it's possible to return a copy of LocalCacheSetting because it is abstract?

We could consider returning the underlying shared_ptr though..this is not in the approved proposal but seems like a good amendment?

Hmm fair enough. Maybe returning a const reference is fine. Consider documenting, though, the situations under which the returned reference becomes invalid.

wu-hui

Thanks! I have to admit I did not think about how to implement Android enough!

firestore/src/include/firebase/firestore/local_cache_settings.h

wu-hui · 2023-06-07T19:16:10Z

firestore/src/include/firebase/firestore/settings.h

+   * Returns a reference to the `LocalCacheSettings` instance
+   * used to configure this SDK.
+   */
+  const LocalCacheSettings& local_cache_settings();


I agree with what you said, but I don't think it's possible to return a copy of LocalCacheSetting because it is abstract?

We could consider returning the underlying shared_ptr though..this is not in the approved proposal but seems like a good amendment?

dconeybe

local_cache_settings.h still has an include line for local_cache_settings_main.h. local_cache_settings.h must not rely on anything specific to the ios implementation, since the same .h file must be used for Android.

github-actions · 2023-07-12T15:27:30Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+ */
+class LocalCacheSettings {
+ public:
+  enum class Kind { kMemory, kPersistent };


📝 Documentation issue: warning: Member Kind (enumeration) of class firebase::firestore::LocalCacheSettings is not documented.

github-actions · 2023-07-12T15:27:30Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+
+  virtual ~LocalCacheSettings() = default;
+
+  virtual Kind kind() const = 0;


📝 Documentation issue: warning: Member kind() const =0 (function) of class firebase::firestore::LocalCacheSettings is not documented.

dconeybe

I think you got the "core" dependencies out of the public headers. Nice job! I've reviewed the code, but haven't looked deeply at the tests. I'll look at the tests in the next round of review.

dconeybe · 2023-07-17T19:46:12Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+
+  PersistentCacheSettings();
+
+  LocalCacheSettings::Kind kind() const override {


kind() should be public, since it's public in the base class LocalCacheSettings. Alternately, kind() could be made protected in the base class, along with the enum class Kind in the base class. In any case, they should be consistent. Was kind() part of the API proposal?

This comment also applies to MemoryCacheSettings::kind().

dconeybe · 2023-07-17T19:48:12Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+    return LocalCacheSettings::Kind::kPersistent;
+  }
+
+  // Get the corresponding settings object from the core sdk.


nit: "core sdk" -> "underlying sdk" or something like that, since it could be the "Android SDK", not the "Core SDK". You can also just omit this comment IMO since it's pretty obvious from the name what the function is doing.

dconeybe · 2023-07-17T19:49:22Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+  }
+
+  // Get the corresponding settings object from the core sdk.
+  const LocalCacheSettingsInternal& internal() const override;


Will the compiler allow you to specialize the return type to const PersistentCacheSettingsInternal&?

dconeybe · 2023-07-17T19:50:06Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+                         const MemoryCacheSettings& rhs);
+
+  /**
+   * Copies this settings instance, with its `MemoryGarbageCollectorSettins` set


typo: "MemoryGarbageCollectorSettins"

dconeybe · 2023-07-17T19:50:19Z

firestore/src/include/firebase/firestore/local_cache_settings.h

+                         const MemoryCacheSettings& rhs);
+
+  /**
+   * Copies this settings instance, with its `MemoryGarbageCollectorSettins` set


nit: "set" -> "set to"

dconeybe · 2023-07-17T20:17:33Z

firestore/src/common/settings.cc

+    return false;
+  }
+
+  if (*lhs.local_cache_settings_ != *rhs.local_cache_settings_) {


What happens if lhs and rhs have different types of persistence? For example, what if *lhs.local_cache_settings_ is memory local cache and *rhs.local_cache_settings_ is persistent local cache? Which operator== is used to compare them? I'm actually surprised this compiles, and doesn't crash if they are different types.

dconeybe · 2023-07-17T20:18:52Z

firestore/src/main/firestore_main.cc

-  settings.set_persistence_enabled(from.is_persistence_enabled());
-  settings.set_cache_size_bytes(from.cache_size_bytes());
+  // TODO(wuandy): Checking `from.local_cache_settings_` is required, because
+  // FirestoreInternal::settings() overrides used_legacy_cache_settings_. All


What is "used_legacy_cache_settings_"? I think maybe the name has changed. Please update this comment.

dconeybe · 2023-07-17T20:20:02Z

firestore/src/main/firestore_main.h

@@ -48,6 +48,7 @@ class Firestore;
 class ListenerRegistrationInternal;
 class Transaction;
 class WriteBatch;
+class Settings;


nit: Put the forward declaration of Settings in its correct alphabetical ordering among the other forward declarations.

dconeybe · 2023-07-17T20:21:34Z

firestore/src/main/local_cache_settings_main.h

+ public:
+  friend bool operator==(const LocalCacheSettingsInternal& lhs,
+                         const LocalCacheSettingsInternal& rhs) {
+    return &lhs == &rhs || lhs.core_settings() == rhs.core_settings();


Is this checking for &lhs == &rhs strictly required? If no, I recommend removing it since it is a performance optimization that likely is not warranted. Applies here and elsewhere in this file.

dconeybe · 2023-07-17T20:22:44Z

firestore/src/main/local_cache_settings_main.h

+    return &lhs == &rhs || lhs.settings_ == rhs.settings_;
+  }
+
+  PersistentCacheSettingsInternal WithSizeBytes(int64_t size) {


Can WithSizeBytes() be const (as well as other "With" member functions declared/defined in this file)?

wu-hui added 5 commits May 3, 2023 11:03

creating new settings files

ef27076

Compiling..pheww

2021a27

main tests pass.

12086e0

LRU GC compiles

26e9896

LRU tests pass

e7a0f70

wu-hui force-pushed the wuandy/LruGC branch from d4d8544 to e7a0f70 Compare May 8, 2023 16:52

wu-hui added the tests-requested: quick Trigger a quick set of integration tests. label May 8, 2023

github-actions bot added the tests: in-progress This PR's integration tests are in progress. label May 8, 2023

wu-hui added 6 commits May 9, 2023 11:43

Feedback

b93e933

Using new cache api, for future reference.

d65416d

fix firestore_main to use legacy API for now

47f4647

More polishing and documentation.

bfa379f

add equality tests

1692738

Address lint warnings

fa90de0

wu-hui requested a review from dconeybe May 10, 2023 15:08

wu-hui assigned dconeybe May 10, 2023

dconeybe requested changes May 11, 2023

View reviewed changes

dconeybe assigned wu-hui and unassigned dconeybe May 11, 2023

feedback

45a3d66

wu-hui commented May 15, 2023

View reviewed changes

wu-hui assigned dconeybe and unassigned wu-hui May 15, 2023

dconeybe and others added 5 commits May 16, 2023 01:58

Merge branch 'main' into wuandy/LruGC

105f757

Merge branch 'main' into wuandy/LruGC

dc90586

Use original deprecation suppress macro

5be24a4

Use refs instead of shared_ptr in api and remove dead code

a8db140

Better cache config source management + others

d56a2d4

dconeybe assigned wu-hui and unassigned dconeybe Jun 5, 2023

use forward decl for internal classes

d560721

wu-hui assigned dconeybe and unassigned wu-hui Jun 7, 2023

dconeybe requested changes Jun 7, 2023

View reviewed changes

dconeybe assigned wu-hui and unassigned dconeybe Jun 7, 2023

wu-hui added 3 commits June 7, 2023 14:58

remove main/api references

b328d02

No subclassing

c4264fd

No subclassing

d9d5ecd

wu-hui commented Jun 7, 2023

View reviewed changes

wu-hui assigned dconeybe and unassigned wu-hui Jun 7, 2023

dconeybe assigned wu-hui and unassigned dconeybe Jun 12, 2023

wu-hui assigned dconeybe and unassigned wu-hui Jun 13, 2023

dconeybe requested changes Jun 13, 2023

View reviewed changes

dconeybe assigned wu-hui and unassigned dconeybe Jun 13, 2023

wu-hui added 2 commits July 5, 2023 10:50

delete includes from main

b757139

No more core from common

ded9c2e

wu-hui force-pushed the wuandy/LruGC branch from c9aa462 to ded9c2e Compare July 11, 2023 15:06

wu-hui added 2 commits July 11, 2023 11:36

Merge branch 'main' into wuandy/LruGC

9ee3f2e

remove unnecessary include

570e8c0

github-actions bot reviewed Jul 12, 2023

View reviewed changes

dconeybe requested changes Jul 17, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Cache Config and LRU GC #1308

New Cache Config and LRU GC #1308

wu-hui commented May 8, 2023 •

edited

Loading

dconeybe left a comment

wu-hui left a comment

wu-hui commented Jun 2, 2023

dconeybe commented Jun 2, 2023

dconeybe left a comment

dconeybe Jun 7, 2023

wu-hui Jun 7, 2023

dconeybe Jul 17, 2023

wu-hui left a comment

wu-hui Jun 7, 2023

dconeybe left a comment

github-actions bot Jul 12, 2023

github-actions bot Jul 12, 2023

dconeybe left a comment

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023

dconeybe Jul 17, 2023


		virtual ~LocalCacheSettings() = default;

		virtual Kind kind() const = 0;


		PersistentCacheSettings();

		LocalCacheSettings::Kind kind() const override {

New Cache Config and LRU GC #1308

Are you sure you want to change the base?

New Cache Config and LRU GC #1308

Conversation

wu-hui commented May 8, 2023 • edited Loading

dconeybe left a comment

Choose a reason for hiding this comment

wu-hui left a comment

Choose a reason for hiding this comment

wu-hui commented Jun 2, 2023

dconeybe commented Jun 2, 2023

dconeybe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wu-hui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dconeybe left a comment

Choose a reason for hiding this comment

github-actions bot Jul 12, 2023

Choose a reason for hiding this comment

github-actions bot Jul 12, 2023

Choose a reason for hiding this comment

dconeybe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wu-hui commented May 8, 2023 •

edited

Loading