BTree indexes do not properly track deletes

README.md

@@ -1,136 +1,13 @@
-# SpacetimeDB SDK for Unity Engine
+# SpacetimeDB C# SDK
 
 ## Overview
 
-This repository contains the C# and [Unity](https://unity.com/) SDKs for SpacetimeDB. The SDK allows to interact with the database server and is prepared to work with code generated from a SpacetimeDB backend code.
+This repository contains the C# and [Unity](https://unity.com/) client SDKs for SpacetimeDB. These SDKs contain all the tools you need to build native clients for SpacetimeDB modules using C#.
 
 ## Documentation
 
-The Unity SDK uses the same code as the C# SDK. You can find the documentation for the C# SDK in the [C# SDK Reference](https://spacetimedb.com/docs/sdks/c-sharp)
+The Unity SDK uses the same code as the C# SDK. You can find the documentation for the C# SDK in the [C# SDK Reference](https://spacetimedb.com/docs/sdks/c-sharp). For a guided tutorial, see the [C# SDK Quickstart](https://spacetimedb.com/docs/sdks/c-sharp/quickstart).
 
 There is also a comprehensive Unity tutorial/demo available:
 - [Unity Tutorial](https://spacetimedb.com/docs/unity/part-1) Doc
 - [Unity Demo](https://github.com/clockworklabs/SpacetimeDBUnityTutorial) Repo
-
-## Installation
-
-### Unity Demo
-
-Download the [.unitypackage release](https://github.com/clockworklabs/SpacetimeDBUnityTutorial/releases) of our [Unity Part 1 tutorial](https://spacetimedb.com/docs/unity/part-1) demo that includes this SDK as a package manfiest requirement.
-
-### Standalone
-
-1. Open the package manager window in Unity.
-2. Click the "(+)" button in the top-left corner and select "Add package from git URL".
-3. Paste the following URL: `https://github.com/clockworklabs/com.clockworklabs.spacetimedbsdk.git`
-
-## Usage
-
-### SpacetimeDBNetworkManager
-
-The Unity SDK for SpacetimeDB requires that there is a `SpacetimeDBNetworkManager` component attached to a GameObject in the scene. The `UnityNetworkManager` component is responsible for connecting to SpacetimeDB and managing the connection. The `UnityNetworkManager` component is a singleton and there can only be one instance of it in the scene.
-
-### Connecting to SpacetimeDB
-
-To connect to SpacetimeDB, you need to call the `Connect` method on the `SpacetimeDBClient` class. The `Connect` method takes the following parameters:
-
-- `token`: The authentication token to use to connect to SpacetimeDB. This token is generated by the backend code and is used to authenticate the client.
-- `hostName`: The hostname of the SpacetimeDB server. This is the same hostname that you use to connect to the SpacetimeDB web interface.
-- `moduleAddress`: The address of the module to connect to. This is the same address that you use to connect to the SpacetimeDB web interface.
-- `sslEnabled`: Whether to use SSL to connect to SpacetimeDB. This is the same value that you use to connect to the SpacetimeDB web interface.
-
-Example:
-
-```csharp
-using SpacetimeDB;
-
-SpacetimeDBClient.instance.Connect(TOKEN, HOST, DBNAME, SSL_ENABLED);
-```
-
-### AuthToken
-
-The `AuthToken` class is an optional helper class that can be used to store the local client's authentication token locally in the Unity PlayerPrefs.
-
-Example:
-
-```csharp
-using SpacetimeDB;
-
-// called when we receive the client identity from SpacetimeDB
-SpacetimeDBClient.instance.onIdentityReceived += (token, identity, address) => {
-    AuthToken.SaveToken(token);
-    local_identity = identity;
-};
-
-SpacetimeDBClient.instance.Connect(AuthToken.Token, hostName, moduleAddress, sslEnabled);
-```
-
-### Subscribing to tables
-
-To subscribe to a table, you need to call the `Subscribe` method on the `SpacetimeDBClient` class. The `Subscribe` method takes a list of queries as a parameter. The queries are the same queries that you use to subscribe to tables in the SpacetimeDB web interface.
-
-### Listening to events
-
-To listen to events, you need to register callbacks on the `SpacetimeDBClient` class. The following callbacks are available:
-
-- `onConnect`: Called when the client connects to SpacetimeDB.
-- `onConnectError`: Called when the client fails to connect to SpacetimeDB.
-- `onDisconnect`: Called when the client disconnects from SpacetimeDB.
-- `onIdentityReceived`: Called when the client receives its identity from SpacetimeDB.
-- `onSubscriptionApplied`: Called when the client receives the initial data from SpacetimeDB after subscribing to tables.
-
-You can register for row update events on a table. To do this, you need to register callbacks on the table class. The following callbacks are available:
-
-- `OnInsert`: Called when a row is inserted into the table.
-- `OnUpdate`: Called when a row is updated in the table.
-- `OnBeforeDelete`: Called before a row is deleted from the table.
-- `OnDelete`: Called when a row is deleted from the table.
-- `OnRowUpdate`: Called when a row is inserted, updated, or deleted from the table.
-
-Example:
-
-```csharp
-using SpacetimeDB.Types;
-
-PlayerComponent.OnInsert += PlayerComponent_OnInsert;
-PlayerComponent.OnUpdate += PlayerComponent_OnUpdate;
-PlayerComponent.OnDelete += PlayerComponent_OnDelete;
-PlayerComponent.OnBeforeDelete += PlayerComponent_OnBeforeDelete;
-PlayerComponent.OnRowUpdate += PlayerComponent_OnRowUpdate;
-```
-
-You can register for reducer call updates as well.
-
-- `OnREDUCEREvent`: Called when a reducer call is received from SpacetimeDB. (If a) you are subscribed to the table that the reducer modifies or b) You called the reducer and it failed)
-
-Example:
-
-```csharp
-using SpacetimeDB.Types;
-
-Reducer.OnMovePlayerEvent += Reducer_OnMovePlayerEvent;
-```
-
-### Accessing the client cache
-
-The client cache is a local cache of the data that the client has received from SpacetimeDB. The client cache is automatically updated when the client receives updates from SpacetimeDB. 
-
-When you run the CLI generate command, SpacetimeDB will automatically generate a class for each table in your database. These classes are generated in the `SpacetimeDB.Types` namespace. Each class contains a set of static methods that allow you to query the client cache. The following methods are available:
-
-- `int Count()`: Returns the number of rows in the table.
-- `IEnumerable<TableRow> Iter()`: Returns an iterator over the table.
-- `IEnumerable<TableRow> FilterByCOLUMN(ColumnValue)`: Filters the table by the specified column value.
-- `TableRow? FindByCOLUMN(ColumnValue)`: Finds a single item by the specifed column value.
-- `IEnumerable<TableRow> Query(Func<TableRow, bool>)`: Filters the table with the specified predicate.
-
-### Calling Reducers
-
-To call a reducer, you need to call the autogenerated method on the `Reducer` class. The autogenerated method takes the reducer arguments as parameters. The reducer arguments are the same arguments that are expected in your server module.
-
-Example:
-
-```csharp
-using SpacetimeDB.Types;
-
-Reducer.MovePlayer(new StdbVector2(0.0f, 0.0f), new StdbVector2(1.0f, 1.0f));
-```

SpacetimeDB.ClientSDK.csproj

@@ -17,15 +17,15 @@
     <PackageReadmeFile>README.md</PackageReadmeFile>
     <RepositoryUrl>https://github.com/clockworklabs/com.clockworklabs.spacetimedbsdk</RepositoryUrl>
     <AssemblyVersion>1.0.0</AssemblyVersion>
-    <Version>1.0.0-rc4</Version>
+    <Version>1.0.0</Version>
     <DefaultItemExcludes>$(DefaultItemExcludes);*~/**</DefaultItemExcludes>
     <!-- We want to save DLLs for Unity which doesn't support NuGet. -->
     <RestorePackagesPath>packages</RestorePackagesPath>
     <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="SpacetimeDB.BSATN.Runtime" Version="1.0.0-rc4" />
+    <PackageReference Include="SpacetimeDB.BSATN.Runtime" Version="1.0.0" />
 
     <InternalsVisibleTo Include="SpacetimeDB.Tests" />
   </ItemGroup>

SpacetimeDB.ClientSDK.sln

@@ -3,22 +3,26 @@ Microsoft Visual Studio Solution File, Format Version 12.00
 # Visual Studio Version 17
 VisualStudioVersion = 17.6.33717.318
 MinimumVisualStudioVersion = 10.0.40219.1
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "client", "examples~\quickstart\client\client.csproj", "{8F33709C-DEE9-41CC-A477-D6128E3700B1}"
-EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SpacetimeDB.ClientSDK", "SpacetimeDB.ClientSDK.csproj", "{242A8146-A58D-43E9-A2BD-31FFC6851AA6}"
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "tests", "tests~\tests.csproj", "{5CD31104-4719-4CE3-8D39-8BAE0B75C085}"
 EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "client", "examples~\quickstart-chat\client\client.csproj", "{FE261832-1594-DE21-C8C8-2D525680CBD7}"
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "examples~", "examples~", "{E0CADA48-79A1-4490-ACEC-698EC7D2AB43}"
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "btree-repro", "btree-repro", "{4434F63A-3300-476E-82F3-D672B83E68E8}"
+EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StdbModule", "examples~\btree-repro\server\StdbModule.csproj", "{A68A749E-CFE1-45CF-8BDA-929B2E677D72}"
+EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "client", "examples~\btree-repro\client\client.csproj", "{5BB2A377-C44C-4310-8951-BB37F5EB8456}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
 		Release|Any CPU = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(ProjectConfigurationPlatforms) = postSolution
-		{8F33709C-DEE9-41CC-A477-D6128E3700B1}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{8F33709C-DEE9-41CC-A477-D6128E3700B1}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{8F33709C-DEE9-41CC-A477-D6128E3700B1}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{8F33709C-DEE9-41CC-A477-D6128E3700B1}.Release|Any CPU.Build.0 = Release|Any CPU
 		{242A8146-A58D-43E9-A2BD-31FFC6851AA6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
 		{242A8146-A58D-43E9-A2BD-31FFC6851AA6}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{242A8146-A58D-43E9-A2BD-31FFC6851AA6}.Release|Any CPU.ActiveCfg = Release|Any CPU
@@ -27,11 +31,28 @@ Global
 		{5CD31104-4719-4CE3-8D39-8BAE0B75C085}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{5CD31104-4719-4CE3-8D39-8BAE0B75C085}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{5CD31104-4719-4CE3-8D39-8BAE0B75C085}.Release|Any CPU.Build.0 = Release|Any CPU
+		{FE261832-1594-DE21-C8C8-2D525680CBD7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{FE261832-1594-DE21-C8C8-2D525680CBD7}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{FE261832-1594-DE21-C8C8-2D525680CBD7}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{FE261832-1594-DE21-C8C8-2D525680CBD7}.Release|Any CPU.Build.0 = Release|Any CPU
+		{A68A749E-CFE1-45CF-8BDA-929B2E677D72}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{A68A749E-CFE1-45CF-8BDA-929B2E677D72}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{A68A749E-CFE1-45CF-8BDA-929B2E677D72}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{A68A749E-CFE1-45CF-8BDA-929B2E677D72}.Release|Any CPU.Build.0 = Release|Any CPU
+		{5BB2A377-C44C-4310-8951-BB37F5EB8456}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{5BB2A377-C44C-4310-8951-BB37F5EB8456}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{5BB2A377-C44C-4310-8951-BB37F5EB8456}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{5BB2A377-C44C-4310-8951-BB37F5EB8456}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
 	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {35C59D87-64E5-4A6C-B1D5-2241B946E847}
 	EndGlobalSection
+	GlobalSection(NestedProjects) = preSolution
+		{4434F63A-3300-476E-82F3-D672B83E68E8} = {E0CADA48-79A1-4490-ACEC-698EC7D2AB43}
+		{A68A749E-CFE1-45CF-8BDA-929B2E677D72} = {4434F63A-3300-476E-82F3-D672B83E68E8}
+		{5BB2A377-C44C-4310-8951-BB37F5EB8456} = {4434F63A-3300-476E-82F3-D672B83E68E8}
+	EndGlobalSection
 EndGlobal

examples~/btree-repro/client/Program.cs

@@ -0,0 +1,111 @@
+// See https://aka.ms/new-console-template for more information
+
+using System.Diagnostics;
+using System.Runtime.CompilerServices;
+using SpacetimeDB;
+using SpacetimeDB.Types;
+
+const string HOST = "http://localhost:3000";
+const string DBNAME = "btree-repro";
+
+DbConnection ConnectToDB()
+{
+    DbConnection? conn = null;
+    conn = DbConnection.Builder()
+        .WithUri(HOST)
+        .WithModuleName(DBNAME)
+        .OnConnect(OnConnected)
+        .OnConnectError((err) =>
+        {
+            throw err;
+        })
+        .OnDisconnect((conn, err) =>
+        {
+            if (err != null)
+            {
+                throw err;
+            }
+            else
+            {
+                throw new Exception("Unexpected disconnect");
+            }
+        })
+        .Build();
+    return conn;
+}
+
+// We assume we're the only one interacting with the server for this test.
+
+uint waiting = 0;
+bool applied = false;
+
+void OnConnected(DbConnection conn, Identity identity, string authToken)
+{
+    Log.Debug("Connected to btree-repro");
+    conn.SubscriptionBuilder()
+        .OnApplied(OnSubscriptionApplied)
+        .OnError((ctx, err) =>
+        {
+            throw err;
+        })
+        .Subscribe(["SELECT * FROM ExampleData"]);
+
+    conn.Reducers.OnAdd += (ReducerEventContext ctx, uint id, uint indexed) =>
+    {
+        Log.Info("Got Add callback");
+        waiting--;
+        ValidateBTreeIndexes(ctx);
+    };
+
+    conn.Reducers.OnDelete += (ReducerEventContext ctx, uint id) =>
+    {
+        Log.Info("Got Delete callback");
+        waiting--;
+        ValidateBTreeIndexes(ctx);
+    };
+}
+
+const uint MAX_ID = 10;
+
+void ValidateBTreeIndexes(IRemoteDbContext conn)
+{
+    Log.Debug("Checking indexes...");
+    foreach (var data in conn.Db.ExampleData.Iter())
+    {
+        Debug.Assert(conn.Db.ExampleData.Indexed.Filter(data.Id).Contains(data));
+    }
+    var outOfIndex = conn.Db.ExampleData.Iter().ToHashSet();
+
+    for (uint i = 0; i < MAX_ID; i++)
+    {
+        foreach (var data in conn.Db.ExampleData.Indexed.Filter(i))
+        {
+            Debug.Assert(outOfIndex.Contains(data));
+        }
+    }
+}
+
+void OnSubscriptionApplied(SubscriptionEventContext context)
+{
+    Log.Debug("Calling Add");
+    context.Reducers.Add(1, 1);
+    applied = true;
+    waiting++;
+    Log.Debug("Calling Delete");
+    context.Reducers.Delete(1);
+    waiting++;
+}
+
+System.AppDomain.CurrentDomain.UnhandledException += (sender, args) =>
+{
+    Log.Exception($"Unhandled exception: {sender} {args}");
+    Environment.Exit(1);
+};
+var db = ConnectToDB();
+while (!applied || waiting > 0)
+{
+    db.FrameTick();
+    Thread.Sleep(100);
+}
+Log.Info("Success");
+Environment.Exit(0);

examples~/btree-repro/client/client.csproj

@@ -0,0 +1,15 @@
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>net8.0</TargetFramework>
+    <ImplicitUsings>enable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="../../../SpacetimeDB.ClientSDK.csproj" />
+  </ItemGroup>
+
+</Project>