|
| 1 | +--- |
| 2 | +layout: post |
| 3 | +title: STM in Clojure - Testing |
| 4 | +date: 2024-12-22 00:10:00 -0500 |
| 5 | +categories: general |
| 6 | +--- |
| 7 | + |
| 8 | +We develop a small framework for testing transaction interaction. |
| 9 | + |
| 10 | +This is the third in a series on implementing STM in Clojure. |
| 11 | +Previous posts: |
| 12 | + |
| 13 | +- [Part 1: STM in Clojure - Design]({{site.baseurl}}{% post_url 2024-12-22-STM-design %}). |
| 14 | +- [Part 2: STM in Clojure - Code]({{site.baseurl}}{% post_url 2024-12-22-STM-code %}) |
| 15 | + |
| 16 | + |
| 17 | +## Testing transactions |
| 18 | + |
| 19 | +How do we test transactions? Who knows? |
| 20 | + |
| 21 | +In the Clojure test suite, there is a file `clojure/test_clojure/refs.clj`. Leaving out the copyright and author info, here are the tests: |
| 22 | + |
| 23 | +```Clojure |
| 24 | +(ns clojure.test-clojure.refs |
| 25 | + (:use clojure.test)) |
| 26 | + |
| 27 | +; http://clojure.org/refs |
| 28 | + |
| 29 | +; ref |
| 30 | +; deref, @-reader-macro |
| 31 | +; dosync io! |
| 32 | +; ensure ref-set alter commute |
| 33 | +; set-validator get-validator |
| 34 | +``` |
| 35 | + |
| 36 | +Yeah, no. |
| 37 | + |
| 38 | + |
| 39 | + I'd like to do some tests for my `Ref` and `LockingTransaction` code. No question that Clojure would be great place to do this, but at this point in building ClojureCLR.Next, well, we don't have Clojure yet. |
| 40 | + |
| 41 | + I could generate some tests based on some of the illustrative samples found in various places. And I'll do that. But I wanted to some very specific tests. Like checking that doing a set after a commute fails. Or that if T1 commutes a Ref, but T2 gets done before T2, the commute runs on T2's value for the Ref. And so on. |
| 42 | + |
| 43 | + So I decided to develop a modest framework for writing tests of this type. We need to be able to write a script for transaction that can do things like `commute`s and `ref-set`s. We need to be able to coordinate between two different transactions. I decided to use `ManualResetEvent`s that are shared between transactions. One transaction can trigger the MRE, another can wait for the MRE to be set. |
| 44 | + I wanted to be able to specify tests for after the transaction finishes: normal exit vs exception thrown; the values for specific Refs. And I'd like for it to be integrated with the testing framework I'm use (Expecto). I'd like to be able to write a test like this: |
| 45 | + |
| 46 | + ```F# |
| 47 | + testCase "Force one to complete before other see final change" |
| 48 | + <| fun _ -> |
| 49 | + let script1 = |
| 50 | + { Steps = [ RefSet(0,10); Trigger(0); ] |
| 51 | + Tests = [ TxTest.Normal; Ref(0,99)] } |
| 52 | +
|
| 53 | + let script2 = |
| 54 | + { Steps = [ Wait(0); RefSet(0,99); ] |
| 55 | + Tests = [ TxTest.Normal; Ref(0,99)] } |
| 56 | +
|
| 57 | + let scripts = [ script1; script2 ] |
| 58 | + execute scripts |
| 59 | +``` |
| 60 | + |
| 61 | +## Code |
| 62 | + |
| 63 | +I used disciminated unions for the steps in a transaction: |
| 64 | + |
| 65 | +```F# |
| 66 | +type TxAction = |
| 67 | + | RefSet of index: int * value: int |
| 68 | + | CommuteIncr of index: int |
| 69 | + | AlterIncr of index: int |
| 70 | + | Wait of index: int |
| 71 | + | Trigger of index: int |
| 72 | + | SleepMilliseconds of int |
| 73 | +``` |
| 74 | + |
| 75 | +Since I only care about thing running or not, I don't need to pass arbitrary functions to `commute` and `alter`, so I'm only going to increment an integer value. |
| 76 | + |
| 77 | +For `RefSet`, `CommutIncr` and `AlterIncr`, the index specifies a `Ref` in an array of `Refs` shared by all scripts in the test case. Similarly for `Wait` and `Trigger` for a shared array of `ManualResetEvents`. My intention is that each event get used only once. |
| 78 | + |
| 79 | +We'll want to know how the transaction completed: |
| 80 | + |
| 81 | +```F# |
| 82 | +type TxExit = |
| 83 | + | NormalExit |
| 84 | + | ExceptionThrown of ex: exn |
| 85 | +``` |
| 86 | + |
| 87 | +We'll need to specify what tests to perform. We need to test whether the outcome was a normal exit or an exception being thrown. And we'll need to check the final values of the various `Ref`s. |
| 88 | + |
| 89 | +```F# |
| 90 | +type TxTest = |
| 91 | + | Throw of exnType: Type |
| 92 | + | Normal |
| 93 | + | Ref of index: int * value: int |
| 94 | +``` |
| 95 | + |
| 96 | +A script for a transaction is a sequence of actions and a set of tests. |
| 97 | + |
| 98 | +```F# |
| 99 | +type TxTest = |
| 100 | + | Throw of exnType: Type |
| 101 | + | Normal |
| 102 | + | Ref of index: int * value: int |
| 103 | +``` |
| 104 | + |
| 105 | +Now we can code. We will need to examine a set of scripts and determine how many `Ref`s and how many `ManualResetEvent`s to create. The maximum index + 1 will suffice. |
| 106 | + |
| 107 | + |
| 108 | + |
| 109 | +```F# |
| 110 | +let getCount(scripts : TxScript list, indexSelect: TxAction -> int) = |
| 111 | + let maxIndex = |
| 112 | + scripts |
| 113 | + |> List.collect (fun s -> s.Steps) |
| 114 | + |> List.map (fun a -> indexSelect a) |
| 115 | + |> List.max |
| 116 | +
|
| 117 | + maxIndex + 1 |
| 118 | +
|
| 119 | +let getHandleCount(scripts : TxScript list) = |
| 120 | + getCount(scripts, fun a -> |
| 121 | + match a with |
| 122 | + | Wait i -> i |
| 123 | + | Trigger i -> i |
| 124 | + | _ -> -1) |
| 125 | +
|
| 126 | +let getRefCount(scripts : TxScript list) = |
| 127 | + getCount(scripts, fun a -> |
| 128 | + match a with |
| 129 | + | RefSet(i, _) -> i |
| 130 | + | CommuteIncr i -> i |
| 131 | + | AlterIncr i -> i |
| 132 | + | _ -> -1) |
| 133 | +``` |
| 134 | + |
| 135 | +I'm sure there are more elegant ways, but this works. |
| 136 | +From the counts, we can create the arrays we need. |
| 137 | + |
| 138 | +```F# |
| 139 | +let createHandles(scripts : TxScript list) = |
| 140 | + Array.init (getHandleCount(scripts)) (fun i -> new ManualResetEvent(false)) |
| 141 | +
|
| 142 | +let createRefs(scripts : TxScript list) = |
| 143 | + Array.init (getRefCount(scripts)) (fun i -> new Clojure.Lib.Ref(0, null)) |
| 144 | +``` |
| 145 | + |
| 146 | +We'll need to pass an `IFn` that has an invoke of one argument that adds one to the (integer) argument. |
| 147 | +There is a neat trick for building `IFn`'s in F#. The class `AFn` defines a working `IFn` that throws on all `invoke` overloads. We can use an object expression to overload just the `IFn.invoke(arg1)` method. |
| 148 | + |
| 149 | +```F# |
| 150 | +let incrFn = |
| 151 | + { new AFn() with |
| 152 | + member this.ToString() = "a" |
| 153 | + interface IFn with |
| 154 | + member this.invoke(arg1) = (arg1 :?> int) + 1 :> obj |
| 155 | + } |
| 156 | +``` |
| 157 | + |
| 158 | +(I was so excited when I discovered object expressions in F#.) |
| 159 | + |
| 160 | +Jumping to end and working back to the hard spot, we will execute scripts using `execute`. |
| 161 | + |
| 162 | +```F# |
| 163 | +let execute(scripts : TxScript list) = |
| 164 | + let handles = createHandles(scripts) |
| 165 | + let refs = createRefs(scripts) |
| 166 | + runTests(scripts, handles, refs) |
| 167 | + handles |> Array.iter (fun h -> h.Dispose()) |
| 168 | + refs |> Array.iter (fun r -> (r :> IDisposable).Dispose()) |
| 169 | +``` |
| 170 | +We pass a list of scripts execute. It generates the arrays of `Ref`s and event handles, runs the scripts and tests them, and then cleans up. |
| 171 | + |
| 172 | +We have to run the tests and then test when they are all done. We use the `async` mechanism to coordinate running the tasks. We take each script and generate an async script, run them all in parallel and wait for them all to finish. Then we run the tests. |
| 173 | + |
| 174 | +```F# |
| 175 | +let runTests(scripts : TxScript list, handles : ManualResetEvent array, refs : Clojure.Lib.Ref array) = |
| 176 | + let results = |
| 177 | + scripts |
| 178 | + |> List.mapi (fun i s -> createExecuteScriptAsync(i, s, handles, refs)) |
| 179 | + |> Async.Parallel |
| 180 | + |> Async.RunSynchronously |
| 181 | +
|
| 182 | + scripts |
| 183 | + |> List.zip( results |> Array.toList) |
| 184 | + |> List.iter (fun (r, s) -> performTests(s, r, refs)) |
| 185 | +``` |
| 186 | + |
| 187 | +The `Async.RunSynchronously` returns an array of the return values from each async script. We pair scripts and results and send them off to `performTests`: |
| 188 | + |
| 189 | +```F# |
| 190 | +let performTests(script: TxScript, result : TxExit, refs: Clojure.Lib.Ref array) = |
| 191 | +
|
| 192 | + let testExceptionThrown(exType: Type, result: TxExit) = |
| 193 | + let thrownType = |
| 194 | + match result with |
| 195 | + | ExceptionThrown(ex) -> ex.GetType() |
| 196 | + | _ -> null |
| 197 | + Expect.equal thrownType exType $"""Expected exception of type {exType}, {if isNull thrownType then "but exited normally" else $"but got {thrownType}"} """ |
| 198 | +
|
| 199 | + let testNormalExit(result: TxExit) = |
| 200 | + Expect.isTrue (result.IsNormalExit) "Expected normal exit, but exception was thrown" |
| 201 | +
|
| 202 | + let testRefValue(i: int, v: int, refs: Clojure.Lib.Ref array) = |
| 203 | + Expect.equal ((refs[i] :> IDeref).deref()) v "Ref value incorrect" |
| 204 | + |
| 205 | + script.Tests |
| 206 | + |> List.iter (fun test -> |
| 207 | + match test with |
| 208 | + | Throw exType -> testExceptionThrown(exType, result) |
| 209 | + | Normal -> testNormalExit(result) |
| 210 | + | Ref(i, v) -> testRefValue(i, v, refs) |
| 211 | + ) |
| 212 | +``` |
| 213 | + |
| 214 | +I think the testing of return results is a little inelegant, but I'm saving a rewrite for another day. |
| 215 | +Notice here that calls to `Expect.equal` and `Expect.isTrue`. This ties us into the Expecto testing framework. |
| 216 | + |
| 217 | +Finally, the biggie. Generating an async script from a sequence (list) of steps. |
| 218 | +Essentially, we need the equivalent of what the `dosync` macro call does: take the body, wrap it with a `(fn [] ...)` and pass that to `LockingTransaction.runInTransaction`. In the code below, the value of `txfn` is just that: an `IFn` that has a zero-arg `invoke` that iterates through the sequence of actions and executes them. The script proper passes `txfn` to `LockingTransaction.runInTransaction` and returns `NormalExit` or `ExceptionThrown(dx)`, depending. |
| 219 | + |
| 220 | +```F# |
| 221 | +let createExecuteScriptAsync(id: int, script : TxScript, handles: ManualResetEvent array, refs: Clojure.Lib.Ref array) = |
| 222 | + let txfn = { new AFn() with |
| 223 | + member this.ToString() = "a" |
| 224 | + interface IFn with |
| 225 | + member _.invoke() = |
| 226 | + script.Steps |
| 227 | + |> List.iteri (fun stepNum step -> |
| 228 | + match step with |
| 229 | + | RefSet(i, v) -> refs[i].set(v) |> ignore |
| 230 | + | CommuteIncr i -> refs[i].commute(incrFn, null) |> ignore |
| 231 | + | AlterIncr i -> refs.[i].alter(incrFn, null) |> ignore |
| 232 | + | Wait i -> handles.[i].WaitOne() |> ignore |
| 233 | + | Trigger i -> handles.[i].Set()|> ignore |
| 234 | + | SleepMilliseconds ms -> Thread.Sleep(ms) |> ignore |
| 235 | + 12 |
| 236 | + } |
| 237 | + async { |
| 238 | + let result = |
| 239 | + try |
| 240 | + LockingTransaction.runInTransaction(txfn) |> ignore |
| 241 | + NormalExit |
| 242 | + with |
| 243 | + | ex -> ExceptionThrown(ex) |
| 244 | + return result |
| 245 | + } |
| 246 | +``` |
| 247 | + |
| 248 | +And that's the whole enchilada. |
| 249 | + |
| 250 | +Realistically, there are only a few meaningful scenario that we can test this way. |
| 251 | +Some more complicated things, like testing many retries to failure, are just too hard. |
| 252 | +Contemplate how long it takes for a 10,000 retries with 100 millisecond waits on each retry. |
| 253 | +Contemplate how you even detect a retry has happened -- we'd have to set up another mechanism for side-effecting actions such as counter external to the transaction. At least this mechanism allowed me to do some simple testing to check for basic operational validity. That was enough reward for the effort involved. |
| 254 | + |
0 commit comments