VBA - Persistence - Use LSet to serialise user defined type to a byte array

First, a health warning; there is no warranty for this code in this blog post; use at your own risk.

Transmitting State In a Distributed System

Years ago, I encountered a technique from a great technical author called Rockford Lhotka who wrote a classic VB6 book called Professional Visual Basic 6 Distributed Objects; as its title suggests it's concerned with using VB6 to build systems that distributed objects across machines. VBA is a version of VB6 so the techniques found therein apply to Excel Developers. In this blog post (and some planned future posts), I comment upon those techniques; specifically the serialisation/persistence technique.

In a distributed system, at some point one inevitably passes an object as a parameter and this raises the potential problem of excessive network traffic. So, if an instance of Class1 residing on MachineA wants to call an instance of Class2 residing on MachineB and passes as a parameter a pointer to an instance of Class3 which was created on MachineA then MachineB will make network calls to query the state of Class3. If these network calls are excessive then you will want to redesign your system to instead serialise the state of Class3 to a byte stream (or other state vessel), pass the byte stream over the network as the parameter, instantiate a duplicate of Class3 on MachineB initialising its state from the passed parameter byte stream and make local calls to the local duplicate. If you change the state of Class3 you'll need to serialise the state and return it back to Machine1 back so its original instance of Class3 is synchronised.

As you can imagine designing distributed systems can be hard and actually this is not what this blog post is to be about. This blog post is meant to be about the persistence mechanism used by Rockford Lhotka, using the LSet keyword. But if you're interested in this problem, Rockford Lhotka, has gone to become a big name in the problem space of distributed systems, promoting his solution Component-based Scalable Logical Architecture (CSLA) an early version of which is found in the above referenced book.

The key takeaway from this section is that there are techniques to serialise a (VB6/VBA) class's state to a an array of bytes and then instantiate a duplicate. Just bear that in mind as a use case for the code below.

Microsoft seems to frown upon LSet (and by implication Rockford Lhotka's CSLA)

Above, I've written above quite a lot about Rockford Lhotka because his ideas seems to be a little at odds with the the official Microsoft documentation on LSet, here's an eye-catching quote ...

Using LSet to copy a variable of one user-defined type into a variable of a different user-defined type is not recommended.

But this is exactly how Rockford Lhotka achieves his trick of serialisation. We'll move onto an example before discussing the point at issue further.

Sample LSet to Byte Array Code

So how does Rockford Lhotka serialise state in VB6 and VBA? Here is some code which demonstrates an instance of a user-defined-type being serialised and the state then used in turn to create a second identical instance of the same user-defined-type.

modUdtToByteArrayByLSet Standard Module

1. Option Explicit
2.  
3. '* No Warranty, use this code at your own risk!
4.  
5. 'Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)
6.  
7. Public Type udtMyType
8.     aLong As Long
9.     anInt As Integer
10.     aDouble As Double
11.     aFixedString As String * 10
12.     aSecondLong As Long
13.     aSecondDouble As Double
14. End Type
15.  
16. Public Type udtByteArray48
17.     value(0 To 47) As Byte
18. End Type
19.  
20. Private Sub TestSaveToByteArray()
21.     Dim uContiguousType As udtMyType
22.     uContiguousType.aLong = 1
23.     uContiguousType.anInt = 2
24.     uContiguousType.aDouble = 3.141
25.  
26.     uContiguousType.aFixedString = "Hello"
27.     uContiguousType.aSecondLong = -434634634
28.     uContiguousType.aSecondDouble = Sqr(10) * -1
29.  
30.     Dim abSaved() As Byte
31.     abSaved() = SaveToByteArray(uContiguousType, 48)
32.  
33.     Dim uContiguousType2 As udtMyType
34.     LoadFromByteArray abSaved(), uContiguousType2, 48
35.  
36.     Debug.Assert uContiguousType2.aDouble = uContiguousType.aDouble
37.     Debug.Assert uContiguousType2.aFixedString = uContiguousType.aFixedString
38.     Debug.Assert uContiguousType2.aLong = uContiguousType.aLong
39.     Debug.Assert uContiguousType2.anInt = uContiguousType.anInt
40.     Debug.Assert uContiguousType2.aSecondLong = uContiguousType.aSecondLong
41.     Debug.Assert uContiguousType2.aSecondDouble = uContiguousType.aSecondDouble
42.     Stop
43. End Sub
44.  
45. '******************************************************************************************************
46. '*
47. '* Persistence routines, you'll need these three for each type you intend to persist
48. '* and customise the signatures
49. '*
50. '******************************************************************************************************
51. Public Sub LoadFromByteArray(ByRef abBytes() As Byte, ByRef puContiguousType As udtMyType, ByVal lSizeOf As Long)
52.  
53.     CheckSizeOf puContiguousType, lSizeOf
54.     Dim uByteArray48 As udtByteArray48
55.  
56.     'CopyMemory ByVal (VarPtr(uByteArray48.value(0))), ByVal (VarPtr(abBytes(0))), lSizeOf
57.     Dim idx As Long
58.     For idx = 0 To lSizeOf - 1
59.         uByteArray48.value(idx) = abBytes(idx)
60.     Next idx
61.  
62.     LSet puContiguousType = uByteArray48
63. End Sub
64.  
65. Private Sub CheckSizeOf(ByRef puContiguousType As udtMyType, ByVal lSizeOf As Long)
66.     Dim uByteArray48 As udtByteArray48
67.     If Not LenB(puContiguousType) = LenB(uByteArray48) Then Err.Raise vbObjectError, , "#size of byte arrays do not match!"
68.     If Not LenB(puContiguousType) = lSizeOf Then Err.Raise vbObjectError, , "#size of byte arrays do not match size_of!"
69. End Sub
70.  
71. Public Function SaveToByteArray(ByRef puContiguousType As udtMyType, ByVal lSizeOf As Long) As Byte()
72.  
73.     CheckSizeOf puContiguousType, lSizeOf
74.  
75.     Dim uByteArray48 As udtByteArray48
76.     LSet uByteArray48 = puContiguousType
77.     SaveToByteArray = uByteArray48.value
78.  
79. End Function
80.  

So to run the above code run TestSaveToByteArray() whose Stop statement conveniently allows inspection of the Locals Window which should look something like that below which shows two instances of the user-defined type myType with identical contents but the second has been populated from a byte array calculated from the first. Both reading and writing the byte array to the user-defined-typed is done with LSet...

Commentary

So whilst the above code works there is quite a lot with which I am unhappy. Primarily, if your UDT is 48 bytes long then you need to separately define another type, the serialisation type, to have one single member byte array of length 48 bytes. Now, I'm assuming that this technique implies that each class has a UDT to contain the state for the class (for I do not know any trick to access the state of a VBA class, if you know do please comment below). So imagine if you have twenty classes in your project of varying byte lengths then you'd need twenty separate UDTs for serialisation to cope with the varying lengths.

For each class, one would need the three persistence routines given in the code: LoadFromByteArray(), CheckSizeOf() and SaveToByteArray()

I just wonder if we can do better than this.

One immediate change I see is to optimise the copying of the array as given in LoadFromByteArray(). Iterating through each byte in the byte array in VBA is clearly sub-optimal, much better here to copy memory directly because an array of bytes is guaranteed to be contiguous. So there is a replacement for the array loop on lines 57-60, you need to uncomment line 56 and line 03; then comment out line 57-60.

But if you cross the Rubicon and start using CopyMemory() then aren't even more optimisations available? This will be the subject of the following posts.