First, a health warning, the code in this blog comes with no warranty. Microsoft would say that copying memory can have unpredictable results. Nevertheless, if you are fascinated by VBA then you might be interested in the guts of VBA.
In the previous post I showed how LSet can be used to copy the state from a user-defined type (UDT) into a byte array and vice versa. This can be used in distributed systems but also one could save the byte array to a file and then load into a subsequent Excel session. Thus it is possible to persist a (UDT) to file, and if you use classes then you can gather all your class level variables into a UDT. One can begin to see how a whole object graph (i.e. the state of a whole hierarchy of classes) might be persist-able.
At the end of the previous post I introduced the CopyMemory() function and I used it to replace the copying of a byte array.
The copied byte array was located in a companion UDT defined solely for LSet-ting to the real UDT. I wondered if I could cut out the middle step and use CopyMemory() to copy directly from the byte array to the real UDT. I have conducted some experiments and you can so long as you use Byte arrays for your strings. Details of my experimental code follow.
modTestCopyStringToBytes Standard Module
So we need to treat strings differently because they are not inherently contiguous, we need to convert them to a byte array but this is not too difficult. What is important is ensuring that no overwriting of other memory happens. In the procedure TestCopyStringToBytes() below you can see that we find the length of the array and the length of the string we're asked to copy and we truncate the string if the byte array is not large enough. We copy over only safe number of bytes. The code has plenty of comments.
- Option Explicit
- '* No Warranty, use this code at your own risk!
- Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)
- '*
- '* Not used in the main code.
- '* Illustrates how strings have be treated carefully as an array of bytes
- '*
- Private Sub TestCopyStringToBytes()
- '* set up string
- Dim sRHS As String
- sRHS = "hello world"
- '* defining the string length here but it could passed in and thus unknown
- Dim abStringAsBytes(0 To 9) As Byte
- '* standard array length determination, I know it is fixed above but ...
- '* ... in other cases the array could be passed in and be of unknown length
- Dim lMaxLen As Long
- lMaxLen = (UBound(abStringAsBytes) - LBound(abStringAsBytes) + 1)
- '* we'd like to not overwrite the allocated memory so truncate the string
- '* remember strings are two bytes for each character so we divide by 2
- Dim sSafeRHS As String
- sSafeRHS = Left$(sRHS, lMaxLen / 2)
- '* give a truncation warning to avoid shock
- If lMaxLen < Len(sRHS) Then Debug.Print "Warning contents of sRHS will be truncated from '" & sRHS & "' to '" & sSafeRHS & "'"
- '* this next function will copy the memory from the string sSafeRHS to the first byte of the byte array, abStringAsBytes(0)
- CopyMemory ByVal VarPtr(abStringAsBytes(0)), ByVal StrPtr(sSafeRHS), LenB(sSafeRHS)
- '* this next line works thanks to VBA being nice and interpreting a byte array as a string for us
- '* (saves us having to reverse the memory copy operation)
- Debug.Print abStringAsBytes
- End Sub
If you run the code you should get a truncation warning and the output of the copied truncated string like this
Warning contents of sRHS will be truncated from 'hello world' to 'hello'
hello
modCopyMemoryDirectToUdt Standard Module
Here is a a very full listing with plenty of debug code to illustrate what is going on. The code is heavily commented. The indented code in Main() is not strictly necessary but it serves as a guide as to what is going on.
- Option Explicit
- '* No Warranty, use this code at your own risk!
- Private Const mlPAD As Long = 28 '* just for formatting
- Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)
- '*
- '* See here we only need one type per entity/class
- '* DO NOT USE "aFixedString As String * 20" as it is not contiguous!
- '* INSTEAD USE "aFixedStringAsBytes(0 To 19) As Byte"
- '*
- Public Type udtContiguousType
- aLong As Long
- anInt As Integer
- aDouble As Double
- aFixedStringAsBytes(0 To 9) As Byte '
- aSecondLong As Long
- aSecondDouble As Double
- End Type
- '*
- '* The following property procedures could be adjusted to sit in a class
- '* the class could store the string as a byte array in the class's state UDT
- '* to the callers of the class the property could look like just a string
- '*
- Property Let MyFixedString(ByRef uType As udtContiguousType, ByVal sRHS As String)
- '* array length determination
- Dim lMaxLen As Long
- lMaxLen = ByteArrayLen(uType.aFixedStringAsBytes)
- '* we'd like to not overwrite the allocated memory so truncate the string
- '* remember strings are two bytes for each character so we divide by 2
- Dim sSafeRHS As String
- sSafeRHS = Left$(sRHS, lMaxLen / 2)
- '* give a truncation warning to avoid shock
- If lMaxLen < Len(sRHS) Then Debug.Print "Warning contents of sRHS will be truncated from '" & sRHS & "' to '" & sSafeRHS & "'"
- '* this next function will copy the memory from the string sSafeRHS ...
- '* ... to the first byte of the byte array, uType.aFixedStringAsBytes(0)
- CopyMemory ByVal VarPtr(uType.aFixedStringAsBytes(0)), ByVal StrPtr(sSafeRHS), LenB(sSafeRHS)
- End Property
- Property Get MyFixedString(ByRef uType As udtContiguousType) As String
- '* this line works thanks to VBA being nice and interpreting a byte array as a string for us
- '* (saves us having to reverse the memory copy operation)
- MyFixedString = uType.aFixedStringAsBytes
- End Property
- '*
- '* the main code, run this code
- '*
- Private Sub Main()
- Dim bDebug As Boolean
- bDebug = True
- Dim uTypeSrc As udtContiguousType
- Dim uTypeDest As udtContiguousType
- '* set some values just like any other UDT, except for the strings
- uTypeSrc.aLong = 1
- uTypeSrc.anInt = 2
- uTypeSrc.aDouble = 3.141
- uTypeSrc.aSecondLong = -434634634
- uTypeSrc.aSecondDouble = Sqr(10) * -1
- '* we have to treat strings specially
- '* the string's special treatment is abstracted by the property procedures
- MyFixedString(uTypeSrc) = "hello world"
- If bDebug Then
- '* we have to be aware of truncation
- Debug.Assert MyFixedString(uTypeSrc) = Left$("hello world", ByteArrayLen(uTypeSrc.aFixedStringAsBytes) / 2)
- '* for illustration we print the memory addresses so that one can see the udt's memory layout , not strictly necessary
- PrintMemoryAddresses "uTypeSrc", uTypeSrc
- '* these next lines highlight the byte length of each member
- '* not strictly necessary
- Debug.Assert VarPtr(uTypeSrc) = VarPtr(uTypeSrc.aLong)
- Debug.Assert VarPtr(uTypeSrc.anInt) - VarPtr(uTypeSrc.aLong) = 4
- Debug.Assert VarPtr(uTypeSrc.aDouble) - VarPtr(uTypeSrc.anInt) = 4
- Debug.Assert VarPtr(uTypeSrc.aFixedStringAsBytes(0)) - VarPtr(uTypeSrc.aDouble) = 8
- Debug.Assert VarPtr(uTypeSrc.aSecondLong) - VarPtr(uTypeSrc.aFixedStringAsBytes(0)) =
- BytesAlignedLen(ByteArrayLen(uTypeSrc.aFixedStringAsBytes))
- Debug.Assert VarPtr(uTypeSrc.aSecondDouble) - VarPtr(uTypeSrc.aSecondLong) = 4
- '* for illustration we print the memory addresses so that one can see the udt's memory layout , not strictly necessary
- PrintMemoryAddresses "uTypeDest", uTypeDest
- '* more layout illustration
- Debug.Assert VarPtr(uTypeDest) = VarPtr(uTypeDest.aLong)
- End If
- CopyMemory ByVal (VarPtr(uTypeDest.aLong)), ByVal (VarPtr(uTypeSrc.aLong)), LenB(uTypeSrc)
- If bDebug Then
- Debug.Assert MyFixedString(uTypeDest) = Left$("hello world", ByteArrayLen(uTypeSrc.aFixedStringAsBytes) / 2)
- Debug.Assert uTypeDest.aLong = uTypeSrc.aLong
- Debug.Assert uTypeDest.anInt = uTypeSrc.anInt
- Debug.Assert uTypeDest.aDouble = uTypeSrc.aDouble
- Debug.Assert MyFixedString(uTypeDest) = MyFixedString(uTypeSrc)
- Debug.Assert uTypeDest.aSecondLong = uTypeSrc.aSecondLong
- Debug.Assert uTypeDest.aSecondDouble = uTypeSrc.aSecondDouble
- End If
- Stop 'please go look at the Locals window
- End Sub
- '*
- '* standard array length calculation
- '*
- Private Function ByteArrayLen(ByRef abBytes() As Byte) As Long
- ByteArrayLen = (UBound(abBytes()) - LBound(abBytes()) + 1)
- End Function
- '*
- '* each member of a user defined type is 32-bit aligned so this function calcs the aligned byte length
- '*
- Private Function BytesAlignedLen(ByVal cbByteCount As Long) As Long
- If cbByteCount \ 4 = cbByteCount / 4 Then
- BytesAlignedLen = cbByteCount
- Else
- BytesAlignedLen = ((cbByteCount \ 4) + 1) * 4
- End If
- End Function
- '*
- '* this prints the memory addresses so one can see the memory laid out
- '*
- Private Function PrintMemoryAddresses(ByVal sVarName As String, ByRef uType As udtContiguousType)
- Debug.Print()
- Debug.Print Pad("&" & sVarName & "", mlPAD) & " = " & VarPtr(uType)
- Debug.Print Pad("&" & sVarName & ".aLong", mlPAD) & " = " & VarPtr(uType.aLong) & " a Long has 4 bytes"
- Debug.Print Pad("&" & sVarName & ".anInt", mlPAD) & " = " & VarPtr(uType.anInt) &
- " an Int has 2 bytes but is 32-bit aligned so has 4 bytes"
- Debug.Print Pad("&" & sVarName & ".aDouble", mlPAD) & " = " & VarPtr(uType.aDouble) & " a Double has 8 bytes"
- Dim lMaxLen As Long
- lMaxLen = ByteArrayLen(uType.aFixedStringAsBytes)
- Dim lAlignedMaxLen As Long
- lAlignedMaxLen = BytesAlignedLen(lMaxLen)
- Dim sByteLen As String
- sByteLen = " this byte array has " & lMaxLen & " bytes" &
- VBA.IIf(lAlignedMaxLen <> lMaxLen, " but is 32-bit aligned to " & lAlignedMaxLen & " bytes", "")
- Debug.Print Pad("&" & sVarName & ".aFixedStringAsBytes", mlPAD) & " = " & VarPtr(uType.aFixedStringAsBytes(0)) & sByteLen
- Debug.Print Pad("&" & sVarName & ".aSecondLong", mlPAD) & " = " & VarPtr(uType.aSecondLong) & " a Long has 4 bytes"
- Debug.Print Pad("&" & sVarName & ".aSecondDouble", mlPAD) & " = " & VarPtr(uType.aSecondDouble) & " a Double has 8 bytes"
- Debug.Print Pad("&" & sVarName & " total length", mlPAD) & " = " & LenB(uType)
- End Function
- '*
- '* this pads a string to help it looks columnar
- '*
- Private Function Pad(ByVal sText As String, ByVal lLen As Long) As String
- Pad = Left$(sText & String(lLen, " "), lLen)
- End Function
So when the Main() code runs we have two things to look at (1) the Immediate Window and (2) the Locals Window. Here are mine, firstly the Immediate Window
Warning contents of sRHS will be truncated from 'hello world' to 'hello'
&uTypeSrc = 5240168
&uTypeSrc.aLong = 5240168 a Long has 4 bytes
&uTypeSrc.anInt = 5240172 an Int has 2 bytes but is 32-bit aligned so has 4 bytes
&uTypeSrc.aDouble = 5240176 a Double has 8 bytes
&uTypeSrc.aFixedStringAsByte = 5240184 this byte array has 10 bytes but is 32-bit aligned to 12 bytes
&uTypeSrc.aSecondLong = 5240196 a Long has 4 bytes
&uTypeSrc.aSecondDouble = 5240200 a Double has 8 bytes
&uTypeSrc total length = 40
&uTypeDest = 5240128
&uTypeDest.aLong = 5240128 a Long has 4 bytes
&uTypeDest.anInt = 5240132 an Int has 2 bytes but is 32-bit aligned so has 4 bytes
&uTypeDest.aDouble = 5240136 a Double has 8 bytes
&uTypeDest.aFixedStringAsByt = 5240144 this byte array has 10 bytes but is 32-bit aligned to 12 bytes
&uTypeDest.aSecondLong = 5240156 a Long has 4 bytes
&uTypeDest.aSecondDouble = 5240160 a Double has 8 bytes
&uTypeDest total length = 40
Now here is my Locals Window screenshot with the variables expanded to show the members. It shows the contents successfully copied from uTypeSrc to uTypeDest.
Commentary
It is interesting that VBA could potentially participate in a persistence scheme. I am minded to investigate the persistence interfaces of the COM specification such as IPersist to see if VBA could indeed play in this league. One problem is that IPersist has non Automation types so one would need to define a proxy component and a proxy interface.
Final Warranty Warning
I can't give a warranty for this code (nor any code on this blog for that matter).