VBA - WinHttpRequest - No asynchronous chunks

Summary: WinHttp.WinHttpRequest is yet another Http request class that has some features such as chunking, sadly it won't chunk asynchronously.

So someone asked a good question of SO about chunking data from a web service, the questioner complained about missing data. With the object browser it can be seen that WinHttp.WinHttpRequest supports events which can be used to trap chunks of data. It looked promising. However, after some experimentation it does not seem possible to have asynchronous chunking. One can have asynchronous request and receive the whole request or one can have a chunked synchronous request. One cannot have both chunked and asynchronous.

I give code below some that others can check my results. I tested against a Node.js slow and chunky web service from a prior blog post. In order to sink events it is necessary to use the WithEvents keyword in a class module. Here is the class module which I called WHRChunked

Option Explicit

'* Tools->References
'WinHttp        Microsoft WinHTTP Services, version 5.1          C:\WINDOWS\system32\winhttpcom.dll

Private WithEvents moWHR As WinHttp.WinHttpRequest

Public msBufferedResponse As String
Public mbFinished As Boolean

Private Const mbDEFAULT_DEBUG  As Boolean = True
Public mvDebug As Variant

Public Property Get bDebug() As Boolean
    If IsEmpty(mvDebug) Then mvDebug = mbDEFAULT_DEBUG
    
    bDebug = mvDebug
End Property
Public Property Let bDebug(ByVal bRHS As Boolean)
    mvDebug = bRHS
End Property

Public Sub HttpGet(ByVal sURL As String, bAsync As Boolean)
    On Error GoTo ErrHandler

    Set moWHR = New WinHttp.WinHttpRequest
    
    
    mbFinished = False
    msBufferedResponse = ""
    
    moWHR.Open Method:="GET", URL:=sURL, async:=bAsync
    
    moWHR.send
    Debug.Print "send called with bAsync=" & bAsync
SingleExit:
    Exit Sub
ErrHandler:
    Debug.Print "Error (" & Err.Number & ") " & Err.Description
    Stop
    Resume
    
End Sub


Private Sub moWHR_OnError(ByVal ErrorNumber As Long, ByVal ErrorDescription As String)
    Debug.Print "moWHR_OnError"

End Sub

Private Sub moWHR_OnResponseDataAvailable(Data() As Byte)
    
    Dim sThisChunk As String
    sThisChunk = StrConv(Data(), vbUnicode)
    
    Debug.Print "moWHR_OnResponseDataAvailable (" & Len(sThisChunk) & ")"
    
    msBufferedResponse = msBufferedResponse & sThisChunk
    
End Sub

Private Sub moWHR_OnResponseFinished()
    Debug.Print "moWHR_OnResponseFinished"
    mbFinished = True
End Sub

Private Sub moWHR_OnResponseStart(ByVal Status As Long, ByVal ContentType As String)

    Dim v
    v = VBA.Split(moWHR.getAllResponseHeaders, vbNewLine)
    Debug.Print "moWHR_OnResponseStart"

End Sub

And we need some code in a standard module to call into the class, remember this needs the web service from previous blog post.

Option Explicit

Sub Test()
    Dim oWHRChunked As WHRChunked
    Set oWHRChunked = New WHRChunked
    
    oWHRChunked.HttpGet "http://localhost:34957/slowAndChunkyWebService?chunkCount=2", True
    'oWHRChunked.HttpGet "http://localhost:34957/slowAndChunkyWebService?chunkCount=2", False
    
    While oWHRChunked.mbFinished = False
        DoEvents
    Wend
    
    Debug.Print oWHRChunked.msBufferedResponse


End Sub

So to experiment simply swap the above commented line for the other to see the different effects, the evidence is posted to the Immediate window using Debug.Print .

Final thoughts, I'm disappointed by this finding I hope I have it wrong. I must do a comparison table of the different features between MSXML2.XMLHTTP60, MSXML2.ServerXMLHTTP60 and WinHttpRequest.

Node.js - Slow and chunky webservice (deliberately slow)

So I want to test event handling of a library available for VBA developers but to test it I need to first build a web server that is deliberately slow and chunky. I blogged a simple web service previously that chunked a request (taking each chunk to process a portion of the post body). This time I want to chunk the response.

We use setTimeout (just like a browser) to schedule execution of a block of code. We are not reading a file or anything we are simply sending text strings back. We are splitting this out into schedule chunks to fit nicely with Node.js asynchronous non-blocking interleaved execution pattern.

The node.js libraries used are http and url. http handles the request and response streams. url will parse the url including querystring which is required here, we parse out chunkCount from the querystring (and take 1 on default). Parsing the url make its easy to route the url, in the code we are only interested in urls that start with /slowAndChunkyWebService.

Use Visual Studio 2017 community with Node,js installed and create a console app then paste in the following code, then press F5 to start running.

'use strict';

const http = require('http');
const url = require('url');
const port = 34957;

console.log('Slow and chunky web servicen');

const requestHandler = (request, response) => {

    var url_parts = url.parse(request.url, true);
    var query = url_parts.query;

    if (url_parts.pathname == '/slowAndChunkyWebService') {

        var chunkCount = 0;

        try {
            chunkCount = parseInt(query.chunkCount);

            if (typeof (chunkCount) == "undefined") { chunkCount = 1; }

        }
        catch (ex) { chunkCount = 1; }

        console.log('main code about to call myWriteChunk() with chunkCount' + chunkCount + 'n');
        myWriteChunk(response, chunkCount);

    } else {
        console.log(request.url);
        response.end(request.url);
    }
}

function myWriteChunk(response, chunkCount) {
    console.log('myWriteChunk called chunkCount' + chunkCount + 'n')

    var chunky = "foobar".repeat(10);

    response.write(chunky);

    chunkCount--;

    if (chunkCount > 0) {
        setTimeout(function () { myWriteChunk(response, chunkCount) }, 1000)
    } else {
        console.log('about to schedule myResponseEndn')
        setTimeout(function () { myResponseEnd(response) }, 1000)
        
    }
}

function myResponseEnd(response) {
    console.log('myResponseEnd calledn')
    response.end();
}



const server = http.createServer(requestHandler);

server.listen(port, (err) => {
    if (err) {
        return console.log('something bad happened', err);
    }

    console.log(`server is listening on ${port}`);
})

One can test this by going to a browser and typing in the url...

http://localhost:34957/slowAndChunkyWebService?chunkCount=2

In the node.js console the following output should be seen...

Debugger listening on ws://127.0.0.1:48449/40a2b131-8546-4801-adf3-c3b16d0b72a2
For help see https://nodejs.org/en/docs/inspector
Debugger attached.
(node:17292) [DEP0062] DeprecationWarning: `node --inspect --debug-brk` is deprecated. Please use `node --inspect-brk` instead.
Slow and chunky web service

server is listening on 34957
main code about to call myWriteChunk() with chunkCount2

myWriteChunk called chunkCount2

myWriteChunk called chunkCount1

about to schedule myResponseEnd

myResponseEnd called

/favicon.ico

And the browser will show contents only after all of them have been received.

foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar