Showing posts with label Html. Show all posts
Showing posts with label Html. Show all posts

Thursday, 26 August 2021

CSS Grid's 'grid-template-areas' are wonderfully intuitive

Finally, I have found a decent CSS grid layout technology. Twenty years ago, I used to use HTML tables to structure a page. Then we were told not to use tables and switch over to CSS instead but the CSS techniques at the time were inadequate and so I and many other programmers carried on with HTML tables. Now, I am happy to blog about CSS Grid's grid-template-areas which are wonderfully intuitive way to layout a page.

A really good YouTube video is Easily Structure your Layout with CSS Grid's 'grid-template-areas' and I have given the source code for this video below. I have also embedded the sample page into this blog entry, converting as required. You should find that this is a responsive page that will reduce to a column/stack if the browser's width is made narrow. The CSS has a 'mobile first' design in that the default declaration is for the reduced screen mobile stack whilst the media queries further down are where to find the full window declarations.

You really should watch the video in full but for those in a hurry the real essence is in the following extracts, first we have this grid-template-areas CSS property...

grid-template-areas:
    "sidebar header header header"
    "sidebar sect1  sect2  sect3"
    "sidebar main   main   main"
    "sidebar footer footer footer";

Then we have the HTML...

<body>
    <aside></aside>
    <header></header>
    <section></section>
    <section></section>
    <section></section>
    <main></main>
    <footer></footer>
</body>

Then these are tied together by specifying the grid-area property in each HTML element's CSS ...

aside { grid-area: sidebar; }
header { grid-area: header; }
section:nth-of-type(1) { grid-area: sect1; }
section:nth-of-type(2) { grid-area: sect2; }
section:nth-of-type(3) { grid-area: sect3; }
main { grid-area: main; }
footer { grid-area: footer; }

And that's it, full listing below. Speaking personally this will be my go to page when drawing up a web page from scratch. Enjoy!

Sample Page

Code Listings

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <link rel="stylesheet" href="main.css">
</head>
<body>
    <aside></aside>
    <header></header>
    <section></section>
    <section></section>
    <section></section>
    <main></main>
    <footer></footer>
</body>
</html></html>

main.css

body,
html {
    height: 100vh;
}

body {
    margin: 0;
    display: grid;
    grid-template-columns: 100%;
    grid-template-rows: repeat(5, auto);
    grid-template-areas:
        "sect1"
        "sect2"
        "sect3"
        "main"
        "footer";
}

aside {
    grid-area: sidebar;
    background-color: #007fff;
}

header {
    grid-area: header;
    background-color: #71b8eb;
}

section:nth-of-type(1) {
    grid-area: sect1;
    background-color: #B3D8FD;
}

section:nth-of-type(2) {
    grid-area: sect2;
    background-color: #5E86AF;
}

section:nth-of-type(3) {
    grid-area: sect3;
    background-color: #6D9FD2;
}

main {
    grid-area: main;
    background-color: #7DA9D5;
}

footer {
    grid-area: footer;
    background-color: #588EC3;
}

@media only screen and (min-width: 768px) {
    body {
        margin: 0;
        display: grid;
        grid-template-columns: auto 27% 27% 27%;
        grid-template-rows: 8% 30% auto 10%;
        grid-template-areas:
            "sidebar header header header"
            "sidebar sect1  sect2  sect3"
            "sidebar main   main   main"
            "sidebar footer footer footer";
    }
}

Links

Friday, 12 October 2018

VBA - Microsoft.ACE.OLEDB.12.0 - Skip HTML Import Export

This post follows on from the other day's mega-post on Microsoft.ACE.OLEDB.12.0. As part of the investigation into the Excel engine, I discovered how to use the Html Export and Html Import drivers (yes they are separate).

Initially I was excited to find another way to store and retrieve data without requiring a database. After writing some code my enthusiasm has waned and my technical recommendation is that you skip this technology.

Weaknesses with the HTML Import Export Drivers

So, I ought to itemise my concerns here...

Type Inference

So the Excel driver has to sample some rows to guess the type of column (I have yet to find a way to declare the column type) and the Html Import driver does so equally. I came up against this more so with the Html Import driver, I'm guessing it samples fewer rows, anyway I had to change the text in the sample data (compared to the mega post) and give the hex values leading ampersands to enforce its inference as being a string. E.g. I needed to replace 00FF00 to &00FF00 in SetUpSomeData(). I have discovered that ordinary text files can have schema.ini files attached, which would put text files streets ahead of Html files because it obviates the need for type inference.

Html Export is not well-formed Xml

I'd like to nominate Html for a troublesome technology award. Html is in same family as Xml but typically cannot be parsed with Xml parsers because it is not well-formed. The latest version of Html, Html 5, is well-formed so going forward things ought to be better. This driver Microsoft.ACE.OLEDB.12.0 is quite new and ought to be output well-formed Html but it doesn't. This means having to write some large amounts of paring code (see AggregateHtmlFiles() in sample code). I could write that code ten times more concisely with Xml library!

Missed opportunity to export multiple tables to one single file

So I myself had to write code to aggregate tables to one single file. It turned out to be more painful that I imagined. The code below shows HTML parsing logic (see AggregateHtmlFiles() in sample code). If I were to rewrite this I'd scrap that approach and start with an Xml representation of cell block and XSLT transform that into the correct Html. This is a shame, for a while I thought multi-table Html file could make a nice config file.

Sample Code

Anyway, I wrote plenty of experimental code. Instead of throwing away this code, I will deposit here. There is an example of exporting from a sheet to an Html file. There is an example of reading an Html file.

Also, because the export only allows one table per Html file I have written some code to aggregate separate single table files into a multi-table file, see AggregateHtmlFiles(). I did this with the same libraries that Internet Explorer uses and so had to workaround IE bugs, code would have been much simple if the InsertAdjacentHTML method wasn't buggy.

Option Explicit
Option Private Module

'* Tools -> References
'* ADODB  Microsoft ActiveX Data Objects 6.1 Library  C:\Program Files (x86)\Common Files\System\ado\msado15.dll
'* ADOX   Microsoft ADO Ext. 6.0 for DDL and Security C:\Program Files (x86)\Common Files\System\ado\msadox.dll

Private Const msColorsHtmlFile As String = "N:Colors.html" '<---- change this to your working file location
Private Const msCurrenciesHtmlFile As String = "N:Currencies.html" '<---- change this to your working file location
Private Const msTeamsHtmlFile As String = "N:Teams.html" '<---- change this to your working file location
Private Const msAggregatedHtmlFile As String = "N:Aggregated.html" '<---- change this to your working file location

Private Sub SetUpSomeData()
    '* WARNING this will wipe data!
    Dim sht As Excel.Worksheet
    Set sht = ThisWorkbook.Worksheets.Item("Sheet1")
    sht.Cells.Clear

    '*
    '* use our array literal trick, for more tricks tips and 'blue sky thinking'
    '* see http://exceldevelopmentplatform.blogspot.com
    '*
    Dim vColors As Variant
    vColors = [{"Color","RGB";"Red","&FF0000";"Green","&00FF00"}] '* note the addition of the ampersand for type inference

    sht.Range("A1:B3").Value2 = vColors

    Dim vCurrencies As Variant
    vCurrencies = [{"Country","Ccy";"France","Euro";"Japan","Yen"}]

    sht.Range("D1:E3").Value2 = vCurrencies

    Dim vTeams As Variant
    vTeams = [{"Team","Country";"New York Red Bulls","US";"Spartak Moskva","Russia";"Man Utd","England";"Barcelona","Spain";"Bayern Munich","Germany"}]

    sht.Range("G1:H6").Value2 = vTeams

End Sub

Private Sub TestWriteToHtmlFile()
    WriteToHtmlFile ThisWorkbook.Worksheets.Item("Sheet1").Range("A1").CurrentRegion, msColorsHtmlFile
    WriteToHtmlFile ThisWorkbook.Worksheets.Item("Sheet1").Range("D1").CurrentRegion, msCurrenciesHtmlFile
    WriteToHtmlFile ThisWorkbook.Worksheets.Item("Sheet1").Range("G1").CurrentRegion, msTeamsHtmlFile
End Sub

Private Sub WriteToHtmlFile(ByVal rngTable As Excel.Range, ByVal sColorsHtmlFile As String)

    Dim sTableAddress As String
    sTableAddress = "[" & rngTable.Worksheet.Name & "$" & rngTable.Address(False, False, xlA1) & "] """

    Dim oConnExcel As ADODB.Connection
    Set oConnExcel = New ADODB.Connection

    If UBound(Split(ThisWorkbook.Name, ".")) = 0 Then Err.Raise vbObjectError, , "#Workbook needs a file extension, i.e. saved at least once!"

    '*
    '* we're reading from worksheet so we need the Excel engine
    '*
    oConnExcel.Open "Provider=Microsoft.ACE.OLEDB.12.0;" & _
           "Data Source=" & ThisWorkbook.FullName & ";" & _
           "Extended Properties='Excel 12.0 Macro;HDR=YES'"

    Dim sFileNameOnly As String, sFolderOnly As String

    If ParseFileName(sColorsHtmlFile, sFileNameOnly, sFolderOnly) Then

        If Not Dir(sColorsHtmlFile) = "" Then Kill sColorsHtmlFile   '* if it exists then delete before re-exporting

        Dim sCmdText As String
        sCmdText = VBA.Replace("SELECT * INTO [%filename%] in %quotedFolder% ""HTML Export;"" FROM %tableName%", "%filename%", sFileNameOnly)
        sCmdText = VBA.Replace(sCmdText, "%quotedFolder%", """" & sFolderOnly & """")
        sCmdText = VBA.Replace(sCmdText, "%tableName%", sTableAddress)

        '*
        '* HTML Export is specified in the command text, no need for a separate connection
        '*
        Debug.Print sCmdText
        oConnExcel.Execute sCmdText
    End If

End Sub


Private Sub TestReadFromHtmlFile()
    ReadFromHtmlFile msColorsHtmlFile, "Colors"
    ReadFromHtmlFile msCurrenciesHtmlFile, "Currencies"
End Sub


Private Sub ReadFromHtmlFile(ByVal sHtmlFile As String, ByVal sTableName As String)

    Dim oConnHtmlImport As ADODB.Connection
    Set oConnHtmlImport = New ADODB.Connection
    oConnHtmlImport.Open "Provider=Microsoft.ACE.OLEDB.12.0;" & _
           "Data Source=" & sHtmlFile & _
           ";Extended Properties='HTML Import;HDR=YES'"

    Dim rsHtmlImport As ADODB.Recordset
    Set rsHtmlImport = New ADODB.Recordset

    '*
    '* the SQL-table-name is the same as the caption for the HTML table
    '* and not the HTML Title element as written in www.connectionstrings.com
    '*
    '* as with open for Excel I needed to use adOpenStatic
    '*
    rsHtmlImport.Open "SELECT * From [" & sTableName & "]", oConnHtmlImport, CursorTypeEnum.adOpenStatic '* can use CursorTypeEnum.adOpenKeyset
    Debug.Assert rsHtmlImport.RecordCount > 0

    '*
    '* do some work with the recordset here
    '*
    DumpRecordset rsHtmlImport
    'Stop

    '*
    '* when finished close the connection to stop file locks
    '*
    oConnHtmlImport.Close
    Set rsHtmlImport.ActiveConnection = Nothing
    Set oConnHtmlImport = Nothing
    Set rsHtmlImport = Nothing
    'Stop

End Sub

Private Sub TestAggregateHtmlFiles()
    Dim sTransactionFiles(0 To 1) As String
    sTransactionFiles(0) = msCurrenciesHtmlFile
    sTransactionFiles(1) = msTeamsHtmlFile
    
    
    Dim sNewMasterHtml As String
    sNewMasterHtml = AggregateHtmlFiles(msColorsHtmlFile, sTransactionFiles)
    Debug.Print sNewMasterHtml
    
    Dim lFile As Long
    lFile = FreeFile()

    If Dir(msAggregatedHtmlFile) <> "" Then Kill msAggregatedHtmlFile

    Open msAggregatedHtmlFile For Output As #lFile
    Print #lFile, sNewMasterHtml
    Close #lFile
    
End Sub

Private Function AggregateHtmlFiles(ByVal sMasterFile As String, ByRef sTransactionFiles() As String) As String
    '*
    '* check files exists firstly
    '*
    CheckHtmlFileExists sMasterFile
    Dim lTransactionFileLoop As Long
    For lTransactionFileLoop = LBound(sTransactionFiles) To UBound(sTransactionFiles)
        Dim sTransactionFile As String
        sTransactionFile = sTransactionFiles(lTransactionFileLoop)

        CheckHtmlFileExists sTransactionFile
    Next

    Dim oHtml4 As MSHTML.IHTMLDocument4
    Set oHtml4 = New MSHTML.HTMLDocument

    Dim htmlMaster As MSHTML.HTMLDocument
    Set htmlMaster = oHtml4.createDocumentFromUrl(sMasterFile, "")



    While htmlMaster.readyState <> "complete": DoEvents: Wend
    Dim objMasterTable As HTMLTable, objMasterTableList As Object
    Set objMasterTableList = htmlMaster.querySelectorAll("table > caption")
    Set objMasterTable = objMasterTableList.Item(objMasterTableList.Length - 1).parentElement
    'objMasterTable.parentElement

    Dim oMasterBody As MSHTML.HTMLBody
    Set oMasterBody = objMasterTable.parentElement

    For lTransactionFileLoop = LBound(sTransactionFiles) To UBound(sTransactionFiles)

        sTransactionFile = sTransactionFiles(lTransactionFileLoop)

        '*
        Dim htmlTransactionFile As MSHTML.HTMLDocument
        Set htmlTransactionFile = oHtml4.createDocumentFromUrl(sTransactionFile, "")
        While htmlTransactionFile.readyState <> "complete": DoEvents: Wend

        '* get captioned table
        Dim objTransactionTableCaption As HTMLTable
        Set objTransactionTableCaption = htmlTransactionFile.querySelector("table > caption")


        If Not objTransactionTableCaption Is Nothing Then
            '*
            Dim objTransactionTable As HTMLTable
            Set objTransactionTable = objTransactionTableCaption.parentElement


            '* write and add the table element
            Dim objNewTable As HTMLTable
            Set objNewTable = htmlMaster.createElement("TABLE")
            objNewTable.setAttribute "border", "1"
            oMasterBody.appendChild objNewTable


            '* write and add the table caption element
            Dim objNewCaption As HTMLTableCaption
            Set objNewCaption = htmlMaster.createElement("CAPTION")
            objNewTable.appendChild objNewCaption
            objNewCaption.innerText = objTransactionTableCaption.innerText

            '* write the column headers
            Dim objTransTableHeaderRow As Object
            Set objTransTableHeaderRow = objTransactionTable.querySelectorAll("tr > th").Item(0).parentElement

            Dim objNewTableRow As Object
            Set objNewTableRow = htmlMaster.createElement("TR")
            objNewTable.appendChild objNewTableRow

            Dim lColumnCount As Long, lColumnLoop As Long
            lColumnCount = objTransTableHeaderRow.ChildNodes.Length

            For lColumnLoop = 0 To lColumnCount - 1
                Dim objNewTH As Object, objTransTH As Object
                Set objTransTH = objTransTableHeaderRow.ChildNodes.Item(lColumnLoop)
                Set objNewTH = htmlMaster.createElement("TH")
                objNewTH.innerText = objTransTH.innerText
                objNewTableRow.appendChild objNewTH
            Next

            Dim objTransTableDataRow As Object
            Set objTransTableDataRow = objTransTableHeaderRow.NextSibling
            While Not objTransTableDataRow Is Nothing
                
                Set objNewTableRow = htmlMaster.createElement("TR")
                objNewTable.appendChild objNewTableRow
                
                For lColumnLoop = 0 To lColumnCount - 1
                    Dim objNewTD As Object, objTransTD As Object
                    Set objTransTD = objTransTableDataRow.ChildNodes.Item(lColumnLoop)
                    Set objNewTD = htmlMaster.createElement("TD")
                    objNewTD.innerText = objTransTD.innerText
                    objNewTableRow.appendChild objNewTD
                Next

                Set objTransTableDataRow = objTransTableDataRow.NextSibling
            Wend

        End If

    Next lTransactionFileLoop

    AggregateHtmlFiles = htmlMaster.DocumentElement.outerHTML

End Function

Private Sub CheckHtmlFileExists(ByVal sHtmlFile As String)
    If Dir(sHtmlFile) = "" Then Err.Raise vbObjectError, , "#File '" & sHtmlFile & "' does not exist!"
End Sub

Private Sub TestReadCatalogOfHtmlFile()
    ReadCatalogOfHtmlFile msColorsHtmlFile
    ReadCatalogOfHtmlFile msCurrenciesHtmlFile
End Sub

Private Sub ReadCatalogOfHtmlFile(ByVal sColorsHtmlFile As String)
    Dim oConnHtmlImport As ADODB.Connection
    Set oConnHtmlImport = New ADODB.Connection
    oConnHtmlImport.Open "Provider=Microsoft.ACE.OLEDB.12.0;" & _
           "Data Source=" & sColorsHtmlFile & _
           ";Extended Properties='HTML Import;HDR=YES'"

    Dim catDB As ADOX.Catalog
    Dim tblList As ADOX.Table

    Set catDB = New ADOX.Catalog
    Set catDB.ActiveConnection = oConnHtmlImport

    Dim adoxTables As ADOX.Tables
    Set adoxTables = catDB.Tables

    Dim adoxTableLoop As ADOX.Table
    For Each adoxTableLoop In adoxTables
        Debug.Print adoxTableLoop.Name
    Next adoxTableLoop

End Sub

Private Sub DumpRecordset(ByVal rs As ADODB.Recordset)
    '*
    '* Some code to iterate over the recordset
    '*
    rs.MoveFirst

    Dim lFieldCount As Long
    lFieldCount = rs.Fields.Count

    While Not rs.EOF

        Dim sOutputLine As String
        sOutputLine = ""

        Dim sFieldAndValue As String
        Dim lFieldLoop As Long
        For lFieldLoop = 0 To lFieldCount - 1
            sFieldAndValue = rs.Fields.Item(lFieldLoop).Name & ":" & rs.Fields.Item(lFieldLoop).Value

            sOutputLine = sOutputLine & VBA.IIf(Len(sOutputLine) > 0, vbTab, "") & sFieldAndValue
        Next
        Debug.Print sOutputLine
        rs.MoveNext
    Wend

End Sub

Private Sub UnitTestParseFileName()

    Const sExportFile As String = "N:Colors.html"
    Dim sFileNameOnly As String, sFolderOnly As String

    Debug.Assert ParseFileName(sExportFile, sFileNameOnly, sFolderOnly)
    Debug.Assert sFileNameOnly = "Colors.html"
    Debug.Assert sFolderOnly = "N:"

    Debug.Assert ParseFileName("N:folder1folder2Colors.html", sFileNameOnly, sFolderOnly)
    Debug.Assert sFileNameOnly = "Colors.html"
    Debug.Assert sFolderOnly = "N:folder1folder2"

End Sub

Private Function ParseFileName(ByVal sFullFileName As String, ByRef psFileNameOnly As String, ByRef psFolderOnly As String) As Boolean

    Dim vSplit As Variant
    vSplit = VBA.Split(sFullFileName, "")

    Dim lUBound As Long
    lUBound = UBound(vSplit)

    If lUBound > 0 Then
        psFileNameOnly = vSplit(lUBound)
        psFolderOnly = Left(sFullFileName, Len(sFullFileName) - Len(psFileNameOnly))
        ParseFileName = True
    End If

End Function

Sample Output

Sample Export Html Source

<HTML DIR=LTR>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=Windows-1252">
<TITLE>Colors</TITLE>
</HEAD>
<BODY>
<TABLE DIR=LTR BORDER>
<CAPTION>Colors</CAPTION>
<TR>
<TH>Color</TH>
<TH>RGB</TH>
</TR>
<TD DIR=LTR ALIGN=LEFT>Red</TD>
<TD DIR=LTR ALIGN=LEFT>FF0000</TD>
</TR>
<TR>
<TD DIR=LTR ALIGN=LEFT>Green</TD>
<TD DIR=LTR ALIGN=LEFT>00FF00</TD>
</TR>
</TABLE>
</BODY>
</HTML>

Sample Export Html Rendered

Colors
Color RGB
Red FF0000
Green 00FF00

Sample Export of Multiple Tables Html Source

<HTML DIR=LTR>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=Windows-1252">
<TITLE>cannot be empty</TITLE>
</HEAD>
<BODY>

<TABLE>
<TR><TD>

<TABLE DIR=LTR BORDER>
<CAPTION>Colors</CAPTION>
<TR>
<TH>Color</TH>
<TH>RGB</TH>
</TR>
<TD DIR=LTR ALIGN=LEFT>Red</TD>
<TD DIR=LTR ALIGN=LEFT>&FF0000</TD>
</TR>
<TR>
<TD DIR=LTR ALIGN=LEFT>Green</TD>
<TD DIR=LTR ALIGN=LEFT>&00FF00</TD>
</TR>
</TABLE>

</TD><TD>

<TABLE DIR=LTR BORDER>
<CAPTION>Currencies</CAPTION>
<TR>
<TH>Country</TH>
<TH>Ccy</TH>
</TR>
<TD DIR=LTR ALIGN=LEFT>France</TD>
<TD DIR=LTR ALIGN=LEFT>Euro</TD>
</TR>
<TR>
<TD DIR=LTR ALIGN=LEFT>Japan</TD>
<TD DIR=LTR ALIGN=LEFT>Yen</TD>
</TR>
</TABLE>
</TR>
</TABLE>
</BODY>
</HTML>

Sample Export of Multiple Tables Html Rendered

Colors
Color RGB
Red &FF0000
Green &00FF00
Currencies
Country Ccy
France Euro
Japan Yen

Friday, 20 July 2018

Python - HTML - pytidylib does not install HTML Tidy

So last post I wrote Python class to decompile a *.chm compiled help file. Found within is what looks like HTML 3.2 that be should upgraded to either XHTML or HTML5. I had written some VBA code to do this but since Python month on this blog I am keen to find out what a Python developer would do. They would (I should imagine) use the library https://pypi.org/project/pytidylib/ which wraps the venerable HTML Tidy.

pip install pytidylib does not install HTML Tidy

So one installs pytidylib from a command window with admin rights using

pip install pytidylib
C:\Users\Simon\source\repos\foo\bar>pip install pytidylib
Collecting pytidylib
  Downloading https://files.pythonhosted.org/packages/2d/5e/4d2b5e2d443d56f444e2a3618eb6d044c97d14bf47cab0028872c0a468e0/pytidylib-0.3.2.tar.gz (87kB)
    100% |████████████████████████████████| 92kB 1.4MB/s
Installing collected packages: pytidylib
  Running setup.py install for pytidylib ... done
Successfully installed pytidylib-0.3.2

And Using Visual Studio I run a small example program to test the install

from tidylib import tidy_document
document, errors = tidy_document('''<p>fõo <img src="bar.jpg">''',
options={'numeric-entities':1})
print (document)
print (errors)

But unfortunately it complains of not being able to find libtidy which indicates HTML Tidy is not installed for you.

Here is the stack trace

OSError
  Message=Could not load libtidy using any of these names: libtidy,libtidy.so,libtidy-0.99.so.0,cygtidy-0-99-0,tidylib,libtidy.dylib,tidy
  StackTrace:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tidylib\tidy.py:99 in Tidy.__init__
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tidylib\tidy.py:234 in get_module_tidy
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tidylib\tidy.py:222 in tidy_document
C:\Users\Simon\source\repos\CompiledHelpToEbookPythonApp\CompiledHelpToEbookPythonApp\HtmlTidy.py:3 in 

Install HTML Tidy Binaries

It is required to install the HTML Tidy Binaries separately. I got mine from http://binaries.html-tidy.org/. Initially, I took the 32-bit edition which was a mistake and the error persisted. So I took the 64-bit edition, I downloaded tidy-5.6.0-vc14-64b.zip, extracted it and then added the extracted bin folder to my path. Don't forget to restart processes for the environment variables changes to be picked up.

After Successful Install

After successful install this is what is output from the sample program above.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
  <head>
    <title></title>
  </head>
  <body>
    <p>fõo <img src="bar.jpg">
  </body>
</html>

line 1 column 1 - Warning: missing <!DOCTYPE> declaration
line 1 column 1 - Warning: plain text isn't allowed in <head> elements
line 1 column 1 - Info: <head> previously mentioned
line 1 column 1 - Warning: inserting implicit <body>
line 1 column 1 - Warning: inserting missing 'title' element

Press any key to continue . . .

Python - VBA - HTML Help - COM Callable Python class runs *.chm decompiler.

I have many old *.chm compiled help files on my computer and for some reason the help viewer is bust. I am considering writing (Python) code which will upgrade these *.chm files into ebooks. The first module is given here, it will run the Microsoft HTML Help executable (hh.exe) to decompile a *.chm file into its constituent html files all within working subdirectories in the %temp% folder.

Submitting to Code Review

I might very well complete a full application and place in github. This means I need to raise my Python standards and so I have submitted this module to codereview.stackexchange.com

The HelpFileDecompiler Python class

Here is the Python code

class HelpFileDecompiler(object):
    _reg_clsid_ = "{4B388A08-8CAB-4568-AF78-47032744A368}"
    _reg_progid_ = 'PythonInVBA.HelpFileDecompiler'
    _public_methods_ = ['DecompileHelpFileMain']

    def DecompileHelpFileMain(self, sHelpFile):
        import os.path
        eg = ", e.g. 'c:\\foo.chm'"
        if not isinstance(sHelpFile, str):
            raise Exception("sHelpFile needs to be a string" + eg)
        if len(sHelpFile) == 0:
            raise Exception("sHelpFile needs to be a non-null string" + eg)
        if sHelpFile.lower()[-4:] != ".chm":
            raise Exception("sHelpFile needs to end with '.chm' " + eg)
        if not os.path.isfile(sHelpFile):
            raise Exception("sHelpFile " + sHelpFile + " not found")

        self.BuildTmpDirectory()
        sCopiedFile = self.CopyChmFileToTempAppPath(sHelpFile)
        self.DecompileHelpFile(sCopiedFile)

    def BuildTmpDirectory(self):
        import os
        import os.path
        sTempAppPath = os.path.join(os.environ['tmp'], 'HelpFileDecompiler')

        if not os.path.isdir(sTempAppPath):
            os.mkdir(sTempAppPath)

    def CopyChmFileToTempAppPath(self, sHelpFile):
        import shutil
        import os.path
        sDestFile = os.path.join(os.environ['tmp'], 'HelpFileDecompiler',
                                    os.path.basename(sHelpFile))
        shutil.copyfile(sHelpFile, sDestFile)
        return sDestFile

    def DecompileHelpFile(self, sHelpFile):
        import win32api
        import subprocess
        import os.path

        if not os.path.isfile(sHelpFile):
            raise Exception("sHelpFile " + sHelpFile + " not found")

        sDecompiledFolder = os.path.join(
            os.environ['tmp'],
            'HelpFileDecompiler',
            os.path.basename(sHelpFile).split(".")[0])

        if not os.path.isdir(sDecompiledFolder):
            os.mkdir(sDecompiledFolder)

        sHELPEXE = "C:\Windows\hh.exe"
        if not os.path.isfile(sHELPEXE):
            raise Exception("sHELPEXE " + sHELPEXE + " not found")

        # Not allowed to quote arguments to HH.EXE
        # so we take the short path to eliminate spaces
        sHelpFileShort = win32api.GetShortPathName(sHelpFile)
        sDecompiledFolderShort = win32api.GetShortPathName(sDecompiledFolder)

        # so now we can run the decompiler
        subprocess.run([sHELPEXE, '-decompile',
                        sDecompiledFolderShort, sHelpFileShort])


if __name__ == '__main__':
    print ("Registering COM servers...")
    import win32com.server.register
    win32com.server.register.UseCommandLine(HelpFileDecompiler)

    helpFile=("C:\\Program Files\\Microsoft Office 15\\root\\vfs\\"
                "ProgramFilesCommonX86\\Microsoft Shared\\VBA\\VBA7.1\\"
                "1033\\VBLR6.chm")
    test = HelpFileDecompiler()
    test.DecompileHelpFileMain(helpFile)

Here is some test VBA client code

Option Explicit

Sub Test()

    Dim obj As Object
    Set obj = VBA.CreateObject("PythonInVBA.HelpFileDecompiler")
    
    obj.DecompileHelpFileMain "C:\Program Files\Microsoft Office 15\root\vfs\" & _
              "ProgramFilesCommonX86\Microsoft Shared\VBA\VBA7.1\" & _
              "1033\VBLR6.chm"

End Sub

Thursday, 25 January 2018

VBA - XMLHttp Request (XHR) does not parse response as XML

HTML was initially conceived to be like XML in that for every opening tag there is a closing tag and the attributes are enclosed in quotes but in reality it breaks these rules and can rarely be used with an Xml parser. So XML is fussy and HTML is not.

However, take a look at the following code; it uses the XmlHttp request (XHR) object but we should note that it never parses the response as Xml unless you write the code (example code given in separate function). I think this is nice use of XHR. The code goes on to insert the response text as Html into a MSHTML.HTMLDocument and from there can web scrape whatever.

Sub DoNotParseXml()

    Dim oXHR As MSXML2.XMLHTTP60
    Set oXHR = New MSXML2.XMLHTTP60
    
    Dim oHtmlDoc As MSHTML.HTMLDocument
    Set oHtmlDoc = New MSHTML.HTMLDocument
    
    oXHR.Open "GET", "https://coinmarketcap.com/all/views/all/" & "?Random=" & Rnd() * 100, False
    oXHR.setRequestHeader "Content-Type", "text/XML"
    oXHR.send

    If oXHR.Status = "200" Then
        
        '* no parse of 'non well-formed xml' take place
        oHtmlDoc.body.innerHTML = oXHR.responseText
    
        '** do some web scraping with MSHTML.HTMLDocument
    
        '... oHtmlDoc.getElementsByClassName("price")

        
        '* but if we had tried to parse the response text .. it would have errored
        ParseXml oXHR.responseText
    End If
End Sub

Private Function ParseXml(ByVal sText As String) As MSXML2.DOMDocument60
    Dim oDom As MSXML2.DOMDocument60
    Set oDom = New MSXML2.DOMDocument60
    oDom.LoadXML sText
    
    '* it would have errored
    Debug.Assert oDom.parseError = 0

End Function

VBA - Webscraping - jQuery selectors available with MSHTML's querySelector and querySelectorAll

So another SO question about web scraping in VBA. I wrote some code but was not happy with it and so revisited it. It turns out that the jQuery selector syntax can be quite advanced, a bit like XPath for Xml.

Tip #1 When using querySelectorAll() use Early-binding

I have seen some strangeness when using querySelectorAll() when getting the length, the problem goes away if you use early binding type library (Tools->References->Microsoft HTML Object Library)

    Dim oSelectors As MSHTML.IHTMLDOMChildrenCollection
    Set oSelectors = oHtml.querySelectorAll("div.blocoCampos input")
    
    Dim lSelectorResultList As Long
    lSelectorResultList = oSelectors.Length

Note above the selector gets input elements which are children of div elements with the class 'blocoCampos'.

Tip #2 When using querySelectorAll() use item to acquire each element not For Each

I have also seen some strangeness when using querySelectorAll() that errors on the Next line of a For Each loop. So avoid by establishing the length of the result array and then use a standard integer loop and acquire each element with item.

    Dim lSelectorResultLoop As Long
    For lSelectorResultLoop = 0 To lSelectorResultList - 1

        Dim objChild As Object
        Set objChild = oSelectors.Item(lSelectorResultLoop)

Selecting grandchild anchor off second span child of a div with id

Given the following HTML source the questioner wanted to navigate to the anchor links. The anchors do have not id and no class; neither do their parent span elements; but the spans' parent div element have (non-unique) id so we can start the capture there.


<div id="resumopesquisa">

  <div id="itemlistaresultados" style="background-color: #EDEDED">
   <span class="labellinha">Acórdãos de Repetitivos</span>
   <!-- <span>  PIS E ICMS E COFINS E CALCULO E BASE E DE REPETITIVOS.NOTA.
   </span> -->
   
   <span><a href="/SCON/jurisprudencia/toc.jsp?livre=ICMS+BASE+DE+CALCULO+PIS+COFINS&repetitivos=REPETITIVOS&&b=ACOR&thesaurus=JURIDICO&p=true">1
     documento(s) encontrado(s)</a></span>
   
  </div>
  
 
 <div id="itemlistaresultados">
  <span class="labellinha">Acórdãos</span>
  <!-- <span>  PIS E ICMS E COFINS E CALCULO E BASE E DE
  </span> -->
  
  <span><a href="/SCON/jurisprudencia/toc.jsp?livre=icms+base+de+calculo+pis+cofins&&b=ACOR&thesaurus=JURIDICO&p=true">284
    documento(s) encontrado(s)</a></span>
  
 </div>
</div>

So let's build up our jQuery selector expression, first let's get the divs but specifiying their id ( yeah, I know I though ids were unique as well) ...

div#itemlistaresultados

But then we need to get the second child span element of the div, we can do this with jQuery's nth-child selector. We simply add a space between the div expression and the span expression to express the parent child relationship ...

div#itemlistaresultados span:nth-child(2)

Finally we pick out the anchor element with

div#itemlistaresultados span:nth-child(2) a

So we can put this jQuery selector expression into MSHTML's querySelectorAll method (use querySelector for singleton results), here is the VBA

    Set oHtml = ie.Document
    Dim objResultList As MSHTML.IHTMLDOMChildrenCollection
    Set objResultList = oHtml.querySelectorAll("div#itemlistaresultados span:nth-child(2) a")

    Dim lResultCount As Long
    lResultCount = objResultList.Length

    Debug.Print
    Dim lResultLoop As Long
    For lResultLoop = 0 To lResultCount - 1

        Dim anchorLoop As MSHTML.HTMLAnchorElement
        Set anchorLoop = objResultList.Item(lResultLoop)

        Debug.Print achLoop.href

    Next

Tip #3 When not required use late binding to get aggregated interface

So when dealing with an input checkbox then it must be understood that its functionality is defined across a great many number of different interfaces such as MSHTML.HTMLInputElement, MSHTML.IHTMLInputElement and many more. Perhaps the input box is a worst case example because it is a multifaceted definition but for illustration here is what OLEView gives the interfaces implemented by the coclass MSHTML.HTMLInputElement ...

    coclass HTMLInputElement {
        [default] dispinterface DispHTMLInputElement;
        [default, source] dispinterface HTMLInputTextElementEvents;
        [source] dispinterface HTMLInputTextElementEvents2;
        [source] dispinterface HTMLOptionButtonElementEvents;
        [source] dispinterface HTMLButtonElementEvents;
        interface IHTMLElement;
        interface IHTMLElement2;
        interface IHTMLElement3;
        interface IHTMLElement4;
        interface IHTMLUniqueName;
        interface IHTMLDOMNode;
        interface IHTMLDOMNode2;
        interface IHTMLDOMNode3;
        interface IHTMLDatabinding;
        interface IHTMLElement5;
        interface IHTMLElement6;
        interface IElementSelector;
        interface IHTMLDOMConstructor;
        interface IHTMLElement7;
        interface IHTMLControlElement;
        interface IHTMLInputElement;
        interface IHTMLInputElement2;
        interface IHTMLInputTextElement;
        interface IHTMLInputTextElement2;
        interface IHTMLInputHiddenElement;
        interface IHTMLInputButtonElement;
        interface IHTMLInputFileElement;
        interface IHTMLOptionButtonElement;
        interface IHTMLInputImage;
        interface IHTMLInputElement3;
        interface IHTMLInputRangeElement;
    };

So instead of figuring out on which interface of the list above a method is implemented it is better to declare the variable with As Object to use late binding, and then all the methods from all of the interfaces are aggregated onto a IDispatch interface.

Links

Monday, 22 January 2018

VBA - Excel table to HTML

Just a little routine to help me write these articles. Given an Excel range it will generate the HTML for a table with some subtle (i.e grey) styling.

Option Explicit

Function MarkupTable(ByVal rng As Excel.Range)

    Dim s As String
    s = "<table style='border: 1px solid lightgrey;'>"
    
    Dim rngRowLoop As Excel.Range
    For Each rngRowLoop In rng.Rows
        s = s & "<tr>"
    
        Dim rngCellLoop As Excel.Range
        For Each rngCellLoop In rngRowLoop.Cells
        
            s = s & "<td style='border: 1px solid lightgrey;'>" & rngCellLoop.Value2 & "</td>"
        
        Next rngCellLoop
    
    
        s = s & "</tr>"
    Next
    

    s = s & "</table>"

    MarkupTable = s

End Function

Sub TestMarkupTable()
    Debug.Print MarkupTable(ActiveCell.CurrentRegion)
End Sub

sample

0018095CIAccessibleStatus BarMsoCommandBarStatus Bar
003400F0IAccessibleRibbonMsoCommandBarRibbon
00130E1AITextDocument2CalibriRICHEDIT60WFont Selector?
00140D90ITextDocument211RICHEDIT60WFont size selector?
000E0F54ITextDocument2GeneralRICHEDIT60WFormat selector?
000E034CWindowBook2EXCEL7TrueA window on a workbook

Wednesday, 17 January 2018

VBA - WebScraping - firing an HTML button's event from VBA with Click, FireEvent and Window.ExecScript

So, many questions on the StackOverflow VBA thread are concerned with web-scraping and so driving the browser in VBA code is a useful skill. Here we're see that we can push the boundary a little more by firing events in the HTML object model.

Code To Write HTML

Here we write some code to write the html file locally. The code uses the HtmlElementStack class (given separately below) to ensure our HTML is well-formed. There is a script tag which defines the function wired to the button's click handler. But we shall see that because the function is defined in the HTML page's global scope, i.e. Window object, then it can be called with Window.execScript


Private Const msFILENAME As String = "N:\TestJavascript2.html"

Function WriteHTML()

    Dim dicHTMLStack As HtmlElementStack
    Set dicHTMLStack = New HtmlElementStack

    With dicHTMLStack
        .SetFileName msFILENAME 
        
        .OE "html"
        .OE "head"
        .OE "title"
        .WL "Some test html with javascript"
        .CE
        .CE
        .OE "body"
        .OE "div", "id='div1'"
        .WL "Some Text"
        .CE
        .OE "button", "id='button1' type='button' onclick='throwMsgBox()'"
        .WL "Click Me!"
        .CE
        .OE "script", "language='jscript'"
        .WL "function throwMsgBox() { alert('hi there'); }"
        .CE
    End With
    
    Set dicHTMLStack = Nothing

End Function


HtmlElementStack class

This class allows us to be a little lazy when writing html files. We keep a stack, ie. Last In Last Out (LIFO) structure in a Scripting.Dictionary, that records all the open elements that need closing. It also manages its own text stream because we need to write off all pending close elements before the text stream is closed.


Option Explicit

'* Tools->References
' Scripting            Microsoft Scripting Runtime     C:\Windows\SysWOW64\scrrun.dll


Private mdicStack As New Scripting.Dictionary

Private mtxt As Scripting.TextStream
Private msFILENAME As String
Private mfso As New Scripting.FileSystemObject

'Private Sub SetStream(ByVal txt As Scripting.TextStream)
'    Set mtxt = txt
'End Sub

Public Sub SetFileName(ByVal sFileName As String)
    msFILENAME = sFileName
    Set mtxt = mfso.CreateTextFile(msFILENAME)
End Sub

Public Sub Write_(ByVal sText As String)
    mtxt.Write sText
End Sub

Public Sub WL(ByVal sText As String)
    mtxt.WriteLine sText
End Sub

Public Sub OE(ByVal sNodeName As String, Optional ByVal sAttribs As String)
    
    If Not mtxt Is Nothing Then
        If Len(sAttribs) = 0 Then
            mtxt.WriteLine "<" & sNodeName & ">"
        Else
            mtxt.WriteLine "<" & sNodeName & " " & sAttribs & ">"
        End If
    
        
    End If

    mdicStack.Add mdicStack.Count, sNodeName
    

End Sub

Public Sub CE()

    If mdicStack.Count > 0 Then
        Dim sLastNode As String
        sLastNode = mdicStack.Item(mdicStack.Count - 1)
        
        Call mdicStack.Remove(mdicStack.Count - 1)
    
        mtxt.WriteLine ""
    End If

End Sub

Private Sub Class_Terminate()

    While mdicStack.Count > 0
        DoEvents
        CE
        DoEvents
    Wend
    
    mtxt.Close

    Set mtxt = Nothing
End Sub


Code to drive IE and call the click handler function 3 different ways

So in this code we create an instance of IE and navigate to our newly written html file. We call the button's click handler function 3 different ways. Firstly, by navigating to element and call 'Click'. Secondly, by calling the function in the global scope (i.e. off the window object) using ExecScript. Thirdly, similar to first but a looser couple FireEvent method.

I recommend acquiring the element immediately before calling a method because I have witnessed a type of stale reference bug.


Private Const msFILENAME As String = "N:\TestJavascript2.html"

Public Sub TestFire()
    
'* Tools->References
'SHDocVw    Microsoft Internet Controls C:\Windows\SysWOW64\ieframe.dll
    
    Dim oIE As InternetExplorerMedium
    Set oIE = New InternetExplorerMedium
    
    oIE.Visible = True
    oIE.navigate msFILENAME
    While oIE.Busy Or oIE.readyState < 4
        DoEvents
    Wend
    
    
    Stop
    '* recommend re-acquiring element before using as I suspect IE suffers from stale references
    oIE.Document.getElementById("button1").Click
    
    Stop

    '* call the function via the global scope, for html global scope is the *window*
    Call oIE.Document.parentWindow.execScript("throwMsgBox()", "JavaScript")

    Stop
    
    '* recommend re-acquiring element before using as I suspect IE suffers from stale references
    oIE.Document.getElementById("button1").FireEvent "onclick"
    
    Stop
    
    
    oIE.Quit
End Sub


Links

Friday, 1 December 2017

Use MSHTML to parse local HTML file without using Internet Explorer (Microsoft HTML Object Library)

So an excellent question came up today on StackOverflow about the parsing of HTML in VBA for when Internet Explorer is unavailable.

Anyone who has done some web scraping will be familiar with creating an instance of Internet Explorer (IE) and the navigating to a web address and then once the page is ready start navigating the DOM using the 'Microsoft HTML Object Library' (MSHTML) type library. The question asks if IE is unavailable what to do. I am in the same situation for my box running Windows 10.

I had suspected it was possible to spin up an instance of MSHTML.HTMLDocument but its creation is not obvious. Thanks to the questioner for asking this now. The answer lies in the MSHTML.IHTMLDocument4.createDocumentFromUrl method. One needs a local file to work with (EDIT: actually one can put a webby url in as well!) but we have a nice tidy Windows API function called URLDownloadToFile to download a file.

This codes runs on my Windows 10 box where Microsoft Edge is running and not Internet Explorer. This is an important find and thanks to the questioner for raising it.


Option Explicit

'* Tools->Refernces Microsoft HTML Object Library


'* MSDN - URLDownloadToFile function - https://msdn.microsoft.com/en-us/library/ms775123(v=vs.85).aspx
Private Declare PtrSafe Function URLDownloadToFile Lib "urlmon" Alias "URLDownloadToFileA" _
        (ByVal pCaller As Long, ByVal szURL As String, ByVal szFileName As String, _
        ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long

Sub Test()

    Dim fso As Object
    Set fso = CreateObject("Scripting.FileSystemObject")

    Dim sLocalFilename As String
    sLocalFilename = Environ$("TMP") & "\urlmon.html"
    
    Dim sURL As String
    sURL = "https://stackoverflow.com/users/3607273/s-meaden"
    
    
    Dim bOk As Boolean
    bOk = (URLDownloadToFile(0, sURL, sLocalFilename, 0, 0) = 0)
    If bOk Then
        If fso.FileExists(sLocalFilename) Then
        
            '* Tools->References Microsoft HTML Object Library
            Dim oHtml4 As MSHTML.IHTMLDocument4
            Set oHtml4 = New MSHTML.HTMLDocument
            
            Dim oHtml As MSHTML.HTMLDocument
            Set oHtml = Nothing
            
            '* IHTMLDocument4.createDocumentFromUrl
            '* MSDN - IHTMLDocument4 createDocumentFromUrl method - https://msdn.microsoft.com/en-us/library/aa752523(v=vs.85).aspx
            Set oHtml = oHtml4.createDocumentFromUrl(sLocalFilename, "")
            
            '* need to wait a little whilst the document parses
            '* because it is multithreaded
            While oHtml.readyState <> "complete"
                DoEvents  '* do not comment this out it is required to break into the code if in infinite loop
            Wend
            Debug.Assert oHtml.readyState = "complete"
            

            Dim sTest As String
            sTest = Left$(oHtml.body.outerHTML, 100)
            Debug.Assert Len(Trim(sTest)) > 50  '* just testing we got a substantial block of text, feel free to delete
            
            '* this is where the page specific logic now goes, here I am getting info from a StackOverflow page
            Dim htmlAnswers As Object 'MSHTML.DispHTMLElementCollection
            Set htmlAnswers = oHtml.getElementsByClassName("answer-hyperlink")
    
            Dim lAnswerLoop As Long
            For lAnswerLoop = 0 To htmlAnswers.Length - 1
                Dim vAnswerLoop
                Set vAnswerLoop = htmlAnswers.Item(lAnswerLoop)
                Debug.Print vAnswerLoop.outerText
            
            Next
    
        End If
    End If
End Sub