The Excel Development Platform: Video

Monday, 30 September 2019

Video - Parsing an MP4 file

Video files, specifically MP4 files, have a curious head of encoded data at the start of the file which can be seen in a hex editor or even Notepad. Below is a screenshot of a hex editor of just such a header.

(In actual fact this is a DASH fragmented mp4 initialization vector IV file, i.e. the first file in a long stream. So is without any encoded media data but it serves as an example)

I have written some VBA code to load this file and parse it out into what are known as ATOMS. I have placed this code into Appendix A because actually I found better source code in the form of the VLC Media Player code. VLC handles all manner of mp4 files and so is a infinitely better bet.

MP4 Files are based on Apple's QuickTime Movie files

During my reaearch, a fantastic discovery was a treasure trove of documentation regarding Apple's QuickTime File Format Specification. Below is a beautiful diagram showing the nested nature of the data structure. Clicking on the image will take you Apple's website. But a word of warning, whilst MP4 is based on QuickTime it may diverge at some points. In fact there is a warning here which says...

Important: The QuickTime File Format has been used as the basis of the MPEG-4 standard and the JPEG-2000 standard, developed by the International Organization for Standardization (ISO). Although these file types have similar structures and contain many functionally identical elements, they are distinct file types.

Warning: Do not use this specification to interpret a file that conforms to a different specification, however similar.

Despite this Apple's documentation is free and well-written and definitely serves as a starting point.

MPEG Specificaation ISO/IEC 14496 parts 12 & 14

There is an official specification published by International Standards Organisation (ISO). In the document ISO/IEC 14496-12 there is a very good table which also shows the nested box/atom structure. It is reproduced below.

ftyp						*	4.3	file type and compatibility
pdin							8.43	progressive download information
moov						*	8.1	container for all the metadata
	mvhd					*	8.3	movie header, overall declarations
	trak					*	8.4	container for an individual track or stream
		tkhd				*	8.5	track header, overall information about the track
		tref					8.6	track reference container
		edts					8.25	edit list container
			elst				8.26	an edit list
		mdia				*	8.7	container for the media information in a track
			mdhd			*	8.8	media header, overall information about the media
			hdlr			*	8.9	handler, declares the media (handler) type
			minf			*	8.10	media information container
				vmhd			8.11.2	video media header, overall information (video track only)
				smhd			8.11.3	sound media header, overall information (sound track only)
				hmhd			8.11.4	hint media header, overall information (hint track only)
				nmhd			8.11.5	Null media header, overall information (some tracks only)
				dinf		*	8.12	data information box, container
					dref	*	8.13	data reference box, declares source(s) of media data in track
				stbl		*	8.14	sample table box, container for the time/space map
					stsd	*	8.16	sample descriptions (codec types, initialization etc.)
					stts	*	8.15.2	(decoding) time-to-sample
					ctts		8.15.3	(composition) time to sample
					stsc	*	8.18	sample-to-chunk, partial data-offset information
					stsz		8.17.2	sample sizes (framing)
					stz2		2 8.17.3	compact sample sizes (framing)
					stco	*	8.19	chunk offset, partial data-offset information
					co64		8.19	64 -bit chunk offset
					stss		8.20	sync sample table (random access points)
					stsh		8.21	shadow sync sample table
					padb		8.23	sample padding bits
					stdp		8.22	sample degradation priority
					sdtp		8.40.2	independent and disposable samples
					sbgp		8.40.3.2	sample-to-group
					sgpd		8.40.3.3	sample group description
					subs		8.42	sub-sample information
	mvex						8.29	movie extends box
		mehd					8.30	movie extends header box
		trex				*	8.31	track extends defaults
	ipmc						8.45.4	IPMP Control Box
moof							8.32	movie fragment
	mfhd					*	8.33	movie fragment header
	traf						8.34	track fragment
		tfhd				*	8.35	track fragment header
		trun					8.36	track fragment run
		sdtp					8.40.2	independent and disposable samples
		sbgp					8.40.3.2	sample-to-group
		subs					8.42	sub-sample information
mfra							8.37	movie fragment random access
	tfra						8.38	track fragment random access
	mfro					*	8.39	movie fragment random access offset
mdat							8.2	media data container
free							8.24	free space
skip							8.24	free space
	udta						8.27	user-data
		cprt					8.28	copyright etc.
meta							8.44.1	metadata
	hdlr					*	8.9	handler, declares the metadata (handler) type
	dinf						8.12	data information box, container
		dref					8.13	data reference box, declares source(s) of metadata items
	ipmc						8.45.4	IPMP Control Box
	iloc						8.44.3	item location
	ipro						8.44.5	item protection
		sinf					8.45.1	protection scheme information box
			frma				8.45.2	original format box
			imif				8.45.3	IPMP Information box
			schm				8.45.5	scheme type box
			schi				8.45.6	scheme information box
	iinf						8.44.6	item information
	xml						8.44.2	XML container
	bxml						8.44.2	binary XML container
	pitm						8.44.4	primary item reference

VLC Media Player Source Code

The Github page for VLC Media Player is where to find the source code. The atoms can be seen to be defined in libmp4.h, here is an excerpt ...

#define ATOM_ftyp VLC_FOURCC( 'f', 't', 'y', 'p' )
#define ATOM_moov VLC_FOURCC( 'm', 'o', 'o', 'v' )

Also defined in the libmp4.h header file are the C structs ...

typedef struct MP4_Box_data_ftyp_s
{
    uint32_t i_major_brand;
    uint32_t i_minor_version;

    uint32_t i_compatible_brands_count;
    uint32_t *i_compatible_brands;

} MP4_Box_data_ftyp_t;

You can compare the above C code with my VBA code in Appendix A. In the vlc_libmp4.c implementation file you can see the actual code to populate the data structures. So for example, the following code populates the data structure given above.

static int MP4_ReadBox_ftyp( stream_t *p_stream, MP4_Box_t *p_box )
{
    MP4_READBOX_ENTER( MP4_Box_data_ftyp_t, MP4_FreeBox_ftyp );

    MP4_GETFOURCC( p_box->data.p_ftyp->i_major_brand );
    MP4_GET4BYTES( p_box->data.p_ftyp->i_minor_version );

    p_box->data.p_ftyp->i_compatible_brands_count = i_read / 4;
    if( p_box->data.p_ftyp->i_compatible_brands_count > 0 )
    {
        uint32_t *tab = p_box->data.p_ftyp->i_compatible_brands =
            vlc_alloc( p_box->data.p_ftyp->i_compatible_brands_count,
                       sizeof(uint32_t) );

        if( unlikely( tab == NULL ) )
            MP4_READBOX_EXIT( 0 );

        for( unsigned i = 0; i < p_box->data.p_ftyp->i_compatible_brands_count; i++ )
        {
            MP4_GETFOURCC( tab[i] );
        }
    }
    else
    {
        p_box->data.p_ftyp->i_compatible_brands = NULL;
    }

    MP4_READBOX_EXIT( 1 );
}

Use Unix Tool Grep in Windows

I'm very impressed with Windows Subsystem for Linux. I used Grep to search the VLC Media Player source code. I did this by pressing SHIFT and right mouse button clicking in a Windows Explorer window and taking the context menu option 'Open Linux shell here' and type in the following ("hdlr" was my search term, substitute for yours) ...

grep -rl hdlr

So Grep is a fantastic tool which is available to Windows 10 users. There are plenty of other great Unix tools as well, looking forward to using them for my everyday Windows tasks.

Appendix A - VBA Code to read DASH IV

The listing below is my admittedly buggy and work-in-progress code for parsing a DASH IV file. I'll confess that in places I have triaged the code to skip some bytes here and there on a 'fake it 'til you make it' basis. Given its bugs and triages this is definitely not production ready so absolutely no warranty. (As a matter of fact, all code on this blog carries no warranty and you use at your own risk). But writing this code forced me to research the problem and led me to discover the VLC Media Player source code. I give the code below for illustrative purposes.

For better source code one should look at the VLC Media Player source, detailed above.

Anyway at least one can get a handle on the nested nature on the data structure. Here is a screenshot of the locals window after the parsing is complete, you can see the nested atoms.

Option Explicit

Private fso As New Scripting.FileSystemObject

Private Type udtFtyp
    lSize As Long
    type As String * 4
    major_brand As String
    minor_version As Long
    compatible_brand As String
    compatible_brand2 As String
End Type

Private Type udtFree
    size As Long
    type As String * 4
    text As String
End Type

Private Type udtStco
    '* Chunk Offset Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCHAEEA
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    numberofentries As Long
End Type

Private Type udtStsz
    '* Sample Size Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCBBCGB
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    numberofentries As Long
End Type

Private Type udtStsc
    '* Sample-to-Chunk Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCIDAFD
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    numberofentries As Long
End Type

Private Type udtStsdSampleDescriptions
    size As Long
    dataformat As Long
    reserved(0 To 5)
    datareferenceindex As Integer
    additionaldata() As Byte
End Type

Private Type udtStsd
    '* Sample Description Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCHHGBH
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    numberofentries As Long
    sampleDescriptions() As udtStsdSampleDescriptions
End Type

Private Type udtStts
    '* Time-to-Sample Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCGFJII
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    numberofentries As Long
End Type

Private Type udtStbl
    '* Sample Table Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCBFDFF
    type As String * 4
    stsd As udtStsd
    stts As udtStts
    stsc As udtStsc
    stsz As udtStsz
    stco As udtStco
End Type

Private Type udtDrefDetails
    size As Long
    type As String * 4
    version As Byte
    flags As Long ' 3 bytes
    data() As Byte
End Type

Private Type udtDref
    '* Data Reference Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCGGDAE
    size As Long
    type As String * 4
    version As Byte
    flags As Long '3 bytes
    numberofentries As Long
    details() As udtDrefDetails
End Type

Private Type udtDinf
    '* Data Information Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCIFAIC
    size As Long
    type As String * 4
    dref As udtDref
End Type

Private Type udtSmhd
    '* Sound Media Information Header Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCHEIJG
    size As Long
    type As String * 4
    version As Byte
    flags As Long '3 bytes
    balance As Integer
    reserved As Integer
End Type

Private Type udtMinf
    '* Media Information Atoms
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-25647
    type As String * 4
    smhd As udtSmhd
    dinf As udtDinf
    stbl As udtStbl
End Type

Private Type udtSoun
    '* Sound Data (Media Data Atom Type)
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCIBHFD
    size As Long
    type As String * 4
End Type

Private Type udtHdlr
    '* handler reference ('hdlr') atom (Media Atom)
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-DontLinkElementID_147
    size As Long
    type As String * 4
    version As Byte
    flags As Long '3 bytes
    componenttype As String * 4
    componentsubtype As String * 4
    componentmanufacturer As Long
    componentflags As Long
    componentflagsmask As Long
    componentname As String
End Type

Private Type udtMdhd
    '* Media Header Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-SW34
    size As Long
    type As String * 4
End Type

Private Type udtMdia
    '* Media Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCHFJFA
    type As String * 4
    mdhd As udtMdhd
    hdlr As udtHdlr
    soun As udtSoun
    minf As udtMinf
End Type

Private Type udtTkhd
    '* Track Header Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCEIDFA
    size As Long
    type As String * 4
    version As Byte
    flags As Long '3 bytes
    creationtime As Long
    modificationtime As Long
    trackid As Long
    reserved0 As Long
    duration As Long
    reserved1(0 To 1) As Long
    layer As Integer
    alternationgroup As Integer
    volume As Integer
    Reserved2 As Integer
    matrixstructure(0 To 35) As Byte
    trackwidth As Long
    trackheight As Long
End Type

Private Type udtTrak
    '* Track Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCBEAIF
    size As Long
    type As String * 4
    tkhd As udtTkhd
    mdia As udtMdia
End Type

Private Type udtMvhd
    '* Movie Header Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCGFGJG
    size As Long
    type As String * 4
    version As Long
    flags As Long
    creationtime As Long
    modificationtime As Long
    timescale As Long
    duration As Long
    preferredrate As Long
    preferredvolume As Long
    reserved As String
    matrix As String
    previewtime As Long
    previewduration As Long
    postertime As Long
    selectiontime As Long
    selectionduration As Long
    currenttime As Long
    nexttrackid As Long
End Type

Private Type udtMoov
    '* Movie Atom
    '* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-55911
    size As Long
    type As String * 4
    mvhd As udtMvhd
    trak As udtTrak
End Type

Private Type udtWholeFile
    ftyp As udtFtyp
    free As udtFree
    moov As udtMoov
End Type

Dim wholefile As udtWholeFile

Private Sub Start()
    
    Dim sFileName As String
    
    sFileName = TEST_FILENAME

    Dim bytes() As Byte
    bytes() = ReadByteFile(sFileName)

    Dim idx As Long
    idx = Read_ftyp(bytes(), 0)
    idx = Read_free(bytes(), idx)
    idx = Read_moov(bytes(), idx)

End Sub

Private Function Read_ftyp(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "ftyp"

    Dim sBuffer As String
    sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.ftyp.lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    lPosition = lPosition + 8
    wholefile.ftyp.major_brand = BytesToString(bytes(), 4, lPosition)
    wholefile.ftyp.minor_version = BytesToLong(bytes(), 4, lPosition)
    wholefile.ftyp.compatible_brand = BytesToString(bytes(), 4, lPosition)
    wholefile.ftyp.compatible_brand2 = BytesToString(bytes(), 4, lPosition)


    Debug.Print "[ftyp] size=8+" & CStr(wholefile.ftyp.lSize - 8)
    Debug.Print "  major_brand = " & wholefile.ftyp.major_brand
    Debug.Print "  minor_version = " & wholefile.ftyp.minor_version
    Debug.Print "  compatible_brand = " & wholefile.ftyp.compatible_brand
    Debug.Print "  compatible_brand = " & wholefile.ftyp.compatible_brand2

    Read_ftyp = lPosition ' + wholefile.ftyp.lSize
End Function

Private Function Read_free(ByRef bytes() As Byte, ByVal lPosition As Long)
    Dim sBuffer As String
    sBuffer = BytesToString(bytes, 50, (lPosition))
    
    
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "free"
    wholefile.free.size = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    lPosition = lPosition + 8
    wholefile.free.text = BytesToString(bytes(), wholefile.free.size - 8, lPosition)

    Debug.Print "[free] size=8+" & CStr(wholefile.free.size - 8)
    Debug.Print "  text=" & wholefile.free.text

    Read_free = lPosition '+ wholefile.free.size
End Function

Private Function Read_moov(ByRef bytes() As Byte, ByVal lPosition As Long)
    
    Dim sBuffer As String
    sBuffer = BytesToString(bytes, 50, (lPosition))
    
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "moov"

    wholefile.moov.size = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    wholefile.moov.type = "moov"
    lPosition = lPosition + 8

    Debug.Print "[moov] size=8+" & CStr(wholefile.moov.size - 8)

    lPosition = Read_mvhd(bytes(), lPosition)   'mvhd element starts 8 bytes in, i.e. it is first child
    lPosition = Read_trak(bytes(), lPosition)

    Read_moov = lPosition
End Function



Private Function Read_mvhd(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "mvhd"
    
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    Dim idx As Long
    idx = lPosition
    
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, lSize + 12, (idx))

    wholefile.moov.mvhd.size = lSize

    Debug.Print "[mvhd] size=8+" & CStr(lSize - 8)

    'https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-BBCGFGJG

    idx = lPosition + 8
    wholefile.moov.mvhd.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.mvhd.flags = BytesToLong(bytes, 3, idx)
    wholefile.moov.mvhd.creationtime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.modificationtime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.timescale = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.duration = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.preferredrate = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.preferredvolume = BytesToLong(bytes, 2, idx)
    wholefile.moov.mvhd.reserved = BytesToString(bytes, 10, idx)
    wholefile.moov.mvhd.matrix = BytesToString(bytes, 36, idx)
    wholefile.moov.mvhd.previewtime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.previewduration = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.postertime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.selectiontime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.selectionduration = BytesToLong(bytes, 4, idx) '94
    wholefile.moov.mvhd.currenttime = BytesToLong(bytes, 4, idx)
    wholefile.moov.mvhd.nexttrackid = BytesToLong(bytes, 4, idx)

    Debug.Print "  version = " & wholefile.moov.mvhd.version
    Debug.Print "  flags = " & wholefile.moov.mvhd.flags
    Debug.Print "  creationtime = " & wholefile.moov.mvhd.creationtime
    Debug.Print "  modificationtime = " & wholefile.moov.mvhd.modificationtime
    Debug.Print "  timescale = " & wholefile.moov.mvhd.timescale
    Debug.Print "  duration = " & wholefile.moov.mvhd.duration
    Debug.Print "  preferredrate = " & wholefile.moov.mvhd.preferredrate
    Debug.Print "  preferredvolume = " & wholefile.moov.mvhd.preferredvolume
    Debug.Print "  reserved = " & wholefile.moov.mvhd.reserved
    Debug.Print "  matrix = " & wholefile.moov.mvhd.matrix
    Debug.Print "  previewtime = " & wholefile.moov.mvhd.previewtime
    Debug.Print "  previewduration = " & wholefile.moov.mvhd.previewduration
    Debug.Print "  postertime = " & wholefile.moov.mvhd.postertime
    Debug.Print "  selectiontime = " & wholefile.moov.mvhd.selectiontime
    Debug.Print "  selectionduration = " & wholefile.moov.mvhd.selectionduration
    Debug.Print "  currenttime = " & wholefile.moov.mvhd.currenttime
    Debug.Print "  nexttrackid = " & wholefile.moov.mvhd.nexttrackid
    'mvhd is a leaf atom and has no children

    Read_mvhd = lPosition + lSize
End Function

Private Function Read_trak(ByRef bytes() As Byte, ByVal lPosition As Long)
    lPosition = lPosition + 2  '* shim, fake it till we make it
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "trak"
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    
    wholefile.moov.trak.type = "trak"
    wholefile.moov.trak.size = lSize
    lPosition = lPosition + 8
    
    Debug.Print "[trak] size=8+" & CStr(lSize - 8)

    lPosition = Read_tkhd(bytes(), lPosition)   'tkhd element starts 8 bytes in, i.e. it is first child
    lPosition = Read_mdia(bytes(), lPosition)
    
    Read_trak = lPosition
End Function

Private Function Read_mdia(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "mdia"
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    wholefile.moov.trak.mdia.type = "mdia"
    
    lPosition = Read_mdhd(bytes(), lPosition + 8) 'mdhd element starts 8 bytes in, i.e. it is first child
    lPosition = Read_mdia_hdlr(bytes(), lPosition)
    lPosition = Read_minf(bytes(), lPosition)
    
    Read_mdia = lPosition
End Function

Private Function Read_mdhd(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "mdhd"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    wholefile.moov.trak.mdia.mdhd.size = lSize
    wholefile.moov.trak.mdia.mdhd.type = "mdhd"
    
    Read_mdhd = lPosition + lSize
End Function

Private Function Read_minf(ByRef bytes() As Byte, ByVal lPosition As Long)
    lPosition = lPosition + 6
    Debug.Assert Chr$(bytes(lPosition + 0)) + Chr$(bytes(lPosition + 1)) + Chr$(bytes(lPosition + 2)) + Chr$(bytes(lPosition + 3)) = "minf"
    
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.type = "minf"
    '*https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-25647
    lPosition = Read_smhd(bytes, lPosition + 4)
    lPosition = Read_dinf(bytes, lPosition)
    
    Read_minf = lPosition + 44
End Function

Private Function Read_dinf(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "dinf"
    
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))
    
    wholefile.moov.trak.mdia.minf.dinf.size = lSize
    wholefile.moov.trak.mdia.minf.dinf.type = "dinf"
    
    lPosition = Read_dref(bytes, lPosition + 8)
    lPosition = Read_stbl(bytes, lPosition)
    
    Read_dinf = lPosition + lSize
End Function

Private Function Read_stbl(ByRef bytes() As Byte, ByVal lPosition As Long)
    lPosition = lPosition + 6
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))
    
    Debug.Assert Chr$(bytes(lPosition + 0)) + Chr$(bytes(lPosition + 1)) + Chr$(bytes(lPosition + 2)) + Chr$(bytes(lPosition + 3)) = "stbl"
    lPosition = Read_stsd(bytes(), lPosition + 4)
    lPosition = Read_stts(bytes(), lPosition)
    lPosition = Read_stsc(bytes(), lPosition)
    lPosition = Read_stsz(bytes(), lPosition)
    lPosition = Read_stco(bytes(), lPosition)

    Read_stbl = lPosition
End Function

Private Function Read_stsd(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "stsd"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.stbl.stsd.type = "stsd"
    wholefile.moov.trak.mdia.minf.stbl.stsd.size = lSize

    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.stbl.stsd.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.stbl.stsd.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.stbl.stsd.numberofentries = lEntries

    If lEntries > 0 Then
        ReDim wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(0 To lEntries - 1)
        Dim lLoop As Long
        For lLoop = 0 To lEntries - 1
            Dim lDetailsSize As Long
            lDetailsSize = 16777216# * bytes(idx) + 65536# * bytes(idx + 1) + 256# * bytes(idx + 2) + bytes(idx + 3)
            
            wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(lLoop).size = lDetailsSize
            idx = idx + 4
            wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(lLoop).dataformat = BytesToLong(bytes, 4, idx)
            
            idx = idx + 6 ' reserved 6 bytes
            wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(lLoop).datareferenceindex = BytesToLong(bytes, 2, idx)
            
            Dim lDataSize As Long
            lDataSize = lDetailsSize - 24
            If lDataSize > 0 Then
                ReDim wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(lLoop).additionaldata(0 To lDataSize - 1)
                CopyBytes bytes(), wholefile.moov.trak.mdia.minf.stbl.stsd.sampleDescriptions(lLoop).additionaldata, idx, lDataSize
            End If
            idx = idx + lDataSize
        Next lLoop
    End If

    Read_stsd = lPosition + lSize
End Function

Private Function Read_stts(ByRef bytes() As Byte, ByVal lPosition As Long)
    lPosition = lPosition + 9 '* fake it till we make it
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "stts"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.stbl.stts.type = "stts"
    wholefile.moov.trak.mdia.minf.stbl.stts.size = lSize

    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.stbl.stts.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.stbl.stts.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.stbl.stts.numberofentries = lEntries

    Read_stts = lPosition + lSize
End Function

Private Function Read_stsc(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "stsc"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)


    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.stbl.stsc.type = "stsc"
    wholefile.moov.trak.mdia.minf.stbl.stsc.size = lSize

    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.stbl.stsc.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.stbl.stsc.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.stbl.stsc.numberofentries = lEntries


    Read_stsc = lPosition + lSize
End Function

Private Function Read_stsz(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "stsz"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.stbl.stsz.type = "stsz"
    wholefile.moov.trak.mdia.minf.stbl.stsz.size = lSize

    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.stbl.stsz.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.stbl.stsz.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.stbl.stsz.numberofentries = lEntries

    Read_stsz = lPosition + lSize
End Function

Private Function Read_stco(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "stco"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))

    wholefile.moov.trak.mdia.minf.stbl.stco.type = "stco"
    wholefile.moov.trak.mdia.minf.stbl.stco.size = lSize

    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.stbl.stco.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.stbl.stco.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.stbl.stco.numberofentries = lEntries
    
    Read_stco = lPosition + lSize
End Function

Private Function Read_dref(ByRef bytes() As Byte, ByVal lPosition As Long)
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "dref"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))
    
    wholefile.moov.trak.mdia.minf.dinf.dref.size = lSize
    wholefile.moov.trak.mdia.minf.dinf.dref.type = "dref"
    
    Dim idx As Long
    idx = lPosition + 8
    wholefile.moov.trak.mdia.minf.dinf.dref.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.dinf.dref.flags = BytesToLong(bytes, 3, idx)
    Dim lEntries As Long
    lEntries = BytesToLong(bytes, 4, idx)
    
    wholefile.moov.trak.mdia.minf.dinf.dref.numberofentries = lEntries
    If lEntries > 0 Then
        ReDim wholefile.moov.trak.mdia.minf.dinf.dref.details(0 To lEntries - 1)
        Dim lLoop As Long
        For lLoop = 0 To lEntries - 1
            Dim lDetailsSize As Long
            lDetailsSize = 16777216# * bytes(idx) + 65536# * bytes(idx + 1) + 256# * bytes(idx + 2) + bytes(idx + 3)
            
            wholefile.moov.trak.mdia.minf.dinf.dref.details(lLoop).size = lDetailsSize
            wholefile.moov.trak.mdia.minf.dinf.dref.details(lLoop).type = _
                    Chr$(bytes(idx + 4)) + Chr$(bytes(idx + 5)) + Chr$(bytes(idx + 6)) + Chr$(bytes(idx + 7))
            idx = idx + 8
            wholefile.moov.trak.mdia.minf.dinf.dref.details(lLoop).version = BytesToLong(bytes, 1, idx)
            wholefile.moov.trak.mdia.minf.dinf.dref.details(lLoop).flags = BytesToLong(bytes, 3, idx)
            
            Dim lDataSize As Long
            lDataSize = lDetailsSize - 12
            If lDataSize > 0 Then
                Stop
                CopyBytes bytes(), wholefile.moov.trak.mdia.minf.dinf.dref.details(lLoop).data, idx, lDataSize
            End If
            idx = idx + lDataSize
        Next lLoop
    End If
    
    Read_dref = lPosition + lSize
End Function

Private Function Read_smhd(ByRef bytes() As Byte, ByVal lPosition As Long)
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)
    
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "smhd"

    Dim idx As Long
    idx = lPosition + 8

    wholefile.moov.trak.mdia.minf.smhd.type = "smhd"
    wholefile.moov.trak.mdia.minf.smhd.size = lSize
    wholefile.moov.trak.mdia.minf.smhd.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.minf.smhd.flags = BytesToLong(bytes, 3, idx)
    wholefile.moov.trak.mdia.minf.smhd.balance = BytesToLong(bytes, 2, idx)
    wholefile.moov.trak.mdia.minf.smhd.reserved = BytesToLong(bytes, 2, idx)
    
    Read_smhd = lPosition + lSize
End Function

Private Function Read_mdia_hdlr(ByRef bytes() As Byte, ByVal lPosition As Long)
    lPosition = lPosition + 6 '* fake it 'til we make it
    'Dim sBuffer As String
    'sBuffer = BytesToString(bytes, 50, (lPosition))
    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "hdlr"

    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    wholefile.moov.trak.mdia.hdlr.size = lSize
    wholefile.moov.trak.mdia.hdlr.type = "hdlr"
    
    Dim idx As Long
    idx = lPosition + 8
    
    wholefile.moov.trak.mdia.hdlr.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.mdia.hdlr.flags = BytesToLong(bytes, 3, idx)
    wholefile.moov.trak.mdia.hdlr.componenttype = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.mdia.hdlr.componentsubtype = BytesToString(bytes, 4, idx)
    idx = idx + 12 'skip the reserved
    
    Dim sRemainder As String
    sRemainder = BytesToString(bytes, lPosition + lSize - idx, (idx))
    
    wholefile.moov.trak.mdia.hdlr.componentname = sRemainder
    
    Read_mdia_hdlr = lPosition + lSize
End Function

Private Function Read_tkhd(ByRef bytes() As Byte, ByVal lPosition As Long)

    Debug.Assert Chr$(bytes(lPosition + 4)) + Chr$(bytes(lPosition + 5)) + Chr$(bytes(lPosition + 6)) + Chr$(bytes(lPosition + 7)) = "tkhd"
    Dim lSize As Long
    lSize = 16777216# * bytes(lPosition) + 65536# * bytes(lPosition + 1) + 256# * bytes(lPosition + 2) + bytes(lPosition + 3)

    Dim idx As Long
    idx = lPosition + 8

    wholefile.moov.trak.tkhd.size = lSize
    wholefile.moov.trak.tkhd.type = "tkhd"
    wholefile.moov.trak.tkhd.version = BytesToLong(bytes, 1, idx)
    wholefile.moov.trak.tkhd.flags = BytesToLong(bytes, 3, idx)
    wholefile.moov.trak.tkhd.creationtime = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.modificationtime = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.trackid = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.reserved0 = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.duration = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.reserved1(0) = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.reserved1(1) = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.layer = BytesToLong(bytes, 2, idx)
    wholefile.moov.trak.tkhd.alternationgroup = BytesToLong(bytes, 2, idx)
    wholefile.moov.trak.tkhd.volume = BytesToLong(bytes, 2, idx)
    wholefile.moov.trak.tkhd.Reserved2 = BytesToLong(bytes, 2, idx)
    
    idx = idx + CopyBytes(bytes, wholefile.moov.trak.tkhd.matrixstructure, idx, 36)
    
    wholefile.moov.trak.tkhd.trackwidth = BytesToLong(bytes, 4, idx)
    wholefile.moov.trak.tkhd.trackheight = BytesToLong(bytes, 4, idx)

    Debug.Print "[trak] size=8+" & CStr(lSize - 8)

    Read_tkhd = lPosition + lSize
End Function

Private Function CopyBytes(ByRef srcbytes() As Byte, ByRef destbytes() As Byte, ByVal lIdx As Long, ByVal cBytes As Long) As Long
    Dim x
    For x = 0 To cBytes - 1
        destbytes(x) = srcbytes(x)
    Next x
    
    CopyBytes = cBytes
End Function

Private Function BytesToLong(ByRef bytes() As Byte, ByVal cBytes As Long, ByRef plPosition As Long, Optional bLittleEndian As Boolean = False) As Long
    BytesToLong = 0
    Dim lLoop As Long
    If bLittleEndian Then
        For lLoop = cBytes - 1 To 0 Step -1
            BytesToLong = BytesToLong * 256 + bytes(plPosition + lLoop)
        Next
    Else
        For lLoop = 0 To cBytes - 1
            BytesToLong = BytesToLong * 256 + bytes(plPosition + lLoop)
        Next
    End If

    plPosition = plPosition + cBytes
End Function

Private Function BytesToString(ByRef bytes() As Byte, ByVal cBytes As Long, ByRef plPosition As Long) As String
    Dim lLoop As Long
    For lLoop = 0 To cBytes - 1
        BytesToString = BytesToString & Chr$(bytes(plPosition + lLoop))
    Next
    plPosition = plPosition + cBytes
End Function

Private Function ReadByteFile(ByVal sFileName As String) As Byte()
    Debug.Assert fso.FileExists(sFileName)

    Dim fileNum As Integer
    Dim bytes() As Byte

    fileNum = FreeFile
    Open sFileName For Binary As fileNum
    ReDim bytes(LOF(fileNum) - 1)
    Get fileNum, , bytes
    Close fileNum

    ReadByteFile = bytes
End Function

The Excel Development Platform

Monday, 30 September 2019

Video - Parsing an MP4 file

MP4 Files are based on Apple's QuickTime Movie files

MPEG Specificaation ISO/IEC 14496 parts 12 & 14

VLC Media Player Source Code

Use Unix Tool Grep in Windows

Links

Appendix A - VBA Code to read DASH IV

No comments:

Post a Comment

Popular Posts