Missing Settlement Functions

As a condition of the Final Judgment in United States v. Microsoft, CA No. 98-1232, Microsoft is required to disclose “the APIs and related Documentation that are used by Microsoft Middleware to interoperate with a Windows Operating System Product”.

Among the outcomes of the present survey of Windows shell functions is a list of API functions that three important software modules are known to make accessible for use by other modules, and for each function a list of other modules that are known to use the function. These lists would seem well suited to a quick test of whether Microsoft’s API disclosure is credibly complete.

Of course, completeness must be assessed in the sense intended by the judgement. The aim is to extract from the survey’s lists each function for which all the following are true:

• is supplied by a module that qualifies as a component of a Windows Operating System Product;
• qualifies as an API;
• has among its known users at least one module that qualifies as a component of Microsoft Middleware;
• is not documented by Microsoft for the settlement.

The ideal expectation is that no function in the survey should satisfy all these conditions. Note especially that the Joint Status Report on Microsoft’s Compliance with the Final Judgments, dated 3rd July 2003, reports to the court that “to identify the relevant interfaces, Microsoft developed new software tools and drew upon the expertise of more than 50 Windows software developers and program managers.” This seems intended to convey an impression of substantial work, perhaps even of thoroughness, specifically just to enumerate APIs for disclosure. It seems reasonable to expect that the ideal will be at least close to the reality.

Even if some few functions satisfy all the conditions, Microsoft surely has something to explain, if only that slight inaccuracies are inevitable, no matter how hard one works or how much care one takes. If hundreds of functions satisfy all the tests, which they do, then although no appeal to occasional error in good faith would be credible, much might be explained as Microsoft having applied some different interpretation, so that Microsoft enumerated APIs for disclosure by testing against very different conditions. Against that, since Microsoft doesn’t present any such conditions that it tested for, nor say anything in public that might help an assessment of its methods, one might reasonably wonder whether the omissions amount to “knowing, willful or systematic violations”.

Windows

To apply the first test to the survey is to ask which of the three shell modules (COMCTL32.DLL, SHELL32.DLL and SHLWAPI.DLL) that supply the surveyed functions qualify as components of a Windows Operating System Product. I contend that the only sensible answer is that all three do.

All three are distributed with all versions of Windows 2000 and Windows XP. The edition used for the survey is Windows XP Service Pack 1a, which certainly qualifies.

Not only are these shell modules distributed in the Windows package, but Microsoft has itself listed all three as core components of Windows, though admittedly for earlier versions of Windows than specified in the judgement. See for instance the MSDN Library from January 1997 for a list of Core Components given as an appendix to the Designed for Microsoft Windows NT and Windows 95 Logo Handbook for Software Applications. Curiously, this appendix does not persist in later descriptions of the logo requirements.

On the other hand, Microsoft has also listed two of these modules, namely COMCTL32.DLL and SHLWAPI.DLL, as core components of Internet Explorer. That Internet Explorer has privileged access to core components of Windows is, to many people, a good brief summary of how all this trouble started.

Perhaps the last word is given by the Settlement Program Interfaces that Microsoft has documented for compliance with the judgement. These include functions exported from each of these three shell modules.

APIs

Applicability of the second test is trivial. All functions in the survey are formally exported by the three shell modules in question. Each is therefore locatable and callable from any other module that knows what to locate and how to call. That Microsoft does not document all such functions as Application Program Interfaces or document how to use the functions (at all, let alone safely) does not mean that the functions are not APIs.

Middleware

Thus, any function that is exported from any of the shell modules for this survey satisfies the first two tests. To apply the third test, examine the list of modules that are known to use the function. The present interest is with functions for which the list of known users includes even one module that is distributed not only with Windows but also with some other Microsoft product that qualifies as Microsoft Middleware.

If a function has no known users, it is eliminated. Most likely, the function is obsolete.

A function is also eliminated if all its known users seem to be distributed only with Windows. Specially prominent examples of such modules are COMDLG32.DLL, EXPLORER.EXE and SHELL32.DLL. Less prominent are PRINTUI.DLL, PROGMAN.EXE, SYNCUI.DLL and TASKMAN.EXE. These seven examples suffice for eliminating numerous shell functions as being in some sense internal to Windows.

An open question in my mind arises from the judgement’s definition of Microsoft Middleware as excluding any “software code described as part of, and distributed separately to update, a Microsoft Middleware Product”. Does this apply to modules that are redistributed with other products apparently only as setup tools, rather than as lasting elements of the installed product? Of particular interest among such setup tools is GRPCONV.EXE, because it uses surprisingly many undocumented shell functions. I have chosen to treat a function’s use by GRPCONV.EXE as an internal Windows matter, not as use that qualifies under the judgement.

Note that all the eliminations so far are generous to Microsoft’s compliance.

That a user is not known for some function does not mean that the function is not used. The known users obtained for the survey are those for which Microsoft distributes symbol files as customer support for Windows, and the modules listed as users of any one function are those that the symbol files show as formally importing the function. A function may be used in some way that does not generate the corresponding import symbol. Such other ways certainly are known, and some are arguably exotic: for instance, the SHLWAPI function EventTraceHandler is used by MSHTML.DLL because the former creates a named memory mapping in which to store the function’s address and the latter knows the name of the memory mapping. For the tables below, I leave such exotica alone, so that they do not count as usage even though they are arguably the most egregious examples.

What remains after the preceding eliminations is a list of functions each of which has at least one known importer that is distributed with some Microsoft product other than Windows. Moreover, it turns out that for every remaining function, at least one of the known importers is a module that is distributed with Internet Explorer (which I take to include Outlook Express), except for some few modules that support the Microsoft Network (MSN). Indeed, almost all the remaining functions are imported by either BROWSEUI.DLL or SHDOCVW.DLL, and a few more by MSHTML.DLL.

SHDOCVW

If anything, SHDOCVW is not just software code for Internet Explorer: it is the main software code for Internet Explorer. There is even a literal sense to this. Some users start Internet Explorer by running the program IEXPLORE.EXE, and some even consider that this program is the essence of the product. However, IEXPLORE has long been little more than a front end for executing SHDOCVW, and the way that this execution is arranged is that IEXPLORE calls a function named IEWinMain in SHDOCVW (exported as ordinal 101 and apparently, if unsurprisingly, not documented).

Another measure by which SHDOCVW is the essence of Internet Explorer is provided by Microsoft’s own advice on How to Determine Which Version of Internet Explorer is Installed, particularly where a “table lists the different versions of the Shdocvw.dll file and the corresponding versions of Internet Explorer”.

BROWSEUI

There may be more debate about BROWSEUI. For instance, as a rough analogue of SHDOCVW’s providing a sort of WinMain function for Internet Explorer, BROWSEUI has a function named SHParseIECommandLine (exported as ordinal 125 and apparently undocumented) that gets the job of parsing the IEXPLORE command line. However, BROWSEUI parses the command line for the ordinary Windows Explorer too, through the function SHExplorerParseCmdLine (exported as ordinal 107 and also apparently undocumented). On this matter, and many others, it behaves in part as a component of Windows and in part as a component of Internet Explorer. Is BROWSEUI one or the other, in terms of the judgement’s definitions, or is it both?

Perhaps the last word comes indirectly from Microsoft. If BROWSEUI is not a component of Internet Explorer, then presumably it is a component of Windows, so that its use of undocumented shell functions is an internal Windows matter and does not qualify for documentation under the judgement. But if this is so, then the functions that BROWSEUI exports and Internet Explorer calls (as with SHParseIECommandLine) ought to have been documented for the settlement—and none were.

Documentation

The last test is whether the function is documented. A function’s status with respect to Microsoft’s documentation has no controversy. The judgement cites the Microsoft Developer Network (MSDN) specifically and the choice of reference for the survey is the January 2004 edition, already at least a year later than Microsoft’s supposed compliance. If a function was not documented by then, yet ought to have been, then Microsoft has a compliance problem to explain and put right—and the compliance authorities create a hostage for the future if they don’t explain the process in public.

Function Lists

All up, I count 363 shell functions as passing all four tests, so that they ought to have been documented by Microsoft but have not been. A large proportion are perhaps dismissable on the grounds that they repackage otherwise well-documented Windows API functions—but even after dismissing these, I still count 196.

The tables that follow show for each function one or more modules that are known to use the function and which I regard as sufficient to qualify the function for documentation under the judgement. These modules are represented according to the following key.

Unless some argument exists that BROWSEUI.DLL and SHDOCVW.DLL (and to a lesser extent, MSHTML.DLL) are not components of Internet Explorer, which is in turn clearly cited as a Microsoft Middleware Product, one can only wonder at how Microsoft can possibly have omitted from the Settlement Program Interfaces more “shell and common controls” functions than it documented.

COMCTL32 Functions

Microsoft documented 17 COMCTL32 functions as Settlement Program Interfaces. I count another 17 functions as omitted (and for one more, both the ANSI and Unicode forms ought to qualify but Microsoft documents only the Unicode form).

These missing functions are exported from the COMCTL32.DLL that shippred with Windows XP Service Pack 1a, are each used by at least one of the following three DLLs that were all distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• shdocvw.dll (required only for counting the ANSI form of Str_SetPtr as missing)
• webcheck.dll (required only for counting DPA_CreateEx as missing)

yet were not documented by Microsoft as late as the January 2004 edition of the MSDN Library on CD.

Note, however, that they have all been documented subsequently. This post-settlement documentation was not done all at once, but as a process of apparently irregular updates. I have not been willing (even if able) to track this process in detail: doing so should be the (paid) work of the compliance authorities, even if Microsoft seems nowhere to have described this post-settlement documentation as rectifying any incompleteness of the original list.

SHELL32 Functions

Microsoft listed among the Settlement Program Interfaces 110 “shell functions”, almost all of which actually are implemented in SHELL32. Hundreds more SHELL32 functions remain undocumented, but I accept that all but 2 seem either to be used only internally within Windows or to be not used at all. Again, for one more function, Microsoft has documented only the Unicode form, even though both ANSI and Unicode forms surely qualify.

The functions shown below as missing from the Settlement Program Interfaces are exported from the SHELL32.DLL that shipped with Windows XP Service Pack 1a, are each used by at least the following DLL that was distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll

yet were not documented by Microsoft in the January 2004 edition of the MSDN Library on CD.

SHLWAPI Functions

It is with SHLWAPI that Microsoft has, if only as I see it, seriously and perhaps culpably undercounted. Microsoft lists only one SHLWAPI function (namely SHGetViewStatePropertyBag) among the Settlement Program Interfaces. That this might be anywhere near complete is just not credible if SHLWAPI is regarded as a Windows component used by Internet Explorer.

Against this is that Microsoft perhaps deems SHLWAPI to be a component of Internet Explorer but somehow not a component of Windows. With this thinking, bizarre as it would surely seem to many, no SHLWAPI functions used by Internet Explorer would need to be documented for the settlement. Inclusion of SHGetViewStatePropertyBag might have been necessary only because it is used by some Middleware product other than Internet Explorer. Or perhaps it is just that Microsoft was in this one case more generous than the settlement requires. Until Microsoft explains its thinking, which might never happen unless asked by a court, we will never know.

Until then, if I take SHLWAPI as a component of Windows for the purposes of the settlement (even if it is also a component of Internet Explorer), then I count 344 SHLWAPI functions as omitted. Roughly half are so-called wrappers of otherwise well-known API functions, mostly to present a Unicode interface even if the underlying implementation will convert to and from ANSI. Microsoft documents a Microsoft Layer for Unicode (MSLU) that does something similar, though by a significantly different method. Whatever the advantages and disadvantages, these long-standing wrappers in SHLWAPI still find very heavy use by all sorts of modules, including many that surely qualify as components of Microsoft Middleware. Since these wrappers form a set and since many will think them a very uninteresting set, I present them separately, later.

Discounting the wrappers for now, there remain 177 functions that are exported from the SHLWAPI.DLL that shipped with Windows XP Service Pack 1a, are each used by at least one of the following DLLs that were all distributed with that same version of Windows and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• iedkcs32.dll (required only for counting SHGetRestriction as missing)
• mshtml.dll
• msieftp.dll (required only for counting SHMessageBoxCheckEx as missing)
• msnmetal.dll (required only for counting SHCancelTimerQueueTimer as missing)
• msrating.dll (required only for counting SHGlobalCounterIncrement as missing)
• shdocvw.dll
• urlmon.dll (required only for counting StrCmpNC as missing)

yet were not documented by Microsoft in the January 2004 edition of the MSDN Library on CD.

Consultation of the MSDN Library on-line in late 2004 revealed that some of the functions in the table below have been documented since this study was first imagined (and the January 2004 edition chosen as the initial point of reference). More have been documented since. This still leaves more than a hundred as missing.

Unicode Wrapper Functions

The Unicode wrapper functions exported from SHLWAPI are numerous. The January 2004 edition of the MSDN Library on CD troubles to document only nine of them. Since nine is not zero, it is not as if Microsoft has always discounted these wrappers as being entirely unworthy of comment. None, however, were documented for the settlement.

The 167 wrapper functions listed below are all exported from the SHLWAPI.DLL that shipped with Windows XP Service Pack 1a, are each used by at least one of the following DLLs that were all distributed with that same Windows version and with the roughly contemporaneous Internet Explorer 6.0:

• browseui.dll
• iepeers.dll (required only to count StartDocWrapW as missing)
• inetcomm.dll (required only to count DrawTextExWrapW as missing)
• mshtml.dll
• msxml3.dll (required only to count IsCharUpperWrapW as missing)
• shdocvw.dll
• wininet.dll (required only to count SHFlushSFCacheWrap as missing)

yet were not documented by Microsoft as late as the January 2004 edition of the MSDN Library on CD.

Many, but nowhere near all, have been documented since, not by the end of 2004 but in time at least for the January 2007 edition of the Windows Vista Software Development Kit (SDK). Microsoft’s selection is at best capricious. How, for instance, does anyone with the slightest appreciation of logic and consistency decide to document IsCharUpperWrapW but leave IsCharLowerWrapW alone, except if instructed to document only those functions that meet some very strict criteria however absurd may be the outcome.