Geoff Chappell - Software Analyst
Some versions of an Intel® Wireless WiFi Link Driver are seen to flood the System Event Log with hundreds and thousands of the same event in quick succession. The event tells of nothing with system-wide importance, just of the driver’s receipt of an Intel proprietary request. That Intel does this to its customers plausibly has no reason other than thoughtless abuse.
One evening last week I got into my head that since Windows was getting ever more insistent that I should reboot my laptop to complete the installation of who know’s what updates even though a week surely hadn’t yet passed since Microsoft last put me through this disruption of shutting down my work, I would better change the irritation into opportunity. Since I anyway had a growing list of experiments that I had long put off because they too would require a few restarts, now might as well be their time. Part of this exercise had me open the Event Viewer to confirm the several expected signs of the reconfigurations I was trying out and to check that my intended changes hadn’t triggered any unexpected complaints.
Perhaps you ask why I make a story of such a routine exercise as looking at event logs. Doesn’t every capable Windows user keep an eye on at least the System and Application event logs, if not the other Windows Logs, just in the self-interest of being aware of what’s normal for their computer? Therein lies a second part to my story, for I confess that I have fallen out of the habit. These logs just aren't as useful as they were. It’s not that fewer types of events that go into them are useful. If anything, the variety of events that get logged these days is richer and more useful than ever. Yet the System and Application logs are sometimes reduced almost to complete uselessness by mass repetitions of events that cannot conceivably be the slightest use to anyone in such quantity, if at all.
Time was that the Windows Logs could be relied on for a useful record of significant events going back for months and even years. Mind you, in those halcyon days you might reasonably expect that these logs would show no Error or even Warning in a week of normal operation. Nowadays, even a fresh installation of Windows 10 comes with errors and warnings in the Application and System logs just from Microsoft components that complain of being misconfigured. And Microsoft bears much blame for the cluttering, too. Operate your computer in some way that does not fit Microsoft’s notions of Windows as a service and you can easily find that your Application log is all but entirely given over to recurring Information events from some service that Microsoft pretty much insists you must run even if it can hardly ever run usefully, e.g., because you deliberately keep your computer off the Internet. But whether the cause is a service or an application, from Microsoft or not, acting directly or indirectly, the unwelcome practical consequence is that your Application log has soon lost all record of anything significant about other programs even as recently as yesterday. Flooding the System log seems to be less common, thankfully, but it does occasionally get done by some errant driver.
Thus did it happen that I looked quickly at the System event log only to find that it was full of events from Netwtw06—and not of some variety of events but of just one event, 7023, logged every minute. These repeating instances of the one event accounted for just short of 75% of all the events in the System log. This one driver apparently thinks that this one piece of recurring information that is very specific to this driver alone should have over 10 times the weight of all the general-purpose system-level information from the various sources whose names start with Kernel. I make this last comparison to emphasise, especially in case you’re from Intel, how very, very arrogantly rude it is to write so much of one vendor-specific thing to the primary shared repositories of significant general-purpose events.
Windows has long provided for special-purpose events to go to special-purpose logs. It’s long past time that polluting the general-purpose logs was regarded as unprofessional, at best. There is nothing but shame in the standard practice of telling complainants that an event is harmless and can safely be ignored. Though a single event may tell of no harm in and of itself, repeating it by the hundreds and thousands into the general-purpose Windows Logs causes the significant harm of exhausting these ordinarily capacious logs and robbing users of potentially valuable history.
On a Lenovo X1 Carbon with an “Intel(R) Dual Band Wireless-AC 8265” as its built-in wireless network adapter, but presumably on other computers and network adapters, you find the System event log is flooded with events for which:
The relevance of the wireless network adapter is that the source of the event, Netwtw06.sys, is an Intel® Wireless WiFi Link Driver, according to the description in the driver’s resources. If you call up the Properties for your wireless network adapter in the Device Manager and ask for the Driver Details, you will see Netwtw06.sys.
Even without thinking what can be meant by Netwtw06 being named as the Source of the event, you might start typing Netwtw06 into Google. You may find, as I did yesterday, 18th August, 2018, that Google’s type-ahead soon predicts “netwtw06 7023 intel proprietary”. So, you are not alone! Indeed, Wireless driver spamming Windows Event Log on Intel’s community site begins with an observation of this same event recurring “every 9-11 seconds” on an “Intel Dual Band Wireless-AC 8260 chip on a Lenovo P50” in March 2018. Another participant reports 42,762 occurrences of this event. Yet another, now in May, reports its recurrence “every minute”. This “rate of 1 a minute” is observed independently for Intel Wireless Driver generating unnecessary windows events..
That you are not alone is no rescue. These threads at Intel’s site are an embarrassing sample of how the software industry treats its customers. I do not mean anything critical of the individual who “posted on behalf of Intel Corporation” and surely did mean to help. The poor guy was likely just following his training about Intel’s practices, which seem at the very least to be designed so that deflection and obtuseness face no disincentive but initiative perhaps does. It would be sadly naive to hope that the name of an Intel driver, a large-ish number and the text “Intel proprietary” are recognised as specific identifiers to feed into a support database or even to pass directly to the engineering team that would know why the event exists. No, there must first be a standard collection of largely irrelevant information. Then there must be a deflection: “we can’t prevent a driver from triggering events.” And then a shooting of the messenger: “Event Viewer is not an Intel software, and event tracking cannot be disabled from the driver”. Even after the patient, tactful user (gabe_luci) points out that it is Intel’s software that provides the event and Intel’s software developers who can stop doing so, Intel “can’t confirm if these events are part of the driver design.” Seriously? What can be the point to “Intel proprietary” in an Intel driver except that it’s by design? And, of course, the passing on is still not to any engineer who might know immediately what 7023 ever signified but is instead to “our business unit”. It goes on like this for some time. Intel, please, lift your game!
It’s surely best to get to the good news now. The immediate problem goes away in the 126.96.36.199 build. Presumably, later builds will be good too.
The first of the Intel community threads that I cite above does by June 2018 proceed to announce with some authority the release of Intel® PROSet/Wireless Software and Drivers for Windows® 10 version 20.60.0, though without explanation and indeed with curiously little confidence that the new build actually does fix the problem. In what I downloaded from this link yesterday, 18th August, 2018, the NETWTW06.SYS driver has version 188.8.131.52.
Of course, I do not say the problem goes away in version 184.108.40.206 merely because I installed it and have found that the problem does not recur for me. No, I say with certainty that the problem has gone away in this build because I know from inspecting the binary that in some build between the 220.127.116.11 which I had been using and the newly obtained 18.104.22.168, Intel removed the one C++ statement that had the driver write event 7023 to the System log.
If you’re not a reverse engineer, then you can get some (much) lesser assurance by searching these versions of the driver for “7023 - Intel proprietary” as Unicode text. It’s gone from the new driver’s resources. And if that’s too much trouble, then just install the new driver and make do with the observation, which hits us all, that those tens of thousands of Netwtw06 events with Event ID 7023 that were put into your System log by the old driver now come with complaints of how “The description for Event ID 7023 from source Netwtw06 cannot be found.” Yes, having spammed your System log with events that say nothing more helpful than “Intel proprietary”, Intel “fixes” it by making all the existing events be even more unhelpful. Were Intel not one of the innovators of integrated circuits and still the market leader, you might reasonably wonder if they’re incompetent. What we have instead is worse: disregard.
Note also that I venture only that the immediate problem goes away. Whatever caused earlier versions of the driver to write event 7023 to the System log very possibly still occurs. As I explain below, event 7023 means little more than that the driver received a network request from an Independent Hardware Vendor (IHV) extensibility module. For all but a few of the many types of these IHV requests, Intel’s driver writes event 7023. It’s at least a sound proposition that different users were seeing the event from having received different types of these requests and that some were seeing it from multiple types. It’s possible, of course, that Intel has found some sources of these requests and has reduced them or even eliminated them. It may also be possible that IHV requests can be received from sources that have nothing directly to do with Intel or aren’t even Intel’s to control. Either way, the one and only reason that you or anyone else ever got Netwtw06 event 7023 in the System log is that Intel’s Netwtw06 put it there, and the one and only reason that neither you nor anyone else can ever see any repeat of event 7023 after updating to version 22.214.171.124 is that this build simply has no code for writing this event.
The wonder, of course, is that back at the coal face of customer support the new build was presented only as something to try and see: “let us know if the events keep showing up with the same frequency”. Intel’s programmers who work on Netwtw06.sys must have known that the new version removes the code and cannot write the event. That’s as conclusive as things ever are in software, so how could such definiteness not feed back to the affected customers? Again, Intel, please lift your game!
The removed C++ statement is in a routine that Intel apparently names osal::CCmdPropIhvRequest::execute. This name is known because Intel’s programmers carelessly leave it and many others in plain sight as text for Windows Pre-Processor (WPP) tracing. This is remarkable because one of the main attractions of WPP tracing is that the format strings of the trace statements do not bloat the binary. They instead become annotations in the Program Database (PDB) file, which is more commonly referred to as the symbol file. That any one trace statement is at line number x in some source file y and is within some routine z is in the symbol file, safely removed from the prying eyes of reverse engineers. Yet here is an Intel driver in which many of the WPP trace statements pass the name of the containing routine as an argument. I don’t complain about the help, but I do observe that it’s misplaced: if Intel wants to help, then ordinary users are more deserving than reverse engineers.
Of course, that Intel names the routine to suggest execution for an IHV request doesn’t mean that this is what the routine actually does, but inspection establishes a flow of execution from the routine that Intel names osal::CNdis::miniportOidRequest and which Netwtw06 registers with NDIS as its MiniportOidRequest routine for handling what Microsoft’s NDIS documentation refers to as an OID request, and the path from this entry point to osal::CCmdPropIhvRequest::execute is specifically for handling requests whose Oid in the NDIS_OID_REQUEST structure is OID_WDI_IHV_REQUEST (0xE440005D).
This OID_WDI_IHV_REQUEST is intended as distinguishing requests that an IHV extensibility DLL in user mode sends to a WLAN Device Driver Interface (WDI) miniport driver in kernel mode. It’s new for Windows 10. My take on it is that it continues Microsoft’s tidying of an old free-for-all of vendor-specific OIDs in competition wiith system-defined OIDs, though for this purpose it seems curiously under-documented by Microsoft and as much under-discussed in public by others. (A Google search for OID_WDI_IHV_REQUEST today, 26th August, turns up just three matches.)
Though a WDI miniport driver for Windows 10 does not receive vendor-specific OIDs directly in requests to its MiniportOidRequest routine, it can keep to any old scheme of vendor-specific OIDs by packaging them into the vendor-specific information that they pass with the system-defined OID_WDI_IHV_REQUEST. Thus do we get to the initial work of osal::CCmdPropIhvRequest::execute, first with the help of a member function named prepareNicSpecificOidFromWdiIhvRequest to unpack the IHV request into the form of a request whose OID is the less new OID_DOT811_NIC_SPECIFIC_EXTENSION, and then with the help of a routine named oscMiniportTranslateProprietaryOid to unpack the request as Intel presumably intended all along.
The very long table that follows lists the several hundred different OIDs that Netwtw06 explicitly recognises in the unpacking of an IHV request. That I list all these many OIDs is because I want you to take in the scale and variety, if only as you scroll rapidly past.
For each OID in the table, an IHV request that has this OID gets non-trivial OID-specific handling from Netwt06. For a relative handful that are marked “no” in the right-hand column, Netwtw06 proceeds directly to this OID-specific handling. For all the many others in the table, Netwtw06 first writes event 7023 to the System event log. For any IHV request that Netwtw06 receives but whose unpacked OID is not in the table, Netwtw06 also writes event 7023 to the System event log but then fails the request as unsupported (specifically by returning the error code NDIS_STATUS_NOT_SUPPORTED).
The symbols are Intel’s, from two tables that Netwtw06 has specifically for OIDs that are unpacked from IHV requests. For the low-numbered OIDs, these names are the same as defined in Microsoft’s NTDDNDIS.H and WINDOT11.H headers. OIDs whose high byte is 0xFF are vendor-specific. For these, the only known symbols are Intel’s. MEASURMENTS is reproduced correctly: if it is a mis-spelling, it is Intel’s. The driver’s tables of OIDs for IHV requests repeat each OID 0x0D01011B and 0xFF400200, and give two symbolic names each. The OIDs 0xFF400144 and 0xFF400190 also appear twice each in the tables, but with only one name each.
By the way, the list above looks to be the first publication of Intel’s names for its vendor-specific OIDs. More than few suggest interesting functionality that might usefully be examined for exploitability. That they are published here is not to help with that but is just for critical analysis of the nuisance that Intel made of writing its event 7023 into the System log in such unjustifiable quantity.
Please take in the table and its interpretation: Netwtw06 event 7023 in the System event log means just that the Netwtw06 driver received an IHV request—or, if you like, an “Intel proprietary” request—that wasn’t one of a handful of types that are presumably too frequent or unremarkable to be worth logging.
Take in this notion of worth. Intel’s programmers perhaps don’t think that an event in the System log has any cost or needs to pull its weight, but neither can they have believed that mere receipt of each of these very many different types of IHV request has any sort of system-wide significance. Still, let’s be generous. We might, just about, accept a driver writing something of no system-wide significance to the System log, and even writing the same thing over and over to the System log, if the event data at least recorded something distinctive about the cause so that it has diagnostic value.
Do we get at least this diagnostic value for Intel in exchange for Intel’s spamming of our System log? No, different types of IHV request are distinguished by different OIDs, which are only 32-bit, yet Intel’s programmers somehow didn’t trouble to include the OID with the event data. They could have easily enough. They just didn’t. The way that Netwtw06 writes event 7023 to the System log is by calling the documented NDIS function NdisWriteErrorLogEntry. This provides for passing one of 15 defined error codes and some number (up to some limit) of arbitrary unsigned 32-bit values. How many and what they mean is up to the caller. Netwtw06 passes exactly one extra value. You can see it at the end of the EventData for every event 7023. It’s always the same in all your tens of thousands of these events because it’s hard-coded into the driver: 0x56524457 as an integer or WDRV as text. So, not only does Netwtw06 event 7023 in the System event log mean just that the Netwtw06 driver received pretty much any IHV request, but it doesn't even have the (minimal) diagnostic value of recording which type!
Again, we might be generous. Hard-working programmers missed that the function they call provides an opportunity for increased diagnostic value. Not everyone sees everything. But Intel’s programmers don’t have just the sin of omission, having missed what the function might have done for them: they also have a sin of commission, having stretched what the function is meant to do. Think again of those 15 defined error codes. Microsoft doesn’t have the function enforce that only those 15 are permitted. Microsoft’s documentation didn’t start specifying just these 15 until the Windows Driver Kit (WDK) for Windows 7, a decade ago. But it is conspicuous that all 15 are error codes. As events every one of them is at the Error level, not Warning and certainly not Information, let alone Verbose. Although earlier documentation does suggest that the error code for NdisWriteErrorLogEntry can be any “NDIS_STATUS_XXX code describing the I/O error”, Intel’s programmers don’t even keep to that. To write event 7023, they make up their own error codes: in this case 0x40001B6F, being the event ID in combination with 0x40000000 to indicate the Information level. They must have known all along that they were abusing this function to write to the System log things that were not as substantial as the function existed to help with. And if they didn’t know it from stretching what the documentation might mean by an error code or by an “NDIS_STATUS_XXX code”, they surely ought to have known from the documentation’s Comments. A lot of Microsoft’s documentation barely adds to the prototypes from the C-language headers, which is very poor of Microsoft, but on the plus side is that where Microsoft does provide comments and remarks of any substance, they mostly are worth heeding. Microsoft’s documentation of NdisWriteErrorLogEntry from at least as far back as the MSDN Library dated January 1999 is very plain:
Logging an I/O error at every possible opportunity is not particularly helpful to users either, so a miniport should log only critical I/O errors that can help a user or system administrator to debug a network failure for which the NIC is responsible on a particular machine or a configuration resource conflict discovered during driver initialization
There is, of course, no sense in which Netwtw06 event 7023 is any sort of critical I/O error, even in one occurrence, let alone by the hundreds and thousands. If you’ve been spammed by Intel’s driver, then I can’t see any reasonable disagreement that Intel owes you an apology. The bar is so low in this industry, however, that many would count it a reasonable excuse that Intel has been no worse than any other. Still, from the customer support all the way back to the programming, Intel has put on a poor show. Please, Intel, lift your game.