Page TM

The NT Insider
®

The only publication dedicated entirely to Windows® system software development

A publication by OSR Open Systems Resources, Inc. Not endorsed by or associated with Microsoft Corporation.

July - August 2010 Digital Edition Volume 17 Issue 2

Writing Filters Is Hard Work: WDK Community

Bug Bash Contest
Undocumented DFS & RDR
Interactions I t‘s back! After almost ten years
since the first ―Bug Bash‖, OSR is

T he Distributed File System (DFS) is

a Microsoft technology that allows
for an enterprise to configure a single
the casual observer, one would expect
that a file system filter driver writer
would care little to nothing about DFS.
back to kickoff a new edition, now the
WDK Community Bug Bash Contest.

server with a root share that contains Aside from the potential aliasing issues What‘s a Bug Bash Contest? Well,
multiple links. To clients, the links that this architecture introduces, which aren‘t you going to be glad you asked.
appear as regular folders under the root exist in other scenarios anyway, this It‘s easy: you find and report bugs in
share, however they can point to shares seems like the sort of thing that should the Windows Driver Kit (WDK).
either on the same server system or on be handled transparently to the client. That‘s it. That‘s all you have to do.
any other computer in the organization. Unfortunately, due to some architectural
and design decisions made in the From there, the bugs will be triaged
For example, you may access a share implementation of XP‘s DFS client and reported directly to the Microsoft
from your client machine via the path support, dealing with DFS can quickly WDK team—so at a bare minimum,
\\DfsServer\DfsRoot\Documents. This become a hornet‘s nest at the bottom of you are helping to get bugs fixed in
could then end up being a link to an a rat hole. In addition, the Filter the WDK.
e n t i r e l y d i f fe r e n t s e r v e r , s a y Manager mini-filter abstraction attempts
\\FooServer\Share. Thus, ignoring the to hide some of the implementation But wait, there‘s more! It wouldn‘t be
other features and options that DFS details and in the process complicates any fun if we didn‘t run a contest,
provides, DFS is effectively a name things even more. where every valid bug submission
aliasing technology that allows an earned a prize, and where all valid
administrator to redirect client requests Before we begin, we should note that bug submissions were judged against
from one machine to another. substantial changes have been made to one another for eligibility to win one
this space in Vista and thus the of several really worthwhile grand
Frustratingly, over the years we‘ve discussions in this article only apply to prizes (if you will).
regularly run into nagging issues with O/S releases prior to Vista unless noted
DFS‘ interaction with the file system otherwise. Really, in the end, if you read this
filter drivers that we have written. To (Continued on page 18) publication, the WDK is required for
some portion of your job function, and
thus improving upon it can only help
you.
The NT Insider Goes Digital– Good?
And hey, you could use a iPod touch,
You are reading the first electronic version of The NT Insider! We’re an Visual Studio Ultimate with
saving trees, a gazillion dollars in print and mail services and getting the MSDN subscription, or a netbook to
play around with, right?
issue into your hands at least 2-3 weeks earlier than the print edition.
For more, see the center page spread
Does this medium work for you? Why or why not? What can we do better? on pages 16-17.

We want to hear from you! Send comments to [email protected]!

 Page 2

The NT Insider™
Published by
Inside This Issue:
OSR Open Systems Resources, Inc.
105 Route 101A, Suite 19
Amherst, New Hampshire USA 03031
(v) +1.603.595.6500 (f) +1.603.595.6503
Writing Filters is Hard Work: Undocumented DFS & RDR Interactions 1
http://www.osr.com

Consulting Partners
W. Anthony Mason
Seeking International Partners & Contacts 3
Peter G. Viscarola
Executive Editor
Daniel D. Root Peter Pontificates: Pros & Cons of Agile SW Development Methodology 4
Contributing Editors
Mark J. Cariddi
Scott J. Noone
OSR Associate Staff Getting Better: Virtual Storport Tweaks 6
Consultant At Large
Hector J. Rodriguez
Send Stuff To Us:
email: [email protected] The Basics of Debugger Extensions: Short Term Effort, Long Term Gain 8

Single Issue Price: $15.00

Getting Away From It All—The Isolation Driver (Part I) 10
The NT Insider is Copyright ©2009. All rights reserved.
No part of this work may be reproduced or used in any
form or by any means without the written permission of
OSR Open Systems Resources, Inc. (OSR).
Check This Out—A Primer on Signature Checks in Windows 12
We welcome both comments and unsolicited manuscripts
from our readers. We reserve the right to edit anything
submitted, and publish it at our exclusive option.
EResources Close Cousin: Implementing Reader/Writer Locks in User-Mode 14
Stuff Our Lawyers Make Us Say
All trademarks mentioned in this publication are the
property of their respective owners. ―OSR‖, ―The NT
Insider‖, ―OSR Online‖ and the OSR corporate logo are
WDK Community Bug Bash Contest 2010 16
trademarks or registered trademarks of OSR Open Systems
Resources, Inc.

We really try very hard to be sure that the information we Analyst’s Perspective: Debug Smarter 30
publish in The NT Insider is accurate. Sometimes we may
screw up. We‘ll appreciate it if you call this to our
attention, if you do it gently.
OSR Seminar Schedule 32
OSR expressly disclaims any warranty for the material
presented herein. This material is presented ―as is‖ without
warranty of any kind, either expressed or implied,
including, without limitation, the implied warranties of
merchantability or fitness for a particular purpose. The
entire risk arising from the use of this material remains with
you. OSR‘s entire liability and your exclusive remedy shall
OSR USB FX2 Learning Kit
not exceed the price paid for this material. In no event shall
OSR or its suppliers be liable for any damages whatsoever.
Don’t forget, the popular OSR USB FX2 Learning Kit is available in the Store
It is the official policy of OSR Open Systems Resources,
Inc. to safeguard and protect as its own, the confidential at www.osronline.com.
and proprietary information of its clients, partners, and
others. OSR will not knowingly divulge trade secret or
proprietary information of any party without prior written The board design is based on the well-known Cypress Semiconductor USB FX2
permission. All information contained in The NT Insider chipset and is ideal for learning how to write Windows device drivers in general
has been learned or deduced from public sources...often
using a lot of sweat and sometimes even a good deal of (and USB specifically of course!). Even better, grab the sample WDF driver for
ingenuity.
this board, available in the Windows Driver Kit (WDK).
OSR is fortunate to have customer and partner relations that
include many of the world‘s leading high-tech organiza-
tions. As a result, OSR may have a material connection
with organizations whose products or services are dis-
cussed, reviewed, or endorsed in The NT Insider. The NT Insider—Digital Edition
Neither OSR nor The NT Insider is in any way endorsed by If you’re a new to The NT Insider (as in, the link to this issue was
Microsoft Corporation. And we like it that way, thank you
very much.
forwarded to you), you can subscribe at
http://www.osronline.com/custom.cfm?name=login_joinok.cfm.
 Page 3

Seeking International Partners & Contacts

O SR is in the process of growing its practice to more

effectively address both US and international markets.
We‘d be interested in hearing from you if your company has
What does OSR Offer in Return?
Every partnership should strive to be a mutually beneficial
relationship. OSR has made its reputation based on definitive
experience in Windows internals and kernel-mode expertise in Windows internals and kernel-mode software
development and an interest in leveraging that experience in a development. This expertise is provided through the various
partnership with OSR. products and services that are licensed and delivered to a
world-wide audience. OSR is offering the opportunity to
Partnership Areas leverage its expertise, knowledge base, resources, intellectual
While OSR is open to discussing partner proposals involving property, business strategy and revenue, with a partner that
all aspects of the business, currently our focus is on brings complimentary value in its country or region.
international growth in the following areas:
Next Steps
Training Please contact us at [email protected] if your organization
has interest in discussing partner potential with OSR. Tell us
Development a bit about your company, and the general areas in which you
might be interested in partnering.
Toolkits

Regions of Interest
While this is not a partner search that is limited to any specific
country or region, of particular interest are partners in India,
Greater China and Eastern Europe. If you believe your
organization would be a valuable partner, we do want to hear
from you regardless of the market you serve.
What OSR Students Say
What Constitutes “Partner” Potential?
In order to align with international partners that will be a Don’t take our word for it. This is what students say about
value-add and credit to the OSR brand, we are most interested our seminars:
in discussing proposals with companies and individuals with
the following: "The word I would use to describe the class is - great.
I will recommend anyone starting out with device
Technical expertise in Windows internals and kernel drivers to attend this seminar. For me, being a
mode software development and testing
developer who has never written a driver for Windows,
it was a great start."
Experience in providing services or solutions that
involve significant Windows kernel-mode
components ―This was my first OSR seminar and probably won't be
my last. I have always heard good things about them
Clear understanding of the markets served and totally agree. The week was enjoyable, entertaining,
informative and I left Munich feeling 100% more
A history of success in business, accompanied by a confident in tackling more debugging issues.―
long list of happy customers
"[Instructor] is a very good tutor. He presents the
Ownership and staff that demonstrate integrity, moral material with a (loud and) clear voice, often repeating
character and strong sense of business ethics stuff so one won't miss a thing. His long experience in
the field shines through. When he got a question he
Effective communication skills both with partners could often refer to how it was designed and
and customers implemented in the Windows source code itself, and
thus bring a very clear and understandable answer."
Page 4

Peter Pontificates:
Pros & Cons of Agile SW Development Methodology
I know that you, dear reader, have come to expect a certain
level of sophistication and journalistic integrity when it
comes to my monthly pontifications: A gloss of civility, a
A maniacal emphasis on ―just getting something
working‖ as opposed to thinking something through,
designing how it should work, and getting something
certain level of refinement, and an even airing of all sides of working correctly.
an issue. Thus, on those very rare occasions when I have a
personal opinion to present, I feel the need to make my Requiring detailed estimates of how long it‘s going to
viewpoint on the issue under discussion extremely clear. And take to implement some piece of functionality.
so it is for today‘s topic: Agile software development
methodology. An insane reliance on – and limitation to
implementing features for – user stories (AKA
Let me start, then, by clearly, dispassionately, and logically scenarios), which yield fragmented feature sets as
describing my point of view: opposed to designing for well-reasoned
comprehensive functionality.
Agile software development methodology sucks ass. Big
time. Totally. 100%. Entirely. For the life of me, I do not understand why I would ever want
to write code for a complex facility before I have had the
Gosh, I hope I‘ve stated that clearly enough. chance to design that facility and consider how it will interact
with everything else in its environment. For sure, managers
Let me explain further: I have never, not once, not ever, like it. They think designing things is a waste of time
participated in, seen, or even heard of a project in which Agile anyhow. This reminds me of clients who tell me to ―just write
works better than any well-organized alternative. Agile is some code‖ to do so-and-so. I work in kernel-mode, boys. I
without question the refuge of those with small minds, poor often work on complex pieces of larger systems. It doesn‘t
writing skills, and extremely high-levels of tolerance for make a lot of sense to just ―write some code‖ that‘s ―good
incomplete functionality that‘s written in a way that‘s ―good enough‖ – when you do that, you get code that doesn‘t handle
enough‖ to pass whatever tests happen to exist. corner cases, doesn‘t work or play well with others, and is
generally ill-conceived.
Just to make sure that you understand that I mean to slag the
entire, wide-ranging, genre that is Agile, I specifically mean to But Agile is all about ―just writing code.‖ System
include Agile permutations like Scrum. Basically, I mean to architecture? Design? In Agile, that‘s not in the plan. What‘s
include any software development methodology that relies on in the plan is ―code it up and see how it works‖ and ―you can
references to barnyard animals, where you stand-up to have fix it in the next sprint.‖ That‘s hosed. It‘s a waste of time.
meetings, or where you‘re supposed to account for what And while I‘m all about ―plan to write one to throw away‖, I
you‘re doing in units of, like, 45 minutes. sorta think that the word ―plan‖ is important there.
My major objections to Agile come from what I view as the
unmitigated stupidity of its three most important tenets: (Continued on page 5)

Peer Help? Turn to NTDEV/NTFSD/WINDBG

Writing and debugging Windows system software isn’t easy. Sometimes, connecting with the right people at the
right time can make the difference. This might be a subject matter expert, or just another driver dev tackling a
similar project or problem to yours. Either way, your peers from all over the world gather on the lists hosted at
http://www.osronline.com/page.cfm?name=ListServer.

NTDEV—internals and driver development

NTFSD—file system and file system filter development
WINDBG—kernel debugging and crash analysis
 Page 5
On the Agile projects I‘ve worked on, the planning process is
Peter Pontificates... just a big ―everyone‘s in on it but the boss‖ joke:

Boss man: You, copiously large engineering unit over

(Continued from page 4) there, how long will it take you to create the huzzy-bub
for the what‘s-a-ma-whosits?
Ever build a house? You don‘t start building a house before
the architect draws-up the plans. It‘s the planning phase Me: Well, boss man, I thought long and hard about this,
where you get to consider costs, trade-offs, and how one part and after consulting with my team members and the
of the house will relate to and work with other parts. You folks in test, I‘m confident I can have it done mid-way
probably don‘t want the garage to open onto the living room. through Sprint 832.
Or maybe you do. But at least you need to think about it.
And, having thought it through, you begin building. Boss man: Well done! That‘s what I like, a compliant
and focused, if slightly overweight, engineering unit. Do
Now, wait… wait… wait. Before you go all ―that‘s not you have milestones for that?
flexible‖ and start screaming ―waterfall, waterfall, waterfall‖
on me: As you progress through your building project, there Me: Oh, yes, boss man. I do. I definitely do! It‘ll take
will be problems, issues, and changes. You need to deal with 2 days to modify the frobnitz, 3 days to code-up the
those. You go back to the plans, you discuss what‘s changed, blortz-fart, 2 days to test the blortz-fart and frobnitz
and you retro-fit those changes to the plan. Then you resume together, 2 days to modify the frazzle-blow, oh… no
building. need to bore you with the details, great one… I‘ll just
put it all in our Online Agile Super Planner, oh mighty
And so it should be in software engineering. It is vital that master.
you at least design what you want to build, even if this means
your manager-troid will have to wait an extra week to see Boss man: Well done! I like specific milestones. Why
some demo he won‘t understand anyways. And if you‘re can‘t the rest of you give me nice, clear, specific
designing a larger system with multiple interconnected sub- milestones like the fat guy over there with the ponytail.
parts, you must define an architecture for that system. As you Whatever his name is.
progress through your architecture, design, and
implementation, at any point you can revise the plan, revisit Of course, I‘d written all the code already, during the previous
what you‘re trying to do, and take the time to think through sprint. I just hadn‘t checked it in. So, during Spring 832, I
(and document) the consequences of your changes. could update the Online Agile Super Planner to indicate that I
hit all my milestones, while I worked on whatever it was I
And, that brings me to the second thing that makes Agile suck wanted to commit for the next sprint.
so badly: The whole ―estimation‖ process. Every time
somebody insists that I estimate how long it‘s going to take Don‘t try to tell me you do anything different, cuz I know
me to implement some particular functionality, and in Agile that‘s what you do. Everyone I know works this way. When
it‘s not uncommon to have to do these estimates with great you know what you want to build, you just build it. Then,
precision, I realize I am dealing with an idiot. Worse, I realize having already built it, you provide astoundingly precise
that I am being asked to lie. And I don‘t like lying. Well, at estimates for when you‘ll have it done. Then you check in
least not to most people and not usually very much. what you‘ve already written according to the estimates you
provided. Ka-Ching! Goals and objectives: MET.
Why is estimating software development time so hard? Duh! Conformance to process: PERFECT. Ability to estimate:
Truthfully, I can’t even estimate with good precision how long AWESOME. Ability to deliver: EPIC. Now give me my
it’ll take me to go to store and get a case of beer. There could bonus.
be construction, a traffic jam, the store might be out of the
beer I want, I could get a flat tire, I might get distracted and And that leads me to the final Agile precept that fries my
decide I need to get a sandwich before I return home. If I potato: User Stories. Every time I hear somebody say ―I‘ve
can‘t estimate how long it‘ll take me buy a case of beer, how entered that as a user story‖ I want to puke. Why? User
the hell can I possibly estimate how long it‘s going to take me stories are just that: Stories. They‘re data. They‘re not
to implement some functionality that nobody‘s ever done wisdom from the ages. And, the way I see them used in most
before, for some device that‘s just had new silicon fabbed? cases, they‘re not even necessarily fully representative of what
The whole concept is ludicrous, not to mention insulting. the majority of users want or need to do. When your
development process is focused entirely on implementing
And don‘t even get me started on Planning Poker. If you like functionality based on a handful of user stories, you more
being treated like a mentally defective 6 year old, you will often than not end up with a random, inconsistent, poorly
seriously like Planning Poker. If you haven‘t heard of integrated, mess.
Planning Poker, just pray that your manager doesn‘t discover
it. It‘s the kind of thing manager-troids love, because ―it‘s fun (Continued on page 31)
for everybody‖ and leads to ―really good estimates.‖
Page 6

Getting Better
Virtual Storport Tweaks
S torport is a welcome relief to storage driver writers
wishing to develop a driver that exports a virtual device.
This article updates the Virtual Storport Miniport Driver that
emulate fully compliant SCSI devices, so in order to be
compliant the driver was changed to do this.

we developed in the earlier three issues of The NT Insider. No SRB DataTransferLength Update
The biggest violation that busTRACE found was the failure
Another Article? of the Storport Virtual Miniport to update the
Well to be honest, when we completed the last article we put DataTransferLength field in the SRB when completing a
the Virtual Storport Miniport Driver code away and expected command that involved a data transfer. While this isn‘t
not to have to revisit it for a long while. However, we were critical, in order to be properly compliant with the Microsoft‘s
contacted by Mike Berhan of busTRACE Technologies who SRB definition requirement, it must be filled in. Thus, for
used their busTRACE 9.0 product (www.bustrace.com) to see example, if you diff the old and new code you‘ll see that
how accurate our implementation was in emulating a SCSI OsrUserReadData and OsrUserWriteData now set the SRB
adapter/device. DataTransferLength field to the size of the data transfer.

So, How Did We Do? busTRACE also pointed out that when the driver failed a
Well, it seems that we missed some stuff. And while request, it was not setting the SRB‘s DataTransferLength
windows didn‘t complain, we‘d be doing the development filed to zero to indicate that no data was transferred.
community a disservice if we didn‘t clean the errors up and
indicate where those errors were. So the purpose of this article Validating the SRB DataTransferLength
is to tell you the errors that were found and how they were Another interesting busTRACE find was that the Storport
corrected. Of course, we‘ll replace the current download code Virtual Miniport Driver was not validating that the size of the
with the updated version. buffer described by the SRBs‘ DataTransferLength field
was large enough to hold the data requested. What this
So let‘s get into what Mike found, using busTRACE and means is that when the Storport Virtual Miniport gets a
busPROBE (which are terrific tools, by the way). request, for example a read or write, it needs to get the length
of the transfer described in the CDB and ensure that the SRB
Assumption of Pre-Zeroed Data Buffers DataTransferLength field is large enough. Failure to make
For SCSI operations other than read or write, Mike noted their this check would lead to crashes or data corruption since the
busPROBE product found that the Storport Virtual Miniport driver could attempt to access past the end of the allocated
driver did not pre-zero the data buffer which will contain the buffer.
output from the driver. Thus when the driver filled in the
return data, and fields that were not explicitly zeroed were
filled with random data.. (Continued on page 7)

While this may not seem like a big deal, remember that the
Storport Virtual Miniport Driver is trying to create and
Learn to Write KMDF Drivers
Why wouldn’t you? If you’ve got a new device you
need to support on Windows, you should be
considering the advantages of writing a KMDF
driver.

Hands on experience with labs that utilize OSR’s

USB FX2 device makes learning easy—and you get
to walk away with the hardware!

Contact an OSR seminar coordinator at

[email protected].

Figure 1—busTRACE INQUIRY Output (abbrev.)

 Page 7

Getting Better... Figure 2 shows how the SCSIOP_INQUIRY processing

code was corrected to correct the errors that busTRACE
flagged. Notice the setting of the pInquiryData-
(Continued from page 6) >AdditionalLength field as well as the setting of the PSRB-
>DataTransferLength field.
SCSIOP_INQUIRY – Additional Length Field
Not Set Another SCSIOP_INQUIRY issue that busTRACE found was
busTRACE also flagged the Storport Virtual Miniport Driver that the driver was not checking the
for an error processing the SCSIOP_INQUIRY command, EnableVitalProductData bit (VPD bit) and PAGE CODE
because the Miniport wasn‘t filling in the AdditionalLength field on input. This request, shown in Figure 3 (page 22), is
field of the SCSI_INQUIRY_DATA that we were returning. sent by the Storport driver, after it has enumerated a device in
The field was being left as zero which is invalid. In the code, order to enumerate a devices‘ VPD pages. The Storport
it should be set to 0x1F (0x1F plus 5 = 0x24 bytes which is Virtual Miniport Driver completely ignored this bit. Since
the valid length). In the driver the code sets we saw no reason to support this feature the Storport Virtual
AdditionalLength to ―INQUIRYDATABUFFERSIZE-5‖ Miniport driver was changed to fail the SCSIOP_INQUIRY
where INQUIRYDATABUFFERSIZE is 36 and the ―-5‖ request if the VPD bit and the PAGE CODE field is zero.
accounts for the 5 bytes of data (this includes that This was done to be compliant with the SCSI specifications.
AdditionalLength byte) preceed the additional data returned.
Mode Sense Emulation
Figure 1 (previous page) shows the busTRACE output that As with most of the errors in the Virtual Storport Miniport
highlights the error (see Additional Length in the figure). In Driver, the error in handling the Mode Sense command was in
the retail version of the product, the user can float the mouse setting the Mode Data Length field in the returned data. As
over the red highlighted area and busTRACE tells you what Figure 4 (page 22) illustrates the code was returning 0x4 in
firmware bug it has detected the Mode Data Length field of the returned data.
(Continued on page 22)

case SCSIOP_INQUIRY : {// 0x12

PCDB pCdb = (PCDB) &PSrb->Cdb;
PUCHAR pBuffer = (PUCHAR) OsrSpGetSrbDataAddress(pIInfo->OsrSPLocalHandle,PSrb);
PINQUIRYDATA pInquiryData;
if(!pBuffer || PSrb->DataTransferLength < INQUIRYDATABUFFERSIZE) {
status = STATUS_INSUFFICIENT_RESOURCES;
//Irp->IoStatus.Information = 0;
PSrb->SrbStatus = SRB_STATUS_ERROR;
goto completeRequest;
}
pInquiryData = (PINQUIRYDATA) pBuffer;
//
// The media is a regular disk, return the correct information.
//
ASSERT(pIInfo->StorageType == OsrDisk);
pInquiryData->DeviceType = DIRECT_ACCESS_DEVICE;
pInquiryData->DeviceTypeQualifier = DEVICE_CONNECTED;
pInquiryData->DeviceTypeModifier = 0;
pInquiryData->RemovableMedia = TRUE;
pInquiryData->Versions = 2; // SCSI-2 support
pInquiryData->ResponseDataFormat = 2; // Same as Version?? according to SCSI book
pInquiryData->Wide32Bit = TRUE; // 32 bit wide transfers
pInquiryData->Synchronous = TRUE; // Synchronous commands
pInquiryData->CommandQueue = FALSE; // Does not support tagged commands
pInquiryData->AdditionalLength =
INQUIRYDATABUFFERSIZE-5; // Amount of data we are returning
pInquiryData->LinkedCommands = FALSE; // No Linked Commands
RtlCopyMemory((PUCHAR) &pInquiryData->VendorId[0],OSR_INQUIRY_VENDOR_ID,
strlen(OSR_INQUIRY_VENDOR_ID));
RtlCopyMemory((PUCHAR) &pInquiryData->ProductId[0],OSR_INQUIRY_PRODUCT_ID,
strlen(OSR_INQUIRY_PRODUCT_ID));
RtlCopyMemory((PUCHAR)
&pInquiryData->ProductRevisionLevel[0],OSR_INQUIRY_PRODUCT_REVISION,
strlen(OSR_INQUIRY_PRODUCT_REVISION));
status = STATUS_SUCCESS;
PSrb->SrbStatus = SRB_STATUS_SUCCESS;
PSrb->DataTransferLength = INQUIRYDATABUFFERSIZE;
goto completeRequest;
}

Figure 2—SCSIOP_INQUIRY Processing

Page 8

The Basics of Debugger Extensions

Short Term Effort, Long Term Gain
I n order to talk about debugger extensions, we need to first
talk about the Debugger Engine library, (DbgEng). DbgEng
is a generic interface that can be used to manipulate various
going to turn out to be straightforward so don‘t worry. The list
of available client COM interfaces and their uses are as
follows:
debugging targets, which can be things such as crash dumps, a
live kernel, an application, etc. The library provides access to IDebugAdvanced – Thread context, source server, and
all of the types of actions that you might want to perform on a some symbol information
debugging target, such as setting or clearing breakpoints, IDebugClient – Connect to targets, register input/output
reading or writing memory, translating addresses into callbacks, register debugger event callbacks, etc.
symbolic debugging information, receiving events from the IDebugControl – Evaluate expressions, disassemble,
target, and so on. execute commands, add/remove breakpoints, get input,
send output, etc.
The DbgEng library is quite flexible and can actually be used IDebugDataSpaces – Read/write virtual memory,
in standalone applications. If you wrote an application that physical memory, I/O ports, MSRs, etc.
exported all of the features of the DbgEng library, you‘d end IDebugRegisters – Read/write pseudo and hardware
up with a debugger exactly like WinDBG or KD because, registers
well, that‘s what they are. In addition, the DbgEng library is IDebugSymbols – Type information, module
what we‘re going to talk to when we write our debugger information, source file information, etc.
extensions. Thus, in effect we have the same API set but two
IDebugSystemObjects – Thread, process, and processor
different uses. This can lead to a fair amount of confusion if
information
you start looking through the WinDBG SDK samples because
there‘s a mix of applications and extensions supplied. If
you‘re not aware of this fact it can be difficult to find a sample
Callback Objects
As their name implies, callback objects export the COM
to get started with.
interfaces necessary to receive notifications of events
happening on the target. There are three types of callback
Sessions objects: input objects, output object, and debugger event
In either case, DbgEng relies on the concept of sessions. In
objects. These allow you to scan the input supplied to the
order to examine or manipulate the target, a debugging session
debugger engine from the user as well as the output of any
must be active. When a debug event is raised from the target,
debugger commands or the debugging target. In addition to
the target may become acquired at which point the debug
this, callback objects provide a way to receive notification of
session is active. The application or extension is now allowed
process creation, thread creation, breakpoints, etc. The list of
to perform whatever options it wants against the target. Once
available callback COM interfaces and their uses are as
finished with the target, it must be released, at which point the
follows:
session is no longer active and operations are no longer
allowed against the target.
IDebugInputCallbacks – Receive notification of the
Aside from sessions, the other major concept to learn about engine waiting for user input
DbgEng is the set of objects that are available to the IDebugOutputCallbacks – Receive notification of output
application or extension. The objects supplied are broken being sent to debugger engine
down into two basic categories: client objects and callback IDebugEventCallbacks – Receive notification of debug
objects. events

Client Objects Creating Your Own Extension

Client objects are used to interact with the debugger engine Now that we have some of the basics down, we can see how
and provide access to all of the available DbgEng APIs used to actually create a debugger extension. Debugger extensions
to manipulate the target via COM interfaces. The way this are nothing more than DLLs that provide commands via
works is that each client object provides a QueryInterface exports and rely on the DbgEng library and any other Win32
method that is used to create instances of the various COM API of their choosing. They can be built just like any other
interfaces. So, for example, if you wanted to set a breakpoint DLL, though we‘ll use the WDK seeing as how that‘s what
on the target, you‘d call the QueryInterface method on a client the available samples use. The only other thing that you‘ll
object to retrieve an instance of the COM interface that
exports the set breakpoint API. Sounds a little scary, but it‘s (Continued on page 9)
 Page 9
// A debuggee has been discovered for the session.
Debugger Extensions... // It is not necessarily halted.
#define DEBUG_NOTIFY_SESSION_ACTIVE
// The session no longer has a debuggee.
0x00000000
#define DEBUG_NOTIFY_SESSION_INACTIVE 0x00000001
// The debuggee is halted and accessible.
(Continued from page 8) #define DEBUG_NOTIFY_SESSION_ACCESSIBLE 0x00000002
// The debuggee is running or inaccessible.
#define DEBUG_NOTIFY_SESSION_INACCESSIBLE 0x00000003
need is the SDK subdirectory of the WinDBG installation, as
that‘s where the necessary header and library files are located. typedef void (CALLBACK* PDEBUG_EXTENSION_NOTIFY)
(__in ULONG Notify, __in ULONG64 Argument);

Extension DLL Entry Points The entire point of writing this DLL is to create your own
There is a single required entry point for extension DLLs,
debugger commands, and those will also be exports of your
DebugExtensionInitialize. This is where you will do all of
DLL. These commands or, extensions must appear in
your one time initialization for the extension DLL. A
the .DEF file associated with your DLL and the exports must
description of the entry point and the function prototype can
contain only lower case letters. The function prototype of an
be found in dbgeng.h:
extension command is as follows:
// Initialization routine. Called once when the
// extension DLL is loaded. Returns a version and // Every routine in an extension DLL has the
// returns flags detailing overall qualities of the // following prototype. The extension may be called
// extension DLL. A session may or may not be active // from multiple clients so it should not cache the
// at the time the DLL is loaded so initialization // client value between calls.
// routines should not expect to be able to query typedef HRESULT (CALLBACK* PDEBUG_EXTENSION_CALL)
// session information. (__in PDEBUG_CLIENT Client, __in_opt PCSTR
typedef HRESULT (CALLBACK* Args);
PDEBUG_EXTENSION_INITIALIZE)
(__out PULONG Version, __out PULONG Flags); You‘ll note that the function prototype indicates that a pointer
to a DEBUG_CLIENT structure is passed as the first
A simple example of a complete DebugExtensionInitialize is parameter to the extension command. This is actually the
as follows: client object whose QueryInterface method you will use to
extern "C"
gain access to the various client COM interfaces for target
HRESULT manipulation.
CALLBACK
DebugExtensionInitialize(PULONG Version, PULONG
Flags) Let‘s see a simple example command that uses the built in
{ expression evaluator to add two and two together and then
// displays the result. This would be the equivalent of typing ? 2
// We're version 1.0 of our extension DLL + 2 in the WinDBG command prompt.
//
*Version = DEBUG_EXTENSION_VERSION(1, 0);
HRESULT CALLBACK
// mycommand(PDEBUG_CLIENT4 Client, PCSTR args)
// Flags must be zero {
// PDEBUG_CONTROL debugControl;
*Flags = 0; HRESULT hr;
DEBUG_VALUE result;
//
// Done! UNREFERENCED_PARAMETER(args);
//
return S_OK; //
} // Let's do a couple of simple things. First
// thing to do is use the passed in client to
// access the debugger engine APIs.
//
// First, we'll get an IDebugControl so that we
There are two other entirely optional exports that you driver // can print messages.
can provide, DebugExtensionUninitialize and //
DebugExtensionNotify. DebugExtensionUninitialize can be hr = Client->QueryInterface
(__uuidof(IDebugControl),
used to undo anything that you might have done in (void **)&debugControl);
DebugExtensionInitialize:
if (hr != S_OK) {
return hr;
// Exit routine. Called once just before the }
// extension DLL is unloaded. As with
// initialization, a session may or may not be //
// active at the time of the call. // Now we can print.
typedef void (CALLBACK* //
PDEBUG_EXTENSION_UNINITIALIZE) debugControl->Output
(void); (DEBUG_OUTCTL_ALL_CLIENTS,
"mycommand running...\n");
If present, DbgEng calls the DebugExtensionNotify callback //
for session state changes: // Use the evaluator to evaluate an expression
(Continued on page 24)
Page 10

Getting Away From It All

The Isolation Driver (Part I)
I n working with a number of file system filter driver projects
in recent years we have constructed a model that we refer to
as the ―isolation‖ driver – a file system filter (usually a mini-
Streaming content delivery - the file starts out
zero-filled with data, but information is provided in
a ―just in time‖ fashion. For example, some
filter) driver that intercepts operations aimed at some set of packages might contain huge bodies of information
files and/or directories. The filter changes the content (and and rather than install all of the data, only the most
potentially the size) of individual files independent of the essential information is installed before the user can
actual contents of the underlying file. Our goal, in a series of begin using the package. The remaining data is
articles, is to describe the basic model, explore development then streamed in the background, with priority
issues and provide a sample mini-filter that implements a given to information that is needed by the running
model for a data isolation driver. Herein, we first explore application.
common usages, a model, and some high level complications Data deduplication – in this case, the file contents
relevant to an isolation driver. are replaced with some sort of reference to the data,
thus allowing duplication to be eliminated or
Applications removed.
One model for an isolation filter would be a hierarchical Data versioning – in this model, the original file
storage manager. Such a component is responsible for remains, but any changes that have been made to
moving infrequently used data from online storage to nearline that file are ―layered‖ on top and the filter driver
storage devices (traditionally tape, but tape has fallen out of provides the unified (or snapshot-in-time) view of
favor in recent years). Another model might be to think of a the file.
typical cache system, where one can keep recently used and/or The key to each of these is that they work best if the filter
modified data in the cache, while older data is moved to some controls the actual view of the file in order to ensure that
larger pool of storage. memory mappings of the file work properly.

It turns out that ―data isolation‖ models are actually applicable Basic Model
to a variety of different uses including (but certainly not The basic model for an isolation driver is that it controls the
limited by): cache. This might seem to be more complicated than a
―typical‖ filter. In fact in our experience, any filter that finds
Encryption filters - they want to show a view of itself in the position of trying to manipulate the cache when it
the data that is different than the data contained in is owned by the underlying file system driver becomes much
the file. more complicated.
Compression filters - they want to show a view of
the data that is both different in size AND content
from the data contained in the underlying file. (Continued on page 11)

Windows File System Development Seminar

Whether developing file systems, file system mini-filters or related components that require interaction and
support from the Windows file system interface, OSR’s Developing File Systems for Windows seminar can help.
There has been no equal to this seminar in the more than 13 years OSR has been presenting and updating it!

NEXT OFFERING: Santa Clara, CA—26-29 October 2010

For more information, visit www.osr.com/fsd.html, or contact the OSR seminar department at [email protected].
 Page 11
sitting on the network redirector, since the
Getting Away... redirector may change caching policy ―in flight‖).

The model we suggest will work better is to ensure that your

(Continued from page 10) filter is not tied to the cache that belongs to the underlying file
system driver. We achieve this in the isolation model by:
So, our initial and forthcoming sample implementation will
start with the simplest of data isolation drivers – one that Utilizing Shadow File Objects; and
merely replaces the actual contents of the file with the desired Providing our own Section Object Pointer (SOP)
contents of the file. In our model, we consider three different structure
―consumers‖ for our file (see Figure 1):
While this might sound unusual, we observe that using a
The provider that gives the filter the actual file different SOP structure is how NTFS implements transactions
contents. This could be implemented in the filter on its files – thus a file has a distinct ―view‖ inside the
itself, although we typically find that some (or all) of transaction versus outside the transaction.
this functionality is provided by a user mode service or
application of some sort. The isolation filter needs to The other complicating factor here is whether or not your
coordinate access with the provider in such cases to filter needs to modify the size of the file. If the file size does
ensure that the provider can access the underlying not change, you have a simple isolation filter (which is what
native file. we‘ll build in later parts of this series). The complication is if
The target that is the application for which we wish to you need to change the size of the file. Why? Because
alter the view. This would normally be some ―typical‖ applications will retrieve those sizes via a directory
application, for example Word or Acrobat. enumeration operation and not by opening the file and
The ignored application for which we do not wish to retrieving the size. While it might come as a surprise to some,
alter the view. This would normally be some specific this really doesn‘t work 100% of the time for application
application for which we do not want to provide the (Continued on page 25)
isolated view. For example, many encryption filters
do not decrypt the contents of a file when the
application is Windows Explorer.

For our sample, the native file will be the same size as the File System Development Kit
presented view.

Issues to Consider
Over the years we have seen a number of different filters that
How’s this for a problem: our customers feel little
try to: pressure to upgrade to new, functional releases of
our FSDK because it’s so stable. Now THAT’s a
Flush/purge the underlying file system‘s cache. problem we like to have.
Acquire locks (typically via the common header)
in hopes that the underlying file system uses that If your commercial, file system-based solution
specific locking model (a dangerous assumption, in
requires this same level of stability, why not consider
fact).
Uses undocumented fields in the underlying file
working with a codebase that has become the gold-
system‘s control blocks to determine caching standard for successful implementations of Windows
behavior (this is a problem for encryption filters file systems, and a company with the experience and
reputation to back it up.

Contact the OSR sales team at [email protected] or

visit www.osr.com/fsdk.html to find out more about
the FSDK and how OSR can help you achieve this
same level of success with your FSD.

Figure 1—Basic Isolation Filter Model

Page 12

Check This Out

A Primer on Signature Checks in Windows
By Tim Roberts Vista and beyond). In either case, the user can simply proceed
with the installation with the appropriate mouse click. Once

I find an enormous amount of confusion and a fair amount of

misinformation about driver signing in Windows. It's a
topic that hasn't traditionally had a lot of attention, because in
you have passed this check, you can disable, enable, unplug
and replug forever, and the signature will not be rechecked,
unless there is a new device that matches your INF file. Your
the past, many of us just ignored driver signing. Those of us CAT can be signed with any valid code-signing certificate.
who had to learn it did so once, and then used that recipe
repeatedly. Check 2: KMCS Check
The second signature check is the kernel-mode code signing
The most important point to realize is that there are two very (KMCS) check. This happens only on x64 architecture
different signature checks that Windows performs. It's easy to systems, but it happens every time the driver is loaded, and it
miss this point, as evidenced by many newsgroup and forum applies to all kernel modules. In other words, unlike the
postings, but it's important to remember because the two check for the WHQL signature or known publisher described
checks require different handling. previously, this check occurs regardless of whether the driver
being loaded is plug-and-play. During driver installation, and
Check 1: Signature Check for Trusted Vendor each time thereafter when the system is booted, the check is
The first signature check occurs to determine if the vendor performed. When you unplug/replug, the driver is checked
supplying the driver is known and trusted. This check occurs (Continued on page 13)
once, when your driver is installed. Specifically, the check
occurs when your driver is copied into the driver store, or
when a new device is detected that requires your INF to be
executed. This check happens on all Windows versions since
Windows 2000. It occurs on all architectures, so it is not one
of those checks that only apply to x64-based platforms.
However, it‘s important to note that this check applies only to
plug-and-play devices.

This signature check only involves the catalog (CAT) file. If

you do not have a CAT file, or if your CAT file is not signed,
you get the dreaded red-flag "unsigned driver" warning. If
your CAT is signed with a non-WHQL signature, you get a
"do you trust this publisher" warning shown in Figure 1 (on Figure 1—Trusted Publisher Warning

Custom Software Development—Experience, Expertise

...and a Guarantee
In times like these, you can’t afford to hire a fly-by-night Windows driver developer. The money you think you’ll
save in hiring inexpensive help by-the-hour, will disappear once you realize this trial and error method of
development has turned your time and materials project into a lengthy ―mopping up‖ exercise...long after your
contract programmer is gone.

Consider the advantages of working with OSR. If we can be of value-add to your project, we’ll tell you. If we
can’t, we’ll tell you that too. You deserve (and should demand) definitive expertise. You shouldn't pay for
inexperienced devs to attempt to develop your solution. What you need is fixed-price solutions with guaranteed
results. Contact the OSR Sales team at [email protected] to discuss your next project.
 Page 13
/v – verbose output
Check This Out... /ac – to select the cross certificate
/sha1 – to indentify your code signing certificate‘s thumbprint
/t to embed a timestamp so the signature does not expire
(Continued from page 12)
Where to Go from Here?
again. Even if your driver package is correctly installed, the Hopefully, this short article has help clarify some of the more
driver will not be loaded unless it passes the KMCS check. confusing aspects of driver signing.
Attaching a kernel debugger disables the KMCS check.
Microsoft has published a thorough hands-on tutorial
The KMCS signature can be present either in your SYS file, describing the KMCS process on the WHDC web site (see [2]
or in your CAT file. Non-plug-and-play drivers usually won't further below). By carefully following the steps outlined in
have a CAT file, so the SYS file must be signed. If you sign this document, you‘re guaranteed to be able to successfully
the CAT file only, you cannot replace the SYS file (for sign your driver. Of course, this assumes you have the correct
example, during debugging). The CAT file contains a certificate (as described above)!
checksum of the matching SYS file. For this reason, I usually
sign both the CAT and the SYS. For drivers used during [1] http://www.microsoft.com/whdc/driver/install/drvsign/
system boot, Microsoft strongly suggests that you sign the crosscert.mspx
SYS file, because it can take too long to locate the [2] http://www.microsoft.com/whdc/driver/install/drvsign/
corresponding CAT file with the limited disk driver stack used kmcs-walkthrough.mspx
during boot.

Kernel-mode code signing check requires a very specific type

of signature, and that the driver be signed in a very particular
way. Specifically: Tim Roberts ([email protected]) is a developer with
Providenza & Boekelheide, Inc., a consulting firm in
You must use a valid Class 3 Code Signing beautiful Tigard, Oregon. He's been programming for 35
Certificate – an ordinary certificate, such as used for years, and living in the Windows world since version 3.0.
SSL, is not acceptable
Tim spends way too much time with computers.
The Class 3 Code Signing Certificate must be issued
by a Microsoft-approved Certification Authority (see
[1] at the conclusion of the article for a list)
At the time of signing, you must reference the ―cross-
certificate‖ for the specific Certification Authority
that issued the Class 3 Code Signing Certificate
you‘re using.
Why An OSR Seminar?
When you sign your SYS and/or CAT file, you must select
both your code-signing certificate, and the associated cross Well, some might say that if you haven’t received
certificate (using the /ac parameter of signtool). This training from OSR, you just don’t know all the reasons
produces a certificate "chain" that ends at the "Microsoft Code why you should take an OSR seminar. As a recent
Verification Root". That certificate chain must be present
either in the SYS file or the CAT file or both, but wherever it attendee of our WDM Driver Lab put it:
exists, it must be a complete chain with cross-certificate.
―I heard about the WDM seminar and OSR from my
My "signtool" command line looks like this: teammates at [company]. When I mentioned driver
training, everyone said, ―OSR‖ without hesitating. We
signtool sign appreciate the courses and detailed coverage of the
/v
/ac MSVC-GlobalSign.cer topics included.‖
/sha1 xxxxxxxxxx
/t http://timestamp.verisign.com/scripts/timestamp.dll We know Windows from the source code out. We’ve
objfre_wlh_amd64\amd64\mydriver.sys been teaching devs around the world how to write
Windows drivers for more than 15 years. Is it time to
Where:
give us a try and see for yourself? Visit:
www.osr.com/seminars.
Page 14

EResources Close Cousin

Implementing Reader/Writer Locks in User Mode
B eing primarily kernel mode programmers at OSR, we
have gotten into the habit of using Reader/Writer locks.
These locks, known as Executive Resources (EResources),
EResource will fail. An EResource can be acquired by a
thread recursively as long as the acquisition type (i.e. shared
or exclusive) is compatible with the current acquisition type of
are great because they can be used at IRQLs less than the lock. What this means is that if a thread holds a lock
DISPATCH_LEVEL and allow shared readers and exclusive exclusive and tries to require the lock shared, the acquisition
writers. In addition they also allow a thread recursive access, will be granted. If, however, the thread holds the lock shared
as long as the acquisition type is compatible with the lock‘s and tries to reacquire the lock exclusive, a deadlock will occur
current acquisition level. (as it does in kernel mode) since the desired acquisition type is
incompatible with the currently held acquisition type.
That being said, it has always baffled us, at OSR, why Threads other than the current thread can attempt to acquire
something similar to EResources was never available in user the EResource also and will only be allowed access if the
mode. We recently had a need to move a piece of software mode of the request is compatible with the current state of the
that previously ran in kernel mode to user mode and that lock. So, for example, if Thread A owns the EResource
software happened to use EResources as its primary locking shared, Thread B will also be allowed access to the
model. Our choices were either to reengineer the locking EResource if it attempts to acquire it shared. If it tries to
model used by the code or to implement EResource acquire the EResource exclusively, access will be denied (i.e.
semantics in user mode. Our choice was to implement it will either be blocked if the user specified to wait for the
EResource semantics in user mode and this article documents EResource, or a status will be returned indicating that the
our implementation. EResource was not acquired).

Overview Other features of EResources are:

We‘re going to assume that readers of this article are familiar
with EResources, but realize that some people are not, so for If an EResource is held exclusively, it can be
the unfamiliar here‘s an overview….. converted to shared access
Users can check to see if an EResource is currently
As we mentioned previously an EResource is a reader/writer held exclusively or shared
lock. This lock can be acquired in 1 of 2 modes, either shared Users can check to retrieve the number of Shared
or exclusive As the modes imply this lock can be held Waiters or Exclusive Waiters for an EResource.
simultaneously by multiple readers (EResource acquired in EResources must be initialized before being used
shared mode) or by one writer (EResource acquired in and must be deleted when they are finished being
exclusive mode). used.

When a thread attempts to acquire an EResource, it can Now that the basics have been explained, let‘s see the
specify whether or not it wants to wait if the specified functions available to us.
EResource is not immediately available. If it elects to wait,
the thread will be put into a wait state until such time as the
lock becomes available, otherwise the acquisition of the (Continued on page 15)

Windows Internals & Software Drivers

Attention security researchers, government contractors and engineers involved in security and threat analysis
modeling! The next offering of our Windows Internals & Software Drivers seminar has been scheduled.

15-19 November, Santa Clara, CA

For a look at the outline, pricing and registration information, visit www.osr.com/swdrivers.html.
 Page 15
has a set of ―C‖ interfaces that we listed in Table 1 that are
EResources... defined in the include file ―UMERESOURCE.H‖. These
interfaces work on an OSRERESOURCE structure and are
backed by a ―C++‖ OSREResource class which is defined in
(Continued from page 14) ―ERESOURCE.H‖. The OSREResource class and
associated OSRERESOURCE_STATE structure are defined
in Figure 2 (page 26).
Functions
This class contains:
In order to use OSRERESOURCEs (our version of
ERESOURCEs) there are a set of functions that the user has m_CriticalSection which is used by the code to
available to them. The functions are listed in Figure 1. synchronize access to the OSREResource data
structures, ensuring that only one thread is accessing
To use an OSRERESOURCE the caller would call them at one time.
OSRInitializeResource in some initialization routine, and m_LockStateMap table which maps a Thread
then call one of the OSRAcquireResourceXXXLite identifier to its resource state (i.e. the mode in which
functions where XXX is either shared or exclusive. After it acquired the lock). This resource state is an
doing some work it would then call OsrReleaseResource to OSRERESOURCE_STATE structure, defined in
release the held resource. Finally, when done with the Table 2, which is composed of 2 fields, State and
resource, the user would call OSRDeleteResourceLite to AcquireCount. State is used to keep track of how
delete the resource. the resource was acquired, ACQUIRED_NONE,
ACQUIRED_SHARED, or ACQUIRED_
So as you can see OSRERESOURCEs are pretty easy to use. EXCLUSIVE, and AcquireCount is used to keep
Since this is a user mode library free for anyone to use, I guess track of the number of times the thread has acquired
it would be a good thing to discuss what is going on the resource (remember that resources can be
underneath the covers of the OSRERESOURCE acquired recursively).
implementation, so that is what we‘ll discuss next. m_WaitEventHandle – the handle to the notification
event that all threads wait on when trying to acquire
Underneath the Covers of OSRERESOURCEs the resource.
In this section we will discuss how we implemented m_SharedCount – number of shared holders of the
OSRERESOURCEs. Our implementation creates a Dynamic
Link Library called ―UMERESOURCE.DLL‖ This DLL (Continued on page 26)

Function Name: Purpose:

OSRInitializeResource Initializes a resource for use. If not done, the resource is unusable
OSRDeleteResourceLite Deletes an resource, the resource must not be held by any thread when deleted.
OSRAcquireResourceExclusiveLite Attempts to acquire the resource exclusively
OSRAcquireResourceSharedLite Attempts to acquire the resource shared
OSRAcquireSharedStarveExclusive Attempts to acquire the resource shared and starves any thread waiting to get the
resource exclusively
OSRIsResourceAcquiredExclusiveLite Returns TRUE if the resource is held exclusively
OSRIsResourceAcquiredSharedLite Returns TRUE if the resource is held shared or exclusive
OSRReleaseResourceLite Releases the held resource
OSRGetCurrentResourceThread Returns the current thread identifier
OSRConvertExclusiveToSharedLite Converts a currently held exclusive reservation into a shared reservation
OSRGetSharedWaiterCount Returns the number of threads waiting for shared access
OSRGetExclusiveWaiterCount Returns the number of threads waiting for exclusive access
OSRReleaseResourceForThreadLite Release the resource on behalf of the input thread identifier
OSRReleaseResource Release the resource held by the current thread

Figure 1—OSRERESOURCE Functions

Page 16

WDK Community Bug Bash Contest 2010

Help Eradicate Bugs!
Prizes?Yes, prizes.
Specifically, every valid submission of a
bug against the WDK earns an award. Every valid
submission. In addition, all valid submissions are
then eligible to win one of two First Prizes, or one
of three Second Prizes, as judged by the organizers.

Soooo, how does it all work?

1. Find bugs.

2. Report the bug(s). You‘ve got six months, but

remember, only the first submission of the same
bug will be valid! While you‘re only eligible for
one award, keep submitting to better your
chances to win a First or Second place prize!

3. We‘ll validate submissions as they come in. If

rejected, we‘ll let you know. If valid, we‘ll be
contacting you to arrange for your award to be
W e can‘t be more thrilled to be organizing
another event to help bring the Windows
driver development community together for a good
shipped.

4. We‘ll submit your bug directly into Microsoft‘s

cause—in this case, to help eradicate bugs in the bug-tracking system. Feel free to check the Bug
Windows Driver Kit (WDK). Thus, on behalf of Bash Bug List from time to time for updates.
sponsors OSR, and ITT Defense & Information
Solutions, and in cooperation with Microsoft, we‘re 5. At the close of the contest, we‘ll get together
pleased to announce the WDK Community Bug and vote on the submissions—and award the
Bash Contest! First and Second Prizes.

Purpose: help fix bugs in the Windows Where do I start?

Driver Kit (WDK) How about at the Bug Bash Homepage
Bugs? Yes, bugs. Kit bugs, doc bugs, sample bugs, (http://www.osronline.com/page.cfm?
utility bugs—if it‘s provided as part of the WDK name=bugbash). From there, you‘ll want to make
we (and our friends on the WDK team at Microsoft) sure to check out the contest rules, FAQ, Bug List
want to know about it. This is your chance. You and of course the Bug Submission pages.
have to work with this toolset for your job. Why not
take a bit of effort to report bugs so that they can Happy Hunting!
get fixed for you and future Windows kernel devs.
Earning goodwill not enough of an incentive? How
about…
 Page 17

WDK Community Bug Bash Contest 2010

Get Your Stuff!
E veryone who submits a valid bug, gets cool
stuff. Really!
Even BETTER: If your bug is chosen to be among
THE BEST that were submitted, you can win
yourself some VERY cool prizes: an HP Mini-210
If you‘re the first to report a bug in a sample driver Netbook, a copy of Visual Studio Ultimate with
that‘s been annoying you for ages, you‘ll get MSDN Subscription, or an Apple iPod Touch.
yourself a Technical Award package with a 4GB
USB Flash Drive, a Windows 7 Keychain/Bottle The idea behind the WDK Community Bug Bash is
Opener, and a cool WDK Community Bug Bash T- that everybody benefits. The community gets
Shirt. involved, bugs get fixed, and we all have a good
time.
Or, say there‘s some hideous grammatical error in
the WDK Docs that you noticed a while See the Bug Bash Homepage at http://
back. Report it, and get an Editorial Award www.osronline.com/page.cfm?name=bugbash
package with a 1GB USB Flash Drive, a WDK for details, official contest rules and to submit your
Community Bug Bash Mouse Pad and one of those bugs. Now… get to work submitting those bugs!
cool Windows 7 Keychain/Bottle Openers.

Report both a technical bug AND an editorial bug,

and earn yourself BOTH awards!

The WDK Community Bug Bash Contest is sponsored by:

In cooperation with:
 Page 18
configuration is shown in Figure 1 (note that the calling of the
Writing Filters... redirectors for the name resolution step is not depicted).

The introduction of DFS in this picture complicates things, as

(Continued from page 1) shown in Figure 2 (next page). With DFS, things begin
identically to the non-DFS case behavior, with the open of
\\DfsServer\DfsRoot first arriving at MUP‘s dispatch entry
The Players: DFS Client, MUP, and Network point for create operations, MupCreate. MUP then does a
Redirectors probe of the server to determine if it is a DFS server (details
In order to understand the complexities of the problems we‘ve of this probe are outside of the scope of this article), and
experienced, we‘ll need to first get a feel for the way things receives a response that indicates that \\DfsServer does in fact
work without any filters involved. During the boot process, support DFS. At this point, MUP then reparses this open of
special drivers called network redirectors (typically either the DFS root back to its own driver, by replacing the name in
monolithic network file system or mini-redirector drivers) the file object with \Device\WinDfs\Root\DfsServer\DfsShare
register with the Multiple UNC Provider (MUP) driver by and returning STATUS_REPARSE. \Device\WinDfs\Root
calling FsRtlRegisterUncProvider (or FsRtlRegisterUnc happens to be another device object that the MUP driver
ProviderEx in Vista and later), providing the name of their creates to handle opens targeted to DFS servers.
redirector device object as part of that registration process. For
example, in the case of the SMB mini-redirector driver, the As a result of this reparse, processing of the open restarts back
name provided to MUP is \Device\LanmanRedirector. Other at the MupCreate function. However, this time the create is
popular redirector device object names are targeted to the DFS device object and proceeds down a
\Device\WebDavRedirector and \Device\NetWareRedirector, different code path in MUP that handles DFS related activity.
for WebDAV and Novell Netware support, respectively. At the start of processing, MUP finds the appropriate
redirector to use to communicate with this DFS server and
Upon receipt of this request, MUP uses the device object to prepares to send this create request to the DFS server via the
extract and store the name of the redirector device as well as a redirector. Due to the fact that DFS runs over SMB, normally
pointer to the redirector device object. Later, when a user the redirector device chosen will be \Device\Lanman
attempts to open a name in the form of \\FooServer\Share, Redirector. If this operation completes successfully, MUP
MUP begins calling each of the redirector drivers to determine completes the create IRP back to the original caller.
if this path represents a server that the redirector supports. If
the redirector indicates that the server is indeed one of its own, Note that the end result of this is different than in the non-DFS
then the name resolution stops and MUP reparses the open case. In the non-DFS case, the create operation was reparsed
(using the standard STATUS_REPARSE method for to the appropriate redirector. In the DFS case, the create is
Windows) to the appropriate redirector device object. This reparsed back to MUP, passed to the redirector, and then
restarts the create operation targeted at the appropriate completed. Due to these differences, in the non-DFS case the
redirector device and MUP, with its job done at this point, is resulting file object points to a redirector device object and in
no longer in the picture. A simplified depiction of this the DFS case the resulting file object points to a MUP device
object.

1) NtCreateFile(“\Device\Mup\FooServer\Share”) This Is Where It Starts To Get

Messy...
\FileSystem\Mup \Filesystem\MrxSmb If you had a hard time following the above,
it‘s going to get a bit worse. As it turns out,
LanmanRedirector must have knowledge
\Device\ 4) IoCompleteRequest of the fact that the create operation is
\Device\ LanmanRedirector
\Device\
MUP
WinDfs\ targeted to a DFS server. Unfortunately,
Root
the IRP_MJ_CREATE IRP structure is
already entirely packed and there‘s no
room for custom parameters. To
3) TDI accommodate the additional information
the DFS developers had to find a creative
2) “\Device\LanmanRedirector\FooServer\Share” way to pass the extra information.
STATUS_REPARSE
To achieve this the DFS and redirector
designers decided to use some fields of the
\\FooServer\Share file object passed along with the create
operation. Under normal circumstances,
Figure 1—The Lay of the Land, Sans DFS (Continued on page 19)
 Page 19

Writing Filters... MUP then attempts to open \\DfsServer\DfsRoot\DfsLink on

the DFS server via the redirector. Because this is not an actual
share on the server but a link to another server, the DFS server
(Continued from page 18) will return a special status of STATUS_PATH_NOT_
COVERED to the client. MUP responds to this special status
the FsContext and FsContext2 fields of the file object are set by sending what is called a, ―DFS referral packet‖ via the
to NULL when the I/O Manager sends the create operation to redirector to the server for the failing path. The response to
the file system. Before completion of the create IRP, it is the this referral packet is the actual location of the DFS link, for
job of the file system to set the FsContext field to a structure example \\FooServer\Share.
that is unique to the stream and the FsContext2 field to a
structure that is unique to this open instance of the stream. MUP is now at the end of its name resolution process and it
knows the real path that is to be used for this create operation.
Based on the knowledge that nothing is usually put there However, MUP is also now back to square one in processing
before calling the file system, MUP stores a magic value of the create request. In other words, MUP must process the open
0xFF444653 (0xFF‖MUP‖) into FsContext2 and a pointer to a of \\FooServer\Share as if this is what the user initially
data structure in FsContext before passing the request to the opened. This is due to the fact that MUP is not sure which
redirector. Attempting to find more details about the magic redirector to use to communicate with this server/share
value brings up practically nothing, though you can find the combination.
following definitions in the LanmanRedirector sources that
used to be provided with the WDK: Therefore, at this point MUP sends the create IRP back to
itself at MupCreate. Processing at this point proceeds exactly
#define DFS_OPEN_CONTEXT as it did in the non-DFS case, with MUP finding the
0xFF444653
appropriate redirector for this open, pre-pending the redirector
typedef struct _DFS_NAME_CONTEXT_ { device name to the path, and returning STATUS_REPARSE.
UNICODE_STRING UNCFileName;
LONG NameContextType; For example, in this case the resulting name might end up as,
ULONG Flags; \Device\LanmanRedirector\FooServer\Share.
} DFS_NAME_CONTEXT, *PDFS_NAME_CONTEXT;

(Continued on page 20)

And some further details on their usage
throughout the source of the sample. 1) NtCreateFile(“\Device\Mup\DfsServer\DfsRoot”)

As we‘ll soon learn, this approach creates

issues for file system filters that need to \FileSystem\Mup
\Filesystem\MrxSmb
filter the SMB redirector. However, before
we point out the complexities this creates 5) IoCompleteRequest
for filters, let‘s continue our investigation
of how this all works without filters \Device\
\Device\ \Device\
WinDfs\ LanmanRedirector
involved. MUP
Root 3) IoCallDriver

What About Accessing a Link?

So far, we‘ve only seen accessing a
standalone UNC share and the root of a
4) TDI
DFS server. Accessing a link of a DFS
server complicates this situation even
further and brings to light another feature 2) “\Device\WinDfs\Root\DfsServer\DfsRoot”
of the MUP DFS implementation. STATUS_REPARSE

When a user attempts to open

\\DfsServer\DfsRoot\DfsLink, everything
proceeds as it did previously. The create \\DfsServer\DfsRoot
first arrives at MUP and MUP determines
that this is a DFS server, so MUP reparses
the create back to itself at its DFS root
device object. DFS then finds the
appropriate redirector device for \\FooServer\Share
\\DfsServer\DfsRoot, which will again be
\Device\LanmanRedirector. Figure 2—And Now, with DFS
 Page 20
a couple of fields of the file object. In order for this design to
Writing Filters... work in all cases, those fields must be propagated to whatever
file object is sent to the redirector. This is a monster problem
for a filter that wants to perform its own open of the file or
(Continued from page 19) directory with IoCreateFileSpecifyDeviceObjectHint or
FltCreateFile. These APIs generate their own file objects
You would believe that at this point MUP would return with no opportunity for the caller to modify the resulting file
STATUS_REPARSE back to the I/O Manager and the object before it is sent to the target device. Thus, a filter
resulting file object would point to \Device\Lanman performing these types of operations breaks the chain between
Redirector, just as in the non-DFS case that we first looked at. DFS and the redirector, leading to unexpected results from the
However, that is not at all what happens. Instead, MUP create.
internally handles the reparse processing and forwards the
request to the appropriate redirector device object. Upon This is also a case where writing your filter as a mini-filter
successful completion, MUP will then (finally) complete the will create surprising complications. For reasons unknown to
user request with success, resulting in the file object pointing us, before calling the mini-filters in the create path, Filter
to the MUP DFS device object. Manager will set the FsContext field of the file object to
NULL, even if it was not NULL on entry to the Filter
Why This Creates Additional Complexity for Filters Manager filter device. Once all of the mini-filters are called,
Why does filtering DFS turn out to be such a big pain for filter the value is restored before calling the underlying FSD with
writers? the request. Thus, even if you could figure out a way to get the
fields set appropriately in the file object, the values are hidden
The first issue that you‘ll have when dealing with DFS is load from your filter during pre-create processing.
ordering. When the redirector drivers register with MUP via
FsRtlRegisterUncProvider, MUP opens the target device And Don’t Ever Try to Return STATUS_
name and caches the returned device object. Thus, if your REPARSE…
filter is not instantiated on the redirector device at the time of If you noticed above, MUP heavily relies on
this open your filter will be bypassed for the communication STATUS_REPARSE in order to perform its work. We snuck
between DFS and the redirector. Mini-filters luck out a bit something by you in a previous section as well when we said:
here because you just need to make sure that Filter Manager is
loaded and attached early, this ensures that the Filter Manager You would believe that at this point MUP would return
device object will be in the call chain and then you can just STATUS_REPARSE back to the I/O Manager and the
attach your filter instance later. Usual load ordering rules can resulting file object would point to \Device\Lanman
be used to make sure that fltmgr.sys is loaded early enough in Redirector, just as in the non-DFS case. However, that’s
the boot process. Prior to Vista, in order to guarantee that not at all what happens. Instead, MUP internally
Filter Manager will actually attach early in the boot process handles the reparse processing and forwards the request
make sure that the Filter Manager AttachWhenLoaded to the appropriate redirector device object.
registry value is correctly set to 1.
What we omitted in this explanation how MUP decides to
With your filter properly inserted between MUP and the handle this particular instance of STATUS_REPARSE itself.
redirector you have a new set of issues to handle. Remember
that DFS and the redirector have a secret handshake stuffed in (Continued on page 21)

Design & Code Reviews

Have a great product design, but looking for extra security to validate internal operations before bringing it to
your board of directors? Or perhaps you’re in the late stages of development of your driver and are looking to
have an expert pour over the code to ensure stability and robustness before release to your client base.

A small investment in time and money in either service can ―save face‖ in front of those who will be contributing
to your bottom line. OSR has worked with both startups and multi-national behemoths. Consider what a team of
internals, device driver and file system experts can do for you. Contact OSR Sales — [email protected].
 Page 21
Not All Stories Have a Happy Ending
Writing Filters... Unfortunately, we continue fighting with DFS to this day and
have open bugs for configurations that do not work and, quite
possibly, might never be made to work. Complicating things
(Continued from page 20)
for us is that MUP and DFS have been redesigned on
Believe it or not, what MUP actually does is dig into the Windows Vista and later such that filters are no longer sit in
device object to which that the create IRP was sent, finds the between MUP and the redirector. Because the design has
driver object, and captures the driver object name. This driver moved forward, we have no chance of getting any changes
object name is then compared to the hardcoded value, made to legacy platforms. Until every last one of our clients
―\Filesystem\Mup‖. If the names match, MUP knows that he has upgraded to Vista and later, we‘re going to be stuck trying
just reparsed the create back to himself and can handle it
to hammer filter drivers into an architecture that clearly wasn‘t
internally. However, if the names do not match then MUP
allows the reparse to travel back to the I/O Manager. designed to interact with other components in the system.

This means that if you reparse a DFS open to something like a

shadow stack (a technique we have used in some of our
layered file system work), DFS will entirely step out of the
way of the create processing. The result of this is that DFS
create operations no longer pass through the DFS client code,
which can again lead to unexpected results.

Note that this also prevents one from attaching a filter to

\Device\Mup, which you might want to do in an attempt to
avoid sitting between the DFS code and the redirector.
Because this specially-handled create IRP will be sent to your
filter device object instead of directly to the MUP device
object, the driver name check will fail due to the fact that the
driver name in the target device will be your filter driver‘s
name. The end result will be a broken DFS client that can no
longer communicate with any DFS servers.

BSOD? Unexplained Hang? - Use OSR’s Problem Analysis Service

An opportunity to ―make your problem go away‖ for $2,499
You may be a dev working on an actual driver, you may be on a tiered product support team, or you may be in
corporate IT, but the commonality you have is that—plain and simple—analyzing Windows OS crashes and hangs
is just plain troublesome. We see it all the time from customers, and from folks trolling the various OSR forums
looking for help.

If you have to debug and analyze such problems all the time, there are fundamentals and techniques you can
learn with training and experience. This is exactly what we offer as part of our 5-day Kernel Debugging & Crash
Analysis seminar. But, if you aren’t charged with regularly root-causing such problems and/or don’t have the
luxury of time to build up knowledge and a repertoire of techniques, having an expert set of eyes do the work for
you, and in a timely fashion, could be a big win. Think about that hang that has been plaguing your development
staff for weeks. Or that customer-reported BSOD that has them ready to walk out the door. Think it’s time to
escalate?

This is what OSR offers as part of its Problem Analysis Service. For a $2,499 fixed-price, submit your issue and
receive initial triage within 2 business days and a report within a week!

More information on this service, including a sample Problem Analysis Report is available at
www.osr.com/analyze.html or by contacting OSR’s Sales Department at [email protected].
 Page 22
an SRB SenseInfoBuffer has been specified, the driver
Getting Better... should be filling it correctly.

Figure 5 (page 23), shows the routine ProcessScsiCommand

(Continued from page 7) Error that sets the sense data that is returned when a
command error occurs. This data will be returned if the input
SRB contains a SenseInfoBuffer. Of course, your
If there are only four bytes actually valid, then the "Mode implementation of this may vary depending on what your
Data Length" should be set to 3 (i.e. 3 bytes are valid driver needs to accomplish.
following the Mode Data Length field) since the Mode Data
Length field is not included in the count. Notice that the driver ―OR‘d‖ SRB_STATUS_AUTOSENSE
_VALID to SRB_STATUS_ERROR to indicate to the caller
Prevent/Allow Medium Handling that the SenseInfoBuffer contains valid data.
When the Storport Virtual Miniport received a SCSI operation
that it did not handle, the driver set the SRB status to When you look at the new code, you will see that the driver
SRB_STATUS_ERROR and completed the request. While has been modified to call the routine in Figure 5 whenever it
this isn‘t necessarily wrong, the Storport Virtual Miniport receives a command that it does not handle.
Driver is trying to emulate a device capable of handling
CDBs. Therefore, the Miniport should really be responding as Adapter Type
a real conformant device would. What this means is that, if One interesting thing that Mike pointed out was that the
AdapterInterfaceType for the Virtual
Storport Driver was showing up as
―Fibre‖. This AdapterInterfaceType
i s a f i e l d i n t h e
PORT_CONFIGURATION_
INFORMATION block that is set up
in our HwFindAdapter routine.
Figuring that the field was not
initialized the driver was modified to
set the field to PNPBus. To our
surprise setting this field had no effect.
Since we were stumped as to why, we
contacted James Antognini of
Microso ft WDK support who
informed us that Storport gets the
Figure 3—SCSIOP_INQUIRY CDB setting from the field
―\Registry\Machine\System\Current
ControlSet\Services\<driver
name>\Parmeters\BusType‖. This
field is a 32-bit REG_DWORD. Thus
we modified the INF file to set the
value of this key to 0xE
(BusTypeVirtual) and it worked. If
you don‘t set a value, it defaults to
BusTypeFibre.

Crash
Alas, during his testing Mike managed
to get the driver to crash. This
problem has been fixed in the new
code drop. It turns out that the code
in CreateConnection that added a
new connection to the list of current
connections did not synchronize
access to the ConnectionList. Thus
if you added and removed connections

(Continued on page 23)

Figure 4—busTRACE Mode Sense Output
 Page 23
with the SCSI specification and thus make it act more
Getting Better... properly like a physical device. This compliance helps
increase the driver's chance of interoperating with all sorts of
software, and that's a certainly good thing.
(Continued from page 22)
Sample code to OSR’s Virtual Storport Miniport Driver can be
quickly enough you would cause the list to become corrupted. downloaded from http://www.osronline.com/OsrDown.cfm/
The fix was simple; just hold the ConnectionListLock around osrvmmemsample.zip?name=osrvmmemsample.zip&id=558
the adding of the new entry to the list in
―CreateConnection‖.

Summary
In our article series Writing a Virtual Storport Miniport Driver
we described key aspects of the architecture, design, and OSR would like to thank Mike Berhan of busTRACE Technologies
implementation of this type of driver. We've built on that for taking the time to test and analyze the OSR Virtual Storport
series, and made the driver more compliant, with the tweaks Miniport Driver and for providing us with a copy of busTRACE so
described in this article. And while these tweaks might not be that we could find and fix the issues discussed in this article.
absolutely critical, they do make the driver more compliant Without the help of Mike and his fine tool, we wouldn't have
known about the driver's failure to conform to the specifications.
NTSTATUS ProcessScsiCommandError(PSCSI_REQUEST_BLOCK PSrb)
{
NTSTATUS status = STATUS_SUCCESS;
if(PSrb->SenseInfoBuffer && PSrb->SenseInfoBufferLength) {
PSENSE_DATA pSense = (PSENSE_DATA) PSrb->SenseInfoBuffer;
RtlZeroMemory(pSense,PSrb->SenseInfoBufferLength);
pSense->ErrorCode = 0x70;
pSense->Valid = 0;
pSense->SenseKey = SCSI_SENSE_ILLEGAL_REQUEST;
pSense->AdditionalSenseLength = 0x15;
pSense->AdditionalSenseCode = SCSI_ADSENSE_ILLEGAL_COMMAND;
pSense->AdditionalSenseCodeQualifier = 0;
PSrb->ScsiStatus = SCSISTAT_CHECK_CONDITION;
PSrb->SrbStatus = SRB_STATUS_AUTOSENSE_VALID | SRB_STATUS_ERROR;
} else {
status = STATUS_UNSUCCESSFUL;
PSrb->SrbStatus = SRB_STATUS_ERROR;
}
return status;
}

Figure 5—ProcessScsiCommandError

OSR: Just Ask

Ask us to cogently explain the Windows I/O Manager to a couple dozen Windows developers of varied background
and experience. Ask us how to address latency issues in a given design of a driver. Ask us to look at a post-
mortem system crash, determine its root cause, and suggest a fix. Ask us to design and implement a solution that
plays well with Windows, even if it has no business being a Windows solution in the first place.

Ask us to perform any of the above activities for your company, and you will be pleased with the definitive answer
or result we provide. Ask us almost anything about user-mode development, Linux or where the world economy
will be in five years, and what you will get is an opinion, laughter or both.

So, the only question WE have is, ―How can we help you?‖ Contact: [email protected]
 Page 24
//
Debugger Extensions... hr = debugSymbols->GetFieldOffset
(module,
typeId,
"CurrentThread",
&fieldOffset);
(Continued from page 9)
if (hr != S_OK) {
// debugSymbols->Release();
hr = debugControl->Evaluate("2 + 2", return hr;
DEBUG_VALUE_INT32, }
&result,
NULL); debugControl->Output
(DEBUG_OUTCTL_ALL_CLIENTS,
if (hr != S_OK) { "Offset of CurrentThread is %d\n",
debugControl->Release(); fieldOffset);
return hr;
} debugSymbols->Release();
return S_OK;
debugControl->Output(DEBUG_OUTCTL_ALL_CLIENTS, }
"Result is %d\n",
result.I32);
Dealing with 32-bit vs. 64-bit
// Note that when you‘re writing a debugger extension
// Done with this. command, your code is always running on the host machine.
//
debugControl->Release(); Also note that the pointer size of the host machine does not
necessarily match the pointer size of the target machine, due
return S_OK;
} to the fact that 32-bit hosts can debug 64-bit targets and vice
versa. In order to deal with this, debugger extensions treat all
Hopefully the steps followed are fairly straightforward at this addresses from the target as 64-bit values. Thus you‘ll note
point. We‘ve taken the passed-in client object, created an that the DbgEng APIs express pointer addresses as ULONG64
instance of IDebugControl, and then executed some methods values. Any 32-bit value used in your extension command
on it to perform actions. must be sign extended out to a full 64-bit value.

To further drive home the pattern, we can see how we‘d get Alternative Debugger Extension Interfaces
the offset of a field of a data structure. For that, we need an To add to the confusion when it comes to writing a debugger
instance of IDebugSymbols: extension, there are two alternative interfaces that you can use
to write your debugger extensions. The first is the WdbgExts
HRESULT CALLBACK interface, which is the legacy debugger extension interface
myothercommand(PDEBUG_CLIENT4 Client, PCSTR args)
{ that existed before DbgEng came around. This interface is still
PDEBUG_SYMBOLS debugSymbols; available in the newest versions of the debugger, however it
HRESULT hr; has two drawbacks. First, it is not the forward moving API
ULONG fieldOffset;
ULONG typeId; thus it is frozen in time and will provide no new features.
ULONG64 module; Second, this interface does not have any support for writing
UNREFERENCED_PARAMETER(args); standalone applications, thus it won‘t port to any kind of
automated analysis tool that you might write. If you‘re
// interested in learning more about the legacy extension model,
// Let's find the offset of the CurrentThread see this article on OSR Online.
// field of the PRCB
//
hr = Client->QueryInterface The other interface available to you is the EngExtCpp
(__uuidof(IDebugSymbols), interface. This is actually just a C++ wrapper library around
(void **)&debugSymbols);
DbgEng to simplify common tasks such as manipulating typed
if (hr != S_OK) { data. Unlike WdbgExts, this is a fully supported interface and
return hr;
} can be used alongside the direct DbgEng calls that we‘ve
discussed in this article.
//
// We need the "type identifier" and module
// containing the symbol that we're interested Go forth!
// in.
// Hopefully we‘ve been able to clear up the cloud of mystery
hr = debugSymbols->GetSymbolTypeId("nt!_KPRCB", that hangs over writing debugger extensions and have set you
&typeId, on a journey of creating your own.
&module);
if (hr != S_OK) { Source to a debug extension (!uniqstack) can be downloaded
debugSymbols->Release();
return hr; from: http://www.osronline.com/OsrDown.cfm/apexts.zip?
}
name=apexts.zip&id=559
//
// Now we can get the offset.
 Page 25
access directory (e.g., ―C:\Windows‖) the performance of the
Getting Away... system can become so poor that your driver will be deemed
―unusable‖ as a result. In such situations we‘ve been forced to
add caching to the size correction process. Even that will not
(Continued from page 11) make the first enumeration of a directory ―fast‖ however.
Directory size correction is an area we will leave for a future
programs. In Figure 2 we show what appears to be a pair of document and discussion, because it is itself surprisingly
files – one of these is 16 bytes long and the other is 54 bytes complicated.
long. In fact, there really is only one file as we have created a
hard link. If we access the file via the hard link, we can see Summary
the size change (in Figure 3). In future installments as we build our sample, we will focus
on the mechanisms necessary to manage our shadow file
This is because NTFS actually stores the file sizes within the objects, contexts and section object pointer structures.
directory entry – but when a file contains hard links, only the Stay tuned!
link that is actually used is updated. Despite this, applications
rely upon correct size information and in some cases they use
it for validation.

You can address this by adding ―directory size correction‖

however the overhead for directory size correction can be
surprisingly harsh, particularly if it requires opening the file to
determine what the correct size should be. OSR’s DMK: ―File and Folder‖
For example, we have found in our own work that adding any Encryption for Windows
additional I/O overhead in directory enumeration can have
dramatic performance impact. When this occurs in a high Several commercially shipping products are a
testament to the success of OSR’s most recent
development toolkit, the Data Modification Kit.

With the hassle of developing transparent file

encryption solutions for Windows on the rise, why
not work with a codebase and an industry-
recognized company to implement your encryption or
other data-modifying file system solution? Your
competition is benefitting from this huge, competitive
advantage, why aren’t you?

Visit www.osr.com/dmk.html, and/or contact OSR to

Figure 2 discuss the DMK and your needs.

Phone: +1 603.595.6500 Email: [email protected]

Subscribe to The NT Insider

Digital Edition
If you’re a new to The NT Insider (as in, the link to
this issue was forwarded to you), you can subscribe
at:

http://www.osronline.com/custom.cfm?name=login_j
oinok.cfm
Figure 3
 Page 26
m_SharedWaitersCount – number of threads
EResources... waiting to acquire the resource in shared mode.
m_ExclusiveWaitersCount – number of threads
waiting to acquire the resource in exclusive mode.
(Continued from page 15) m_Initialized – Boolean indicating whether or not
the resource was initialized by a call to
resource. Since threads can hold the resource OSRInitializeResource.
recursively this does not necessarily reflect the
number of threads holding the resource The code which implements this class uses the
simultaneously. m_CriticalSection to synchronize access to the rest of the
m_StarveExclusiveCount – number of threads OSREResource data, while the m_LockStateMap is used to
waiting to acquire the resource in shared mode, but keep track of each holder of the resource, the mode of
want to starve exclusive waiters. acquisition (State), and the number of times that the
m_ExclusiveCount – number of exclusive holders of acquisition was made since a holder could acquire the
the resource, which is usually one, but since a thread resource recursively (AcquireCount). The data members‘
can acquire resource lock recursively, this count m_SharedCount and m_ExclusiveCount are also used to
would reflect that. keep track of the acquisition counts.

// This structure keeps track of the state of the resource. We need For any thread which tries to
// to know how the resource was acquired and how many times it was
// recursively acquired. acquire the OSREResource and
// elects to wait for the resource,
typedef struct _OSRERESOURCE_STATE
{ the m_WaitEventHandle is the
union { handle to the event that the
struct { thread will wait on. This event
ULONG State; // Acquisition State
ULONG AcquireCount; // Count of times acquired is called a ―NotificationEvent‖.
}; What this means is that unlike a
ULONGLONG Reserved;
}; ―SynchronizationEvent‖ where
} OSRERESOURCE_STATE, *POSRERESOURCE_STATE; only the first waiting thread is
woken up, when the event is set,
class OSREResource : public CObject all waiting threads are woken up.
{
public: This means that each awakened
OSREResource(); thread has the possibility of
virtual ~OSREResource(); acquiring the resource. Consider
enum { the scenario where a thread holds
ACQUIRED_NONE, the resource exclusively and 10
ACQUIRED_SHARED,
ACQUIRED_EXCLUSIVE threads are waiting to acquire the
} ACQUIRED_STATE; resource shared. In our
BOOLEAN AcquireResourceExclusive(BOOLEAN Wait); implementation all the shared
BOOLEAN AcquireResourceShared(BOOLEAN Wait); waiters will be awakened when
BOOLEAN AcquireSharedStarveExclusive(BOOLEAN Wait); the exclusive holder releases the
BOOLEAN IsResourceAcquiredExclusive();
BOOLEAN IsResourceAcquiredShared(); resources and will be able to
VOID ReleaseResource(); acquire the resource.
VOID ConvertExclusiveToShared();
VOID ReleaseResourceForThread(ERESOURCE_THREAD ResourceThreadId);
ULONG GetSharedWaiterCount(); Since the source code is readily
ULONG GetExclusiveWaiterCount();
NTSTATUS InitializeResource(); available for download, we don‘t
NTSTATUS DeleteResource(); want to spend a lot of time going
private: through it. What we do want to
mutable CRITICAL_SECTION m_CriticalSection; do however is go over
typedef std::map<DWORD,OSRERESOURCE_STATE> CMapThreadToLockState;
AcquireResourceShared and
CMapThreadToLockState m_LockStateMap; ReleaseResource to at least give
you a feel for how the code
volatile DWORD m_SharedCount;
volatile DWORD m_StarveExclusiveCount; works.
volatile DWORD m_ExclusiveCount;
volatile DWORD m_SharedWaitersCount;
volatile DWORD m_ExclusiveWaitersCount; AcquireResourceShared
HANDLE m_WaitEventHandle; Figure 3 (page 27) contains the
BOOLEAN m_Initialized;
}; AcquireResourceShared
function. As the name implies,
Figure 2—OSREResource Class and OSRERESOURCE_STATE Structure this function attempts to acquire
(Continued on page 27)
 Page 27

EResources... This routine first looks to see if the calling thread already
holds the resource by looking for the threads‘ Id in the
m_LockStateMap. If the thread is found, then the code
(Continued from page 26) updates the ownership counts and returns TRUE to the caller
indicating that the resource has been acquired.
the OSREResource shared and if it cannot, waits if the user
indicated wait for the resource to become available. If the thread Id was not found in the m_LockStateMap, the
Remember that our code acquires m_CriticalSection to code then checks to see if the resource is held exclusively or if
ensure proper synchronization when access the class data there are threads waiting for the resource exclusively. If there
structures. (Continued on page 28)

BOOLEAN OSREResource::AcquireResourceShared(BOOLEAN Wait)

{
DWORD cThread = GetCurrentThreadId();
ULONG loopCount = 0;
EnterCriticalSection(&m_CriticalSection);

m_SharedWaitersCount++;
while(TRUE) {
// See if we already own the lock.
//
CMapThreadToLockState::iterator ite = m_LockStateMap.find(cThread);
if(ite != m_LockStateMap.end()) {
//
// We already own it, let's see what access we already have.
//
ite->second.AcquireCount++;
if(ite->second.State & ACQUIRED_SHARED) {
// We already have it shared, so just bump the count.
//
m_SharedCount++;
} else if(ite->second.State & ACQUIRED_EXCLUSIVE) {
// We have it exclusive, so just give it to the caller exclusive
//
m_ExclusiveCount++;
}
m_SharedWaitersCount--;
LeaveCriticalSection(&m_CriticalSection);
return TRUE;
} else if(m_ExclusiveCount || m_ExclusiveWaitersCount) {
// Someone else owns it exclusive or there are exclusive waiters, so we have to wait.
//
if(!Wait) {
m_SharedWaitersCount--;
LeaveCriticalSection(&m_CriticalSection);
break;
}
LeaveCriticalSection(&m_CriticalSection);
WaitForSingleObject(m_WaitEventHandle,INFINITE);
// Add a wait loop to prevent this thread from hogging the CPU. The event gets set
// when the holder releases the lock and we have to ensure that everyone gets a chance
// to run before continuing.
//
loopCount++;
if(loopCount == 5) {
Sleep(200);
loopCount = 0;
}
EnterCriticalSection(&m_CriticalSection);
continue;
} else {
// Nobody has it or wants it, so we will take it. We reset the event, so that it is no
// longer signaled. Anyone who was waiting for the event should already be woken up.
//
OSRERESOURCE_STATE state;
state.State = ACQUIRED_SHARED;
state.AcquireCount = 1;
m_SharedCount++;
m_SharedWaitersCount--;
m_LockStateMap.insert(std::make_pair(cThread,state));
ResetEvent(m_WaitEventHandle);
LeaveCriticalSection(&m_CriticalSection);
return TRUE;
}
}
return FALSE;
} Figure 3—AcquireResourceShared
 Page 28

EResources... There is one thing that we must highlight here and it is that the
loop code follows the WaitForSingleObject. Keep in mind
that there could be many threads, with various scheduling
(Continued from page 27) priorities, wanting to acquire the OSREResource in different
ways, all waiting for it to become free. If we automatically
are, then this indicates that the caller cannot acquire the gave acquisition of the OSREResource to the first thread
resource. What is done next depends on the input Wait through the code, we could potentially starve other waiters,
parameter. If this parameter is TRUE, then the caller will since we know that highest priority will almost always get the
wait on the m_WaitEventHandle. If this parameter is CPU first. We didn‘t want to keep track of the waiters in a
FALSE, the code will return to the caller indicating that the list and heuristically pick the next owner, so we decided to
resource was not acquired. make it somewhat random (BTW, if you don‘t like it, change
it and let us know what your solution is and why it is better!).
Finally, if the resource is not held exclusively, or if there are Therefore we have all threads coming out of the
no threads waiting for the resource exclusively, then the code WaitForSingleObject sleeping for about 200 milliseconds, to
inserts an entry into the m_LockStateMap. This entry let all threads that were blocked have time to run through the
indicates how the resource was acquired, updates the acquire code again and reassess their access to the lock.
appropriate counts in the class, and returns to the caller
indicating that the resource was acquired. That‘s all there is to acquiring the OSREResource shared, so
let‘s see how you release it.

ReleaseResource
VOID OSREResource::ReleaseResource() Figure 4 contains the code for
{
DWORD cThread = GetCurrentThreadId(); ReleaseResource. As the name
implies, this code releases an owned
EnterCriticalSection(&m_CriticalSection);
OSREResource. Remember that
// our code acquires m_Critical
// See if we own the lock. We'd better.....
// Section to ensure proper
CMapThreadToLockState::iterator ite = m_LockStateMap.find(cThread); synchronization when accessing the
if(ite != m_LockStateMap.end()) {
class data structures.
//
// We own it. Decrement the ownership count based upon The first step for the code is to
// our access.
// attempt to find the calling thread in
if(ite->second.State & ACQUIRED_SHARED) { the m_LockStateMap, which keeps
m_SharedCount--;
} else { track of the threads that are currently
m_ExclusiveCount--; holding the OSREResource. If the
} calling thread is not found in the
// m_LockStateMap structure, the
// Look at our ownership count in the lowpart of the code will raise a 0xC0000002
// large integer. If it is 1, then this is our
// last reference to the lock, so we will delete exception to indicate that the
// ourselves from the ownership map. OSREResource was not held by the
//
if(ite->second.AcquireCount == 1) { caller. If the caller was found in the
// m_LockStateMap, the code will
// erase us, we no longer own the lock.
// decrement either the m_Shared
m_LockStateMap.erase(ite); Count or m_ExclusiveCount field,
//
depending upon how the caller had
// Wake up anyone waiting on access to the lock. acquired the OSREResource.
//
SetEvent(m_WaitEventHandle);
} else { The code then looks at whether or
// not this is the calling thread‘s last
// We still have outstanding references....
// reference to the OSREResource,
ite->second.AcquireCount--; which would indicate that the
} OSREResource is now free to be
LeaveCriticalSection(&m_CriticalSection);
} else { acquired by other threads. It checks
RaiseException(0xC0000002,EXCEPTION_NONCONTINUABLE,0,NULL); for the last reference by looking at
}
} the AcquiredCount field of the
returned OSRERESOURCE
Figure 4—ReleaseResource _STATE structure. If the
(Continued on page 29)
 Page 29
These were added to aid in debugging problems that you may
EResources... run into and are shown in Figure 5. As for deadlocks, the
only way to debug those is to examine each thread in your
application and see what it is waiting for in hopes that you can
(Continued from page 28) determine the deadlock.

AcquireCount is not 1, then the code knows that the Summary

OSREResource is still not available to other acquirers. If So there you have it, an OSR implementation of Executive
however the AcquireCount field is 1, then the code knows Resources in user mode. As you can see the code is pretty
this is the last reference to the OSREResource, at which point simple, and provides the advantages of ERESOURCEs to
it can delete the calling threads entry from the user mode, i.e. a true reader/writer lock.
m_LockStateMap, and can signal the OSREResource’s
event (m_WaitEventHandle). Signaling the event will You will find the code associated with this article at:
cause all the other threads waiting for this OSREResource to http://www.osronline.com/OsrDown.cfm/osreresource.zip?
be awakened so that they can attempt to acquire the
name=osreresource.zip&id=561
OSREResource.

Problems
As we mentioned earlier, the code will raise different
exceptions if it determines that something bad is going on.

Exception Code: Explanation:

0xC0000001 IsResourceAcquiredShared –invalid lock state, could indicate corruption
0xC0000002 ReleaseResource – resource not owned by caller
0xC0000003 ConvertExclusiveToShared – resource not owned exclusive
0xC0000004 [NOT USED]
0xC0000005 CovertExclusiveToShared – resource not owned by caller
0xC0000006 ReleaseResourceForThread – resource not owned by input thread
0xC0000007 DeleteResource – resource still in use
0xC0000008 AcquireResourceExclusive – already have the lock shared
0xC0000009 Resource not initialized
0xC000000A Resource already initialized

Figure 5—Exceptions Thrown by UMERESOURCEDLL

OSR’s Corporate, On-site Training

Save Money, Travel Hassles, Gain Customized Expert Instruction
We can:
Prepare and present a one-off, private, on-site seminar for your team to address a specific
area of deficiency or to prepare them for an upcoming project.
Design and deliver a series of offerings with a roadmap catered to a new group of recent hires
or within an existing group.
Work with your internal training organization/HR department to offer monthly or quarterly
seminars to your division or engineering departments company-wide.

To take advantage of our expertise in Windows internals, and in instructional design, contact an OSR seminar
consultant at +1.603.595.6500 or by email at [email protected].
Page 30

Analyst’s Perspective
Debug Smarter
O K, OK, I‘ll admit it…Debugging crash dumps can get
tedious fast. In a recent system hang that I analyzed, there
were 1,200 threads in the system. Given no details on what
So, I think it‘s time for a regime change. Let‘s promote
debugging to first class citizenship and spend the time
necessary to make our lives easier. Let‘s declare August 2010
was going on at the time of the hang, my eyes would probably the Month of Debugging Smarter and start rethinking how we
start bleeding before I found the threads that were interesting. approach solving our debugging problems. See P 8 of this
And that wasn‘t even a big terminal server machine with lots issue for an article on the basics of writing a debugger
of sessions running, so it was fairly tame by modern extension and, to get your creative juices flowing, there‘s even
standards. the source of a kernel mode implementation of !uniqstack for
you to play with. This command scans all of the threads in the
This is of course why we rely on our automated tools to do the system and, when finished, provides the !thread output of the
heavy lifting for us. When the system crashes, we don‘t go threads in the system with unique call stack sequences. As an
look up the bugcheck code and start trying to decode trap example of how many threads this eliminates, my 1,200
frames or context records on the stack, we instead rely on ! thread system ended up only having 105 unique call chain
analyze –v to do this work for us. In my 1,200 thread case, I sequences, which is certainly much more manageable.
didn‘t bother taking a detailed look at every thread in the
system but instead relied on !stacks 2 to give me a summary So no more excuses about not knowing how to write your own
of threads so that I could quickly scan for something that extension! And, if you do write your own extension, email me
looked ―interesting.‖ at [email protected] to let me know about it. Hopefully I can gather
submissions from The NT Insider readership and put together
What I think we often lose sight of though is that these some interesting tidbits for our next issue.
commands aren‘t magic. At some point in time, someone
thought that their job would be made easier if there was a
command that would quickly provide summary analysis
information at the time of a crash. Someone else thought
sifting through the output of !process 0 7 was also far too Analyst’s Perspective is a column by OSR consulting
painful and came up with a command to provide a summary associate, Scott Noone. When he’s not root-causing
view. complex kernel issues he’s leading the development and
instruction of OSR’s Kernel Debugging & Crash Analysis
Why is it that we don‘t all think this way? Instead, we tend to seminar. Comments on this article, or suggestions for a
rely on the existing commands and then complain that they future submission can be addressed to [email protected].
don‘t work the way we‘d like. Between WinDBG‘s scripting
capabilities and its debugger extension support it‘s hard to say
that there is anything it can‘t do. The common responses to
that though are usually, ―the scripting language is too cryptic‖ Kernel Debugging & Crash Analysis
or, ―I don‘t know how to write an extension.‖ But, in reality,
any language is cryptic until you take the time to learn it and You’ve seen our articles where we delve into analyses
most of us didn‘t know how to put pants on at some point in of various crash dumps or system hangs to determine
our lives. However, one day we decided that it was an
root cause. Want to learn the tools and techniques
important skill and decided to look at some examples and
practice (if you still don‘t know how to put pants on, I yourself? Consider attendance at OSR’s Kernel
apologize and you are deemed exempt from the remainder of Debugging & Crash Analysis seminar. This hands-on
this perspective). training provides fundamentals as well as valued
techniques that are used and espoused internally at
The other trap that we all fall into is that we just don‘t allow OSR.
ourselves the time to write a script or extension that will
potentially save us hours down the road. When debugging a
difficult problem, we tend to get tunnel vision and refuse to The next offering of this seminar is to be held in:
tear ourselves away from the problem to do something so
frivolous. I definitely get dragged down into this one, and by Portland, OR 18-22 October
the time I‘ve figured the problem out the extension idea leaves
my mind. Until I get the next dump of course, at which point I For more information, visit www.osr.com/debug.html
really wish I had written that extension…
 Page 31
in Bolivia and needs a safe place to store his secret
Peter Pontificates... communications.‖ You code to these particular stories. No,
you don‘t get a chance to think through the overall experience
for any user. This is Agile software development. You don‘t
(Continued from page 5) get to think. You‘re not allowed to design. You‘re allowed to
―get some code working‖ so you can try things out. And that
User stories are the software equivalent of task based code just needs to meet the user stories, and pass the tests that
documentation – another stoopid fad that I truly despise. But, were so lovingly crafted and stored with those stories.
you know, manager-troids love task-based documentation for Anything else? Well, that‘s for next sprint.
the same reasons they love user stories for software
development: It‘s easy to understand, it‘s easy to demonstrate So, in Agile what you get are ridiculously incomplete
that it‘s correct (just follow the steps and see if they work), requirements, driving a development process that emphasizes
and you can get an idiot to do it. The problem? It doesn‘t sloppy implementation over design, that‘s tracked using a
consider anything that‘s not specifically required for you to long list of bogus and irrelevant milestones. The only thing
achieve the task. For example, you can write a document that‘s left to wonder about is why everybody doesn‘t think
chapter entitled ―How to Peel an Onion‖ that describes the Agile development sucks ass.
process in detail. It will cover all the steps. But, on finishing
the chapter, the reader hasn‘t learned anything about onions,
peeling, or anything else. They‘ve just learned how to peel an
onion, and even then only under the conditions that the
chapter contemplates, using the steps the chapter prescribes. Peter Pontificates is a regular opinion column by OSR
consulting partner, Peter Viscarola. Peter doesn’t care if you
And so it is for user stories. Let‘s say you get a couple of user agree or disagree, but you do have the opportunity to
stories like ―Bob is a church-going motorcycle rider that needs respond and perhaps even see your comments or a rebuttal
to store digital copies of his ‗special‘ magazines in encrypted in a future issue. Send your own comments, rants or
form on his flash drive‖ and ―Hector served with Che Guevara distortions of fact to: [email protected].

Training Custom Development

OSR training services consist of public and private At OSR, we're experts in Windows system
seminars on a variety of topics including Windows software: Windows device drivers, Windows file
internals, driver development, file system systems, and most things related to Windows
development and debugging. Public seminar internals. It’s all we do. As a result, most OSR
presentations are scheduled and presented in a solutions can be proposed on a firm, fixed-price
variety of locations around the world, and basis. Clients will know the cost of a project phase
customized, private presentations are delivered to and deliverable dates before they have to make a
corporate clients based on demand. commitment.

Consulting Toolkits
In consultative engagements, OSR works with OSR software development toolkits provide
clients to determine needs and provide options to solutions that package stable, time-testing
proceed with OSR, or suggest alternative technology, with support from an engineering staff
solutions external to OSR. “Consulting" assistance that has helped dozens of customers deliver
from OSR can be had in many forms, but no successful solutions to market.
matter how it is acquired, you can be assured that
we'll be bringing our definitive expertise, industry
experience, and solid reputation to bear on our
engagement with you.

More information on OSR products and services can be found at the www.osr.com.
 Page 32
OSR OPEN SYSTEMS RESOURCES, INC. PRESORTED
105 State Route 101A, Suite 19 FIRST CLASS MAIL
Amherst, New Hampshire 03031 USA U.S. POSTAGE
PAID
(603)595-6500 ♦ Fax (603)595-6503 PERMIT # 456
MANCHESTER, NH

RETURN SERVICE REQUESTED

First Class

The NT Insider™ is a subscription-based publication

Subscribe to The NT Insider—Digital Edition

If you’re a new to The NT Insider (as in, the link to this issue was forwarded to you), you can subscribe at:
http://www.osronline.com/custom.cfm?name=login_joinok.cfm

New OSR Seminar Schedule!

Seminar Dates Location

Writing WDM Drivers (Lab) 16-20 August Seattle, WA

Writing WDF Drivers (Lab) 27-September-October 1 Santa Clara, CA

Kernel Debugging & Crash Analysis (Lab) 18-22 October Portland, OR

Developing File Systems for Windows 26-29 October Santa Clara, CA

Internals and Software Drivers (Lab) 15-19 November Santa Clara, CA

Writing WDM Drivers (Lab) 6-10 December Boston, MA

Course outlines, pricing, and how to register, visit the www.osr.com/seminars!