XP64: 32-bit app, 64-bit driver access problems

[catbot]

Hey everybody. I'm looking for any help in solving my problem, and MSDN and Microsoft support sites don't have info on my specific problem (at least that I can find).

Background: I have a 32-bit app, with inline assembly, that deploys a driver only at run-time (when the user specifies). I am attempting to get my app to run on xp64 with as little work as needed (I plan to build a complete 64-bit app later, but would like to get it working so I can use it for debugging other apps).

Current situation: I have re-written and compiled my driver (.sys) as 64-bit (all inline assembly moved to .asm file). My 32-bit app has been recompiled with all inline assembly converted to intrinsics and generic assembly. This recompiled app with the old 32-bit driver works correctly on windows xp.

My problem: My 32-bit app with the 64-bit driver fails to run on xp64 fails. I have debug code in my driver to figure out if it actually works, but it is never accessed. Below are the steps I go through to deploy the driver for use.

1. Copy driver to the drivers folder (I ask the system to write the file to the drivers folder)
- the driver is placed in the WOW64\drivers directory

2. OpenSCManager
- I open the service control manager so i can start a service for accessing my driver (this appears to work correctly because I get a success result)

3. CreateService
- Create a service for accessing my driver (result is also good here)

4. CreateFile
- Get a handle to my driver file from step 1 (this is where I get an error that the file doesn't exist, even though I can see it on the system)
- I can't find any info on whether or not the call to CreateFile is different on xp64 than the 32-bit version, or if the FsRedirection of WOW64 is affecting the call.

5. Pass the handle to the created service (I never get here)

6. Use the driver to access system/device info

7. Remove the driver


Any help that can be provided is greatly appreciated.

Thanks,

CatBot

 

[Steel Chicken]

a total noob thought, but I thought WOW was for 32bit emulation. so if you are trying to use 64bit code in a 32bit emulated environment, wouldn't that be a problem?

 

[catbot]

Obviously I'm also a noob on this topic, but The way I thought it worked was that if you were running a 32-bit app, WOW intercepted all system calls. Since I'm technically running a 32-bit app that's trying to deploy a driver compiled with the AMD64 flag, I assumed WOW would still intercept everything the app does (but not necessarily what the driver needs to do).

When I launch the program, I can verify that the driver is copied to the SysWOW64\Drivers folder, but the error I get back from the CreateFile call is ERROR_FILE_NOT_FOUND (error code 2 from GetLastError check)...maybe this error is related to trying to mix a 32-bit app and 64-bit driver. I was hoping I wouldn't have to do it yet, but maybe I do have to go full 64-bit for everything.

An interesting side note: I got an email from a friend that ran into a similar problem, but his fix doesn't work for me. Supposedly, if you attempt to test your 32-bit app on XP64, and you fail to load your driver (which will probably happen to most people the first time), the OS keeps track of this failure. Then, if you re-compile the driver, re-use the same name as your 32-bit driver (the one that failed to load), your app will continue to fail. His solution was to delete the HKLM\System\CurrentControlSet\Services\<drivername> key, and then everything worked out. (I believe this only works if you have a similar process to the one I have above of creating a service to deploy/use your driver).