SCSIPORT

This is a description of my expierience implementing ScsiPort on OZONE. ScsiPort is the interface between a Microsoft OS and the corresponding SCSI controller chip drivers. So, for example, if you have an Adaptec 7880-based controller, it comes with a file called aic78xx.sys. The interface that aic78xx.sys is written to is called ScsiPort. You can dig up a more-or-less complete description of ScsiPort on msdn.microsoft.com.

The reason for doing this is, well, have you attempted to get doc for an Adaptec controller? Maybe I'm just not good at it, but all I got was the run-around. So now I can use the surplus-priced motherboard's built-in 7880 chip with OZONE (you don't think I would actually spend money on an Adaptec board after that cr*p, do you?). Plus, now I can (theoretically) use any PCI SCSI controller card that has an MS driver.

So here's what I basically did to get this to work:

1. Used a dumpbin /all aic78xx.sys command on my Win2K system to find out which routines the aic78xx.sys binary actually uses. Fortunately, there are only a couple dozen, and they are all ScsiPort... routines. One of the other scsi drivers uses some Hal... routines (I can't remember which other one I looked at though).
2. Studied the ScsiPort... routines on msdn.microsoft.com
3. Wrote my ScsiPort... routines. It is basically a subroutine library consisting of the various ScsiPort... routines.
4. Wrote a disassembler that converts the aic78xx.sys file into an equivalent aic78xx.s file that can be assembled with the gas assembler. Originally, it just disassembled it into a bunch of .long and .byte directives, ie, no instruction decoding. But I later revised it to do some instruction decoding (by ripping off gdb's disassemble routine) for debugging purposes. All the .long/.byte's are still there, the decoded instructions are in there as comments.
5. Put it all together as a loadable image, consisting of these modules:
   • A simple main program which locks the loadable image's pages in memory then calls aic78xx's DriverEntry routine (its starting address) then returns.
   • The disassembled aic78xx file
   • My ScsiPort routines
6. So I load and run that image, it scans the PCI bus for aic-type chips and creates an OZONE scsi-type device table entry. I can now use my standard scsi tools to access disks or whatever on the scsi bus.

Here are the fun things I had to do to make it work:

• My OS was written using the gcc compiler under Linux. Needless to say, the aic78xx.sys image was most likely made with Microsoft compilers and linkers. Well, gcc uses a 'caller pops call arguments' paradigm, whereas Microsoft uses a 'callee pops call arguments' paradigm. So the result is that my ScsiPort routines have to have little bits of asm 'glue' in them to call the aic78xx Hw... routines, and 'glue' on the ScsiPort... entrypoints. One exception is the ScsiPortNotification routine, it uses the 'caller pops call arguments' paradigm, apparently because of the varargs calling format.

   

• I had to cut-and-paste the various struct definitions from various places. A good source for them is to download the Win2K DDK from the msdn site. The structs on the website (HW_INITIALIZATION_DATA for example) are out-of-date. You may be able to use them as is by simply including them. I figured I'd probably spend more time chasing undefined symbols that way so I extracted only what was needed. Mistakes I made:
  • SCSI_PHYSICAL_ADDRESS is a 64-bit quantity, not 32-bit
  • Gcc didn't like the convoluted definition of AccessRanges in the PORT_CONFIGURATION_INFORMATION struct. I assumed that ACCESS_RANGE (*AccessRanges)[] meant there were two levels of indirection (as in char *argv[]) thus equivalent to ACCESS_RANGE **AccessRanges, meaning that it was a pointer to an array of pointers to the ACCESS_RANGE structs. Well, it should simply be ACCESS_RANGE *AccessRanges, ie, a pointer to an array of ACCESS_RANGE structs.

   

• All access to I/O registers is made through callbacks from the aic78xx driver to the ScsiPort routines. Now it is possible for the controller's I/O registers to be mapped to memory space, but all the port access routines are provided with is a virtual address of the I/O register to be accessed. Well, I/O space can't be mapped to a virtual address, so I tell the aic78xx driver that the I/O registers are mapped at virtual address FFFFxxxx (which is where I put my pagetables). It would be impossible for the controller to have any real memory mapped registers there, so this is safe. Then, when my port access routines see a register virtual address of FFFFxxxx, they know to use in/out instructions instead of a normal memory access to get to the controller's registers.

  So when routine ScsiPortGetDeviceBase gets a call with the InIoSpace set, it simply returns IoAddress+0xFFFF0000. Then when any of the ScsiPort... port access routines see an address that is .ge. 0xFFFF0000, they use in/out instructions. If InIoSpace is clear, it allocates a system pagetable entry to map the controller's registers with, and sets the attributes to disable caching.

   

• There are exciting learning opportunities awaiting you in the ScsiPortInitialize routine.
  • From the doc, it looked like it gets called once for each different bus type supported by the controller. I only want to do PCI support, so I thought that the aic78xx driver would only call it once with AdapterInterfactType indicating PCIBus. This lead to two mistakes.
    1. So I set my ScsiPortInitialize routine up to call HwFindAdapter thinking it would scan the PCI bus for a suitable controller and report back if it found one or not. It also has a parameter to tell me if I should call it again to find another adapter. I figured it would return the bus/dev/func of the adapter it found in the ConfigInfo struct. Booby prize for that. HwFindAdapter will really only check to see if the adapter you give it in the ConfigInfo struct is one that it supports. Ok, so I changed my ScsiPortInitialize routine to scan the PCI bus and call HwFindAdapter for every SCSI adapter I find on the PCI bus.
    2. So my ScsiPortInitialize routine got called 42 times (isn't that the correct temperature Farenheit for serving Dom Perringon '53)? So I looked at the sample code for the initio 9100uw controller, and it calls ScsiPortInitialize once for each device-id/vendor-id combination that it supports, and it passes the vendor-id/device-id in the HwInitializationData. I had an old struct definition from the website that didn't have those locations in it. So I got the new struct from the NTDDK download and I use it to scan the PCI Bus with, then just call HwFindAdapter for adapters with matching vendor-id/device-id.
  • The BusInformation passed to HwFindAdapter can be NULL as far as I can tell. Apparently neither the ini90u sample driver nor aic78xx require it.
  • You have to fill the AccessRanges array for the ConfigInfo struct you pass to HwFindAdapter by scanning the base address array of the PCI config info for the device. I originally assumed the HwFindAdapter routine filled the array in, after all, it should know how to read its controller's PCI config registers and know the sizes, etc.
  • Should HwFindAdapter successfully return, I create the OZONE device table entry for the scsi controller then I call aic78xx's HwInitialize routine to start the controller chip.

   

• Queuing of requests is another fun subject. Now that it's apparently done and apparently working, it seems more-or-less simple. But from initially reading the doc, I made these assumptions:
  • RequestComplete merely indicates that the I/O request is complete. It does not have any implications as to whether the aic78xx driver is ready to accept any new requests.
  • NextRequest indicates that the aic78xx driver is able to accept one request for any Lu.
  • NextLuRequest indicates that the aic78xx driver is able to accept a request for the indicated Lu as well as for any Lu. So I thought it could accept up to two requests, as long as at least one was for the indicated Lu.
  Needless to say that is all wrong. It apparently means:
  • RequestComplete indicates an I/O request is complete.
  • NextRequest indicates that the aic78xx driver is able to accept one request for any Lu that does not currently have any I/O's in progress on it. So requests may have to wait for RequestComplete callbacks before they can be started.
  • NextLuRequest indicates that the aic78xx driver is able to accept one request for any Lu that does not currently have any I/O's in progress on it, or a request for the indicated Lu (even if it has I/O's in progress).
  What I ended up with was the following:

    Per hba 'active' flag
      - gets set just before calling HwStartIo
      - cleared when I get NextRequest or NextLuRequest callback
      - when this flag is set, it blocks any request 
        from being passed to HwStartIo
  
    Per hba 'pending' request queue
      - new requests stay here if they come in when either 
        the hba active flag is already set
        or the corresponding lu active flag is already set
  
    Per lu 'active' flag
      - gets set just before calling HwStartIo
      - gets cleared when I get NextLuRequest callback
        *or* when I get RequestComplete callback and there
             aren't any more requests in progress on this lu
      - when this flag is set, it blocks requests 
        for this lu from being passed to HwStartIo
  
    Per lu 'in progress' request queue
      - requests get placed here just before calling HwStartIo 
      - they get removed when I get the RequestComplete callback
  
  When a new request comes in, it goes on the end of my hba's pending q
  Then if both my hba's active and corresponding lu's active flags are clear, 
    move it from my hba pending q and put on my lu in-progress q
    set my hba active flag
    set my lu active flag
    pass request to HwStartIo
  
  When I get a NextLuRequest callback,
    clear lu's active flag
    fall through to NextRequest processing
  When I get a NextRequest callback,
    clear hba's active flag
    scan hba's pending request queue for a request on an inactive lu
    if found,
      move it from hba pending q to lu in-progress q
      set hba and lu active flags
      pass request to HwStartIo
  
  When I get a RequestComplete callback,
    remove request from lu's in-progress queue
    post the request as complete back to my OS
    if lu's in-progress queue now empty,
      clear lu's active flag
      if hba active flag is clear,
        scan hba's pending request queue for a request on an inactive lu
        if found, 
          move it from hba pending q to lu in-progress q
          set hba and lu active flags
          pass request to HwStartIo
  

• Ok, so let's say aic78xx driver calls me with RequestComplete and I can now start another I/O because both the hba and the lu are idle. Well, it will lose the request if I call HwStartIo directly from my ScsiPortNotification routine. So what I do instead is set up a software interrupt to cause HwStartIo to be called when I'm completely out of any of the aic78xx routines.

The process took about two weeks from start to where I'm at now, and there are about 3000 lines of original code. If you want to look at my sources for this stuff, they are here.

Here's what's left to 'finish up':

• Write and test the code for timeouts and aborts
• Make a PE image loader to load the aic78xx.sys file directly. But the disassembly method has the advantage that I can statically link it to my loader and kernel so I can boot from a SCSI disk.
• Change the software interrupt HwStartIo paradigm to simply just checking the queue after every top-level call to the aic78xx driver (there aren't many). This should be more efficient.
• Maybe test it with the MS driver for the Lsilogic 875 board I have in that computer. I currently have a native driver for that board and I use it to access my system disk. Max said the MS driver for that board is dogmeat though, so I'm not going to make that a permanent change.

Thanks to Max on alt.os.development for the original challenge and for answering my questions.