Software & Application Miscellaneous: ZwReadFile slows after a few hundred MBs

  • anonymous / 205 / Sun, 31 Jan 2010 22:31:00 GMT / Comments (5)
  • I've got a thread that processes IRP_MJ_READ/WRITEs. From the code that follows, if I use pReadBuffer with ZwReadFile then everything is smooth but if I simply use pCurrentAddress then my whole system slows to a crawl after a few hundred megs. Can anyone shed some light?
    Writes don't seem to be affected if I use pCurrentAddress.

    -Hao

    pCurrentAddress = MmGetSystemAddressForMdlSafe(pIrp->MdlAddress, HighPagePriority);
    pReadBuffer = ExAllocatePool(PagedPool, pIrpStack->Parameters.Read.Length);
    if(pReadBuffer == NULL)
    {
    pIrp->IoStatus.Information = 0;
    pIrp->IoStatus.Status = STATUS_INSUFFICIENT_RESOURCES;
    IoCompleteRequest(pIrp, IO_NO_INCREMENT);
    }
    RtlZeroMemory(pReadBuffer, pIrpStack->Parameters.Read.Length);

    ZwReadFile(FileHandle, NULL, NULL, NULL, &IoStatusBlock,
    pReadBuffer, pIrpStack->Parameters.Read.Length,
    &pIrpStack->Parameters.Read.ByteOffset, NULL);
    RtlCopyMemory(pCurrentAddress, pReadBuffer, IoStatusBlock.Information);
    ExFreePool(pReadBuffer);
  • Keywords:

    zwreadfile, slows, few, hundred, mbs, software, application

  • http://software.itags.org/software-application/240752/«« Last Thread - Next Thread »»
    1. offhand i'd guess you were leaking something. Try !poolused checking out in
      the debugger - it should show you if there's a lot of memory being used for
      some particular pool tag.

      since you already have an MDL that you're using, why not just build
      READ/WRITE IRPs directly rather than doing this indirection back through the
      NT API?
      -p
      --
      This posting is provided "AS IS" with no warranties, and confers no rights.
      "Hao Nguyen" <howser56...yahoo.com> wrote in message
      news:2453ba04.0408262224.3af2dd2a...posting.google.com...
      > I've got a thread that processes IRP_MJ_READ/WRITEs. From the code
      > that follows, if I use pReadBuffer with ZwReadFile then everything is
      > smooth but if I simply use pCurrentAddress then my whole system slows
      > to a crawl after a few hundred megs. Can anyone shed some light?
      > Writes don't seem to be affected if I use pCurrentAddress.
      > -Hao
      > pCurrentAddress = MmGetSystemAddressForMdlSafe(pIrp->MdlAddress,
      > HighPagePriority);
      > pReadBuffer = ExAllocatePool(PagedPool,
      > pIrpStack->Parameters.Read.Length);
      > if(pReadBuffer == NULL)
      > {
      > pIrp->IoStatus.Information = 0;
      > pIrp->IoStatus.Status = STATUS_INSUFFICIENT_RESOURCES;
      > IoCompleteRequest(pIrp, IO_NO_INCREMENT);
      > }
      > RtlZeroMemory(pReadBuffer, pIrpStack->Parameters.Read.Length);
      > ZwReadFile(FileHandle, NULL, NULL, NULL, &IoStatusBlock,
      > pReadBuffer, pIrpStack->Parameters.Read.Length,
      > &pIrpStack->Parameters.Read.ByteOffset, NULL);
      > RtlCopyMemory(pCurrentAddress, pReadBuffer,
      > IoStatusBlock.Information);
      > ExFreePool(pReadBuffer);

      peter | Tues, 20 May 2008 08:02:00 GMT |

    2. "Hao Nguyen" <howser56...yahoo.com> wrote in message
      news:2453ba04.0408262224.3af2dd2a...posting.google.com...
      > I've got a thread that processes IRP_MJ_READ/WRITEs. From the code
      > that follows, if I use pReadBuffer with ZwReadFile then everything is
      > smooth but if I simply use pCurrentAddress then my whole system slows
      > to a crawl after a few hundred megs. Can anyone shed some light?
      > Writes don't seem to be affected if I use pCurrentAddress.

      This is not related to your problem, but why are you wasting your time clearing
      the buffer before the read?

      You're not calling MmGetSystemAddressForMdlSafe more than once for the same
      buffer, are you? In fact, why are you calling MmGetSystemAddressForMdlSafe, at
      all? Unless you're not in the requesting process's context, it is not needed;
      just use IRP->UserBuffer

      -Brian

      Brian Catlin, Sannas Consulting 310-944-9492
      Windows Network, Video, WDM Device Driver Training & Consulting
      See WWW.AZIUS.COM.bad for courses and scheduling
      REMOVE .BAD FROM EMAIL AND WEB ADDRESS

      > -Hao
      > pCurrentAddress = MmGetSystemAddressForMdlSafe(pIrp->MdlAddress,
      > HighPagePriority);
      > pReadBuffer = ExAllocatePool(PagedPool,
      > pIrpStack->Parameters.Read.Length);
      > if(pReadBuffer == NULL)
      > {
      > pIrp->IoStatus.Information = 0;
      > pIrp->IoStatus.Status = STATUS_INSUFFICIENT_RESOURCES;
      > IoCompleteRequest(pIrp, IO_NO_INCREMENT);
      > }
      > RtlZeroMemory(pReadBuffer, pIrpStack->Parameters.Read.Length);
      > ZwReadFile(FileHandle, NULL, NULL, NULL, &IoStatusBlock,
      > pReadBuffer, pIrpStack->Parameters.Read.Length,
      > &pIrpStack->Parameters.Read.ByteOffset, NULL);
      > RtlCopyMemory(pCurrentAddress, pReadBuffer,
      > IoStatusBlock.Information);
      > ExFreePool(pReadBuffer);

      brian | Tues, 20 May 2008 08:03:00 GMT |

    3. "Brian Catlin" <brianc...sannas.org.bad> wrote in message news:<#unnHKEjEHA.1204...TK2MSFTNGP15.phx.gbl>...
      > "Hao Nguyen" <howser56...yahoo.com> wrote in message
      > news:2453ba04.0408262224.3af2dd2a...posting.google.com...
      > > I've got a thread that processes IRP_MJ_READ/WRITEs. From the code
      > > that follows, if I use pReadBuffer with ZwReadFile then everything is
      > > smooth but if I simply use pCurrentAddress then my whole system slows
      > > to a crawl after a few hundred megs. Can anyone shed some light?
      > > Writes don't seem to be affected if I use pCurrentAddress.
      > This is not related to your problem, but why are you wasting your time clearing
      > the buffer before the read?
      >

      Just a habit I have when dealing with buffers. Unless the allocation
      does this automatically?

      > You're not calling MmGetSystemAddressForMdlSafe more than once for the same
      > buffer, are you? In fact, why are you calling MmGetSystemAddressForMdlSafe, at
      > all? Unless you're not in the requesting process's context, it is not needed;
      > just use IRP->UserBuffer
      > -Brian
      >

      I'm only calling MmGetSystemAddressForMdlSafe once for the buffer and
      using DO_DIRECT_IO so unless I'm mistaken, I can't use
      IRP->UserBuffer.

      howser56 | Tues, 20 May 2008 08:04:00 GMT |

    4. "Peter Wieland [MSFT]" <peterwie...online.microsoft.com> wrote in message news:<e59ZEFEjEHA.3928...TK2MSFTNGP11.phx.gbl>...
      > offhand i'd guess you were leaking something. Try !poolused checking out in
      > the debugger - it should show you if there's a lot of memory being used for
      > some particular pool tag.
      > since you already have an MDL that you're using, why not just build
      > READ/WRITE IRPs directly rather than doing this indirection back through the
      > NT API?
      > -p
      >

      I plan on doing that eventually. Using ZwXX is just a little easier
      to get things going initially.

      howser56 | Tues, 20 May 2008 08:05:00 GMT |

    5. "Hao Nguyen" <howser56...yahoo.com> wrote in message
      news:2453ba04.0408271019.64f106ff...posting.google.com...
      > "Brian Catlin" <brianc...sannas.org.bad> wrote in message
      > news:<#unnHKEjEHA.1204...TK2MSFTNGP15.phx.gbl>...
      >> "Hao Nguyen" <howser56...yahoo.com> wrote in message
      >> news:2453ba04.0408262224.3af2dd2a...posting.google.com...
      >> > I've got a thread that processes IRP_MJ_READ/WRITEs. From the code
      >> > that follows, if I use pReadBuffer with ZwReadFile then everything is
      >> > smooth but if I simply use pCurrentAddress then my whole system slows
      >> > to a crawl after a few hundred megs. Can anyone shed some light?
      >> > Writes don't seem to be affected if I use pCurrentAddress.
      >> This is not related to your problem, but why are you wasting your time
      >> clearing
      >> the buffer before the read?
      > Just a habit I have when dealing with buffers.

      An expensive habit

      > Unless the allocation does this automatically?

      No, and it isn't needed. You're told how much of the buffer has been written
      to. Trust it.

      >> You're not calling MmGetSystemAddressForMdlSafe more than once for the same
      >> buffer, are you? In fact, why are you calling MmGetSystemAddressForMdlSafe,
      >> at
      >> all? Unless you're not in the requesting process's context, it is not
      >> needed;
      >> just use IRP->UserBuffer
      >> -Brian
      > I'm only calling MmGetSystemAddressForMdlSafe once for the buffer and
      > using DO_DIRECT_IO so unless I'm mistaken, I can't use
      > IRP->UserBuffer.

      Yes, you can use IRP->UserBuffer, IF you will always be accessing the buffer
      from the context of the requesting process. Since you're passing
      IRP->UserBuffer to another driver, that driver will either lock down the buffer
      (if it does direct I/O), or it will do the copy associated with buffered I/O.

      -Brian

      Brian Catlin, Sannas Consulting 310-944-9492
      Windows Network, Video, WDM Device Driver Training & Consulting
      See WWW.AZIUS.COM.bad for courses and scheduling
      REMOVE .BAD FROM EMAIL AND WEB ADDRESS

      brian | Tues, 20 May 2008 08:06:00 GMT |