Indicating packets from WSK to NetAdapterCx

Lev Stipakov 96 Reputation points
2021-09-06T14:42:24.943+00:00

Hello,

I am working on the virtual network adapter driver which uses NetAdapterCx, WSK and CNG.

Here is what driver does on Tx path:

  • iterate over packets in "post" subsection of NET_RING
  • for each packet, get the first fragment and fragment's MDL. If MDL is NULL (packet was bounced by NetAdapter), allocate MDL from fragment's VA.
  • encrypt MDL chain with CNG using chaining mode (account for plaintext length must be multiplier of block size)
  • send MDL chain with WskSendTo/WskSend, pass NET_PACKET as completion routine context
  • move sent fragments and packets to "drain" subsection by adjusting NextIndex
  • in WSK completion routine, set NET_PACKET::Scratch to 1 to indicate that packet has been sent
  • iterate over packets in "drain" subsection and drain them to OS by adjusting BeginIndex if Scratch is set to 1

The code could be found here: https://github.com/lstipakov/ovpn-dco-win/blob/zerocopy/txqueue.cpp#L139

My question is - what would be the proper way to implement the same "zerocopy" approach on Rx path? At the moment it works like this:

  • packet is received by WskReceiveFromEvent callback
  • packet is decrypted into ciphertext buffer, fetched from pre-allocated "producer" pool (I use DMF_BufferQueue)
  • ciphertext buffer is enqueued into "consumer" queue
  • call NetRxQueueNotifyMoreReceivedPacketsAvailable() to trigger Rx queue's Advance callback
  • in Rx Advance callback, iterate over fragments, dequeue buffer from "consumer" queue and copy buffer content to fragment's VA
  • Buffer is "reused" by placing into "producer" pool

I can do decryption in-place and make WSK retain data by returning STATUS_PENDING from WskReceiveFromEvent callback, but how do I "indicate" data provided by WSK to NetAdapter without memcpying? Can I somehow tell NET_FRAGMENT "hey use this MDL which I got from WSK and decrypted in-place" ?

Ping @Jeffrey Tippet [MSFT]

Windows Hardware Performance
Windows Hardware Performance
Windows: A family of Microsoft operating systems that run across personal computers, tablets, laptops, phones, internet of things devices, self-contained mixed reality headsets, large collaboration screens, and other devices.Hardware Performance: Delivering / providing hardware or hardware systems or adjusting / adapting hardware or hardware systems.
1,566 questions
0 comments No comments
{count} votes

Accepted answer
  1. Lev Stipakov 96 Reputation points
    2021-09-09T12:29:20.693+00:00

    (copied from https://community.osr.com/discussion/comment/302680/#Comment_302680)

    I think I figured it out by reading NetAdapter's code. Here is how I made it work in my driver:

    • When setting datapath capabilities, we use NET_ADAPTER_RX_CAPABILITIES_INIT_DRIVER_MANAGED macro. We also specify EvtAdapterReturnRxBuffer callback.
    • In WskReceiveFromEvent callback we iterate over WSK_DATAGRAM_INDICATION list and enqueue each item into cosumer pool (from DMF_BufferQueue)
        // each datagram indication is one UDP datagram
        for (PWSK_DATAGRAM_INDICATION next; dataIndication != NULL; dataIndication = next) {
            next = dataIndication->Next;
      
            // break list so that we can pass individual WSK_DATAGRAM_INDICATION to WskRelease
            dataIndication->Next = NULL;
      
            // fetch buffer from producer
            OVPN_RX_BUFFER* rxBuffer;
            LOG_IF_NOT_NT_SUCCESS(OvpnBufferQueueFetch(device->DataRxBufferQueue, &rxBuffer));
            rxBuffer->DatagramIndication = dataIndication;
      
            // enqueue buffer to consumer, which is consumed by NetAdapter's RX Advance callback
            OvpnBufferQueueEnqueue(device->DataRxBufferQueue, rxBuffer);
        }
      
        // tell NetAdapter that we have something to consume
        OvpnAdapterNotifyRx(device->Adapter);
      
    • In RX queue's Advance callback we iterate over queue's fragments. We dequeue datagram indication and set fragment properties based on datagram indication's buffer and MDL:
      while (NetFragmentIteratorHasAny(&fi)) {
          OVPN_RX_BUFFER* buffer;
          // nothing has arrived and decrypted yet?
          if (!NT_SUCCESS(OvpnBufferQueueDequeue(bufferQueue, &buffer))) {
              break;
          }
      
          fragment = NetFragmentIteratorGetFragment(&fi);
      
          PWSK_DATAGRAM_INDICATION datagramIndication = buffer->DatagramIndication;
          // return buffer to producer queue
          OvpnBufferQueueReuse(bufferQueue, buffer);
      
          PMDL mdl = datagramIndication->Buffer.Mdl;
      
          // TODO: correctly adjust crypto/ovpn overhead
          fragment->ValidLength = datagramIndication->Buffer.Length - 8;
          fragment->Offset = datagramIndication->Buffer.Offset + 8;
          fragment->Capacity = MmGetMdlByteCount(mdl);
      
          NET_FRAGMENT_VIRTUAL_ADDRESS* virtualAddr = NetExtensionGetFragmentVirtualAddress(&queue->VirtualAddressExtension, NetFragmentIteratorGetIndex(&fi));
          virtualAddr->VirtualAddress = (PUCHAR)(MmGetSystemAddressForMdlSafe(mdl, LowPagePriority));
      
          // TODO: handle case when packet (DataIndication) contains multiple fragments
          NET_PACKET* packet = NetPacketIteratorGetPacket(&pi);
          packet->FragmentIndex = NetFragmentIteratorGetIndex(&fi);
          packet->FragmentCount = 1;
      
          packet->Layout = {};
      
          // NetAdapter will call ReturnRxBuffer callback when it is done with buffers, there we return datagramIndication back to WSK
          returnCtx = NetExtensionGetFragmentReturnContext(&queue->ReturnContextExtension, NetFragmentIteratorGetIndex(&fi));
          returnCtx->Handle = (NET_FRAGMENT_RETURN_CONTEXT_HANDLE)datagramIndication;
      
          NetFragmentIteratorAdvance(&fi);
          NetPacketIteratorAdvance(&pi);
      }
      NetFragmentIteratorSet(&fi);
      NetPacketIteratorSet(&pi);
      
    • In NetAdapter's EvtAdapterReturnRxBuffer callback we call WskRelease on a datagram indication, which is passed as NET_FRAGMENT_RETURN_CONTEXT_HANDLE by NetAdapter:
      _Use_decl_annotations_
      void
      OvpnEvtAdapterReturnRxBuffer(NETADAPTER netAdapter, NET_FRAGMENT_RETURN_CONTEXT_HANDLE rxReturnContext)
      {
          POVPN_ADAPTER adapter = OvpnGetAdapterContext(netAdapter);
          POVPN_DEVICE device = OvpnGetDeviceContext(adapter->WdfDevice);
      
          PWSK_DATAGRAM_INDICATION dataIndication = (PWSK_DATAGRAM_INDICATION)rxReturnContext;
      
          LOG_IF_NOT_NT_SUCCESS(((WSK_PROVIDER_DATAGRAM_DISPATCH*)device->Socket.Socket->Dispatch)->WskRelease(device->Socket.Socket, dataIndication));
      }
      

    Not sure if this is the "best practices", but it works and doesn't crash on iperf3 tests.

    0 comments No comments

0 additional answers

Sort by: Most helpful