Indicating packets from WSK to NetAdapterCx

Question

Hello,

I am working on the virtual network adapter driver which uses NetAdapterCx, WSK and CNG.

Here is what driver does on Tx path:

• iterate over packets in "post" subsection of NET_RING
• for each packet, get the first fragment and fragment's MDL. If MDL is NULL (packet was bounced by NetAdapter), allocate MDL from fragment's VA.
• encrypt MDL chain with CNG using chaining mode (account for plaintext length must be multiplier of block size)
• send MDL chain with WskSendTo/WskSend, pass NET_PACKET as completion routine context
• move sent fragments and packets to "drain" subsection by adjusting NextIndex
• in WSK completion routine, set NET_PACKET::Scratch to 1 to indicate that packet has been sent
• iterate over packets in "drain" subsection and drain them to OS by adjusting BeginIndex if Scratch is set to 1

The code could be found here: https://github.com/lstipakov/ovpn-dco-win/blob/zerocopy/txqueue.cpp#L139

My question is - what would be the proper way to implement the same "zerocopy" approach on Rx path? At the moment it works like this:

• packet is received by WskReceiveFromEvent callback
• packet is decrypted into ciphertext buffer, fetched from pre-allocated "producer" pool (I use DMF_BufferQueue)
• ciphertext buffer is enqueued into "consumer" queue
• call NetRxQueueNotifyMoreReceivedPacketsAvailable() to trigger Rx queue's Advance callback
• in Rx Advance callback, iterate over fragments, dequeue buffer from "consumer" queue and copy buffer content to fragment's VA
• Buffer is "reused" by placing into "producer" pool

I can do decryption in-place and make WSK retain data by returning STATUS_PENDING from WskReceiveFromEvent callback, but how do I "indicate" data provided by WSK to NetAdapter without memcpying? Can I somehow tell NET_FRAGMENT "hey use this MDL which I got from WSK and decrypted in-place" ?

Ping @Jeffrey Tippet [MSFT]

Answer 1

(copied from https://community.osr.com/discussion/comment/302680/#Comment_302680)

I think I figured it out by reading NetAdapter's code. Here is how I made it work in my driver:

• When setting datapath capabilities, we use NET_ADAPTER_RX_CAPABILITIES_INIT_DRIVER_MANAGED macro. We also specify EvtAdapterReturnRxBuffer callback.
• In WskReceiveFromEvent callback we iterate over WSK_DATAGRAM_INDICATION list and enqueue each item into cosumer pool (from DMF_BufferQueue)

    // each datagram indication is one UDP datagram
    for (PWSK_DATAGRAM_INDICATION next; dataIndication != NULL; dataIndication = next) {
        next = dataIndication->Next;
  
        // break list so that we can pass individual WSK_DATAGRAM_INDICATION to WskRelease
        dataIndication->Next = NULL;
  
        // fetch buffer from producer
        OVPN_RX_BUFFER* rxBuffer;
        LOG_IF_NOT_NT_SUCCESS(OvpnBufferQueueFetch(device->DataRxBufferQueue, &rxBuffer));
        rxBuffer->DatagramIndication = dataIndication;
  
        // enqueue buffer to consumer, which is consumed by NetAdapter's RX Advance callback
        OvpnBufferQueueEnqueue(device->DataRxBufferQueue, rxBuffer);
    }
  
    // tell NetAdapter that we have something to consume
    OvpnAdapterNotifyRx(device->Adapter);
  

• In RX queue's Advance callback we iterate over queue's fragments. We dequeue datagram indication and set fragment properties based on datagram indication's buffer and MDL:

  while (NetFragmentIteratorHasAny(&fi)) {
      OVPN_RX_BUFFER* buffer;
      // nothing has arrived and decrypted yet?
      if (!NT_SUCCESS(OvpnBufferQueueDequeue(bufferQueue, &buffer))) {
          break;
      }
  
      fragment = NetFragmentIteratorGetFragment(&fi);
  
      PWSK_DATAGRAM_INDICATION datagramIndication = buffer->DatagramIndication;
      // return buffer to producer queue
      OvpnBufferQueueReuse(bufferQueue, buffer);
  
      PMDL mdl = datagramIndication->Buffer.Mdl;
  
      // TODO: correctly adjust crypto/ovpn overhead
      fragment->ValidLength = datagramIndication->Buffer.Length - 8;
      fragment->Offset = datagramIndication->Buffer.Offset + 8;
      fragment->Capacity = MmGetMdlByteCount(mdl);
  
      NET_FRAGMENT_VIRTUAL_ADDRESS* virtualAddr = NetExtensionGetFragmentVirtualAddress(&queue->VirtualAddressExtension, NetFragmentIteratorGetIndex(&fi));
      virtualAddr->VirtualAddress = (PUCHAR)(MmGetSystemAddressForMdlSafe(mdl, LowPagePriority));
  
      // TODO: handle case when packet (DataIndication) contains multiple fragments
      NET_PACKET* packet = NetPacketIteratorGetPacket(&pi);
      packet->FragmentIndex = NetFragmentIteratorGetIndex(&fi);
      packet->FragmentCount = 1;
  
      packet->Layout = {};
  
      // NetAdapter will call ReturnRxBuffer callback when it is done with buffers, there we return datagramIndication back to WSK
      returnCtx = NetExtensionGetFragmentReturnContext(&queue->ReturnContextExtension, NetFragmentIteratorGetIndex(&fi));
      returnCtx->Handle = (NET_FRAGMENT_RETURN_CONTEXT_HANDLE)datagramIndication;
  
      NetFragmentIteratorAdvance(&fi);
      NetPacketIteratorAdvance(&pi);
  }
  NetFragmentIteratorSet(&fi);
  NetPacketIteratorSet(&pi);
  

• In NetAdapter's EvtAdapterReturnRxBuffer callback we call WskRelease on a datagram indication, which is passed as NET_FRAGMENT_RETURN_CONTEXT_HANDLE by NetAdapter:

  _Use_decl_annotations_
  void
  OvpnEvtAdapterReturnRxBuffer(NETADAPTER netAdapter, NET_FRAGMENT_RETURN_CONTEXT_HANDLE rxReturnContext)
  {
      POVPN_ADAPTER adapter = OvpnGetAdapterContext(netAdapter);
      POVPN_DEVICE device = OvpnGetDeviceContext(adapter->WdfDevice);
  
      PWSK_DATAGRAM_INDICATION dataIndication = (PWSK_DATAGRAM_INDICATION)rxReturnContext;
  
      LOG_IF_NOT_NT_SUCCESS(((WSK_PROVIDER_DATAGRAM_DISPATCH*)device->Socket.Socket->Dispatch)->WskRelease(device->Socket.Socket, dataIndication));
  }
  

Not sure if this is the "best practices", but it works and doesn't crash on iperf3 tests.