Edit

DecryptMessage (Schannel) function

  • Article

The DecryptMessage (Schannel) function decrypts a message. Some packages do not encrypt and decrypt messages but rather perform and check an integrity hash.

This function is also used with the Schannel security support provider (SSP) to signal a request from a message sender for a renegotiation (redo) of the connection attributes or for a shutdown of the connection.

Note

EncryptMessage (Schannel) and DecryptMessage (Schannel) can be called at the same time from two different threads in a single security support provider interface (SSPI) context if one thread is encrypting and the other is decrypting. If more than one thread is encrypting, or more than one thread is decrypting, each thread should obtain a unique context.

Syntax

SECURITY_STATUS SEC_Entry DecryptMessage(
  _In_    PCtxtHandle    phContext,
  _Inout_ PSecBufferDesc pMessage,
  _In_    ULONG          MessageSeqNo,
  _Out_   PULONG         pfQOP
);

Parameters

phContext [in]

A handle to the security context to be used to decrypt the message.

pMessage [in, out]

A pointer to a SecBufferDesc structure. On input, the structure references one or more SecBuffer structures. One of these may be of type SECBUFFER_DATA. That buffer contains the encrypted message. The encrypted message is decrypted in place, overwriting the original contents of its buffer.

When using the Schannel SSP with contexts that are not connection oriented, on input, the structure must contain four SecBuffer structures. Exactly one buffer must be of type SECBUFFER_DATA and contain an encrypted message, which is decrypted in place. The remaining buffers are used for output and must be of type SECBUFFER_EMPTY. For connection-oriented contexts, a SECBUFFER_DATA type buffer must be supplied, as noted for nonconnection-oriented contexts. Additionally, a second SECBUFFER_TOKEN type buffer that contains a security token must also be supplied.

MessageSeqNo [in]

The sequence number expected by the transport application, if any. If the transport application does not maintain sequence numbers, this parameter must be set to zero.

When using the Schannel SSP, this parameter must be set to zero. The Schannel SSP does not use sequence numbers.

pfQOP [out]

A pointer to a variable of type ULONG that receives package-specific flags that indicate the quality of protection.

When using the Schannel SSP, this parameter is not used and should be set to NULL.

This parameter can be the following flag.

Value Meaning
SECQOP_WRAP_NO_ENCRYPT
 The message was not encrypted, but a header or trailer was produced.
Note: KERB_WRAP_NO_ENCRYPT has the same value and the same meaning.

Return value

If the function verifies that the message was received in the correct sequence, the function returns SEC_E_OK.

If the function fails to decrypt the message, it returns one of the following error codes.

Return code Description
SEC_E_INVALID_HANDLE A context handle that is not valid was specified in the phContext parameter. Used with the Schannel SSP.
SEC_E_INVALID_TOKEN The buffers are of the wrong type or no buffer of type SECBUFFER_DATA was found. Used with the Schannel SSP.
SEC_E_MESSAGE_ALTERED The message has been altered. Used with the Schannel SSP.
SEC_E_OUT_OF_SEQUENCE The message was not received in the correct sequence.
SEC_I_CONTEXT_EXPIRED The message sender has finished using the connection and has initiated a shutdown. For information about initiating or recognizing a shutdown, see Shutting Down an Schannel Connection. Used with the Schannel SSP.
SEC_I_RENEGOTIATE The remote party requires a new handshake sequence or the application has just initiated a shutdown. Return to the negotiation loop and call AcceptSecurityContext (Schannel) or InitializeSecurityContext (Schannel), pass SECBUFFER_EXTRA returned from DecryptMessage().

Remarks

Sometimes an application will read data from the remote party, attempt to decrypt it by using DecryptMessage (Schannel), and discover that DecryptMessage (Schannel) succeeded but the output buffers are empty. This is normal behavior, and applications must be able to deal with it.

When you use the Schannel SSP, the DecryptMessage (General) function returns SEC_I_CONTEXT_EXPIRED when the message sender has shut down the connection. For information about initiating or recognizing a shutdown, see Shutting Down an Schannel Connection.

If you are using TLS 1.0, you may need to call this function multiple times, adjusting the input buffer on each call, to decrypt a whole message.

The DecryptMessage (Schannel) function returns SEC_I_RENEGOTIATE when a post-handshake TLS protocol message other than application data is received. Once DecryptMessage (Schannel) function has returned SEC_I_RENEGOTIATE, any further EncryptMessage() and DecryptMessage() calls will fail with SEC_E_CONTEXT_EXPIRED. An application handles this situation by calling AcceptSecurityContext (Schannel) (server side) or InitializeSecurityContext (Schannel) (client side) and passing SECBUFFER_EXTRA returned from DecryptMessage(). After this initial call returns a value, proceed as though your application were creating a new connection. Then the application can continue calling EncryptMessage() and DecryptMessage(). For more information, see Creating an Schannel Security Context.

Requirements

Requirement Value
Minimum supported client Windows XP [desktop apps only]
Minimum supported server Windows Server 2003 [desktop apps only]
Header Sspi.h (include Security.h)
Library Secur32.lib
DLL Secur32.dll

See also