strtok, _strtok_l, wcstok, _wcstok_l, _mbstok, _mbstok_l

Article
11/16/2012

Find the next token in a string, using the current locale or a specified locale passed in. More secure versions of these functions are available; see strtok_s, _strtok_s_l, wcstok_s, _wcstok_s_l, _mbstok_s, _mbstok_s_l.

char *strtok(
   char *strToken,
   const char *strDelimit 
);
wchar_t *wcstok(
   wchar_t *strToken,
   const wchar_t *strDelimit 
);
unsigned char *_mbstok(
   unsigned char*strToken,
   const unsigned char *strDelimit 
);
unsigned char *_mbstok(
   unsigned char*strToken,
   const unsigned char *strDelimit,
   _locale_t locale
);

Parameters

strToken
In the first call to strtok, this parameter is a pointer to a writeable string that contains one or more tokens. Pass NULL for this parameter in subsequent calls to strtok to find the remaining tokens.
strDelimit
Set of delimiter characters.
locale
Locale to use.

Return Value

Returns a pointer to the next token found in strToken, or NULL when no more tokens are found. Each call modifies strToken by substituting a null character for the first delimiter that occurs after the returned token.

Remarks

The strtok function finds the next token in strToken. The set of characters in strDelimit specifies the delimiters of the token to be found in strToken.

wcstok and _mbstok are wide-character and multibyte-character versions of strtok. The arguments and return value of wcstok are wide-character strings; those of _mbstok are multibyte-character strings. These three functions behave identically otherwise.

Security Note:
These functions incur a potential threat brought about by a buffer overrun problem. Buffer overrun problems are a frequent method of system attack, resulting in an unwarranted elevation of privilege. For more information, see Avoiding Buffer Overruns.

On the first call to strtok, the function skips leading delimiters and returns a pointer to the first token in strToken, terminating the token with a null character. More tokens can be broken out of the remainder of strToken by a series of calls to strtok. Each call to strtokmodifies strToken by inserting a null character after the token returned by that call.

To read subsequent tokens from the original strToken parameter, call strtok with NULL for strToken.

The output value is affected by the setting of the LC_CTYPE category setting of the locale; see setlocale for more information. The versions of these functions without the _l suffix use the current locale for this locale-dependent behavior; the versions with the _l suffix are identical except that they use the locale parameter passed in instead. For more information, see Locale.

Note

Each function uses a thread-local static variable for parsing the string into tokens. Therefore, multiple threads can simultaneously call these functions without undesired effects. However, within a single thread, interleaving calls to one of these functions is highly likely to produce data corruption and inaccurate results. When parsing multiple strings, finish parsing one string before parsing the next.

Generic-Text Routine Mappings

TCHAR.H routine	_UNICODE & _MBCS not defined	_MBCS defined	_UNICODE defined
_tcstok	strtok	_mbstok	wcstok
_tcstok	_strtok_l	_mbstok_l	_wcstok_l

_strtok_l and _wcstok_l are not dependent on locale and are not meant to be called directly. They are provided for internal use by _tcstok_l.

Requirements

Routine	Required header
strtok	<string.h>
wcstok	<string.h> or <wchar.h>
_mbstok, _mbstok_l	<mbstring.h>

For additional compatibility information, see Compatibility in the Introduction.

Example

// In this program, a loop uses strtok
// to print all the tokens (separated by commas, tabs,
// or blanks) in the string 
//
#include <string.h>
#include <stdio.h>

char string[] = "A string\tof ,,tokens\nand some  more tokens";
char seps[]   = " ,\t\n"; // note the initial space character
char *token;

int main( void )
{
   printf( "Tokens:\n" );
 
   // Establish string and get the first token:
   token = strtok( string, seps ); // C4996
   // Note: strtok is deprecated; consider using strtok_s instead
   while( token != NULL )
   {
      // While there are tokens in "string"
      printf( " %s\n", token );

      // Get next token: 
      token = strtok( NULL, seps ); // C4996
   }
}

Tokens: A string of tokens and some more tokens

.NET Framework Equivalent

Not applicable. To call the standard C function, use PInvoke. For more information, see Platform Invoke Examples.

Change History

Date	History	Reason
May 2009	Topic cleanup and clarification of strToken parameter.	Information enhancement.