<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:atom="https://www.w3.org/2005/Atom" xmlns:sy="https://purl.org/rss/1.0/modules/syndication/" xmlns:dc="https://purl.org/dc/elements/1.1/" xmlns:content="https://purl.org/rss/1.0/modules/content/" xmlns:wfw="https://wellformedweb.org/CommentAPI/" xmlns:slash="https://purl.org/rss/1.0/modules/slash/"
  version="2.0">
  <channel>
    <title>Twisty Little Passages, All Alike</title>
    <atom:link
      href="https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/feed.xml"
      rel="self"
      type="application/rss+xml" />
    <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/feed.xml</link>
    <description />
    <lastBuildDate>Sun, 12 Apr 2015 22:35:00 GMT</lastBuildDate>
    <language>en-US</language>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <item>
      <title>The final macro</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/the-final-macro</link>
      <pubDate>Thu, 21 Jul 2011 07:00:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/07/21/the-final-macro/</guid>
      <description><![CDATA[The problem

Cleaning up resources is hard.  One has to keep track of all allocated resources and...]]></description>
      <content:encoded><![CDATA[<h1>The problem</h1>

Cleaning up resources is hard.  One has to keep track of all allocated resources and make sure to free each one on each error path.  The difficulty is compounded, some say, with exceptions, which can make error paths non-obvious.  Traditionally, C++ programmers use <a href="https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization">RAII</a> wrappers (e.g. <a href="https://msdn.microsoft.com/en-us/library/ee410601.aspx">unique_ptr</a>
 and <a href="https://msdn.microsoft.com/en-us/library/ezzw7k98.aspx">CComPtr</a>) to automatically clean up resources --- but it's inconvenient to create a new type for each possible cleanup activity.  In exceptional code in particular, we're often forced to either create yet another RAII type, or duplicate cleanup code in the success and exception-rethrowing paths.  It becomes messy quickly.

<h1>The solution</h1>

Other languages provide facilities (like <a href="https://msdn.microsoft.com/en-us/library/zwc8s4fz.aspx">finally</a> and <a href="https://www.lispworks.com/documentation/HyperSpec/Body/s_unwind.htm">unwind-protect</a>) to delay the execution of bits of code until the current task is completed; it's natural to clean up resources in finally blocks.  C++ has never had finally blocks, but the latest language features allow us to create them ourselves.  I've always said that the wondering thing about C++ is its ability to create very powerful, but also easy-to-use abstractions. It's powerful enough to implement features other languages have to have built in. Let's look at an example:

<table style="margin-top:2em">
<tr><td><b>Code</b></td><td><b>Output</b></td></tr>
<tr><td valign="top">
<pre>
<font color="#228b22">void</font>
<font color="#0000ff">foo</font> (<font color="#228b22">int</font> <font color="#a0522d">n</font>)
{
    printf (<font color="#8b2252">"foo %d\n"</font>, n);
}

<font color="#228b22">void</font>
<font color="#0000ff">bar</font> ()
{
    <font color="#228b22">int*</font> <font color="#a0522d">x</font> = <font color="#a020f0">new</font> int (5);
    <b>FINALLY</b> (foo (*x), <font color="#a020f0">delete</font> x);

    printf (<font color="#8b2252">"Hello, world!\n"</font>);

    <b>FINALLY</b> ({
        <font color="#a020f0">if</font> (x) {
            printf (<font color="#8b2252">"Look, a statement!\n"</font>);
        }
    });

    printf (<font color="#8b2252">"Second line\n"</font>);
}

<font color="#228b22">void</font>
<font color="#0000ff">qux</font> ()
{
    <b>FINALLY</b> (printf (<font color="#8b2252">"After throw\n"</font>));
    printf (<font color="#8b2252">"throwing\n"</font>);
    <font color="#a020f0">throw</font> 42;
}

<font color="#228b22">int</font> <font color="#0000ff">main</font> ()
{
    bar ();
    printf (<font color="#8b2252">"bar returned\n"</font>);
    <font color="#a020f0">try</font> {
        qux ();
    } <font color="#a020f0">catch</font> (<font color="#228b22">int</font>) {
        printf (<font color="#8b2252">"caught exception\n"</font>);
    }
    
    printf (<font color="#8b2252">"qux returned\n"</font>);
}
</pre></td><td valign="top"><pre>
Hello, world!
Second line
Look, a statement!
foo 5
bar returned
throwing
After throw
caught exception
qux returned
</pre></td></tr></table>

<p>The code above isn't witchcraft.  It's standards-conforming† code. Here's the bit that makes it work:</p>

<pre>
<font color="#7a378b">#include</font> <font color="#8b2252">&lt;utility&gt;</font>

<font color="#a020f0">template</font>&lt;<font color="#a020f0">typename</font> <font color="#228b22">FunctorT</font>&gt;
<font color="#a020f0">struct</font> <font color="#228b22">FinallyUnwinder</font>
{
    <font color="#0000ff">FinallyUnwinder</font> (<font color="#228b22">FunctorT</font> <font color="#a0522d">Functor</font>)
        : Functor (<font color="#008b8b">std</font>::<font color="#228b22">move</font> (<font color="#a0522d">Functor</font>))
    {}

    <font color="#0000ff">FinallyUnwinder</font> (<font color="#228b22">FinallyUnwinder</font>&amp;&amp; <font color="#a0522d">Other</font>)
        : Functor (<font color="#008b8b">std</font>::move (Other.Functor))
    {}

    ~<font color="#0000ff">FinallyUnwinder</font> ()
    { <font color="#a020f0">this</font>-&gt;Functor (); }
    
    <font color="#a020f0">private</font>:
    <font color="#0000ff">FinallyUnwinder</font> (<font color="#a020f0">const</font> <font color="#228b22">FinallyUnwinder</font>&amp; <font color="#a0522d">other</font>);
    <font color="#228b22">FinallyUnwinder</font>&amp; <font color="#a020f0">operator</font><font color="#0000ff">=</font>(<font color="#a020f0">const</font> <font color="#228b22">FinallyUnwinder</font>&amp; <font color="#a0522d">other</font>);
    <font color="#228b22">FunctorT</font> <font color="#a0522d">Functor</font>;
};

<font color="#a020f0">template</font>&lt;<font color="#a020f0">typename</font> <font color="#228b22">FunctorT</font>&gt;
<font color="#a020f0">typename</font> <font color="#228b22">FinallyUnwinder</font>&lt;<font color="#228b22">FunctorT</font>&gt;
<font color="#0000ff">Finally</font> (<font color="#228b22">FunctorT</font> <font color="#a0522d">Functor</font>)
{
    <font color="#a020f0">return</font> FinallyUnwinder&lt;<font color="#228b22">FunctorT</font>&gt; (<font color="#008b8b">std</font>::move (Functor));
}

<font color="#7a378b">#define</font> <font color="#0000ff">UTX_CAT2</font>(<font color="#a0522d">a</font>,<font color="#a0522d">b</font>) a ## b
<font color="#7a378b">#define</font> <font color="#0000ff">UTX_CAT</font>(<font color="#a0522d">a</font>,<font color="#a0522d">b</font>) UTX_CAT2 (a, b)
<font color="#7a378b">#define</font> <font color="#0000ff">UTX_GENSYM</font>(<font color="#a0522d">ar</font>) UTX_CAT (ar, __LINE__)

<font color="#7a378b">#define</font> <font color="#0000ff">FINALLY</font>(...)                          \
    <font color="#228b22">auto</font> <font color="#0000ff">UTX_GENSYM</font> (utx_finally) =           \
        Finally ([&amp;] () { __VA_ARGS__ ; });

</pre>

<p>This code relies on <a href="https://en.wikipedia.org/wiki/Variadic_macro">variadic macros</a>, <a href="https://en.wikipedia.org/wiki/C%2B%2B0x#Lambda_functions_and_expressions">lambda functions</a>, and <a href="https://en.wikipedia.org/wiki/C%2B%2B0x#Rvalue_references_and_move_constructors">move constructors</a>, all of which are very recent features.
</p>
<p> To see how it all fits together, let's look at the preprocessed version of one of the lines above:</p>

<pre>
<font color="#228b22">auto</font> <font color="#a0522d">utx_finally65</font> = Finally ([&amp;] () { printf (<font color="#8b2252">"After throw\n"</font>) ; });;</pre>

<p>Here, we're creating a lambda functor object, capturing all variables from the parent function by their addresses (which is fine, because this lambda will never run after its parent function returns), and stashing the result away in an automatic variable; its type is specific to the functor being assigned, and its name is created via macro magic.  The compiler understands what's going on and can inline this entire process, producing code that's just as efficient as the hand-coded equivalent.


<h1>Conclusion</h1>

Using powerful primitives built into C++, we can create new expressive constructs, and we don't need to give up any performance in the process.  The above macro, while tricky to implement, is an convenient alternative to explicit RAII. While <a href="https://www.boost.org/doc/libs/1_44_0/libs/scope_exit/doc/html/index.html">previous attempts</a> to enjoyed some success, it's only with C++11 that we're able to provide a natural and convenient syntax.

<h1>Notes</h1>
<p>
† - variadic macros are technically supported in C99, not C++, but support for variadic macros in C++11 compilers is universal.
</p>]]></content:encoded>
    </item>
    <item>
      <title>Against BOOL parameters</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/against-bool-parameters</link>
      <pubDate>Thu, 02 Jun 2011 07:30:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/06/02/against-bool-parameters/</guid>
      <description><![CDATA[BOOL parameters are evil

Boolean parameters are everywhere. In public APIs all the way down to...]]></description>
      <content:encoded><![CDATA[<h1>BOOL parameters are evil</h1>
<p>
Boolean parameters are everywhere. In public APIs all the way down to private internal functions, code like this is ubiquitous:
</p>
<pre>
SaveData ("Bleg", TRUE);
</pre>
<p>
What does <code>TRUE</code> mean here? What does <code>TRUE</code> mean when dropped in the middle of a <a href="https://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx">larger parameter list</a>? A BOOL parameter is like a semantic bomb: hen you see a literal truth value, you have to stop what you're doing, refer to the context, the documentation of the function, or perhaps <a href="https://en.wikipedia.org/wiki/Divination">more desperate measures</a> to figure out what effect the word has. Everyone who's been programming for more than a few weeks has been frustrated in this spot.
</p>
<p>
It's because of this frustration that I'm going to be a heretic and say "no" to BOOL. If you find yourself writing "BOOL" in a parameter list, stop yourself. You'll be happier later:
</p>
<p>
<pre>
SaveData ("Bleg", SAVE_DATA_DO_NOT_OVERWRITE);
</pre>
</p>
<h1>A better alternative</h1>
<p>
<b>BOOL is just a poorly-named enumeration.</b> Consider <a href="https://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx">CreateEvent</a>. It has two BOOL parameters: bManualReset and bInitialState. If instead of BOOLs, there parameters were enumeration values &#x2014; say EVENT_MANUAL_RESET, EVENT_AUTO_RESET, EVENT_SIGNALED, EVENT_RESET &#x2014; then calling code would be much easier to understand. The effect of the call would be immediately apparent from the text of the program. It's important to note that the generated machine code is equally efficient†: the processor doesn't care whether an integral value comes from a boolean or an enumeration. By the time the processor gets to it, its type information is gone.
</p>
<p>
Using enums is better than BOOL, but we can actually do better still: each of CreateEvent's two BOOL parameters conveys only a single bit of information. We can combine them into a single flags parameter, resulting in code that remains clear to a human reader while being more extensible and arguably more efficient than the original. Our new flags parameter can be extended to support more behavior without changing the function signature, cutting down on API churn.
</p>
<h1>Conclusion</h1>
<h2>Don't</h2>
<ul><li>Use BOOL, bool, __bool, MyBool, and so on in function parameter lists.</li></ul>
<ul><li>Use many parameters when only a few would do.</li></ul>
<h2>Do</h2>
<ul><li>Replace BOOLs with either enumerations or flag parameters.</li>
<li>Give meaningful names to values passed to functions. TRUE and FALSE tell the reader nothing.</li>
</ul>
</p>

<p>
<small>
† Yes, a BOOL is one byte wide, while an enumeration (on Windows) is
four bytes wide. The difference doesn't matter in practice, but
even if it did, one could specify BYTE‡ in a function prototype instead of an enumeration
type, but nevertheless document that callers should specify 
<i>named</i> constants for that argument .
</small>
</p>

<p>
<small>
‡ Yes, it's possible to pass meaningful named values for a BOOL argument too. But the word
"BOOL" is just an invitation to pass TRUE and FALSE. At least BYTE
says, "I'm different".
</small>
</p>]]></content:encoded>
    </item>
    <item>
      <title>If you write UI, click this link</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/if-you-write-ui-click-this-link</link>
      <pubDate>Mon, 23 May 2011 13:01:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/05/23/if-you-write-ui-click-this-link/</guid>
      <description><![CDATA[]]></description>
      <content:encoded><![CDATA[<img src="https://i.imgur.com/7710d.png" />]]></content:encoded>
    </item>
    <item>
      <title>On assuming your customers are innumerate</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/on-assuming-your-customers-are-innumerate</link>
      <pubDate>Tue, 10 May 2011 13:42:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/05/10/on-assuming-your-customers-are-innumerate/</guid>
      <description><![CDATA[So I was shopping for a couch this week and saw one I liked. I asked about delivery...]]></description>
      <content:encoded><![CDATA[<p>
So I was shopping for a couch this week and saw one I liked. I asked about delivery options.
</p>
<p>
The salesperson replied, "It's an $80 base fee for delivery plus a $20 per-piece charge."
</p>
<p>Considering this price a little high, I asked, "Well, when could I come pick it up my self?"
</p>
<p>The salesperson cheerfully said, "The warehouse is open all day on weekdays, so any time is good. I should just let you know that there's a $25 pickup fee, though."
</p>
<p>"<b>What</b>? Why not just add $25 to the base price and make the base delivery fee $55?" 
</p>
<p>"Good idea. I'll bring it up with my manager," the salescritter lied.
</p>
<p>
I decided to shop elsewhere. Anyone capable of shenanigans like that is probably being underhanded with respect to other parts of his business. How on earth is this stuff legal?
</p>]]></content:encoded>
    </item>
    <item>
      <title>Breaking code on purpose</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/breaking-code-on-purpose</link>
      <pubDate>Wed, 27 Apr 2011 07:20:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/27/breaking-code-on-purpose/</guid>
      <description><![CDATA[Breaking code is useful.  When we run into an impossible situation, it's better to fail early and...]]></description>
      <content:encoded><![CDATA[<p>
Breaking code is useful.  When we run into an impossible situation, it's better to fail early and predictably, especially in a debug build, where the users of which are likely to notice and kindly sent a bug report.  (Most of the time, these bug reports will even lack profanity.)  On the other hand, in production code, it's sometimes best to muddle through as best we can in these situations.  A few months ago, a colleague of mine came up with a nifty way to mark "this can't happen" cases:
</p>
<pre>
BOOL
FrobWidget (
    Widget* WidgetToFrob
    )
{
    switch (WidgetToFrob-&gt;Color) {
        case WIDGET_COLOR_BLUE:
            assert (!"Blue widgets cannot be frobbed");
            return FALSE;
        ....
    }

    return TRUE;
}

</pre>
<p>
Doing it that way is clear, concise, and prettier than <code>assert (false)</code> and <code><a href="https://msdn.microsoft.com/en-us/library/k089yyh0(v=VS.100).aspx">abort</a></code>.  The string literal won't be included in binaries built without assertions.
</p>

<p>
Granted, we sometimes need to break code in production builds too.  <code>abort ()</code> is the traditional way to do it, but <code><a href="https://msdn.microsoft.com/en-us/library/ff543626(VS.85).aspx">DbgBreakPoint</a></code> is a better choice.  If no debugger is attached, the current process will exit unexpectedly, but if a debugger <i>is</i> attached, the debugger can step through a <code>DbgBreakPoint</code> call.  Unlike <code><a href="https://msdn.microsoft.com/en-us/library/ms679297(VS.85).aspx">__debugbreak</a></code>, <code>DbgBreakPoint</code> isn't a compiler intrinsic, so it'll do the right thing on any compiler.  Unlike <code><a href="https://msdn.microsoft.com/en-us/library/ms679297(VS.85).aspx">DebugBreak</a></code>, <code>DbgBreakPoint</code> will compile down to a single instruction when the compiler supports the proper intrinsic, making it easier to NOP out in a debugger.  <code>DbgBreakPoint</code> is the best choice for breaking code on purpose.
</p>]]></content:encoded>
    </item>
    <item>
      <title>Everyone quotes command line arguments the wrong way</title>
      <link>https://docs.microsoft.com/archive/blogs/twistylittlepassagesallalike/everyone-quotes-command-line-arguments-the-wrong-way</link>
      <pubDate>Sat, 23 Apr 2011 12:52:00 GMT</pubDate>
      <dc:creator><![CDATA[Daniel Colascione]]></dc:creator>
      <guid
        isPermaLink="false">https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/</guid>
      <description><![CDATA[Background

At one time or another, we all need to pass arbitrary command line arguments to some...]]></description>
      <content:encoded><![CDATA[<h1>Background</h1>
<p>
At one time or another, we all need to pass arbitrary command line arguments to some program.  These arguments could be filenames, debugging flags, hostnames, or any other kind of information: the point is that we are to take a string and make sure some child program receives exactly that string in its <a href="https://msdn.microsoft.com/en-us/library/bky3b5dh.aspx">argv array</a> no matter what this string contains.  The task is <a href="https://www.perseus.tufts.edu/Herakles/stables.html">harder than it appears</a>.
</p><p>
For better or for worse<sup>1</sup>, Windows knows about only <a href="https://msdn.microsoft.com/en-us/library/ms683156%28v=vs.85%29.aspx">one command line string</a> for each process.  Because one string is not terribly useful, libraries conspire to provide the illusion of multiple command line arguments: before creating a subprocess, a program combines all argument strings into one command line string, and the newly-born subprocess, before calling main, <a href="https://msdn.microsoft.com/en-us/library/17w5ykft%28v=vs.85%29.aspx">splits this string into arguments</a> and passes the arguments as <code>argv</code>.  In principle, each program can parse the command line string differently, but most use the <a href="https://blogs.msdn.com/b/oldnewthing/archive/2010/09/17/10063629.aspx">convetion</a> that <a href="https://msdn.microsoft.com/en-us/library/bb776391%28v=vs.85%29.aspx">CommandLineToArgvW</a> and the Microsoft C library understand.  This convention is a <a href="https://secure.wikimedia.org/wikipedia/en/wiki/Bijective_function">good one</a> because it provides a way to encode <i>any</i> command line argument as part of a command line string without losing information.
</p><p>
The problem is that there is no ArgvToCommandLineW.  How do we construct an argument string understood by CommandLineToArgvW?
</p>
<h2>Test program</h2>
For exposition's sake, we'll be using this small program to generate the example output below:
<pre>
#include &lt;stdio.h&gt;
int __cdecl
wmain(int, wchar_t** argv)
{
    for (int arg = 0; argv[arg]; ++arg) {
        wprintf (L"%d: [%s]\n", arg, argv[arg]);
    }
}
</pre>

<h1>Popular solutions: clear, simple, and wrong</h1>

<h2>The C runtime library is useless</h2>
<p>
Our first instinct should be look for a library function that's already solved the problem.  Were we to conduct this search, we would quickly find functions provided by the C runtime specifically for running subprocesses: the <a href="https://msdn.microsoft.com/en-us/library/431x4c1w.aspx">_exec</a> and <a href="https://msdn.microsoft.com/en-us/library/20y988d2.aspx">_spawn</a> families.  These functions <i>appear</i> to be precisely what we need: they take an arbitrary number of distinct command line arguments and promise to launch a subprocess.  Unfortunately and counter-intuitively, these functions do not quote or process these arguments: instead, they're all concatenated into a single string, with arguments separated spaces, and this string is then passed to the child, where it's eventually interpreted by CommandLineToArgvW.  This approach works only for arguments that do not themselves contain spaces.
</p>
<p>
Thus, if we run child.exe like this:
</p>
<pre>_spawnlp (_P_WAIT, L"child.exe", L"child.exe", L"argument 1", L"argument 2", NULL)</pre>
<p>
child.exe receives <i>five</i> command line parameters:
</p>
<pre>
0: [child.exe]
1: [argument]
2: [1]
3: [argument]
4: [2]
</pre>

<p>
That's not what we want! So, while the C runtime process-launching functions appear to be what we need, they're actually useless for solving the command line argument problem.
</p>

<h2>Adding quotation marks is insufficient</h2>

<p>
Having seen the problems with the previous approach, one might suggest that we heed the advice provided in the the C runtime documentation and surround arguments containing spaces with double-quote characters.  This solution is also wrong.
</p>

<p>
Recall<sup>2</sup> that the <a href="https://msdn.microsoft.com/en-us/library/17w5ykft%28v=vs.85%29.aspx">CommandLineToArgV convention</a><sup>3</sup> stipulates that arguments containing spaces be surrounded by double quotation marks.  Following the above approach and surrounding arguments with quotation marks produces good results for simple cases, and many people stop here.
</p>
<pre>
child.exe argument1 "argument 2"  "\some\path with\spaces"
</pre>
<p>
Is correctly interpreted as:
</p>
<pre>
0: [child.exe]
1: [argument1]
2: [argument 2]
3: [\some\path with\spaces]
</pre>
<p>
So far, so good: but what if our arguments are more complex? Bear in mind that our convention <i>also</i> stipulates that we precede a double quotation mark that is <i>part</i> of an argument with a backslash, and that we precede with another backslash a backslash that precedes a quotation mark which itself actually terminates the argument and is not included as part of that argument<sup>4</sup>.  So, if we follow our simplistic approach above and use this as our command line:

<pre>
child.exe argument1 "she said, "you had me at hello""  "\some\path with\spaces"
</pre>

<p>
then the child sees:
</p>

<pre>
0: [child.exe]
1: [argument1]
2: [she said, you]
3: [had]
4: [me]
5: [at]
6: [hello]
7: [\some\path with\spaces]
</pre>

<p>
We don't want that either! The problem becomes more insidious when quotes are unbalanced:
</p>

<pre>
child.exe argument1 "argument"2" argument3 argument4
0: [child.exe]
1: [argument1]
2: [argument2 argument3 argument4]
</pre>

<p>Arguments ending with backslashes also lead to undesired interpretations:</p>

<pre>
child.exe "\some\directory with\spaces\" argument2

0: [child.exe]
1: [\some\directory with\spaces" argument2]
</pre>

<p>
Many popular programs (including command shells, the authors of which really should know better) use this simple approach.  Developers test only with simple argument strings, leaving users confused and puzzled when their command lines are occasionally mangled.
</p>

<h1>The correct solution</h1>

We've seen that properly quoting an arbitrary command line argument is non-trivial, and that doing it incorrectly causes subtle and maddening problems.  The function below properly quotes an argument; translate it into your language and coding style of choice.

<pre>
void
ArgvQuote (
    const std::wstring& Argument,
    std::wstring& CommandLine,
    bool Force
    )
    
/*++
    
Routine Description:
    
    This routine appends the given argument to a command line such
    that CommandLineToArgvW will return the argument string unchanged.
    Arguments in a command line should be separated by spaces; this
    function does not add these spaces.
    
Arguments:
    
    Argument - Supplies the argument to encode.

    CommandLine - Supplies the command line to which we append the encoded argument string.

    Force - Supplies an indication of whether we should quote
            the argument even if it does not contain any characters that would
            ordinarily require quoting.
    
Return Value:
    
    None.
    
Environment:
    
    Arbitrary.
    
--*/
    
{
    //
    // Unless we're told otherwise, don't quote unless we actually
    // need to do so --- hopefully avoid problems if programs won't
    // parse quotes properly
    //
    
    if (Force == false &&
        Argument.empty () == false &&
        Argument.find_first_of (L" \t\n\v\"") == Argument.npos)
    {
        CommandLine.append (Argument);
    }
    else {
        CommandLine.push_back (L'"');
        
        for (auto It = Argument.begin () ; ; ++It) {
            unsigned NumberBackslashes = 0;
        
            while (It != Argument.end () && *It == L'\\') {
                ++It;
                ++NumberBackslashes;
            }
        
            if (It == Argument.end ()) {
                
                //
                // Escape all backslashes, but let the terminating
                // double quotation mark we add below be interpreted
                // as a metacharacter.
                //
                
                CommandLine.append (NumberBackslashes * 2, L'\\');
                break;
            }
            else if (*It == L'"') {

                //
                // Escape all backslashes and the following
                // double quotation mark.
                //
                
                CommandLine.append (NumberBackslashes * 2 + 1, L'\\');
                CommandLine.push_back (*It);
            }
            else {
                
                //
                // Backslashes aren't special here.
                //
                
                CommandLine.append (NumberBackslashes, L'\\');
                CommandLine.push_back (*It);
            }
        }
    
        CommandLine.push_back (L'"');
    }
}
</pre>

<p>
To construct a command line string for a program from arbitrary arguments, we encode each argument (including the program name) with the above function and follow all but the last with a single space.  We can then pass the resulting string as the lpCommandLine parameter to <a href="https://msdn.microsoft.com/en-us/library/ms682425%28v=vs.85%29.aspx">CreateProcess</a> and be confident that the child process will decode each argument to exactly the arguments we were initially given.
</p>

<h1>cmd.exe</h1>

<p>
One might conclude that we're done: after all, we now understand how to send arbitrary strings through CreateProcess so that they emerge unchanged on the other side --- but life is not that simple.  Often, we can't directly supply our command line CreateProcess, but instead must pass it through a level of indirection before reaching our intended child process.  The most common indirection is a trip trough the venerable <a href="https://msdn.microsoft.com/en-us/library/bb490880.aspx">cmd.exe</a>, which we encounter when we use the <a href="https://msdn.microsoft.com/en-us/library/277bwbdz.aspx">system</a> function, construct a script for later execution, write a makefile for <a href="https://msdn.microsoft.com/en-us/library/2kzfk8c7.aspx">nmake</a>, and do many other things. Because the quoting rules for CommandLineToArgvWcmd's differ from those of cmd, we cannot give a command line intended for the former to the latter and and expect our arguments to survive the trip.  Because this difference is subtle, it's easy to forget about it and leave latent bugs in programs that interact with cmd.
</p>

<p>
cmd is essentially a text preprocessor: given a command line, it makes a series of textual transformations, then hands the transformed command line to CreateProcess.  Some transformations replace environment variable names with their values.  Some transformations, such as IO redirection, have useful side effects.  Some transformations, such as those triggered by the &amp;, ||,  &amp;&amp; operators, split command lines into several parts.  It's important to note that cmd doesn't know or care about command line <i>arguments</i> per se: to cmd, the world is made up of whole command lines.  Like the post office delivering postcards, cmd doesn't try to understand what is handles: it merely does its limited job (via these transformations), then leaves it another program to figure out what the final command line means.
</p> 

<p> 
All of cmd's transformations are triggered by the presence of one of the <i>metacharacters</i>  <code>(, ), %, !, ^, ", &lt;, &gt;, &amp;,</code> and <code>|</code>. <code>"</code> is particularly interesting: when cmd is transforming a command line and sees a <code>"</code>, it copies a <code>"</code> to the new command line, then begins copying characters from the old command line to the new one without seeing whether any of these characters is a metacharacter. This copying continues until cmd either reaches the end of the command line, runs into a variable substitution, or sees another <code>"</code>.  In the last case, cmd copies a <code>"</code> to the new command line and resumes normal processing.  This behavior is <i>almost</i>, but not <i>quite</i> like what CommandLineFromArgvW does with the same character<sup>4</sup>; the difference is that cmd does not know about the <code>\"</code> sequence and begins interpreting metacharacters earlier than we would expect.  It should be apparent why the commands below produce the indicated output:
</p>

<pre>
C:\&gt; child "hello world" >\\.\nul

C:\&gt; child "hello"world" >\\.\nul
0: [child]
1: [helloworld &gt;\\.\nul]

C:\&gt; child "hello\"world" >\\.\nul
0: [child]
1: [hello"world]
2: [&gt;\\.\nul]

C:\&gt;
</pre>

<p>
If we relying on cmd's <code>"</code>-behavior to protect arguments, quotation marks will produce unexpected behavior.  If we pass untrusted data as command line parameters, then the bugs caused by this convention mismatch become a <b>security issues</b>:
</p>

<pre>
C:\&gt; child "malicious argument\"<b>&amp;whoami</b>"
0: [child]
1: [malicious-argument"]
<b>ntdev\dancol</b>
</pre>

<p>Here, cmd is interpreting the <code>&amp;</code> metacharacter as a command separator because, from its point of view, the <code>&</code> character lies outside the quoted region. whoami, of course, can be replaced by any number of harmful commands.  Note that this command is properly formatted for use with CreateProcess: it's the passing through cmd that causes trouble.
</p>

<h2>A better method of quoting</h2>

<p>
While the <code>"</code> metacharacter cannot fully protect metacharacters in our command lines against unintended shell interpretation, the <code>^</code> metacharacter can.  When cmd transforms a command line and sees a <code>^</code>, it ignores the <code>^</code> character itself and copies the next character to the new command line literally, metacharacter or not.  That's why <code>^</code> works as the line continuation character: it tells cmd to copy a subsequent newline as itself instead of regarding that newline as a command terminator.  If we prefix with <code>^</code> every metacharacter in an argument string, cmd will transform that string into the one we mean to use.

<pre>
C:\&gt; child ^"malicious argument\^"^&amp;whoami^"
0: [child]
1: [malicious argument"&amp;whoami]
</pre>

<p>
It's important to also <code>^</code>-escape <code>"</code> characters: otherwise, cmd would literally copy <code>^</code> characters appearing between <code>"</code> pairs, mangling the argument string in the process:
</p>
<pre>
C:\&gt; child "malicious-argument\^"^&amp;whoami"
0: [child]
1: [malicious-argument\^&amp;whoami]
</pre>

<p>
In effect, because cmd's <code>"</code> handling is useless for our purposes, we use <code>^</code> to tell cmd to not attempt to be smart about quote detection.
</p>

<h1>Conclusion</h1>

<p>In general, we can safely pass arbitrary command line arguments to programs, provided we take a few basic precautions.</p>
<p><b>Do</b>:</p>
<ol><li>Always escape all arguments so that they will be decoded properly by CommandLineToArgvW, perhaps using my ArgvQuote function above.</li>
<li>After step 1, then <b>if and only if</b> the command line produced will be interpreted by cmd, prefix each shell metacharacter (or each character) with a <code>^</code> character.</li></ol>
<p><b>Do not</b>:</p>
<ol><li>Simply add quotes around command line argument arguments without any further processing.</li><li>Allow cmd to ever see an unescaped <code>"</code> character.</li></ol>

<h3>Notes</h3>

<p>
<sup>1</sup> Worse.<br />
<sup>2</sup> You <i>did</i> follow my links above, yes?<br />
<sup>3</sup> I know you didn't.<br />
<sup>4</sup> Just to be clear: CommandLineFromArgvW neither knows nor cares about cmd's metacharacters and looks <b>only</b> for <code>"</code> and <code>\</code>.<br />
</p></p></p>]]></content:encoded>
    </item>
  </channel>
</rss>