The entire code can be found on my Github: https://github.com/guyalster/SkipList

SKIP LIST, so what is it all about?

A skip list is a dynamic and randomized data structure that allows storing keys in a sorted fashion. The skip list is in essence, a linked list with multiple levels. Each level is itself a linked list and with pointers from the upper levels to the lower levels. The thing that is special about this data structure is that its expected runtime for all operations (insertion, deletion, searching) is O(Log(N)). So how is this runtime achieved? Well, that's a result of the randomness of the skip list and how it is built. But before we talk about how we achieve this runtime lets first answer a more fundamental question. Why do we even need it? We already have binary search trees which are great data structures for storing sorted keys (and values) and achieving a logarithmic runtime when they are balanced. In fact, they are deterministic in behavior and have a guaranteed logarithmic runtime (rather than an expected one). The answer to this question will motivate us to continue this quest. First, implementing a balanced binary search tree can be a very tricky business. Anyone who has tried to implement an AVL/RB tree has seen the complications. On the other hand, as I'll show later in this post, implementing a skip list is much easier. But there is one more important advantage to skip lists over their tree counterparts. That is synchronization. Locking in skip lists can be done in a much more efficient way than in trees, but most importantly, synchronization in skip lists can be implemented without any locking at all while guaranteeing consistency. However, this is a subject that I will not get into in this post as it is also very complicated and requires much more insight and depth. However, mentioning this is crucial to justify why this data structure is so important.

How to build a skip list:

So now we come to the juicy part. How to build a skip list. Well, the best analogy to a skip list I've heard came from an MIT Data Structures & Algorithms class. There the lecturer explained skip lists in terms of the NYC subway system. The NYC subways have multiple lines which go in the same direction. However, one of them passes through every station on its way to the destination, while others pass through some of the stations. The express line passes through the least amount of stations. If you want to get from station A to M, the best strategy is to take the express train from station A directly to station F. Then take the next train from F to J that also stop at stations H and I. Then take the slowest train from I to M that stops at every station. This, of course, is a better strategy than taking the slowest train from A to M that stops in all stations on its way. This is the same idea of how a skip list works and where it derives its efficiency from.  So let's examine how we build the skip list from the ground up. We start with the lowest level (that is the slowest train lane), we then toss a coin, and if its "heads" we insert the element in the next upper level. We continue to do this until we draw "tail". The actual way the skip list will be implemented is slightly different but in essence, is the same as I just described. The idea is that when we want to insert a new node to the skip list, we first generate a number between 0 to MAX_SKIP_LEVELS (which is a constant defined by the user). This number will determine in how many levels this new node will appear. Inserting the new node starts from the topmost level that is equal to the random level we generated (the express lane) and we move to the right in that lane until we reach a node that has a key that is greater than our new node. That will be our insertion point. From that point, we continuously insert our node while adjusting the "next" pointers until and including the last level (e.g. level 0). Searching of an existing key will be done in a similar fashion: We'll start from the topmost level of the skip list, going right (using the next pointer) until we hit a node with a key >= our key. We then go down to the next level until we reach the topmost level that contains our node. As you can see from these, the runtime of insertion, searching, and deletion is proportional to the number of nodes we have in each level. What is the expected number of nodes we have in the level i? It is easy to see based on the random process that it will be about half of the nodes that appear in the previous (bottom) layer. This should sound familiar at this point. We notice that in essence. a skip list resembles very much a balanced binary search tree. The main difference is that a key appears in the parent as well as in the child nodes all the way down to the leafs.  Using this intuition, we can now see why those operations of the skip list are expected to be O(Log(N)) where N is the number of elements stored in the list.

Skip List Data Structure and APIs:

In GitHub all of the above are located in the SkipList.h At the top, we have the definition of a SkipNode. The SkipNode has only two fields: the current level of that node and a pointer to pointers to other SkipNodes. The pointer to pointers can be viewed as a dynamic array of lists. This array represents all the levels that this node participates in. This seems strange at first. How many different pointers will the array contain? This of it like this: Image Node N1 with key K1, appears in 10 Levels. Imagine that Node N2 with key k2 is greater than k1 and appears in 5 levels while Node N3 with key k3 which is also greater than k1 appears in 11 levels. N1's array will have the first 5 pointers in its array pointing to N2 and the next 5 pointers pointing to N3. Let's assume that we're looking for N2 in the SkipList. When we reach level 10 we'll get to N1 which is less than N2. we will see that at level 10 through 6 it points to N3 which is greater than N2, so we'll go down in the levels until we reach level  5, where we'll find N2 as the next pointer of N1 and return it.

The next definition is of a SkipList structure. Notice that this structure contains only three function pointers. This structure is the C equivalent of a C++ interface with pure virtual functions. Who will implement this interface?  That will be a structure that is defined in SkipList.c. The reason it is defined in the .c file rather than in the .h file is for encapsulation purposes. How does the user of the library get a pointer to the SkipList interface? This is done using the initialization function defined just below the structure. But there is something interesting about that function (InitializeSkipList). Notice that it accepts three parameters: size_t - This is the maximal number of levels. SkipCompare - This is a type of a function pointer defined above and which receives two skip lists and returns the larger of them. ValCompare - a function pointer which accepts a pointer to a key (void*) and a skipNode* and returns if that key is greater than the key in that SkipNode. So, it is the responsibility of the user to tell the library how to compare the values in the SkipNode by providing the comparators to the library upon initialization. But wait a minute! that's weird! Shouldn't the library know itself how to compare these Nodes ? after all, the nodes are defined here. But if you'll take a closer look, you'll notice how none of the structures defined above contain keys and values. In fact, they are very stripped down. So where are the keys and the values that are supposed to be stored inside each SkipNode? That is the trick here. This is a design decision I made when I wrote this library. If I were to write this code in C++, I would use templates to define the SkipNode in a generic way that is independent of the key type it operates on.  But in C I don't have that luxury. In C the only way for me to create generic types is by defining them as void *. Instead of doing that, I decided to take a different approach. I decided to adopt the approach that is often used in operating system kernels, which is to have the data structure be embedded inside the main user structure. That is, instead of having the  SkipNode contain the user's types, the user will hold a SkipNode inside his/hers structures.

For example:

But how will this work? Since the library only operates on SkipNodes, how will we be able to get the containing structure? This is a critical piece of the puzzle. in the SkipList.h file you will find a macro defined as: This Macro is not simple to understand so I suggest examining it carefully to see what it does. However, I will mention that it uses the relative location of the SkipNode variable inside the enclosing structure to get the address of the enclosing structure. This is a worthwhile trick to master by itself.

So far we have seen the structures that are exposed to the users of the library. But as you probably noticed none of them contain the metadata to manage the skiplist. Furthermore, ideally, we would like to have a sentinel node which does not contain any keys or values but is used to get access to the rest of the skip list. This sentinel structure is defined in the .C file (again for the purposes of encapsulation).

Are you seeing the object-orientedness here? The first variable is the SkipList interface pointer. When the user calls the initialization function: InitializeSkipList(...) he expects to get a pointer to a SkipList structure. We will return to the user the address of the Sentinel object cast to be a SkipList pointer. This is classic C style polymorphism. Notice that the interface function pointers inside the SkipList structure all accept a pointer to a SkipList structure as well. The user will pass back to each one of the API functions the pointer to the SkipList. The implementation will cast that pointer back to be SkipNodeSentinal and will have the access it needs to all the metadata of the skip list. Again, classic C style polymorphism. The rest of the variables inside the SkipNodeSentinal are not worth emphasizing as they are all part of the metadata of the skip list.

I highly recommend going over the code in my GitHub. I really tried to put a lot of emphasis on good software engineering and to make the interface as clean as possible. Enjoy.

Let's start first with the original problem. We would like to implement the function: int  strstr(const string &Str, const string& Pattern); 

Most of us have had the chance of implementing this function at some point or another during our careers. The brute force solution of this function entails two loops, one over Str and one over pattern. Let's see it:

What is the runtime complexity of this algorithm? Since we are running in two loops and for each i we are trying all j's, we have a worst-case complexity of O(N*M) where N is the size of Str and M is the size of Pat. Can you spot where the inefficiency comes from?  The problem is that if we can't find a match we have to go back (with i) and start over from where we left off when we started the match. Why are we doing this? let's look at two examples:

* str = "abcbcefg" and pat = "bce".  In this case we start the match at i = 1 and j = 0, we continue to i = 2 and j = 1, but then at i = 3 and j = 2 we have a mismatch. Our algorithm will detect this mismatch and will set "i" to be 2. But what's the point? for i = 2 we already know that there will be no match. In fact, based on strwe should not be setting "i" to a previous index, but we could just as well continue advancing i.

* str = "abababc" pat = "ababc". In this case, we will start matching at i = 0 and will reach i = 4 when we detect a mismatch. We will then return to i = 1 and start again. When we reach i = 2 we will detect that the strings match. The observation is, that we should not have even bothered returning to i = 1 we could have set "i"to 2. But, if we look even closer we may notice that after we matched str = "abab" and pat = "abab" we already have one pair of "ab" in pat which we know that can be used for future matches.

The two examples are a great segway to explain how KMP works and why it is so efficient. KMP starts off by building a prefix table for pat. This prefix table will be used to indicate for each index "i" how much we have matched thus far and use this information to jump back only by the amount needed.

Lets see how the prefix table will look like for different patterns:

pat = "ababc" will result in the table: [0,0,1,2,0]

pat = "ississi" will result in the table: [0,0,0,1,2,3,4]

pat = "abcd" will result in the table: [0,0,0,0]

Notice what is happening here ? when we detect a prefix that we have seen (such as the second "ab" in "ababc") the numbers in the table indicate the size of that prefix. So if we start matching "abababc" with "ababc" for i = 4 and j = 4 we see that 'a' is different from 'c', and hence we have a mismatch. Using the table above we see that the table at index j - 1 = 3 has the value 2. We can have "j" jump back two indexes from 4 to 2, and continue our match from there. We don't have to touch "i" anymore, we just modify "j" based on the maximal size of the prefix that we have seen thus far.

Lets see the code to generate the Prefix table:

We allocate the prefix table to be the size of "Pat". We use two indexes to fill the table. The first is "i", it will indicate the end of the current prefix. The second is "j" which will be used to find the next occurance of the prefix. Notice that "j" starts from index 1. As we are traversing we check to see whether Pat[i] == Pat[j]. If they are, then we know that there is a prefix of size i + 1 that we have encountered at index "j". We update the table at index "j" to indicate this prefix, and update "i" as well. If they are not the same, then we have two cases: First, if i == 0, it means that we have not found any prefix thus far and we just increment "j". However, if  i > 0, we keep "j" where it is and make "i" go back to the index where the prefix  has started and compare it again with "j" . To better explain this scenario I will show an example:

Pat = "aabaabaaaab". The prefix table for this will be: [0,1,0,1,2,3,4,5,2,2,3]. When j = 1, we have our first match and i is increment to 1. However, for j = 2 we have a mismatch Pat[i = 1] != Pat[j = 2] and "i" goes back to 0. When j = 3, things start becoming more interesting. We set PrefixT[3] = 1. For j = 4 we have a prefix of size 2 matching, "i" becomes 2 and PrefixT[4] = 2. For j = 5 we have a prefix of size 3 matching (e.g. "aab") we update "i" and "j again. In fact we continue to have more matches up to j = 8 when Pat[j = 8] = 'a' and Pat[i = 5] = 'b'. At that point since i > 0 we don't advance j but we move i to be i = PrefixT[i - 1] = 2. Pat[i = 2] = 'b'  while Pat[j = 8] = 'a' which is again a mismatch. We set i = Prefix[i - 1] = 1. In this case we have Pat[i = 1] = 'a', which is equal to Pat[j = 8], and so PrefixT[8] = i + 1= 2. We increase i by 1 to i = 2 and increase j by 1 to j = 9. Since Pat[i = 2] == Pat[j == 9] we set PrefixT[9] = i + 1 = 3.  What does it tell us ? It tells us that at index 9 (the end of Pat) we have been able to match a prefix of size 3 of Pat (e.g. "aab"). So even though we had a slight mismatch at index j = 8 and i = 5, we used this prefix table to jump back as much as needed based on the previous prefix match so that we can continue matching afterwards.

We are now ready to see the full algorithm and how it uses the prefix table.

Notice how we use the Prefix Table only when a mismatch occurs and if "j" which in this case points to pat is greater than 0. The runtime complexity of this new version in the worst case will be O(N) the size of str.

Now we have reached the second part. In this part we will show how to solve an interesting problems using KMP algorithm.

The problem asks us to convert a string to a palindrome by appending at its beginning the minimum number of letters. For example: "aacecaaa" can be turned into a palindrome by appending a single "a" at its beginning. Furthermore, consider the following string: "abcd" to turn it into a palindrome we need to append "dcb" at its beginning. What about the string "abba"? Since it is already a palindrome, we needn't append anything at its beginning. The problem asks us to return the palindrome string after we appended the letters to it.

With a brute force approach, we may try to find the longest palindrome from the beginning. Everything after that consists of letters that we need to append (in reverse order) to the beginning.  Finding the longest palindrome is a task that takes O(N^2) time, unless you are familiar with Manacher's algorithm in which case you can turn it into a O(N) time algorithm. Manacher's algorithm is very complicated and so we will instead use a different approach that will leverage KMP and will take us o(N) time as well.

An important observation about palindromes is that their prefix (up to the center) is equal to the reverse of their suffix (from the center until the end). It would be nice given a string Str to know how much are the prefix and suffix (from the center) of Str alike. For example, "aacecaaa", we notice that this is almost a palindrome. The last "a" of the string is making it a non-palindrome. All we need to do is to append that "a" to the beginning of the string and it becomes a palindrome. In the case of  "abcd" there is no suffix that is equal to a prefix. Hence, we need to append "dcb" to the beginning. Whenever you need to deal with prefixes KMP should be one of the things that come to mind immediately. In this case, we will use it in a sophisticated way.

Given a string W will we generate a new string W#Wr (where Wr is the reverse of W and # is a symbol that is not in the alphabet of W). The reason we need the special character # will be clear later. By reversing W and appending it W# we can create the KMP prefix table and find out if Wr contains at its some shared characters with the prefix of W. If it does, we know that all the letters before it (e.g. from # to the value at the end of the prefix table) are letters we need to add at the beginning of W to turn it into a palindrome. Lets look again at "aacecaaa".

We will create a new string as explained above: "aacecaaa#aaacecaa" the corresponding prefix table for it is: [0,1,0,0,0,1,2,2,0,1,2,2,3,4,5,6,7]. Notice that the value at the last entry in the table is 7. This means that we have a match of size 7 with the prefix of our original string, and you can actually verify it since we have already seen that "aacecaa" is indeed of size 7. to find the letters we need to append we can either go from the # to the index of string length - 7, or we can only look at the original string and take the substring starting at index 7. In our algorithm, we will use the latter. Now we need to address the special character "#" that we added. The reason for it is not obvious. Consider the string "aaaaa". If we don't add the "#" the value at the end of the prefix table will be much larger than the size of the original string (due to the fact that the string is composed of the same character). By adding the delimiter "#" we are separating the two parts of the string (e.g. W and Wr) making sure only to seek the prefix we are interested in Wr.

We are now ready to see the code in action:

Notice how short and concise this code is. I urge you to try and implement it by seeking the longest palindrome and comparing the code. That is it for this post. It was long, but I hope it will help you internalize this algorithm and use it when you need it.

Imagine that you have a set of N numbers. You would like to generate a random subset of size K, such that any such subset has equal probability to be generated. In other words, every element of the set has equal probability to end up in that subset. The algorithm to do this is actually very straight forward, however, the proof of why it works is not so trivial. Hence I will describe the algorithm first show the code and then I'll add the formal mathematical proof for it which I wrote in LaTex due to the need for mathematical notation.

First thing is first: The Algorithm. The algorithm works as follows: we start by taking the first K elements from the set. We then iterate from indexes K+1 to N and generate a random number between {0 - i +1} where i is the current index we are traversing. If the random number we get is between 0 to k (say index j) we replace the number in that index with the number at i. Otherwise, we continue to the next number.

Code:

The resulting subset will be in the first K elements of the vector V. I've attached to this post a pdf file with the proof I wrote for the correctness of the method. I think that reading it and understanding it is worth your while. 

Reservoir Sampling - Once and for all

ReservoirSampling1.pdf

This is an interesting problem because it challenges us to rethink our hypothesis and strengthen it.

In this problem, we are given a set of balloons with numbers on them. The numbers represent the cost of each balloon. If we pop a balloon i, we get coins by the following formula:    B[i - 1]*B[i]*B[i + 1].

Where B[i] represents the cost of balloon i and B[i + 1], B[i - 1] the corresponding costs of the right and left balloons to the ith balloon.

For the sake of this problem, we may assume that B[-1] and B[N + 1] (N = the last balloon) are both 1. The problem asks you to maximize your profit by popping the balloons. 

Notice however, that once you pop a balloon i, balloons i - 1 and i + 1 become neighbors of each other. 

Let's try to solve this problem with induction:

P(n < N) - we know how to solve the problem for n < N balloons and get the max profit for them.

P(1) -  The base case deals with only a single balloon, in which case the answer is straightforward - we just pop it and collect the coins for it.

P(N) - Assuming we have N balloons, how do we reduce the problem to smaller subproblems ? We have to pop one or more of the balloons to reduce the problem size. However, which balloon should we pop first ? This is tricky. When we pop a balloon we modify the rest of the system. Also,which one maximizes our profit ? At this point, we notice that we need to change our perspective.

What if instead of choosing a balloon to pop first, we actually choose a balloon to pop last ? so we reduce P(N) to P(N - 1) based on the last balloon to pop. This way, our subproblems are not affected by our decision. But which balloon should we save for last ? anyone of them may end up with a different cost. In such a case we are left with no choice but to try every one of them as the last balloon to pop and pick the one that maximizes our profit.

We'll solve this problem using dynamic programming. The recurrence relation is as follows:

D[i,j] = max{over all k: D[i,k - 1] + D[k + 1,j]  + (B[i - 1] * B[k] * B[j + 1])}

notice how this recurrence relation maps to our hypothesis: D[i,j] represents P(N) and D[i,k -1],D[k + 1,j] represent our hypothesis: P(n < N). We multiply B[k] (the last balloon to pop) with B[i - 1] and B[j + 1] because we assume that the rest of the balloons betwee i to k and between k to j have been popped already. 

So now, writing the code should be straight forward:

Let's first describe the problem: Given a series of words and a page width (line length), left justify the words to fit them in the lines based on the page width.  More formally: given S = { Wi } for i >= 0 and Pw >= 0 the page width we need to return: R = { {Wi} } i> 0 such that  Sum(|Wi|) + Sum(Wi)  <= Pw.

As usual, we'll turn to our inductive approach for solving the problem. Let's define Cost(i,j) - the cost of putting Wi ... Wj in a single line. The cost function is what we need to actually minimize. We'll define it more specifically later on, so for now, let's assume that we are given this cost function and that if Wi... Wj can't be put on the same line it will return infinity, otherwise, it will return a score such that 0 <= score < infinity.

Hypothesis: P(n < N) - We know how to get the minimum score for arranging words Wi ... Wn (inclusive) in multiple lines in a justified manner (e.g. with the minimal accumulative score).

Base case P(1)  = Score(1,1).

How do we reduce P(N) to P(n < N)? well, let's start by fitting some K words in the last line. The score for those words is Score(WN - k + 1, WN). Once we have determined the cost of these words we are left with N - K < N words to fit in the rest of the lines. based on our hypothesis we know how to score them!

So the algorithm works as follows:

Initialization: P(-1) = INFINITY

1. For each  0 <= i <= N:

    2. For each 0 <= j <= i:

        3. P(i) = min(P(i),P(j - 1) + Score(j,i))

You may have already noticed that the code is arranged in a dynamic programming fashion. In fact, TeX uses this exact approach to justify text .The only thing left to be shown is how the Score(i,j) function works. The truth is, that the score function is very straightforward. It takes the difference between the page width and the sum of word length i...j (including spaces) and raises it to some power. We'll use the third power in our scoring function. Furthermore, we would like to be able to keep track of the words we put in each line. After all, the point of this exercise is to justify the text, and not just hand out scores. To do so, we'll use an additional data structure Parent(i) that will save back pointers to the previous line. As we determine the minimum j (from above) we also update Parent(i) = j - 1. this way we can retrieve the words to be arranged in each line  (in reverse order). Now let's see some actual code:

Let's look on how the main function uses Justify to print a set of words in a justified manner:

As a final comment I will just note that you can take a slightly different (but equivalent) approach, and instead of filling the last line and calculate the score of the first lines, you first try to fill the first line and calculate the score for the rest of the line. I personally find dynamic programming to be easier to grasp when you look at it from end to beginning rather than the other way. 

Before we talk about how to do this, lets remind ourselves of what a postorder traversal is. A postorder traversal traverses the tree in the following fashion: 

Left subtree -> Right subtree -> Root. 

Now, before we go about devising the algorithm for a postorder traversal without recursion and without any data structures, lets think how we can solve a simpler problem. Lets try to do the same but with an inorder traversal (e.g. Left subtree->Root->Right subtree).

Since we can't use any additional data structures, we must use the tree's internal structure to keep track of where we are and how we visit each node. For this problem there is a very elegant solution that I will present now, and that you can easily find on the internet. Notice that once we are done traversing the Left subtree we need to jump back to the Root, but since we are not using any additional data structures we need to make sure that we have a way to get back to the Root node. I will first present the code, and then explain it. 

The algorithm runs in a loop so long as Root is not NULL. The first thing we do is get a pointer to the left child of Root. if that child is NULL, we need to print Root's value and move to the right child of Root (by making Root = Root->right). Left is the the left child of Root. Now comes the interesting part. We look at the right subtree of Left and we take the path all the way to the right. The node that will be used as the right most node of Left is R. In line 31 we run in a loop so long as R->right is not NULL (hence we need to go more to the right) or R is not equal to Root. At first the second part of this condition seems weird, but it will soon be all clear. At first, you may assume that R will not be equal to Root and so the loop in line 31 ends when R->right is equal to NULL. When that loops end, we check again to see on which of the conditions we stopped. The first time we stop because R->right is NULL. what we do is that we make R->right point back to Root. This is the most critical part of the algorithm, because this is how we are able to get back to Root after we finished traversing the left subtree. Once R->right is set to point to Root, we set Root to be equal to Left and continue the loop. Since we are going to be doing this continuously on every iteration, at some point we will reach back that node that was pointed to by R. Root will be equal to that node at that point. We will then realize that Root->left is NULL, print that node and then take the right path (as in line 26). Where will we end up when we take that right path ? well, obviously because we pre-set the right path of that node to point back up to the Root of the tree, we will end up there. Root will point once again to the original root of the tree. Now here is where things get interesting again. Left will again point to the left subtree of the original Root of the tree and R will now traverse the right subtree of Left as before. However, this time the condition in line 31 will stop due to the fact that R->right will be equal to Root. We will end up in else clause at line 41. In this case what do we know? We know that we have just completed traversing the left subtree of the original tree, and that we need to print the Root and move to the right subtree. That is exactly what lines 43 - 45 do. However, in addition, they also clean up the "mess" we created, by setting R->right to be NULL again. This is fine, because after this point we will no longer be traversing this subtree again. Root will be set to Root->right and we will do everything we did all over again, but this time to the right subtree. Notice that the algorithm will end after Root points to the right most node in the right most subtree. If you find this to be a little tricky, I suggest going over the code, and playing on a paper with a "tree" of your own, so that you can convince yourself that this works. 

We are now ready to see how to do a postorder traversal in a similar fashion. However, I must warn you that the postorder traversal, although similar in principle, is still more complicated. We'll understand why soon. Before that, lets ask ourselves what was it about the above solution that helped us reach our goal ? The answer is that by setting the rightmost node of the left subtree to point to the Root we achieved a way to get back to the root of the original tree. This also served as a "marker" for us that we completed traversing the left subtree and that it is time to print Root's value and move on to the right subtree. A key observation is that once we set Root to point to Root->right we no longer need to traverse Root again and we only focus on the right subtree. In other words, we forget about Root and move on.

When we do postorder traversal however, things are much more complicated. This is because Root gets printed last. We can't just forget about it and move to another subtree. Once we are done traversing the left subtree we need to traverse the right subtree and only at the end we print Roots value. This fact is what causes the solution above to break if we try to apply it in similar fashion to the postorder case. However, let's not despair. We can still use the above solution, with some modifications. But, the devil is in the details, so pay close attention. Again, I will present first the code, and explain it in detail afterward.

Ok, this code is much longer. But the structure should look familiar. You will immediately notice that in the first lines I create a dummy node named "Dummy" and make Root be its left subtree. The reason I do this is because without it I will not have a way to get back to Root (the original root of the tree) after I complete traversing the right subtree. By adding this dummy node, I'm turning Root into a left subtree of another tree, and based on our previous solution we already saw that getting from a left subtree back to the root of a tree is simple, by using a back pointer. Once we have set Root to be the left subtree of the dummy node, we set Root to point to the dummy node (in line 21). At this point we start our loop. We have Left point again to the left subtree of Root. If Left is NULL, we need to move to the right subtree (without printing Root). Otherwise, we do the traversal of the right subtree of Left in lines 35 - 39 just as we did in the inorder traversal. Notice that we also count the levels we traverse on the right path. This count will be used later on when we revisit this subtree again. On the first time and just as in the inorder case R->right will be NULL at some point (no back pointer). We will end up in lines 43-44. In those lines we will set the back pointer up to Root and set Root to be Left. So far, nothing fancy at all. The fun part comes after we complete traversing the entire left subtree and we end up with Root pointing to what was pointed to by R (with the back pointer). Root->left will be NULL and so we'll set Root to be Root->right. However, Root->right is our back pointer that will lead us back to our original Root of the original tree (in this case it will be the dummy node). So now, we have Root pointing to the dummy node again. Again, we have Left pointing to the left subtree (which is in this case the original tree we were supposed to traverse) and we start taking the right path using R in lines 35-39. However, this time the loop will terminate because R->right will be equal to Root (or Dummy if you will). Lets stop for a second and think what does it mean ? what it means is that we completed traversing the Left subtree (of the left subtree of Dummy) and we still haven't completed the right subtree (of the left subtree of dummy). But this is where things really get tricky. We have to print the right subtree in a bottom-up fashion, starting from R going up all the way to Left. This is where "Count" from lines 33 and 37 comes to play. When we reach the point that we realize that R->right is equal to Root (line 46) we loop so long as Count >= 0. For each Count we continuously go down the right path from Left but each time stopping one level above the level we were in the previous loop. When we stop we print the value of that node and continue looping. Once we are done with this loop we can clean up the "mess" just as we did in the inorder case, and move to the right. Notice that since Dummy has only a left subtree, moving to the right means we essentially end the traversal and break from the main loop in line 23. The last thing we do just before exiting is to delete Dummy.

That's it. A little long, and slightly tedious, but hopefully you were able to keep up and get the gist of it. 

The first problem is stated as followed: "Given a string S, return the minimum number of characters required to be deleted from S, in order to turn it into a Palindrome".

By definition, the minimum characters solution will give us also the maximal size palindrome, but lets focus first on the minimum.

Lets define the problem as P(N) - the minimum number of characters required to be deleted to create a palindrome from string S of size N. if S is a palindrome, then the answer should be 0. if S has all unique characters, then the answer should be N - 1 (since a string of size 1 is a palindrome). Lets assume that we know how to solve this problem for any string n < N. How can we solve P(N + 1) ? there are two options here: the first is to think how we can extend the solution from n < N to N + 1. The second is to think how we can reduce N + 1 to n < N. in this case the second option will be easier. consider the N+1'th character of S. it either participates in the palindrome or it doesn't. It can participate in the palindrome in two cases: the first, it is equal to the first character of S, in which case we need to see what P(N - 2) is equal to. In the second case, it can participate in a palindrome that starts at the second character of S for example S = BS'A  |S| = N + 1, and S' = AS'' , |S'| = N - 2. In this case we need to check P(N - 1) and add 1 to its result (since we decided to delete the first character of S). The last option is that the N+1'th character is not present in the solution, so we need to delete it. In this case we need to check again P(N - 1) and add 1 to it (since we are about to delete the N + 1'th character of S). 

Notice that our problem definition in terms of just N is not very robust. Instead of defining the problem in terms of just N,let adds an additional parameter: the beginning of the string.

P(i,n < N) - we know how to find the minimal number of character deletions for  any sub string of S starting at index i and of size n < N.

Base case: P(i,1)  = 0

P(i,N) = MIN{S[i] ==  S[i + N - 1] ? P(i + 1, N - 2) : INFINITY ,P(i + 1,N - 1) + 1, P(i,N - 1) + 1}

The above equation is merely a formalization of what we spoke about in the paragraph above (using the ternary operator).

How does the code look like for this:

In such a problem in which there is the optimal substructure property, dynamic programming is called for.

The run time complexity of this code is O(N^2). 

I'll come back with some additional problems related palindromes soon enough.

No, you won't hear from me in this post how Python is the best language I've encountered.You won't write a line of code in Python that will make you shed a tear of happiness because you just wrote the most amazing line of code in your life - this is saved for C++. But if you want to get things done quickly and neatly, then Python is there for you. The abundance of libraries and the ease at which you can do things in the language makes me smile. The language has been created for lazy programmers as myself. I skimmed through a python book once to the point where the syntax made sense to me. From that point on, anything I needed I found on line. I would like to share in this post some of the nice things I was able to achieve with Python that made my life much happier.

In my previous group at Microsoft, we worked for a long time on a highly impact-full feature. The design and development of this feature took over a year. It was an amazing ride, and for such an amazing ride you take with you only the best tools. So my obvious choice was C++. By the end of the project I had written tens of thousands of  C++ lines of code. the feature itself was shipped as a DLL, and was intended to be called by another layer of the stack. Our biggest concern was testing. How do you verify the correctness of so many scenarios supported by this huge DLL. More specifically, we wanted to find a way to test our DLL in isolation (without taking dependencies on other layers of the stack). We realized that a testing harness was called for, and my initial approach (given that it was a native C++ DLL) to write this testing harness in C++ as well.  However, call it luck or call it coincidence, it so happened that I got my hands on a Python book. As I mentioned before, I skimmed through this book, learning all the necessary stuff I wanted to learn about the language. I immediately suspected that since the interpreter was written in C, the language might have a good way of interfacing with native code. Indeed, my suspicions were correct. Python in fact has a great way of interacting with native code (the ctypes library), and specifically interacting with native DLLs. After a small conversation with my manager, I decided to give it a shot. 

After a month worth of Python coding I was finally ready to demo my testing harness. It was in fact a great success. It was even more successful than I originally anticipated for it to be. Using this new harness I was able to write many scenario (integration) tests that usually required a full fledged system with all the layers of the stack available. The testing harness allowed myself and others working on this project to write python scripts that would call the DLL (as a blackbox) and test the specific scenario, enabling to validate the correctness of the system. Not only that, the testing harness proved itself extremely useful for finding systematic bugs that would otherwise take a very long time to reproduce or detect. It was only then that I realized the power of a great scripting language. It was only then that I realized the importance of using the right tool for the right task. As much as I love C++, Python was the best tool to use at that point, and I'm happy I chose it.

Since then, I've adopted Python as a step brother to C++. If I need to get *** done, I don't think about it twice. I go for Python. A nice IDE that I found to be extremely useful is the PyCharm IDE, I highly recommend it. As a last story about how I find Python to be useful, I'll tell about a manual task that my new manager (at my new team at Bing) asked me to do. My manager asked me to go through pairs of URLs (page url and image url) and figure out some properties about them. Unfortunately, that task could not be automated, and required that I open each pair of URLs in a browser and look at them to detect some anomalies.  However, given that I'm a lazy programmer (yea, yea I already said that), I figured that the fact that the task can't be automated does not mean that the process of reviewing them can't be automated. So I immediately wrote a python script that read the hundreds of URL pairs from a file, and opened the browser with each pair for me, awaiting a keyboard stroke to move on to the next pair. Yep, in less than an hour of work, I was able to make my life so much easier and so much more comfortable. 

To conclude. I highly recommend picking up python. It is a great tool for most of the stuff that you want to automate. For additional resources on the language I also recommend reading the book: "Automate the boring stuff with Python: Practical  Programming for Total Beginner" by Al Sweigart. Enjoy.

The description of this problem is as followed:

As the description says, the most intuitive way to solve this problem is to start enumerating all permutation until we reach the K'th one. However, this would make the problem not very interesting. We would like to solve it differently. We'll start by observing all the enumerations for N = 3 (above). We see that the numbers 1,2,3 appear as the leading digits two times each. For each instance in which 1 is the leading digit, the digits 2 and 3 appear as the second digits only once, and as the third (and last) digits only once. What if we would have had N = 4 ?  In this case 1,2,3,4 would appear as the leading digits six times each (4*6 = 4!). For each instance that they appear as leading digits, we have the rest of the digits appearing 2 times each as the next one in line (just like the case of N = 3) and the leftover digits appearing as second to last 1 time each yet again ....

This observation should hint us to a potential solution. We can find the next leading digit based on K and N. If N = 3 and K = 4 it means that the permutation that we are looking for starts with 2. Why ? Because we have observed that each leading digit appears two times in the total permutations of the numbers 1 - 3. In other words since we have buckets of size 2 for each leading digit, we can find the leading digit by (K-1/Bucket Size) Notice that we need to scale K from 1 - K to 0 - (K-1) scale. So we can find the first leading digit... what about the next leading digit ? lets continue with our example of N = 3 K = 4. Once we found the bucket that the K'th number falls into, we can get rid of the leading digit as we will not need it anymore. We are left with digits 1 and 3 in this order. Since we have two digits we can be in either the permutation 2 1 3 or the permutation 2 3 1. Which one is it ? luckily we can apply the same logic using K to find which of these two we need to choose. Since our buckets are of size 2, K - 1 % 2 will tell us which of the two options we need to choose. In this case we choose the second permutation. We will apply the same logic exactly to find the last digit.

The logic is now clear. However, looking closely it seems that we still need to order the permutations in some way so that we can choose the right permutation based on K. Do we really need to order the permutations ? Actually we don't. We only need to enumerate the digits in their natural ordering  - 1 2 3 4 ... N . We pick the next digit based on K and the bucket size which is of size N!/N = N-1! . Once we find our digit, we remove it from the list and leave the other digits in tact. We return to choose the next digit from the list by performing the logic described above, but this time we adjust the bucket size to be (N-1!)/N-1 = N-2! and using (K - 1)% (Old Bucket Size) as our new K. We continue this iterative process until we are left with no digits.

Now that we understand the logic, lets look at the code:

To start lets define an expression tree. This is a binary tree in which every inner node represents an operator and every leaf represents an operand. The input for the problem is an infix expression of the form: "2" "*" "6" "-" "(" "23" "+" "7" ")" "/" "(" "1" "+" "2" ")". The quotation marks are deliberate, as each operand/operator is a separate string. The set of strings is received in a vector. 

We are given the definition of the tree structure as followed: 

If we look carefully at the definition of the tree we notice that we need to transform the infix notation into a prefix notation. The reason is that the tree itself is built from the prefix notation. The problem then comes down to turning the infix expression into a corresponding prefix expression. To do so we realize that in order to transform the infix expression to its equivalent prefix notation we need to scan the infix backwards (from end to start). This way we put the operands last and the operators first with special care to operator precedence. 

To implement the design we'll leverage two stacks. The first will hold only operators: "+,-,*,/,(,)". The stack will hold them according to their precedence. More precisely, "+" or "-" can follow only ")" and other "+" and "-" already pushed to the stack. However, if the top of the stack contains "*" or "/" then we need to handle them first before pushing the "+" or "-" to the stack. Also, when we encounter "(" that means that we just finished processing an inner expression. In this case we have to first handle all the operators that have been pushed after the matching ")". When we encounter: ")","*","/" we needn't do anything special and we can just push them into the stack. 

The second stack that we'll use is an expression stack. this stack will contain all of our (Tree Node) expressions as we build them. Since we are building a binary tree, we have to make sure that every operator (from the operator stack) that is being handled, will consume two expressions from the expression stack.

The run time of this algorithm is O(N), for N - the size of the input vector.