Difficulties with non-nullable types (part 4)

Article
04/27/2005

So far i've discussed two interesting problems that arise when you try to add non-nullable types to C# and .Net. The first problem was that non-nullable instance fields might be accessed before being initialized, but we were able to come up with a good model (from C++ no less) that would allow that to work. The second problem was with non-nullable static fields. Unfortunately, there doesn't appear to be a great solution in place to enable these. Oh that i wish those were the only things you had to deal with when trying to add these types. So what's next on the menu?

Well, let's consider the following code:

 public class Test {
    void ProcessAllFiles(string[] fileNames) {
        foreach (string file in fileNames) {
            ProcessFile(file);
        }
    }

    void ProcessFile(string! fileName) {
        //Do stuff with fileName, assured that is isn't null
    }
}

As you can imagine, this would not be compile. ProcessFile needs a non-null string, but instead we're passing it a string that could be null. There are a couple things we could do about this. One would be to replace the code with:

     void ProcessAllFiles(string[] fileNames) {
        foreach (string file in fileNames) {
            if (file != null) {
                ProcessFile(file);
            }
        }
    }

Now we know that "file" is not null and that it's safe to call ProcessFile with it. Alternatively we might have a construct like so:

     void ProcessAllFiles(string[] fileNames) {
        foreach (string file in fileNames) {
            ProcessFile( !!file );
        }
    }

The !! construct will throw if the expression it is applied to is null, and if exception still continues then it means that it's not null. I'm not a huge fan of this construct. I don't think it reads well, and i think it's likely to be confused with ! quite a bit. But it's definitely something to think about.

But all this is really to make up for the fact that what you'd really like to do is write this code:

     void ProcessAllFiles(string![] fileNames) {
        foreach (string! file in fileNames) {
            ProcessFile(file);
        }
    }

Now everything is typesafe and the signature of ProcessAllFiles is set up so that it can't accept any null file names in the array that's passed in. Seems great right? Well, the problem now arises of how someone can create one of these arrays of a non-nullable type. Let's take a look at how the code might look:

     void DoSomeStuff() {
        string![] fileNames = new string![2];
        fileNames[0] = "foo.txt";
        fileNames[1] = "bar.txt";

        ProcessAllFiles(fileNames);
    }

Unfortunately, this won't work. When you instantiate the array it's going to be filled with nothing but nulls. If you accidently use it before you've filled in all elements then someon eis going to be passed null when that should be impossible in the type system. So what can you do about that. Well you could limit it so that when you initialize an array of non-nullable elements you pass in all elements at construction time to ensure that it's completely filled with non-null types. i.e.:

     void DoSomeStuff() {
        string![] fileNames = new string![] { "foo.txt", "bar.txt"};

        ProcessAllFiles(fileNames);
    }

However, this isn't a very attractive solution. What if you have an array where you dynamically determine the set of elements at compile time. i.e. something like this:

     void DoSomeStuff(int count) {
        string![] fileNames = new string![count];
        for (int i = 0; i < count; i++) {
           fileNames[i] = "foo" + i;
        }

        ProcessAllFiles(fileNames);
    }

You can't represent the construction of this array with a predefined set of elements since the number of elements is dependent on the passed in "count" value. So what can be done about this? Well as before we could not allow you to have arrays of non-null values. But that would be extremely unfortunate. Consider the enormous amount of APIs out there that look something like:

     string[] GetFoos() { ... }

that you would really want to look like:

     string![]! GetFoos() { ... }

(i.e., a non-null array of non-null strings). The double-bang is a bit ugly, but not that bad.

So supporting this is definitely something that you'd want. I've given it some thought, and the only solution i've been able to find is to introduce an intrinsic array constructor supplied by the .Net runtime that will create the array and fill it with all the non null values. This ensures that all values will be set by the time the array is used. I think that this would look something like this:

 public delegate T Creator<T>();

//This class is treated specially by the runtime and some type 
//system rules are relaxed when it is executing.
public class ArrayCreator<T> where T : class {
    public T[] New(int length, ArrayInitializer<T> initialize) {
        //The runtime will be fine with these elements not being initialized
        //even if T is a non-null type.
        T[] array = new T[length];

        for (int i = 0; i < length; i++) {
            array[i] = create();
        }

        return array;
    }
}

Note: we might allow a little more flexibility with "Creator" such as passing the index of the element being created. For now, this simple example will suffice.

You can now use this like so:

     void DoSomeStuff(int count) {
        int i = 0;
        string![] fileNames = ArrayCreator<string!>.New(count, delegate {
            return "foo" + i++;
        });

        ProcessAllFiles(fileNames);
    }

It could work... but yech... it's hideous. Maybe we could some up with some nicer syntax to help out here, but nothing is leaping to mind. Do you guys have any suggestions for a nicer method of handling this?

---

Edit: Michael astutely noticed that the check of "if (fileName != null)" would be insufficient for determining at compile time if fileName was actually null or not. The reason for this is that the != operator might have been overridden, and so we can trust the result of it. However, what we could add would be some sort of intrinsic operator like null() that would do this work for us and which would allow the compiler to determine using flow control if a variable was null or not. So you could have:

if ( ! null(fileName)) {
string! nonNullFileName = fileName;
...

and have the compiler and runtime be ok with that.

Comments

Anonymous
April 26, 2005
well, you can't really solve your array problem.

you need to bind the size of the array to compile-time, and check that each element has been assigned a not-null string.

the only way is if you can only specific these strings at compiletime (using 'notnull' keyword):

---
notnull string[] message = new string[]{ "test", "hello", thisisANotNullString, thisIsANormalString };
--

this would result in a compile-time error on the "thisISANormalString".

this is the best you can do, IMO.

you say you might like to get notnull strings dynamically from somewhere, but how? any dynamic input can be null either that, or you'll get a runtime exception.

the only place you can get a 'not-null' string from is compile-time.
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
Doug: "Arrays of non-nullables are an interesting idea, and might be worth considering, but I'm not sure they buy much. Especially since just having non-nullables would presumably buy us constructs like List<string!>. "

You beat me to (part 5) of my series.

How, exactly, do you implement List<string!>?

Right now List<T> has a T[] inside. But if you can't make a array of non-nullable types, how can you instantiate List<string!>? :)

I'll still blog about that, but nice catch seeing that problem.
Anonymous
April 26, 2005
Michael: "well, you can't really solve your array problem. "

But we can :)

I showed an example of solving it. However, the issue of solving it in an attractive manner is not quite clear :)
Anonymous
April 26, 2005
Doug: "I think big part of the value of having non-nullable types would be this very "problem" of forcing you to explicitly decide between these behaviors. Presently it is all too easy to either inadvertently pick one or the other without really thinking about it or to get in a situation where you throw a null-reference exception in off-the-beaten-path cases because you didn't consider what would happen if a parameter was null. "

I completely agree. :)
Anonymous
April 26, 2005
Michael: "any dynamic input can be null either that, or you'll get a runtime exception"

That's not true. If you have a:

Creator<string!> c = ...;

then each invocation of 'c' is guaranteed to return a non-null string at runtime.
Anonymous
April 26, 2005
> That's not true. If you have a:

what I meant was how would you get dynmaic affirmably 'not null' input?

from a file? nope.

from client typing it in? nope.

from a request/session object? nope. all of these can be null.

perhaps from a normal string type? nope, because you can't convert b/w string and string! without a possible compile-time error. thus, you may as will create your own application-specific datatype:

---
class NotNullString {
public NotNullString(string s){
...
}
}
---
Anonymous
April 26, 2005
further (sorry it's a little off-topic but there doesn't seem to be a general area to discuss this 'thing')

what is the relationship of a nullable type to a not nullable type?

is this valid?

--
public string overrideTest(){ }
public string! overrideTest(){ }
--

i.e. is it a subclass of the base type? can you cast between them?
Anonymous
April 26, 2005
Michael: You can think of it like venn diagram:

String
------------------------------
|...........null.............|
|.|------------------------|.|
|.|........................|.|
|.|........String!.........|.|
|.|........................|.|
|.--------------------------.|
|............................|
------------------------------

So, yes "String!" is-a "String", while "String" isn't necessarily a "String!"

As "String!" is "narrower" it would be certainly reasonable to override in the manner you suggested (if we had covariant return types).
Anonymous
April 26, 2005
Also: "what I meant was how would you get dynmaic affirmably 'not null' input? "

Simple, you get the input in a possible null form, and then assign to the non-null form after confirming that it is, in fact not null.

like in the example i put above:

foreach (string file in fileNames) {
....if (file != null) {
........ProcessFile(file);
....}
}

this could be rewritten as:

foreach (string file in fileNames) {
....if (file != null) {
........string! nonNUllFile = file;
........ProcessFile(nonNUllFile);
....}
}
Anonymous
April 26, 2005
Ok. that venn diagram didn't come out. It should be a circle called "String!" with all non-null strings in it, surrounded with another circle that contains "null" in it which is called "String".
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
Michael: "perhaps from a normal string type? nope, because you can't convert b/w string and string! without a possible compile-time error. "

Yes, this would be fine. You could use !! to do it, or you could make sure that all that flow paths to the use of a variable had proved it to not be null. (similar to how you need to prove that a variable is assigned to a method before it is used).
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
> or you could make sure that all that flow
> paths to the use of a variable had proved it
> to not be null.

indeed, but 'any old assignment' is far different from 'assignment of a value whom is not null'. the compiler needs to resolve the actual value of the variable somehow; IOW it needs to be, say, a 'const' variable, or some function returning a 'string!'.

what I was getting at is - where does this flow start? how do normal, everyday strings get to be 'non null' strings? as per a previous post, I don't believe it's possible to convert b/w nullable and not nullable without a compile-time error.
Anonymous
April 26, 2005
I said:
> and not nullable without a compile-time
> error.

forgive me, this should say 'runtime error'.
Anonymous
April 26, 2005
Michael: "indeed, but 'any old assignment' is far different from 'assignment of a value whom is not null'."

Not especially. All assignments are necessarily tracked anyways (for type safety and for definite assignement). Now the rule simply becomes: "the variable must be definitely assigned, and all assignments that flow in before it's use must assign a provably non null value. When determining null/non-null, flow through a conditional that uses the intrinsic null() operator can be used to 'prove' that in one path of the flow the variable is not null, and in the other it is".

If it makes you fele more comfortable. imagine the intrinsic was written:

if null (fileName) {

} else {

}

Now the compile can provably say that in the entrance to the "else" that fileName is not null (presuming that it's a local variable and not say a field which might be mutable from some other thread).

"the compiler needs to resolve the actual value of the variable somehow;"

No, it doesn't. It merely needs to split betwene "null" and "non-null". Far easier than determing the actual value :)

And given type modifiers (like ! ), and intrinsics which tell the compiler information about what's going to happen at runtime, this can be done.

"IOW it needs to be, say, a 'const' variable, or some function returning a 'string!'."

Yup. Those, along with intrinsics can accomplish this.

"what I was getting at is - where does this flow start? how do normal, everyday strings get to be 'non null' strings? as per a previous post, I don't believe it's possible to convert b/w nullable and not nullable without a compile-time error. "

I recommend reading the sections of the C# language spec on definite assignment. They go into painful detail on this :)
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
Michale: "sorry, but I still don't see that working.

how does the compiler know that 'if' statement has been implemented? can it be used in conjunction with other statements inside the 'if' block? "

I'm not sure what you mean? Could you give an example.

"and even if it doesn't, how does the complier ensure that, after it lets you inside, you actually assign the variable you just checked? "

Flow analysis. :)

"seems like you need an entirely new construct to perform this 'conversion'. "

Yes. Specifically: null() :)

As i said, just consider it to be like this:

if null (fileName) {

} else {
....//compiler now knows that if fileName is a local variable that it's not null.

....//as long as there are no other assigments between the above and below then it's fine to do this.

....string! nonNullFile = fileName;
}

or, if you have this:

if null (fileName) {

} else {
....fileName = SomeMethodWhichReturnsNonNullString();
....string! nonNullFile = fileName;
}

then you're still good. All paths between the declaration and the use can be (and are) mapped out. You simply need to ensure that all asignments on that path are proven to be non-null (like the method call above), or that the value could not be null along that flow (using the intrinsic).

Note, the compiler could also do this for casts. i.e.:

if (someLocalVar is string) {
....string v = someLocalVar; <-- no cast needed

also, there can be an operator like !! which can be used to assign a null into a non-null, and which can throw if the value was null.

Ok. bed time for me. Hope to hear from you tomorrow!
Anonymous
April 26, 2005
"convertToNotNull(myvariable, variableToAssignTo); "

Yup, that's the !! operator in my examples.
Anonymous
April 26, 2005
> No, it doesn't. It merely needs to split
> betwene "null" and "non-null". Far easier
> than determing the actual value :)

Are you able to explain (or point me somewhere where it is explained) how the c# compiler can do this easier? I can't really imagine how that would work ... I can understand how it confirms assignment; but what is actually assigned requires runtime knowledge, doesn't it?
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
> You would get an error message telling you
> that you would have to assign it into a
> local value first like so:

hmm, so your suggestion is that you can't use these 'intrinsics' (to use your word) on any member variables without first copying them?

What about methods? who's result can change after each invocation?

For example,

---
if( !null(fooMethod()) ){
msg! = fooMethod();
}
---

Clearly this would be (should be) illegal. would you be forced to change it to:

---
Object temp = fooMethod();
if( !null(fooMethod()) ){
msg! = temp;
}
---

?

Also, I think the threaded issue is important, (again, when we talk about member-variables)...
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 26, 2005
The comment has been removed
Anonymous
April 27, 2005

Instead of "ProcessFile( !!file );", I would suggest "ProcessFile( (string!)file );
" since this construct would already be in the language

2) Is is simple to build a string![]:
string![]! MakeArrayOfNonNullableStrings()
{
string[] result = new string[2];
string[0] = "..";
string[1] = "..";
return (string![]!)result;
}
The rationale is that non-nullability is an invariant on the instances of the type, and that invariants are meaningful only in stable states, in particular after full initialization.
So to me, it is perfectly acceptable to have a runtime check at this moment (and only at this moment). Factory methods are an ideal pattern to encapsulate this behavior.

3) In some cases, the compiler might event be so smart as to detect that the runtime check is unnecessary, e.g. in the examble above.

4) Another solution would be to have an array type that can grow:
string![]! MakeArrayOfNonNullableStrings()
{
string![]! result = new string![0];
string.Add("..");
string.Add("..");
return result;
}

5) To be more efficient the array could specify a Capacity property:
string![]! result = new string![0:2]
0 = initial Length
2 = max Capacity
result.Add() would increase Length (with a check that Length <= Capacity)
result.Remove() would decrease Length (with a check that Length >= 0)
operator[int i] would be allowed for 0 <= i <= Length only

Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Cyrus -

As you may remember from all my discussions with Eric when I was desperately asking for nonnull types, I use them daily at the moment. I have a template which back then I called nonnull, but have actually renamed to ""

so, if I want a collection of nonnull strings, I have to cough up
<List<_<String>>>, which is pretty ugly, but at least gives me full compile time protection.

I've laughed at your previous issues, because I've thought anyone who writes code like that (bidirectional relationships between classes) deserves all the runtime errors they are going to get

but this is an issue I have actually run into, so thought I'd add my 2c :)

First off - when it comes down to it, do we really need arrays of non-null types? Arrays at the end of the day are just a performance optimization, and I think we are largely beyond the need to optimize at that level, unless you're writing a game or something.

However, I like to prepare my collections in one go - so I sort of have the same problem. I can think of three solutions - which are sort of AND's rather than OR's - because you'd want to use all three depending on the situation.

a) The default value. Really, the only thing I'm ever going to use an array for is if I'm doing something highly performant, and so most of the time I'm thinking an array of numbers or something. If you init an array of int's, you get all 0's. That sounds good to me. If I initialize an array of the object "Complex" - I want just a fixed set of zeros. Does this make sense - yes - because often its "ok" for values to be "wrong", or "unpopulated", but what we are trying to avoid is runtime errors.. If I'm displaying a list of strings, I don't mind if there's a few empty strings in there for values that haven't been populated yet, because it's not going to break anything.

for instance:
string![]! x = new string![]! ("");

b) like someone above said - CASTING - given that the API's aren't done using non-null strings, I find that casting into nonnull is something I do a lot... but it's ok - because the programmer has made a conscious decision that "I know this is not null" - and my template will throw a run-time error if they are stupid enough to make a mistake, because the cast operator checks. So, there is nothing wrong with

string [] x = FillOutArray();
string![]! y = (string![]!) x;

which will throw an error if x or any
element of x is null

c) Finally, and most importantly, we are now in a world with iterators! To me, that gives us the best way of doing it. If we take an iterator, that means we ahve the choice - we can prepare a collection in advance, and pass it's iterator to initialize an array, or we can pass the functionality to build rows, for ultimate performance. Either way, the solution is elegant and readable.

string![]! x = new string![]! (myListOfStrings);
or
string![]! x = new string![]! ( StringListCreator );
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
"The reason for this is that the != operator might have been overridden"
Another mistake on the big part of C#. Value equality and reference equality should have been kept completely distinct (different operators). Reference equality shouldn't be overloadable, only value.
Anonymous
April 27, 2005
Here's another interesting question: how does non-nullable types interact with definite assignment verification.

Example:

void Main()
{
string! message = "this will be basically ignored"; // should this initialization be needed?

WeirdSignature(ref message);

Console.WriteLine(message);
}

void WeirdSignature(out string! x)
{
return "hello world!";
}
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
I'll second DrPizza's comment, off topic as it is :) - let's ditch out parameters - they were always a bad idea - and I've never used one.

As for multiple return values - we have them now! All you have to do is define several templates: tuple2, tuple3, tuple4

Then you can just do:

tuple2< string, int> MyFunction( blah)
Anonymous
April 27, 2005
Michael: "
Okay, but surely then this must result in a runtime error if bar is null. And runtime errors (i.e checking for nulls) is the very thing you are trying to remove, no?"

Yes, this specific intrinsic would result in an exception (which may or may not be what you want).

If you don't want the exception then use the null() intrinsic to inform the compiler that the value is/isn't null down a certain flow path.
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
MIchael: "hmm, so your suggestion is that you can't use these 'intrinsics' (to use your word) on any member variables without first copying them? "

That's correct. You need to know absolutely at compile time that the value is not null. Because of threading, flow control cannot help you with data that might be accessed by other threads which might change the value after checking it. By making a local copy you can prevent anything but the current thread from changing it and you can actually do flow analysis.

"What about methods? who's result can change after each invocation?

For example,

---
if( !null(fooMethod()) ){
msg! = fooMethod();
}
---

Clearly this would be (should be) illegal. would you be forced to change it to:

---
Object temp = fooMethod();
if( !null(fooMethod()) ){
msg! = temp;
} "

No. It would have to be:

Object temp = fooMethod();
if( !null(temp) ){
....msg! = temp;
} "

---

"Also, I think the threaded issue is important, (again, when we talk about member-variables)... "

It is. That's why you would not be allowed to use the null() intrinsic on member variables.

You would be allowd to use !! as that would first copy the value to a temp local (thus preventing access from another thread), and then throw or assign as necessary.
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
"Your solution isn't exactly correct (at the moment) as your 'Creator' doesn't return a non-nullable string. "

Actually... (although i may be wrong), i believe that it is the case taht a string concatenated onto anythign always returns a non-null string.

If that knowledge were in the compiler then the above code would be fine.

If it's possible to get null when concatenting a string and something, then you're right, we'd have to cast to (string!).
Anonymous
April 27, 2005
Joc:

" 1) Instead of "ProcessFile( !!file );", I would suggest "ProcessFile( (string!)file );
" since this construct would already be in the language"

Good idea. I like that better too.

"2) Is is simple to build a string![]:
string![]! MakeArrayOfNonNullableStrings()
{
...string[] result = new string[2];
...string[0] = "..";
...string[1] = "..";
...return (string![]!)result;
}

The rationale is that non-nullability is an invariant on the instances of the type, and that invariants are meaningful only in stable states, in particular after full initialization."

I like that idea too. I'll definitely have to give that some more thought.

"So to me, it is perfectly acceptable to have a runtime check at this moment (and only at this moment). Factory methods are an ideal pattern to encapsulate this behavior."

Yup.

"4) Another solution would be to have an array type that can grow:
string![]! MakeArrayOfNonNullableStrings()
{
...string![]! result = new string![0];
...string.Add("..");
...string.Add("..");
...return result;
}"

perhaps. although it's would be somewhat of a new construct at that point. Almost an ArrayList... but not. :)

"5) To be more efficient the array could specify a Capacity property:
string![]! result = new string![0:2]
0 = initial Length
2 = max Capacity
result.Add() would increase Length (with a check that Length <= Capacity)
result.Remove() would decrease Length (with a check that Length >= 0)
operator[int i] would be allowed for 0 <= i <= Length only"

Yup. That will certainly solve a problem that i'll be presenting in a later post.

You guys are great! :)
Anonymous
April 27, 2005
Doug: "
Regarding how to implement List<string!> since List<T> has a T[] in it: "

I'm going to hold off talking about that until next post, since i think it's so interesting :)
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Doug:

"Here's another interesting question: how does non-nullable types interact with definite assignment verification.

Example:

void Main() {
...string! message = "this will be basically ignored"; // should this initialization be needed?

...WeirdSignature(out message);

...Console.WriteLine(message);
}

void WeirdSignature(out string! x) {
...return "hello world!";
}"

First, i think you meant to write:

"void WeirdSignature(out string! x) {
...x = "hello world!";
}"

If you have that then you would not need that initializer. instead you could write:

void Main() {
...string! message; //no initialization needed

...WeirdSignature(out message);

...Console.WriteLine(message);
}

There is a difference between not being assigned and being assigned something of the wrong type. Not being assigned is not equivalent to be assigned null (for locals).

However, if you had this:

"void WeirdSignature(ref string! x) {
...x = "hello world!";
}"

then you would get a compile error at:

...WeirdSignature(ref message);

saying that message was being used before being assigned to.
Anonymous
April 27, 2005
> Yes, this specific intrinsic would result in
> an exception (which may or may not be what
> you want).

Well, then, aren't you just shifting the non-null checking off to someone else?

What are they supposed to do with that knowledge? They can't call the method that they wanted too... the code now looks like:

---
try {
string! nonnull;
try {
nonnull = !!otherString;
} catch(CantConvertNullToNonNullRefException){
//
// note: can't call method
//
return;
}
---

instead of the previous (and more readable, imo)

---
try {
fooMethod(someString);
} catch(NullReferenceException e){
print("oops, param N was null: ");
}
---

Seems like all we are achieving then is NOT the elimination of null-checks, but the shifting of them to someone else ... (the caller - who may or may not be fully aware of why null isn't allowed).
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Ahh yes, I meant to assign to the out parameter and not to return (since returning there made no sense). Sorry for the brain fart.

I agree that it should work the way you say (exactly the same as the existing definite assignment rules). Cool.

Thanks for the correction.
Anonymous
April 27, 2005
Michael:

"> Yes, this specific intrinsic would result in
> an exception (which may or may not be what
> you want).

Well, then, aren't you just shifting the non-null checking off to someone else?

What are they supposed to do with that knowledge? They can't call the method that they wanted too... the code now looks like:

---
try {
...string! nonnull;
...try {
......nonnull = !!otherString;
...} catch(CantConvertNullToNonNullRefException){
...//
...// note: can't call method
...//
...return;
} "

NO. You would instead write this:

if (!null(otherString)) {
//can't call method
return;
} else {
fooMethod(otherString);
}
Anonymous
April 27, 2005
> NO. You would instead write this:
>
> if (!null(otherString)) {
> //can't call method
> return;
> } else {
> fooMethod(otherString);
> }

(I think you meant to write that the other way around, but..)

Still, you've got the same problem - the logical consideration of 'null' is done by someone who doesn't exactly know why they are doing it; they only do it because the compiler asks them to.

Consider the API user, who's now had to make so many of the 'non null' call structures.. He's had to duplicate this annoying null-consideration code everywhere because the API designer didn't want to have to deal with nulls.

Seems a little weird, doesn't it?
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
also a final comment while you're not around to argue with me :)

I don't exactly see why the language needs to solve a problem that the programmer can solve with a custom datatype ('not nullable wrapper') - i.e. NotNullableString.
Anonymous
April 27, 2005
Micahel: " also a final comment while you're not around to argue with me :)

I don't exactly see why the language needs to solve a problem that the programmer can solve with a custom datatype ('not nullable wrapper') - i.e. NotNullableString."

How do you implement that custom datatype?
Anonymous
April 27, 2005
Michael: use <dots> to indent things. :)
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Michael: "Well, it doesn't eliminate them, it shifts them somewhere else. You still have the same class of bugs, but they are shifted to the caller rather then the implementor. ".

It eliminates them from runtime and moves them to compile time. If the developer can't take care of it at that point, then there's not much that can be done about that :)

And it is the caller who should be making sure that they're passing valid data to people. If they're not passing valid data they're doign something wrong. :)
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Michael: "But, I still think that I would prefer a not-null wrapper instead of a not-null "modifier" like "!". "

Fair enough :)

If i call something NotNull i'd like it to be not null... but that's just me ;-)
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Michael: "Agreed. But, in my opinion there is enough of a difference for it to be confusing and annoying that I no longer received the runtime warning information I previously got.

Particularly because documentation is very very rare (IME) but at least there is a hope of having a decent exception message (even stack trace) to look at. With the not-null we completely loose the stack trace. "

With the not-null we completely loose the stack trace.

With the not-null we completely loose the stack trace.

With the not-null we completely loose the stack trace.

exactly :)

With the not-null no exception will ever get thrown, so you won't get or need a stack trace.
Anonymous
April 27, 2005
Michael: FYI, it's fantastic chatting with you about this. I'm always interested in hearing what people think abuot this stuff and expecially if they don't like it (since it's hard to work against your own biases). Is there an email address that i can reach you at? If so, let me know by sending me a message through the "contact" link at the top of the blog.

Thanks much!
Anonymous
April 27, 2005
Michael:
"Right, but to me what they (API) are saying is: "We can't handle null, you have to guess why and deal with it". [If there is no doc]. So I sit back and go, "well, what do I do now? I wanted to pass null.." "

Why on earth would you want to pass null...

You wanted to get the runtime exception? You wanted things to fail?
Anonymous
April 27, 2005
Michael:
"I would suggest this is where we differ. I don't 'get rid of them' from my app. I find myself happy with nulls, and sometimes I deal with them, by ignoring them (i.e. continuing on with whatever operation I was going to perform, but disregarding that option) or other such things. "

And you can do exactly that with the new model. Nothing stops you. :)

If you want to keep on going, then just keep on going.

I think i need to see an example of code of yours that would suddenly not work right if an API switched to using non-null arguments.
Anonymous
April 27, 2005
> I don't know why this would be a runtime
> error.

Quite right, I think I messed that example up :)

I'll have another go at it soon, but I was trying to show that by having (string!) as a more-restrictive subtype of "string" we can get problems with LSP. I still believe this is true, but let me look into it a little :)
Anonymous
April 27, 2005
> exactly :)

> With the not-null no exception will ever get
> thrown, so you won't get or need a stack
> trace.

Right, but sometimes I like the stracktrace!! :)

For example, consider if the api calls was something like:

[1]me -> [2]api -> [3]api -> [4]me -> [5]end

if null was blocked at '2' because API writer realised '4' would fail if it was pass null, I may never notice my error in coding '4'. (okay, this is stretching).

But what I'm saying is that sometimes stacktrace is good, and if it's eliminated due to some short-sighted developer not wanting to deal with nulls, id be a little upset.
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
Regarding my LSP concern, It's not an issue, I realised, because we are dealing with assigment to a reference only; not anything within the object itself.
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
> I just cannot fathom your objection to this

I don't neccessarily object (well I do, actually - I don't think a feature like this is needed), just having a discussion about the possible problems it might cause.
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
In any case, I think we've almost worn out this discussion :) I'm beginning to repeat myself ...

My only point is that it isn't entirely obvious that string! is a subtype of string and what this will entail (re-directing of previously well-understood compiletime bindings, etc) I can just imagine this causing confusion ...

That, combined with the fact that I don't really think this feature is neccessary [I work with in the finance world with databases too (I think someone else mention this as a pro for the idea); and never find the need to match my c# code with my db schema (i never allow nulls anyway, in the schema)] I just think it's another complication that the language doesn't need...

it's definately been fun :)
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 27, 2005
The comment has been removed
Anonymous
April 28, 2005
The comment has been removed
Anonymous
April 28, 2005
"I don't neccessarily object (well I do, actually - I don't think a feature like this is needed), just having a discussion about the possible problems it might cause."

Let us not talk of what is "needed" but instead of what we "want" to help us produce better programs in less time. Talk of what is "needed" leads only to Turing machines, and I don't want to program a Turing machine, thank you very much.

Your complaints about not knowing why someone wants something to be non-null are a complete red herring. It doesn't matter why they want it. All that matters is that they do want it.

Without non-null types there are two ways you'll find out that they don't want a null. The good way is for the documentation to tell you. That way you can explicitly check for nullness yourself and explicitly make sure that you don't pass in a null. The bad way is to get a null pointer exception at runtime, crashing the plane, making the nuclear reactor melt down, and emptying all the money out of your bank account. That's something we rather want to avoid.

So, we propose a mechanism by which a developer can say, "this value should never be null". When we do this, we no longer have to worry about the documentation--which is just as well, as it's normally wrong or obsolete or both. And we can't ever get a runtime exception that crashes our plane or melts down our nuclear core or bankrupts us.

Brilliant. There's no downside to this (there are some implementation difficulties where we need to pick a rule, as Cyrus has been discussing, but that's just a question of deciding which way we want the language to work and just sticking with it). And it makes our software more reliable and easier to develop.

If you don't want this kind of checking, you shouldn't be using C# (or any other statically typechecked language) at all.
Anonymous
April 28, 2005
Darren:
> Why? Nulls are inidious and evil.

See, this is where we differ. I don't feel this way about nulls.

> But the BIG problem with nulls is that
> unlike other bugs in your system, you get NO
> INFORMATION WHATSOEVER as to what went wrong

I disagree here too; typically you should be checking if something is null (if it's possible) and throw an exception if it is and you can't handle it. This is the type of error I would like to see. If not, I personally find the stracktrace incredibly helpful.

> That's not just a GOOD thing, it's a
> NECESSARY thing.

For me it is neither.
Anonymous
April 28, 2005
The comment has been removed
Anonymous
April 28, 2005
The comment has been removed
Anonymous
April 28, 2005
DrPizza:

You won't be convincing me of anything, and I can see that I won't be convincing you of my point of view.

That's fine, I don't really mind.

Clearly, you are a little bothered by my position - namely that 'null' variables don't bother me, and in my programs I have succesfully dealt with them in the past, and will continue to do so in the future.

The 'pass the buck' nature of this proposal doesn't really 'sit well' with me, but if it does get added to the language, then so be it.

I do think I have explained my position here (although it is a little difficult to follow arguments in this form, but that's all we've got), and I do understand your position too; I just don't this feature is needed.
Anonymous
April 28, 2005
"Clearly, you are a little bothered by my position - namely that 'null' variables don't bother me, and in my programs I have succesfully dealt with them in the past, and will continue to do so in the future. "
Isn't it easier to deal with bugs by not making them in the first place?

"The 'pass the buck' nature of this proposal doesn't really 'sit well' with me,"
It doesn't "pass the buck". If a null pointer exception gets thrown, the caller has to deal with it. If a compile-time error gets generated, the caller has to deal with it. You've got to deal with it either way. Just, with non-nullable types, the callee can prohibit you from doing something statically known not to work. With a runtime null pointer exception, he can't.

"I do think I have explained my position here (although it is a little difficult to follow arguments in this form, but that's all we've got),"
I really don't think you have. I'd really like to see a succinct explanation of:
1) why you want to be able to pass nulls to functions that can't deal with them
2) why you don't complain that you can't pass integers to functions wanting strings (because it's the same thing, and you've not thus far complained about it)
3) why you believe that static detection of failures is a bad thing

"and I do understand your position too; I just don't this feature is needed. "
Again, nothing is needed beyond a paper tape and tape reader, or a particular combinator, or subtract and branch if not zero, or whatever other Turing complete primitive mechanism you want to use. I don't think any of us are interested in what's "needed".
Anonymous
April 29, 2005
"Clearly, you are a little bothered by my position ..."

Michael - I am bothered by your position. Why? It's scary and it's wrong.

You have been given the question "would you like to catch the error on your machine, before you ship the software, or just have the software go down in production"

and you are answering "I'll take production please"

I find that very disturbing.
Anonymous
April 29, 2005
> Michael - I am bothered by your
> position. Why? It's scary and it's wrong.

Nice to see you have an open mind about the idea :)

I will respond to DrPizza in-depth soon.
Anonymous
April 29, 2005
Michael: "
> 2) why you don't complain that you can't pass integers
> to functions wanting strings (because it's the same
> thing, and you've not thus far complained about it)

Yeah, Right. "

Actually Michael, i'm confused about this as well since it's the same thing to me too.

In the int/string case it's the type system coming in and telling you that you're passing in the wrong type. it's up to you to figure out what you want to do in that case.

In the string/string! case it's the type system... yadda yadda yadda.

They're identical. So i'm not sure why you're against one, but for another.
Anonymous
April 29, 2005
> Isn't it easier to deal with bugs
> by not making them in the first place?

Indeed it is.

> It doesn't "pass the buck".

Yes it does. The API says: "I'm not going to handle this variable
for the 'null' case, so I will force you to do it".

> If a null pointer exception
> gets thrown, the caller has to deal with it.

True, but they can choose to let it propogate. In this case they can't, they are forced to either re-throw, or something.

> Just, with non-nullable types, the callee can prohibit
> you from doing something statically known not to
> work. With a runtime null pointer exception, he can't.

Uhh, actually he can - and just did - prohibit you from doing something: executing the rest of his function.

Clearly there is a difference between the two systems; but - again - I think it's something the LANGUAGE should stay out of. If I want to restrict my types to some certain subset of what is allowed by the language, I would prefer it to be done inside a new custom object, not with new language operators. The new custom object method has so many more advantages, I really don't see your disagreement to it.

> 1) why you want to be able to pass nulls to functions
> that can't deal with them

Nope - My position is that the functions should deal with them.

> 2) why you don't complain that you can't pass integers
> to functions wanting strings (because it's the same
> thing, and you've not thus far complained about it)

Yeah, Right.

> 3) why you believe that static detection of failures is a bad thing

I never said that; it's not a bad thing. It's great.

> Again, nothing is needed beyond a paper tape and tape reader,

You, then, are an advocate of adding as many features you can think of to a language? I'm not. I'd hope to keep a language relatively simple, [note: a LANGUAGE not the languages API] so that it is easier to understand.

With a small set of common 'blocks' it's easier to read and understand other peoples code (and your own). I think this feature (and the nullable type feature that this seems to be spawned from) fit into this category.
Anonymous
April 29, 2005
The comment has been removed
Anonymous
April 29, 2005
Michael: "I would prefer it to be done inside a new custom object, not with new language operators. The new custom object method has so many more advantages, I really don't see your disagreement to it. "

I'm confused :(

We went over your custom object and we saw that it wasn't up to the job.

You wanted to pass a non-null object but it wasn't able to actually prevent you from doing that. (as i showed trivially that you could still pass null).

So, what was the benefit?

In your system you have a type called NonNullString which isn't checked and which doesn't prevent null bugs. In my system you have a type called "string!" which is checked and which does prevent null bugs :)
Anonymous
April 29, 2005
ps: by the looks of things, I've missed the invitation to the NHA meetings (Null Haters' Anonymous) ... Seriously, what does everyone have against the literal? I find myself using it alot as a marker for various states. It's truly very rare for me to get a NullReferenceException.

But I suppose this request is in response for the 'nullable' operator.

I wonder, would this proposal be so widely accepted if that one didn't exist?
Anonymous
April 29, 2005
The comment has been removed
Anonymous
April 29, 2005
> I'm confused :(
>
> We went over your custom object and we
> saw that it wasn't up to the job.

To me it still did the job; even though you could null out the actual custom object you couldn't null out the string. (which is the goal). It also allows me to apply more rules to the format of the string (i.e. N chars long, etc). AND it would hopefully come with a little documentation, explaining it's purpose.

All +'s over the operator approach, IMHO.

> You wanted to pass a non-null object but
> it wasn't able to actually prevent you from
> doing that. (as i showed trivially that you
> could still pass null).

As above, all the I wanted to accomplish was a non-null 'wrapped' object (i.e. the object the wrapper holds). The holder object can be null, but I don't care; I'll check for that (in my API).

> So, what was the benefit?

As above ...
Anonymous
April 29, 2005
The comment has been removed
Anonymous
April 29, 2005
Michael: "To me it still did the job; even though you could null out the actual custom object you couldn't null out the string."

but that's absolutely not true :)

if the custom object is null then the string is null. So you've violated the condition that the string not be null!

All you've done is transitively push the nullness and nonnullness of the string off to another object. But that's completely useless to me since now all the null problems i was haivn with the string are just transfered to your object.

I want a system where i cannot actually get null. You're asking for a non-language way to get that, and basically the problem is that non-language way does not exist.
Anonymous
April 29, 2005
The comment has been removed
Anonymous
April 29, 2005
Michael: "
As above, all the I wanted to accomplish was a non-null 'wrapped' object (i.e. the object the wrapper holds). The holder object can be null, but I don't care; I'll check for that (in my API). "

But i want a non-null holder object for my non-null value.

How do you accomplish that? :)
Anonymous
April 29, 2005
> the entire point of htis is so that you can't null out the object. And
> if you can null out the object...

Not from my point of view it wasn't. I assumed the point was to ensure an object was in a specific state when you received it (i.e. in this case: not null).

> so absolutely nothing has been gained.

For me it has; I've ensured the value of the string INSIDE that object, AND i've applied some application specific rules to it. I've gained HEAPS more: clearer, easier to deal with, documented, application specific, etc.
Anonymous
April 29, 2005
The comment has been removed
Anonymous
April 29, 2005
Michael: "So you can be sure, do to your logic, at runtime there will be no states like this. All your "!" does is enforce this at compile-time."

Yes.

Exactly :)

That was the point of all of this. To take something that normally happened at runtime (which is incredibly bad), and move it to compile time (where i can handle it).

"My suggestion is this can be done with a custom-object (hence rules are enscapsulated)."

But, once again, it can't. Your system pushes it back to runtime. Thus violating the need to get it at compile time.

---

Let me give you an example. When we ship software, it's enormously expensive to fix a bug in it. I mean, ridiculously ridiculously expensive.

Imagine getting a NullReferenceException after we've shipped VS that could have been stopped by having the compiler tell us we were doing somethign bad that would normally fail at runtime.

Now i've saved myself and my company a ton of money and i've made the product better for my customers. That's a good thing :)
Anonymous
April 29, 2005
> How do you accomplish that? :)

Well, we can't :)

But in reality, that doesn't concern ME very much, but I guess it concerns you :)

PS: I'd love to stay and chat, but I have to go ...

I'll look forward to lots of disagreement when I get back ... :)
Anonymous
April 30, 2005
"I see what you are saying (I think) that you can't prevent an object from being null; but I think it's at the same level as preventing it from being in ANY state. "
It isn't. Preventing it from being non-existant is not the same as preventing it from existing but being in a particular state.
Anonymous
April 30, 2005
The comment has been removed
Anonymous
April 30, 2005
The comment has been removed
Anonymous
May 01, 2005
You are probably not going to read this, with all these comments...

I hope I never see this in production. The nullables are bad enough, and we certainly don't need this stuff. It must be simpler to teach people to unit test?

Most of all: I can't see why you want to do this. Isn't compile time checking a red herring? Don't you see what all this compile time checking and type safety things are doing to the language?

You are slowly killing your product. What was a simple language, mostly elegant with a few quirks, is now turing into a complex langague, not so elegant and with many quirks.
Anonymous
May 02, 2005
The comment has been removed
Anonymous
May 03, 2005
The comment has been removed
Anonymous
May 03, 2005
"Specifically, you'd have to forbid the use of the "." and "[]" operators on nullable (reference) types. "
Why?

The aim isn't to remove null pointer exceptions when something might legitimately be null.

It's to remove null pointer exceptions when the contract of a function says that an argument can't be null.
Anonymous
May 03, 2005
> For me the language doesn't exist just to sit there
> looking pretty. it exists so i can write good programs.
> If it doesn't allow me to do that, then it's not a very good tool.

I disagree (a little). A language needs to be 'independant' enough
so that it is good for EVERYONE. Thus, it has a constrained set
of features so that there aren't overly complicated and useless
contrstructs in it for those that don't like them. I think the problem
is that these features (nullable & not nullable) are really for a
small set of programmers, it just so happens that those programmers
are the ones in control of the language.

Sadly, I think these features (and probably more) will be added, and
it will probably be enough to stop me from upgrading. It seems to me
that c# is a little 'feature happy'. Personally, (and profressionally)
I don't think I'll be upgrading to the next c#.

At the very least it would be nice if these features could come as some
sort of extension or something, (as they are compile-time only, not run-time)
like a 'strict compiler' option.

Anyway, I did promise I wouldn't reply again ... oops :)
Anonymous
May 04, 2005
"I disagree (a little). A language needs to be 'independant' enough so that it is good for EVERYONE. Thus, it has a constrained set of features so that there aren't overly complicated and useless contrstructs in it for those that don't like them."
In what sense are non-nullable types "useless"?

"I think the problem is that these features (nullable & not nullable) are really for a small set of programmers, it just so happens that those programmers are the ones in control of the language. "
I don't agree at all. I think that MOST programmers have run into at least one null pointer exception in their career, and I would wager rather more than one. Non-nullable types would solve these very common exceptions in the majority of situations. And nothing you've proposed is anything like as effective in this regard.

"Sadly, I think these features (and probably more) will be added, and it will probably be enough to stop me from upgrading. It seems to me that c# is a little 'feature happy'. Personally, (and profressionally) I don't think I'll be upgrading to the next c#. "
Personally, I think that C# is very feature-poor. C# 2 will resolve some of the issues, but certainly not all.

"At the very least it would be nice if these features could come as some sort of extension or something, (as they are compile-time only, not run-time) like a 'strict compiler' option. "
Why?
Anonymous
May 04, 2005
The comment has been removed
Anonymous
May 04, 2005
The comment has been removed
Anonymous
May 04, 2005
I have not followed this discussion but http://www.research.microsoft.com/specsharp/ seems closesly related. Among other things it "statically enforces non-null types".
Anonymous
May 09, 2005
The comment has been removed
Anonymous
May 12, 2005
"Most important to me is that I don't need it. If I could turn off the check for missing downcasts, I would do so. I have unit tests which ensures my code behaves as expected. A bonus is that the same tests removes the need for many compiletime checks. "

Compiler-enforced checks are quicker to write than unit tests, and more accurate than unit tests.

You don't unit test things that are too trivial to break, and mechanisms such as non-nullable types increase the number of things that fall into that category.
Anonymous
May 23, 2005
The comment has been removed
Anonymous
January 28, 2006
After reading all this, I like the idea of adding FXCOP-style warnings to help programmers identify places where members are called on possibly null objects.
Anonymous
July 25, 2007
There is no-doubt that the C#2 nullable-types is a cool feature. However I regret that C# don't support

Partager via

Difficulties with non-nullable types (part 4)

Comments

Ressources supplémentaires