Training
Module
Build your first Orleans app with ASP.NET Core 8.0 - Training
Learn how to build cloud-native, distributed apps with Orleans.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
There are broadly two kinds of serialization used in Orleans:
The majority of this article is dedicated to grain call serialization via the serialization framework included in Orleans. The Grain storage serializers section discusses the grain storage serialization.
Orleans includes an advanced and extensible serialization framework which can be referred to as Orleans.Serialization. The serialization framework included in Orleans is designed to meet the following goals:
int
to/from long
, float
to/from double
)High-fidelity representation of types is fairly uncommon for serializers, so some points warrant further elaboration:
Dynamic types and arbitrary polymorphism: Orleans doesn't enforce restrictions on the types that can be passed in grain calls and maintain the dynamic nature of the actual data type. That means, for example, that if the method in the grain interfaces is declared to accept IDictionary but at runtime, the sender passes SortedDictionary<TKey,TValue>, the receiver will indeed get SortedDictionary
(although the "static contract"/grain interface did not specify this behavior).
Maintaining object identity: If the same object is passed multiple types in the arguments of a grain call or is indirectly pointed more than once from the arguments, Orleans will serialize it only once. On the receiver side, Orleans will restore all references correctly so that two pointers to the same object still point to the same object after deserialization as well. Object identity is important to preserve in scenarios like the following. Imagine grain A is sending a dictionary with 100 entries to grain B, and 10 of the keys in the dictionary point to the same object, obj
, on A's side. Without preserving object identity, B would receive a dictionary of 100 entries with those 10 keys pointing to 10 different clones of obj
. With object identity-preserved, the dictionary on B's side looks exactly like on A's side with those 10 keys pointing to a single object obj
. Note that because the default string hash code implementations in .NET are randomized per-process, ordering of values in dictionaries and hash sets (for example) may not be preserved.
To support version tolerance, the serializer requires developers to be explicit about which types and members are serialized. We have tried to make this as pain-free as possible. You must mark all serializable types with Orleans.GenerateSerializerAttribute to instruct Orleans to generate serializer code for your type. Once you have done this, you can use the included code-fix to add the required Orleans.IdAttribute to the serializable members on your types, as demonstrated here:
Here is an example of a serializable type in Orleans, demonstrating how to apply the attributes.
[GenerateSerializer]
public class Employee
{
[Id(0)]
public string Name { get; set; }
}
Orleans supports inheritance and will serialize the individual layers in the hierarchy separately, allowing them to have distinct member ids.
[GenerateSerializer]
public class Publication
{
[Id(0)]
public string Title { get; set; }
}
[GenerateSerializer]
public class Book : Publication
{
[Id(0)]
public string ISBN { get; set; }
}
In the preceding code, note that both Publication
and Book
have members with [Id(0)]
even though Book
derives from Publication
. This is the recommended practice in Orleans because members identifiers are scoped to the inheritance level, not the type as a whole. Members can be added and removed from Publication
and Book
independently, but a new base class cannot be inserted into the hierarchy once the application has been deployed without special consideration.
Orleans also supports serializing types with internal
, private
, and readonly
members, such as in this example type:
[GenerateSerializer]
public struct MyCustomStruct
{
public MyCustom(int intProperty, int intField)
{
IntProperty = intProperty;
_intField = intField;
}
[Id(0)]
public int IntProperty { get; }
[Id(1)] private readonly int _intField;
public int GetIntField() => _intField;
public override string ToString() => $"{nameof(_intField)}: {_intField}, {nameof(IntProperty)}: {IntProperty}";
}
By default, Orleans will serialize your type by encoding its full name. You can override this by adding an Orleans.AliasAttribute. Doing so will result in your type being serialized using a name that is resilient to renaming the underlying class or moving it between assemblies. Type aliases are globally scoped, and you cannot have two aliases with the same value in an application. For generic types, the alias value must include the number of generic parameters preceded by a backtick, for example, MyGenericType<T, U>
could have the alias [Alias("mytype`2")]
.
Members defined in a record's primary constructor have implicit ids by default. In other words, Orleans supports serializing record
types. This means that you cannot change the parameter order for an already deployed type, since that breaks compatibility with previous versions of your application (in the case of a rolling upgrade) and with serialized instances of that type in storage and streams. Members defined in the body of a record type don't share identities with the primary constructor parameters.
[GenerateSerializer]
public record MyRecord(string A, string B)
{
// ID 0 won't clash with A in primary constructor as they don't share identities
[Id(0)]
public string C { get; init; }
}
If you don't want the primary constructor parameters to be automatically included as Serializable fields, you can use [GenerateSerializer(IncludePrimaryConstructorParameters = false)]
.
Sometimes you may need to pass types between grains which you don't have full control over. In these cases, it may be impractical to convert to and from some custom-defined type in your application code manually. Orleans offers a solution for these situations in the form of surrogate types. Surrogates are serialized in place of their target type and have functionality to convert to and from the target type. Consider the following example of a foreign type and a corresponding surrogate and converter:
// This is the foreign type, which you do not have control over.
public struct MyForeignLibraryValueType
{
public MyForeignLibraryValueType(int num, string str, DateTimeOffset dto)
{
Num = num;
String = str;
DateTimeOffset = dto;
}
public int Num { get; }
public string String { get; }
public DateTimeOffset DateTimeOffset { get; }
}
// This is the surrogate which will act as a stand-in for the foreign type.
// Surrogates should use plain fields instead of properties for better performance.
[GenerateSerializer]
public struct MyForeignLibraryValueTypeSurrogate
{
[Id(0)]
public int Num;
[Id(1)]
public string String;
[Id(2)]
public DateTimeOffset DateTimeOffset;
}
// This is a converter that converts between the surrogate and the foreign type.
[RegisterConverter]
public sealed class MyForeignLibraryValueTypeSurrogateConverter :
IConverter<MyForeignLibraryValueType, MyForeignLibraryValueTypeSurrogate>
{
public MyForeignLibraryValueType ConvertFromSurrogate(
in MyForeignLibraryValueTypeSurrogate surrogate) =>
new(surrogate.Num, surrogate.String, surrogate.DateTimeOffset);
public MyForeignLibraryValueTypeSurrogate ConvertToSurrogate(
in MyForeignLibraryValueType value) =>
new()
{
Num = value.Num,
String = value.String,
DateTimeOffset = value.DateTimeOffset
};
}
In the preceding code:
MyForeignLibraryValueType
is a type outside of your control, defined in a consuming library.MyForeignLibraryValueTypeSurrogate
is a surrogate type that maps to MyForeignLibraryValueType
.MyForeignLibraryValueTypeSurrogateConverter
acts as a converter to map to and from the two types. The class is an implementation of the IConverter<TValue,TSurrogate> interface.Orleans supports serialization of types in type hierarchies (types which derive from other types). In the event that a foreign type might appear in a type hierarchy (for example as the base class for one of your own types), you must additionally implement the Orleans.IPopulator<TValue,TSurrogate> interface. Consider the following example:
// The foreign type is not sealed, allowing other types to inherit from it.
public class MyForeignLibraryType
{
public MyForeignLibraryType() { }
public MyForeignLibraryType(int num, string str, DateTimeOffset dto)
{
Num = num;
String = str;
DateTimeOffset = dto;
}
public int Num { get; set; }
public string String { get; set; }
public DateTimeOffset DateTimeOffset { get; set; }
}
// The surrogate is defined as it was in the previous example.
[GenerateSerializer]
public struct MyForeignLibraryTypeSurrogate
{
[Id(0)]
public int Num;
[Id(1)]
public string String;
[Id(2)]
public DateTimeOffset DateTimeOffset;
}
// Implement the IConverter and IPopulator interfaces on the converter.
[RegisterConverter]
public sealed class MyForeignLibraryTypeSurrogateConverter :
IConverter<MyForeignLibraryType, MyForeignLibraryTypeSurrogate>,
IPopulator<MyForeignLibraryType, MyForeignLibraryTypeSurrogate>
{
public MyForeignLibraryType ConvertFromSurrogate(
in MyForeignLibraryTypeSurrogate surrogate) =>
new(surrogate.Num, surrogate.String, surrogate.DateTimeOffset);
public MyForeignLibraryTypeSurrogate ConvertToSurrogate(
in MyForeignLibraryType value) =>
new()
{
Num = value.Num,
String = value.String,
DateTimeOffset = value.DateTimeOffset
};
public void Populate(
in MyForeignLibraryTypeSurrogate surrogate, MyForeignLibraryType value)
{
value.Num = surrogate.Num;
value.String = surrogate.String;
value.DateTimeOffset = surrogate.DateTimeOffset;
}
}
// Application types can inherit from the foreign type, assuming they're not sealed
// since Orleans knows how to serialize it.
[GenerateSerializer]
public sealed class DerivedFromMyForeignLibraryType : MyForeignLibraryType
{
public DerivedFromMyForeignLibraryType() { }
public DerivedFromMyForeignLibraryType(
int intValue, int num, string str, DateTimeOffset dto) : base(num, str, dto)
{
IntValue = intValue;
}
[Id(0)]
public int IntValue { get; set; }
}
Version-tolerance is supported provided the developer follows a set of rules when modifying types. If the developer is familiar with systems such as Google Protocol Buffers (Protobuf), then these rules will be familiar.
Base
class can declare a field with id 0
and a different field can be declared by Sub : Base
with the same id, 0
.int
& uint
are invalid.int
to long
or ulong
to ushort
are supported.ulong
to ushort
are only supported if the value at runtime is less than ushort.MaxValue
.double
to float
are only supported if the runtime value is between float.MinValue
and float.MaxValue
.decimal
, which has a narrower range than both double
and float
.Orleans promotes safety by default. This includes safety from some classes of concurrency bugs. In particular, Orleans will immediately copy objects passed in grain calls by default. This copying is facilitated by Orleans.Serialization and when Orleans.CodeGeneration.GenerateSerializerAttribute is applied to a type, Orleans will also generate copiers for that type. Orleans will avoid copying types or individual members which are marked using the ImmutableAttribute. For more details, see Serialization of immutable types in Orleans.
✅ Do give your types aliases using the [Alias("my-type")]
attribute. Types with aliases can be renamed without breaking compatibility.
❌ Do not change a record
to a regular class
or vice-versa. Records and classes are not represented in an identical fashion since records have primary constructor members in addition to regular members and therefore the two are not interchangeable.
❌ Do not add new types to an existing type hierarchy for a serializable type. You must not add a new base class to an existing type. You can safely add a new subclass to an existing type.
✅ Do replace usages of SerializableAttribute with GenerateSerializerAttribute and corresponding IdAttribute declarations.
✅ Do start all member ids at zero for each type. Ids in a subclass and its base class can safely overlap. Both properties in the following example have ids equal to 0
.
[GenerateSerializer]
public sealed class MyBaseClass
{
[Id(0)]
public int MyBaseInt { get; set; }
}
[GenerateSerializer]
public sealed class MySubClass : MyBaseClass
{
[Id(0)]
public int MyBaseInt { get; set; }
}
✅ Do widen numeric member types as needed. You can widen sbyte
to short
to int
to long
.
int.MaxValue
cannot be represented by a short
field, so narrowing an int
field to short
can result in a runtime exception if such a value were encountered.❌ Do not change the signedness of a numeric type member. You must not change a member's type from uint
to int
or an int
to a uint
, for example.
Orleans includes a provider-backed persistence model for grains, accessed via the State property or by injecting one or more IPersistentState<TState> values into your grain. Before Orleans 7.0, each provider had a different mechanism for configuring serialization. In Orleans 7.0, there is now a general-purpose grain state serializer interface, IGrainStorageSerializer, which offers a consistent way to customize state serialization for each provider. Supported storage providers implement a pattern that involves setting the IStorageProviderSerializerOptions.GrainStorageSerializer property on the provider's options class, for example:
Grain storage serialization currently defaults to Newtonsoft.Json
to serialize state. You can replace this by modifying that property at configuration time. The following example demonstrates this, using OptionsBuilder<TOptions>:
siloBuilder.AddAzureBlobGrainStorage(
"MyGrainStorage",
(OptionsBuilder<AzureBlobStorageOptions> optionsBuilder) =>
{
optionsBuilder.Configure<IMySerializer>(
(options, serializer) => options.GrainStorageSerializer = serializer);
});
For more information, see OptionsBuilder API.
Orleans has an advanced and extensible serialization framework. Orleans serializes data types passed in grain request and response messages as well as grain persistent state objects. As part of this framework, Orleans automatically generates serialization code for those data types. In addition to generating a more efficient serialization/deserialization for types that are already .NET-serializable, Orleans also tries to generate serializers for types used in grain interfaces that are not .NET-serializable. The framework also includes a set of efficient built-in serializers for frequently used types: lists, dictionaries, strings, primitives, arrays, etc.
Two important features of Orleans's serializer set it apart from a lot of other third-party serialization frameworks: dynamic types/arbitrary polymorphism and object identity.
Dynamic types and arbitrary polymorphism: Orleans doesn't enforce restrictions on the types that can be passed in grain calls and maintain the dynamic nature of the actual data type. That means, for example, that if the method in the grain interfaces is declared to accept IDictionary but at runtime, the sender passes SortedDictionary<TKey,TValue>, the receiver will indeed get SortedDictionary
(although the "static contract"/grain interface did not specify this behavior).
Maintaining object identity: If the same object is passed multiple types in the arguments of a grain call or is indirectly pointed more than once from the arguments, Orleans will serialize it only once. On the receiver side, Orleans will restore all references correctly so that two pointers to the same object still point to the same object after deserialization as well. Object identity is important to preserve in scenarios like the following. Imagine grain A is sending a dictionary with 100 entries to grain B, and 10 of the keys in the dictionary point to the same object, obj, on A's side. Without preserving object identity, B would receive a dictionary of 100 entries with those 10 keys pointing to 10 different clones of obj. With object identity-preserved, the dictionary on B's side looks exactly like on A's side with those 10 keys pointing to a single object obj.
The above two behaviors are provided by the standard .NET binary serializer and it was therefore important for us to support this standard and familiar behavior in Orleans as well.
Orleans uses the following rules to decide which serializers to generate. The rules are:
Orleans supports transmission of arbitrary types at runtime and therefore the in-built code generator cannot determine the entire set of types that will be transmitted ahead of time. Additionally, certain types cannot have serializers generated for them because they are inaccessible (for example, private
) or have inaccessible fields (for example, readonly
). Therefore, there is a need for just-in-time serialization of types that were unexpected or could not have serializers generated ahead of time. The serializer responsible for these types is called the fallback serializer. Orleans ships with two fallback serializers:
MyPrivateType
contains a field MyType
which has a custom serializer, that custom serializer will be used to serialize it.The fallback serializer can be configured using the FallbackSerializationProvider property on both ClientConfiguration on the client and GlobalConfiguration on the silos.
// Client configuration
var clientConfiguration = new ClientConfiguration();
clientConfiguration.FallbackSerializationProvider =
typeof(FantasticSerializer).GetTypeInfo();
// Global configuration
var globalConfiguration = new GlobalConfiguration();
globalConfiguration.FallbackSerializationProvider =
typeof(FantasticSerializer).GetTypeInfo();
Alternatively, the fallback serialization provider can be specified in XML configuration:
<Messaging>
<FallbackSerializationProvider
Type="GreatCompany.FantasticFallbackSerializer, GreatCompany.SerializerAssembly"/>
</Messaging>
The BinaryFormatterSerializer is the default fallback serializer.
Warning
Binary serialization with BinaryFormatter
can be dangerous. For more information, see the BinaryFormatter security guide and the BinaryFormatter migration guide.
Exceptions are serialized using the fallback serializer. Using the default configuration, BinaryFormatter
is the fallback serializer and so the ISerializable pattern must be followed in order to ensure correct serialization of all properties in an exception type.
Here is an example of an exception type with correctly implemented serialization:
[Serializable]
public class MyCustomException : Exception
{
public string MyProperty { get; }
public MyCustomException(string myProperty, string message)
: base(message)
{
MyProperty = myProperty;
}
public MyCustomException(string transactionId, string message, Exception innerException)
: base(message, innerException)
{
MyProperty = transactionId;
}
// Note: This is the constructor called by BinaryFormatter during deserialization
public MyCustomException(SerializationInfo info, StreamingContext context)
: base(info, context)
{
MyProperty = info.GetString(nameof(MyProperty));
}
// Note: This method is called by BinaryFormatter during serialization
public override void GetObjectData(SerializationInfo info, StreamingContext context)
{
base.GetObjectData(info, context);
info.AddValue(nameof(MyProperty), MyProperty);
}
}
Serialization serves two primary purposes in Orleans:
The serializers generated by Orleans are suitable for the first purpose due to their flexibility, performance, and versatility. They are not as suitable for the second purpose, since they are not explicitly version-tolerant. It is recommended that users configure a version-tolerant serializer such as Protocol Buffers for persistent data. Protocol Buffers is supported via Orleans.Serialization.ProtobufSerializer
from the Microsoft.Orleans.OrleansGoogleUtils NuGet package. The best practices for the particular serializer of choice should be used to ensure version tolerance. Third-party serializers can be configured using the SerializationProviders
configuration property as described above.
.NET feedback
.NET is an open source project. Select a link to provide feedback:
Training
Module
Build your first Orleans app with ASP.NET Core 8.0 - Training
Learn how to build cloud-native, distributed apps with Orleans.