CoreML Namespace

The CoreML namespace provides relatively high-level access to machine learning model runtimes.

Classes

MLArrayBatchProvider	An IMLBatchProvider backed by an array.
MLCustomLayer_Extensions	Default implementations for optional methods in the IMLCustomLayer protocol.
MLCustomModel	Default implementation of IMLCustomModel.
MLCustomModel_Extensions	Default implementations for optional methods in the MLCustomModel protocol.
MLDictionaryConstraint	Contains a value that constrains the type of dictionary keys.
MLDictionaryFeatureProvider	An implementation of IMLFeatureProvider that is backed by a NSDictionary.
MLFeatureDescription	A developer-meaningful description of a MLModel feature.
MLFeatureValue	An immutable value and MLFeatureType for a feature.
MLImageConstraint	Contains constraints for an image feature.
MLImageSize	Describes one acceptable image size for the CoreML model inputs.
MLImageSizeConstraint	Description of the constraint on image sizes for a CoreML model.
MLModel	Encapsulates a trained machine-learning model.
MLModelConfiguration
MLModelDescription	A developer-meaningful description of the MLModel.
MLModelErrorExtensions	Extension methods for the CoreML.MLModelError enumeration.
MLModelMetadata	A DictionaryContainer that holds metadata related to a MLModel.
MLMultiArray	Represents an efficient multi-dimensional array.
MLMultiArrayConstraint	Contains constraints for a multidimensional array feature.
MLMultiArrayShapeConstraint	Describes the constraints on the shape of the multidimensional array allowed by the model.
MLPredictionOptions	Contains a value that indicates whether to restrict prediction computations to the CPU.
MLSequence	Encodes a sequence as a single input.
MLSequenceConstraint	A constraint on sequences of features.

Interfaces

IMLBatchProvider	Interface defining the protocol for providing data in batches to the model.
IMLCustomLayer	Interface defining methods necessary for a custom model layer.
IMLCustomModel	Interface defining a custom CoreML model.
IMLFeatureProvider	An interface that defines input or output features and allows access to their values.

Enums

MLComputeUnits
MLFeatureType	Enumerates the kinds of features supported by CoreML.
MLImageSizeConstraintType	Enumerates the form of a MLImageSizeConstraint.
MLModelError	Enumerates errors that may occur in the use of Core ML.
MLMultiArrayDataType	Enumerates the types of values stored in a MLMultiArray.
MLMultiArrayShapeConstraintType	Enumerates the form of a MLMultiArrayShapeConstraint.

Remarks

The CoreML namespace, introduced in iOS 11, allows runtime querying of a broad variety of machine-learning models produced by frameworks such as scipy-learn, TensorFlow, and Azure Custom Vision cognitive services. CoreML does not support on-device modification of models or weights, however it does support loading a model from a URL, so developers could use that to download updated models.

CoreML relies on a "model" that is distributed as a single .mlmodel file. This model is compiled into a usable form by either using integrated tools in Xcode, Xamarin Studio, or at the command-line. At the command line, an .mlmodel file can be compiled with xcrun coremlcompiler compile model.mlmodel outputfolder . A compiled model takes the form of a directory called modelname.mlmodelc . The model is then loaded at runtime with code similar to:

var bundle = NSBundle.MainBundle;
var assetPath = bundle.GetUrlForResource("MarsHabitatPricer", "mlmodelc");
NSError mlErr;
model = MLModel.Create(assetPath, out mlErr);

Models in CoreML are fairly "opaque box" and do not have an extensive API. Rather, the iOS developer must know the input and output names and types the model expects. For instance, an image-recognition model might expect a CVPixelBuffer of size 227x227 identified as "image" and might have two outputs: a string identified as "classLabel" and a NSDictionary with NSString keys and double

values in the range [0,1] representing the confidence of the prediction.

Developers must convert their native representations into CoreML-compatible instances of IMLFeatureProvider . The easiest way to do this is with a MLDictionaryFeatureProvider :

CVPixelBuffer pixelBuffer = // from image or video stream
var imageValue = MLFeatureValue.Create(pixelBuffer);

var inputs = new NSDictionary<NSString, NSObject>(new NSString("image"), imageValue);

NSError error, error2;
var inputFp = new MLDictionaryFeatureProvider(inputs, out error);
if(error != null)
{
	ErrorOccurred(this, new EventArgsT<string>(error.ToString()));
	return;
}
var outFeatures = model.GetPrediction(inputFp, out error2);
if(error2 != null)
{
	ErrorOccurred(this, new EventArgsT<string>(error2.ToString()));
	return;
}

var predictionsDictionary = outFeatures.GetFeatureValue("classLabelProbs").DictionaryValue;

A more complex, but more flexible, way is to implement IMLFeatureProvider :

public class MarsHabitatPricerInput : NSObject, IMLFeatureProvider
{
	public double SolarPanels { get; set; }
	public double Greenhouses { get; set; }
	public double Size { get; set; }

	public NSSet<NSString> FeatureNames => new NSSet<NSString>(new NSString("solarPanels"), new NSString("greenhouses"), new NSString("size"));

	public MLFeatureValue GetFeatureValue(string featureName)
	{
		switch (featureName)
		{
			case "solarPanels":
				return MLFeatureValue.Create(SolarPanels);
			case "greenhouses":
				return MLFeatureValue.Create(Greenhouses);
			case "size":
				return MLFeatureValue.Create(Size);
			default:
				return MLFeatureValue.Create(0);
		}
	}
}

Getting a prediction occurs synchronously, with a call to GetPrediction :

NSError prErr;
IMLFeatureProvider outFeatures = model.GetPrediction(pricerInput, out prErr);
double result = outFeatures.GetFeatureValue("price").DoubleValue;

CoreML currently supports:

Type	Variants	Produced by:
Neural networks	Convolutional, feed-forward, recurrent	Caffe, Keras, Azure Custom Vision
Tree ensembles	Random forests, boosted trees, decision trees	scikit-learn, XGBoost
SVMs	Scalar and multiclass	scikit-learn, LIBSVM
Generalized linear models	Linear and logistic regression	scikit-learn
Pipeline models	Sequentially chained models	scikit-learn

Apple has open-sourced (3-clause BSD License) python tools to create CoreML models: https://pypi.python.org/pypi/coremltools

See also

• https://github.com/xamarin/ios-samples/blob/master/ios11/CoreML
• https://github.com/xamarin/ios-samples/blob/master/ios11/CoreMLAzureModel
• https://github.com/xamarin/ios-samples/blob/master/ios11/CoreMLImageRecognition
• https://github.com/xamarin/ios-samples/blob/master/ios11/CoreMLVision