Model caching and StoreModel lifecycle in Foundry Local

  • Applies to: Foundry Local on Azure Local

This article explains how the StoreModel custom resource definition (CRD) tracks model caching in the cluster's local OCI registry, how cache jobs download and store model artifacts, and how inference pods retrieve cached models.

Important

  • Foundry Local is available in preview. Preview releases provide early access to features that are in active deployment.
  • Features, approaches, and processes can change or have limited capabilities before general availability (GA).

What StoreModel does

StoreModel is an internal CRD the operator uses to track whether model artifacts are downloaded and cached in the cluster's local OCI registry. You never create StoreModel resources directly. The inference operator creates and manages them as part of the ModelDeployment reconciliation process.

StoreModel lifecycle

The StoreModel status phase shows where model caching is in its lifecycle and whether the model is ready for use.

Phase Description
Pending StoreModel created, cache job not yet started.
Storing A Kubernetes Job is running to download model artifacts and push them to the local registry.
Available Model artifacts are cached. The status.storeRef field contains the local OCI path.
Error The cache job failed. The operator deletes the StoreModel so the next deployment attempt can retry.

How caching works

When the operator processes a ModelDeployment:

  1. It generates a deterministic StoreModel name from the model source, alias, compute type, framework, and version.
  2. It checks if a StoreModel CR with that name already exists.
  3. If the StoreModel is Available, the operator reads the cached OCI path from status.storeRef and proceeds.
  4. If no StoreModel exists, the operator creates one, which triggers a cache Job.
  5. The operator raises a temporary retry (with a configurable poll interval) until the StoreModel reaches Available.
  6. If the StoreModel enters Error, the operator deletes it and marks the ModelDeployment as failed.
  7. A configurable timeout prevents indefinite waiting.

The inference pods use an init container (the model-store-retriever) that pulls model files from the local registry into the pod's filesystem before the main inference container starts.

How caching fits into reconciliation

The caching step occurs early in the ModelDeployment reconciliation flow, after model resolution but before child resource creation:

  1. The operator validates the ModelDeployment spec and resolves the model source.
  2. The operator ensures the model is cached locally by creating a StoreModel CR and waiting for the cache job to complete.
  3. Once the StoreModel is Available, the operator proceeds to select the container image, generate API key secrets, and create the remaining child resources (Deployment, Service, ConfigMap, and others).

For the full reconciliation flow, see Inference operator and model lifecycle.

Related content

  • Last updated on