Azure function container fail to start in AKS

Samko Yoon 1 Reputation point
2021-05-21T07:57:49.56+00:00

I've deployed azure python function image(mcr.microsoft.com/azure-functions/python:3.0-python3.8) to AKS. It has one blob trigger function and it works fine locally but when it's deployed to AKS, it's throwing below error.

info: Host.Startup[314]
      Loading functions metadata
info: Host.Startup[315]
      1 functions loaded
info: Host.Triggers.Warmup[0]
      Initializing Warmup Extension.
info: Host.Startup[503]
      Initializing Host. OperationId: '96345dde-4fe4-44bc-9687-4f92a13abe67'.
info: Host.Startup[504]
      Host initialization: ConsecutiveErrors=7, StartupCount=8, OperationId=96345dde-4fe4-44bc-9687-4f92a13abe67
info: Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService[0]
      LoggerFilterOptions
      {
        "MinLevel": "None",
        "Rules": [
          {
            "ProviderName": null,
            "CategoryName": null,
            "LogLevel": null,
            "Filter": "<AddFilter>b__0"
          },
          {
            "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider",
            "CategoryName": null,
            "LogLevel": "None",
            "Filter": null
          },
          {
            "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider",
            "CategoryName": null,
            "LogLevel": null,
            "Filter": "<AddFilter>b__0"
          }
        ]
      }
info: Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService[0]
      FunctionResultAggregatorOptions
      {
        "BatchSize": 1000,
        "FlushTimeout": "00:00:30",
        "IsEnabled": true
      }
info: Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService[0]
      SingletonOptions
      {
        "LockPeriod": "00:00:15",
        "ListenerLockPeriod": "00:01:00",
        "LockAcquisitionTimeout": "10675199.02:48:05.4775807",
        "LockAcquisitionPollingInterval": "00:00:05",
        "ListenerLockRecoveryPollingInterval": "00:01:00"
      }
info: Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService[0]
      QueuesOptions
      {
        "BatchSize": 16,
        "NewBatchThreshold": 8,
        "MaxPollingInterval": "00:01:00",
        "MaxDequeueCount": 5,
        "VisibilityTimeout": "00:00:00"
      }
info: Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService[0]
      BlobsOptions
      {
        "CentralizedPoisonQueue": false
      }
info: Microsoft.Azure.WebJobs.Hosting.JobHostService[0]
      Starting JobHost
info: Host.Startup[401]
      Starting Host (HostId=sandboxdeployment55b745968bxggd9, InstanceId=44dd9602-2325-49b0-9d79-a12cb0dbd034, Version=3.0.15725.0, ProcessId=1, AppDomainId=1, InDebugMode=False, InDiagnosticMode=False, FunctionsExtensionVersion=(null))
info: Host.Startup[314]
      Loading functions metadata
info: Host.Startup[0]
      Loading proxies metadata
info: Host.Startup[0]
      Initializing Azure Function proxies
fail: Host.Startup[515]
      A host error has occurred during startup operation '96345dde-4fe4-44bc-9687-4f92a13abe67'.
Autofac.Core.DependencyResolutionException: An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = ProxyHandler (ReflectionActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.ProxyHandler], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = ApiInspectorHandler (ReflectionActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.Handlers.ApiInspectorHandler], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = DebugLogStorageService (DelegateActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.Logging.IDebugLogService], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = FileSystemStorage (DelegateActivator), Services = [DetailedTrace (Microsoft.Azure.AppService.Proxy.Common.Infra.IBlobStorage)], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> Read-only file system (See inner exception for details.) (See inner exception for details.) (See inner exception for details.) (See inner exception for details.)
 ---> Autofac.Core.DependencyResolutionException: An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = ApiInspectorHandler (ReflectionActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.Handlers.ApiInspectorHandler], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = DebugLogStorageService (DelegateActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.Logging.IDebugLogService], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = FileSystemStorage (DelegateActivator), Services = [DetailedTrace (Microsoft.Azure.AppService.Proxy.Common.Infra.IBlobStorage)], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> Read-only file system (See inner exception for details.) (See inner exception for details.) (See inner exception for details.)
 ---> Autofac.Core.DependencyResolutionException: An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = DebugLogStorageService (DelegateActivator), Services = [Microsoft.Azure.AppService.Proxy.Runtime.Logging.IDebugLogService], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = FileSystemStorage (DelegateActivator), Services = [DetailedTrace (Microsoft.Azure.AppService.Proxy.Common.Infra.IBlobStorage)], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> Read-only file system (See inner exception for details.) (See inner exception for details.)
 ---> Autofac.Core.DependencyResolutionException: An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = FileSystemStorage (DelegateActivator), Services = [DetailedTrace (Microsoft.Azure.AppService.Proxy.Common.Infra.IBlobStorage)], Lifetime = Autofac.Core.Lifetime.RootScopeLifetime, Sharing = Shared, Ownership = OwnedByLifetimeScope ---> Read-only file system (See inner exception for details.)
 ---> System.IO.IOException: Read-only file system
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at Microsoft.Azure.AppService.Proxy.Client.IoC.PersistenceModule.GetLogLocation(String type) in C:\source\src\Microsoft.Azure.AppService.Proxy.Client\IoC\PersistenceModule.cs:line 57
   at Microsoft.Azure.AppService.Proxy.Client.IoC.PersistenceModule.<Load>b__0_0(IComponentContext ctx) in C:\source\src\Microsoft.Azure.AppService.Proxy.Client\IoC\PersistenceModule.cs:line 25
   at Autofac.RegistrationExtensions.<>c__DisplayClass5_0`1.<Register>b__0(IComponentContext c, IEnumerable`1 p)
   at Autofac.Builder.RegistrationBuilder.<>c__DisplayClass0_0`1.<ForDelegate>b__0(IComponentContext c, IEnumerable`1 p)
   at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   --- End of inner exception stack trace ---
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.<Execute>b__5_0()
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
   at Autofac.Core.Resolving.InstanceLookup.Execute()
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)
   at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.ResolveNamed[TService](IComponentContext context, String serviceName, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.ResolveNamed[TService](IComponentContext context, String serviceName)
   at Microsoft.Azure.AppService.Proxy.Client.IoC.DiagnosticsModule.<>c.<Load>b__0_0(IComponentContext c) in C:\source\src\Microsoft.Azure.AppService.Proxy.Client\IoC\DiagnosticsModule.cs:line 19
   at Autofac.RegistrationExtensions.<>c__DisplayClass5_0`1.<Register>b__0(IComponentContext c, IEnumerable`1 p)
   at Autofac.Builder.RegistrationBuilder.<>c__DisplayClass0_0`1.<ForDelegate>b__0(IComponentContext c, IEnumerable`1 p)
   at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   --- End of inner exception stack trace ---
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.<Execute>b__5_0()
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
   at Autofac.Core.Resolving.InstanceLookup.Execute()
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Activators.Reflection.AutowiringParameter.<>c__DisplayClass0_0.<CanSupplyValue>b__0()
   at Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
   at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   --- End of inner exception stack trace ---
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.<Execute>b__5_0()
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
   at Autofac.Core.Resolving.InstanceLookup.Execute()
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Activators.Reflection.AutowiringParameter.<>c__DisplayClass0_0.<CanSupplyValue>b__0()
   at Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
   at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   --- End of inner exception stack trace ---
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
   at Autofac.Core.Resolving.InstanceLookup.<Execute>b__5_0()
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
   at Autofac.Core.Resolving.InstanceLookup.Execute()
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.Core.Container.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)
   at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context)
   at Microsoft.Azure.AppService.Proxy.Client.ProxyClientFactory.Create(String proxiesJson, ILogger logger) in C:\source\src\Microsoft.Azure.AppService.Proxy.Client\ProxyClientFactory.cs:line 66
   at Microsoft.Azure.WebJobs.Script.ProxyFunctionProvider.LoadProxyMetadata(String proxiesJson, Dictionary`2 functionErrors, ILogger logger) in /src/azure-functions-host/src/WebJobs.Script/Host/ProxyFunctionProvider.cs:line 123
   at Microsoft.Azure.WebJobs.Script.ProxyFunctionProvider.ReadProxyMetadata(String scriptPath, ILogger logger, Dictionary`2 functionErrors) in /src/azure-functions-host/src/WebJobs.Script/Host/ProxyFunctionProvider.cs:line 110
   at Microsoft.Azure.WebJobs.Script.ProxyFunctionProvider.LoadFunctionMetadata() in /src/azure-functions-host/src/WebJobs.Script/Host/ProxyFunctionProvider.cs:line 83
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at Microsoft.Azure.WebJobs.Script.ProxyFunctionProvider.GetFunctionMetadataAsync() in /src/azure-functions-host/src/WebJobs.Script/Host/ProxyFunctionProvider.cs:line 56
   at Microsoft.Azure.WebJobs.Script.FunctionMetadataManager.AddMetadataFromCustomProviders(IEnumerable`1 functionProviders, List`1 functionMetadataList) in /src/azure-functions-host/src/WebJobs.Script/Host/FunctionMetadataManager.cs:line 202
   at Microsoft.Azure.WebJobs.Script.FunctionMetadataManager.LoadCustomProviderFunctions(List`1 functionMetadataList) in /src/azure-functions-host/src/WebJobs.Script/Host/FunctionMetadataManager.cs:line 195
   at Microsoft.Azure.WebJobs.Script.FunctionMetadataManager.LoadFunctionMetadata(Boolean forceRefresh, Boolean includeCustomProviders) in /src/azure-functions-host/src/WebJobs.Script/Host/FunctionMetadataManager.cs:line 148
   at Microsoft.Azure.WebJobs.Script.FunctionMetadataManager.GetFunctionMetadata(Boolean forceRefresh, Boolean applyAllowList, Boolean includeCustomProviders) in /src/azure-functions-host/src/WebJobs.Script/Host/FunctionMetadataManager.cs:line 83
   at Microsoft.Azure.WebJobs.Script.ScriptHost.GetFunctionsMetadata() in /src/azure-functions-host/src/WebJobs.Script/Host/ScriptHost.cs:line 346
   at Microsoft.Azure.WebJobs.Script.ScriptHost.InitializeAsync(CancellationToken cancellationToken) in /src/azure-functions-host/src/WebJobs.Script/Host/ScriptHost.cs:line 277
   at Microsoft.Azure.WebJobs.Script.ScriptHost.StartAsyncCore(CancellationToken cancellationToken) in /src/azure-functions-host/src/WebJobs.Script/Host/ScriptHost.cs:line 257
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation, Int32 attemptCount, JobHostStartupMode startupMode) in /src/azure-functions-host/src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs:line 278
info: Microsoft.Azure.WebJobs.Hosting.JobHostService[0]
      Stopping JobHost

I think this has to do with lacking write access to /home/site/wwwroot but not sure how to resolve this...

Below is my dockerfile

# To enable ssh & remote debugging on app service change the base image to the one below
# FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8-appservice
FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
    AzureFunctionsJobHost__Logging__Console__IsEnabled=true

COPY requirements.txt /

# Install pip dependencies and remove them after the pip install
RUN apt-get update \
    && apt-get install -y --no-install-recommends gcc \
    && rm -rf /var/lib/apt/lists/* \
    && pip install -r /requirements.txt \
    && apt-get purge -y --auto-remove gcc

# Adding company PKI Certificates to System Trusted Store
COPY Certificates/der/*.crt /build/pki/
COPY Certificates/pem/*.crt /build/pki/pem/
RUN cp /build/pki/pem/*.crt /usr/local/share/ca-certificates/ && update-ca-certificates

# Microsoft.AspNetCore.Server.Kestrel issue
EXPOSE 8080
ENV ASPNETCORE_URLS=http://*:8080
ENV DOTNET_CLI_TELEMETRY_OPTOUT=1
ENV COMPlus_EnableDiagnostics=0

COPY . /home/site/wwwroot

Any guide would be appreciated!

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,909 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.