Can't see my NGINX logs in Azure app services, works fine locally [supervsisord, Docker, Django, app-service]

Alexander Lindgren 31 Reputation points
2022-01-24T09:35:50.577+00:00

I have a Dockerized Django application, which I'm orchestrating with Supervisor, which is not optimal but needed when hosting on Azure app services as their multi-app support with docker-compose is still in preview mode (aka. beta).

According to best-practises I have configured each application within supervisord to emit the logs to STDOUT. It works fine when I create the Docker image locally, run it and check the docker logs. However, when I have deployed it to Azure app services and check the logs, my web-application (Gunicorn) is logging as expected, however, the logs from NGINX don't appear at all.

I have tried different configurations in my Dockerfile for linking the log files generated by NGINX (linking to both /dev/stdout and /dev/fd/1 for example) and I have also gone into the the nginx.conf config and trying to log out directly to /dev/stdout. But whatever I do it work fine locally, but on Azure the logs don't show any NGINX-logs. I've pasted relevant configuration files, where you can see the commented lines with the options I've tried with. Hope someone can help me figure this one out.

EDIT:
I've also tried logging the NGINX app to a log-file in the system, which also works fine locally, but not in Azure app-services. I tried deactivating the "user nginx" part in nginx.conf as I though it could have something to do with permissions, but that didn't help either.

EDIT 2:
I also tried creating the log files in my home-directory in the web-app at Azure, thinking it may had to do with not being able to create logs in other directories - again, it works locally, but the logs in Azure are empty.

Dockerfile

FROM python:3.8
ENV PYTHONUNBUFFERED 1

###################
# PACKAGE INSTALLS
###################

RUN apt-get update 
RUN apt-get install -y pgbouncer
RUN apt-get update && apt-get install -y supervisor
RUN apt-get install nano
RUN apt-get install -y git 
RUN apt-get install curl

# Supervisor-stdout for consolidating logs
RUN pip install git+https://github.com/coderanger/supervisor-stdout 


###################
# AZURE SSH SETUP
###################

# Install OpenSSH and set the password for root to "Docker!". In this example, "apk add" is the install instruction for an Alpine Linux-based image.
RUN apt-get install -y --no-install-recommends openssh-server \
     && echo "root:Docker!" | chpasswd 

# Copy the sshd_config file to the /etc/ssh/ directory
COPY ./bin/staging/sshd_config /etc/ssh/

# Copy and configure the ssh_setup file
RUN mkdir -p /tmp
COPY ./bin/staging/ssh_setup.sh /tmp
RUN chmod +x /tmp/ssh_setup.sh \
    && (sleep 1;/tmp/ssh_setup.sh 2>&1 > /dev/null)


##############
# NGINX SETUP
##############

ENV NGINX_VERSION 1.15.12-1~stretch
ENV NJS_VERSION   1.15.12.0.3.1-1~stretch
RUN set -x \
    && \
    NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62; \
    found=''; \
    for server in \
        hkp://keyserver.ubuntu.com:80 \
        hkp://p80.pool.sks-keyservers.net:80 \
        pgp.mit.edu \
    ; do \
        echo "Fetching GPG key $NGINX_GPGKEY from $server"; \
        apt-key adv --keyserver "$server" --keyserver-options timeout=10 --recv-keys "$NGINX_GPGKEY" && found=yes && break; \
    done; \
    test -z "$found" && echo >&2 "error: failed to fetch GPG key $NGINX_GPGKEY" && exit 1; \
    apt-get remove --purge --auto-remove -y gnupg1 && rm -rf /var/lib/apt/lists/* \
    && dpkgArch="$(dpkg --print-architecture)" \
    && nginxPackages=" \
        nginx=${NGINX_VERSION} \
        nginx-module-xslt=${NGINX_VERSION} \
        nginx-module-geoip=${NGINX_VERSION} \
        nginx-module-image-filter=${NGINX_VERSION} \
        nginx-module-njs=${NJS_VERSION} \
    " \
    && echo "deb https://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list.d/nginx.list \
    && apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests -y \
                        $nginxPackages \
                        gettext-base \
    && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list

COPY ./conf/nginx/staging.conf /etc/nginx/conf.d/default.conf
COPY ./conf/nginx/nginx.conf /etc/nginx/nginx.conf

# Linking logs to be able to print errors and logs to STDOUT
#RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log
RUN ln -sf /dev/fd/1 /var/log/nginx/access.log && ln -sf /dev/fd/2 /var/log/nginx/error.log

##########################
# DJANGO APPLICATION SETUP
##########################

# install app
RUN mkdir /var/app && chown www-data:www-data /var/app
WORKDIR /var/app
COPY ./requirements.txt /var/app/
RUN pip install -r requirements.txt
COPY . /var/app/

#############
# SUPERVISORD
#############

COPY ./bin/staging/supervisord_main.conf /etc/supervisor/conf.d/supervisord_main.conf
COPY ./bin/staging/prefix-log /usr/local/bin/prefix-log


##########
# VOLUMES
##########

VOLUME /var/logs

########
# PORTS
########

# Expose ports (Added from previous dockerfile)
EXPOSE 80 2222 

#########################
# SUPERCRONIC (CRON-TABS)
#########################

ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.12/supercronic-linux-amd64 \
    SUPERCRONIC=supercronic-linux-amd64 \
    SUPERCRONIC_SHA1SUM=048b95b48b708983effb2e5c935a1ef8483d9e3e

RUN curl -fsSLO "$SUPERCRONIC_URL" \
 && echo "${SUPERCRONIC_SHA1SUM}  ${SUPERCRONIC}" | sha1sum -c - \
 && chmod +x "$SUPERCRONIC" \
 && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
 && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic

#############
# PERMISSIONS
#############

RUN ["chmod", "+x", "/var/app/bin/staging/entrypoint_main.sh"]
RUN ["chmod", "+x", "/usr/local/bin/prefix-log"]

############
# ENTRYPOINT
############

ENTRYPOINT ["/var/app/bin/staging/entrypoint_main.sh"]

supervisord_main.conf

[supervisord]
logfile=/var/logs/supervisord.log   ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB               ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10                  ; # of main logfile backups; 0 means none, default 10
loglevel=info                       ; log level; default info; others: debug,warn,trace
pidfile=/var/logs/supervisord.pid
nodaemon=true                       ; Run interactivelly instead of deamonizing
# user=www-data

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[inet_http_server]
port = 127.0.0.1:9001

[supervisorctl]
serverurl = http://127.0.0.1:9001
#serverurl=unix:///var/run/supervisor.sock

[program:nginx]
#command=/usr/local/bin/prefix-log /usr/sbin/nginx -g "daemon off;"
command=/usr/sbin/nginx -g "daemon off;"
directory=./projectile/
autostart=true
autorestart=true
stdout_events_enabled=true
stderr_events_enabled=true
stdout_logfile = /dev/fd/1
stdout_logfile_maxbytes=0
stderr_logfile = /dev/fd/2
stderr_logfile_maxbytes=0

[program:ssh]
command=/usr/local/bin/prefix-log  /usr/sbin/sshd -D
stdout_events_enabled=true
stderr_events_enabled=true
stdout_logfile = /dev/fd/1
stdout_logfile_maxbytes=0
stderr_logfile = /dev/fd/2
stderr_logfile_maxbytes=0

[program:web]
user=www-data
command=/usr/local/bin/prefix-log gunicorn --bind 0.0.0.0:8000 projectile.wsgi:application # Run each app trough a SH script to prepend logs with the application name
#command=gunicorn --workers=%(ENV_WORKER_COUNT)s --bind 0.0.0.0:8000 myapp_project.wsgi:application
directory=./projectile/
autostart=true
autorestart=true
stdout_events_enabled=true
stderr_events_enabled=true
stdout_logfile = /dev/fd/1
stdout_logfile_maxbytes=0
stderr_logfile = /dev/fd/2
stderr_logfile_maxbytes=0

nginx.conf

user  nginx;
worker_processes  2; # Set to number of CPU cores, 2 cores under Azure plan P1v3

error_log  /var/log/nginx/error.log warn;
#error_log  /dev/stdout warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    #access_log  /dev/stdout main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

staging.conf

server {
    listen 80 default_server;

    error_log /dev/stdout info;
    access_log /dev/stdout;

    client_max_body_size 100M;

    location /static {
        root /var/app/ui/build;
    }

    location /site-static {
        root /var;
    }

    location /media {
        root /var;
    }

    location / {
        root /var/app/ui/build; # try react build directory first, if file doesn't exist, route requests to django app
        try_files $uri $uri/index.html $uri.html @app;
    }

    location @app {
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto "https"; # assumes https already terminated by the load balancer in front of us

        proxy_pass          http://127.0.0.1:8000;
        proxy_read_timeout  300;
        proxy_buffering    off;
    }

}
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
6,907 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Alexander Lindgren 31 Reputation points
    2022-01-25T12:54:13.703+00:00

    Solved. Had to disable the WEBSITES_PORT 8000 in the app, which made the app go straight to Gunicorn instead of going trough NGINX.

    0 comments No comments