Unable to configure Azure Monitor OpenTelemetry-based auto-instrumentation for Java application running on Azure VM

Tejas Shah 21 Reputation points
2022-03-23T20:36:02.04+00:00

I am trying to configure Azure Monitor OpenTelemetry-based auto-instrumentation for Java application running on Azure VM. I have my application running on JBoss on one server and Pivotal Tc Server on another server. I tried to configure Azure monitor on both servers using the article below but not able to see anything on the Application Insights resources after configuring. My Java application is running on Azure vm but it's not running on Azure App Service. Based on the article below, I think I should be able to do a codeless monitor my application in Application Insights resource after configuring but I am not able to generate the applicationinsights.log in the same folder where I have placed applicationinsights-agent-3.2.8.jar and applicationinsights.json. Any help is appreciated.

https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-in-process-agent#autocollected-requests-dependencies-logs-and-metrics

OS - Windows 2019 Datacenter
Java Application running on JBoss or Pivotal Tc Server.
JVM configured for JBoss domain or using tomcat setup for TcServer - https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-arguments

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,867 questions
{count} votes

Accepted answer
  1. AnuragSingh-MSFT 20,676 Reputation points
    2022-03-29T12:02:54.093+00:00

    @TejasShah-0975, thank you for the detailed response above. It helped me a lot to configure a test environment on an Azure Windows VM, to test with both servers and here are my findings/recommendations (I am summarizing the findings and step so that it would help others too looking for help on this topic):

    For Azure Monitor OpenTelemetry-based auto-instrumentation for Java applications, the basic requirements are as below:

    1. Point the JVM to the jar file

    2. Set the Application Insights connection string

    One of the most effective ways to troubleshoot issues related to it is to review the applicationinsights.log file, which is created in the applicationinsights**.jar directory. However, in current scenario, the .log file itself was not getting generated. This points to the fact that either the path mentioned is incorrect for the ApplicationInsights**.jar file OR that the supplied -javaagent parameter for JVM did not take into effect.

    ---
    Here are my recommendations/findings for the 2 servers:

    1. JBOSS EAP 7.3 on Windows VM

    There are two operating modes for JBOSS servers - Standalone Server and Managed domain. I observed that you were modifying the host.xml file with -javaagent parameter, but it looks like you are using JBOSS in standalone mode. Therefore, please update the JBOSS_HOME\bin\standalone.conf.bat file to update the JAVA_OPTS environment variable, to also include the -javaagent parameter.

    187961-image.png

    You may use Process Explorer to review the environment variables set for any process. Once the changes above were made, I noticed the following:

    187952-image.png

    After verifying the above changes, I was able to get the applicationInsights.log file generated and verified that the data is getting ingested in the AppInsights workspace.

    ---
    2. Pivotal TC Server

    After going through the documentation and a couple of other reference materials, I came across this link - https://community.pivotal.io/s/article/Customizing-the-environment-or-JVM-options-for-vFabric-tc-Server?language=en_US

    It states that - "When running tc Server as a service on Windows, JVM settings are specified in the conf/wrapper.conf file.".

    The only change you need to make is to add wrapper.java.additional.X. with value as -javaagent... parameter, as shown below:

    187943-image.png

    After making this change and restarting the tc server instance, the required JVM parameter for -javaagent was supplied to process as verified using Process Explorer.

    187760-image.png

    Please note, -javaagent is used for instrumentation/profiling and the appinsights jar file reference must be provided using this switch only. The way this switch is provided to JVM will depend on the Server technology being used. For more information, you may reach out to the respective Server support team about the configuration details.

    If you have any questions specific to ApplicationInsights, please let me know.

    ---
    Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.


1 additional answer

Sort by: Most helpful
  1. Tejas Shah 21 Reputation points
    2022-03-28T20:37:58.45+00:00

    @AnuragSingh-MSFT , yes both steps are performed. Any chance we can connect to help troubleshoot this.

    Here are configurations for two different app servers we use. In neither of the application server configuration, applicationinsights.log is generating right now.

    Article used - https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-arguments

    Application Server #1 JBoss - domain - Reference - Jboss EAP 7.0 - Domain server configuration (https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-arguments ).

    • Point the JVM to the jar file.
      File C:\JBoss\jboss-eap-7.3\domain\configuration\host.xml
      <!--Add Java agent jar file here, tried both relative and absolute path for the jar file. Also placed the applicationinsight.json in the same folder as the jar file-->
      <!--option value="-javaagent:JBOSS_HOME\applicationinsights-agent-3.2.8.jar"/-->
      <option value="-javaagent:C:\JBoss\jboss-eap-7.3\applicationinsights-agent-3.2.8.jar"/> 2. Set the Application Insights connection string. ******applicationinsights.json
      {
      "connectionString": "InstrumentationKey=xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx;IngestionEndpoint=https://centralus-2.in.applicationinsights.azure.com/",
      "role": {
      "name": "QA-test-05"
      },
      "sampling": {
      "percentage": 100
      },
      "jmxMetrics": [
      ],
      "customDimensions": {
      },
      "instrumentation": {
      "logging": {
      "level": "INFO"
      },
      "micrometer": {
      "enabled": true
      }
      },
      "proxy": {
      },
      "preview": {
      "processors": [
      ]
      },
      "selfDiagnostics": {
      "destination": "file+console",
      "level": "INFO",
      "file": {
      "path": "C:\JBoss\jboss-eap-7.3\applicationinsights.log",
      "maxSizeMb": 5,
      "maxHistory": 1
      }
      }
      }

    Application Server #2 Pivotal Tcserver Reference used - https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-arguments
    https://tcserver.docs.pivotal.io/3x/docs-tcserver/topics/postinstall-getting-started.html#postinstall-startstop-tcserver-windows

    1. Point the JVM to the jar file. location of the file -

    File Java Class path - C:\Shaw\domain\Spectrum\conf\wrapper.conf
    wrapper.java.classpath.5=%CATALINA_BASE%\bin\winx86_64\applicationinsights-agent-3.2.8.jar
    File

    • C:\Shaw\domain\Spectrum\bin\setenv.bat
      set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-11.0.13_8
      set AGENT_PATHS=
      set "JAVA_AGENTS=%CATALINA_BASE%\bin\winx86_64\applicationinsights-agent-3.2.8.jar"
      set JAVA_LIBRARY_PATH=
      set JVM_OPTS=-Xmx512M -Xss256M
      set JDK_JAVA_OPTIONS="--add-exports=java.rmi/sun.rmi.registry=ALL-UNNAMED"
      set JAVA_OPTS=%JVM_OPTS% %AGENT_PATHS% %JAVA_AGENTS% %JAVA_LIBRARY_PATH%
      set "CATALINA_OPTS=%CATALINA_OPTS% -javaagent:%CATALINA_BASE%\bin\winx86_64\applicationinsights-agent-3.2.8.jar"
      File
    • C:\pivotal-tc-server-standard-4.1.10.RELEASE\pivotal-tc-server\runtimes\tomcat-9.0.52.A.RELEASE\bin\catalina.bat
      rem Add on applicationinsights.jar file to CLASSPATH
      rem set "CLASSPATH=%CATALINA_HOME%\bin\applicationinsights-agent-3.2.8.jar"
      set "CLASSPATH=%CATALINA_BASE%\bin\winx86_64\applicationinsights-agent-3.2.8.jar"
      :applicationinsights-agent ClassPathDone
      File
    • C:\pivotal-tc-server-standard-4.1.10.RELEASE\pivotal-tc-server\runtimes\tomcat-9.0.52.A.RELEASE\bin\tool-wrapper.bat
      set "CLASSPATH=%CLASSPATH%%CATALINA_HOME%\bin\bootstrap.jar;%CATALINA_HOME%\bin\tomcat-juli.jar;%CATALINA_HOME%\lib\servlet-api.jar;%CATALINA_HOME%\lib\tomcat-util.jar;%CATALINA_BASE%\bin\winx86_64\applicationinsights-agent-3.2.8.jar"
    1. Set the Application Insights Connection string. File - C:\Shaw\domain\Spectrum\bin\winx86_64\applicationinsights.json {
      "connectionString": "InstrumentationKey=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx;IngestionEndpoint=https://centralus-2.in.applicationinsights.azure.com/",
      "role": {
      "name": "sample appspectrum test app"
      },
      "sampling": {
      "percentage": 100
      },
      "jmxMetrics": [
      ],
      "customDimensions": {
      },
      "instrumentation": {
      "logging": {
      "level": "TRACE"
      },
      }
      "selfDiagnostics": {
      "destination": "file+console",
      "level": "INFO",
      "file": {
      "path": "applicationinsights.log",
      "maxSizeMb": 5,
      "maxHistory": 1
      }
      }
      }
    0 comments No comments