How to access fine-grained log data from VSCode?

Question

I'm new to Azure Monitor and I'm trying to determine if it is suitable for my project. I would like to be able to access fine-grained log data from VSCode. I'm an education researcher, so I'm interested in how programmers work rather than application performance or debugging.

I need to be able to see when a VSCode user saves, compiles, and runs their code. Ideally, I'd also like to be able to see keystrokes, when text is highlighted/cut/pasted, and scrolling in a file. I know this data can be sensitive, and I will only be collecting this data from people who have know exactly what data is being collected, agreed to participate in a research study, and signed consent forms.

My hope was that I would be able to turn on some sort of verbose log setting and collect this data locally. This way, we could ask study participants to export a file containing telemetry events from that particular work session. If this isn't possible, I would also consider using Application Insights or another Azure solution to collect data remotely.

So far, I've tested changing the log settings in VSCode to 'trace'. However, it doesn't seem like events like runtime errors or specific keystrokes are being displayed in the output. I've also looked into using the vscode-telemetry-extension, but again I'm not sure if it is capturing things like runtime errors or specific keystrokes. I'd be interested to hear from someone more familiar with the Azure ecosystem how they might approach collecting this kind of fine-grained data. Thanks!

Answer 1

Ayse, thank you for posting this question.

Without any reference to VS code and its capability, please find below some information about monitoring solutions in Azure that might help you:

1. Application Insights - This is specifically used for an application that you are developing and is an Application Performance Management solution. Using this, you can collect basic telemetry events which comprises events, metrics and trace. For details, see Application Insights telemetry data model. To be able to track specific events of interest (as mentioned in the question), custom tracking methods will have to be used in the application itself (here - VS Code), given that such action tracking is available and such traces will have to be sent to Application Insights workspace for future reference. This is not going to work in the current scenario, as you are not developing the application of interest here - VS Code.
2. Another option available, which might work in this scenario, would involve saving the traces/events in a log file to a specific directory on the VM where application is running (in this case VS Code). Azure Monitor Agent can be used to collect custom text-based log files in this case. For details, see Collect text logs with Azure Monitor Agent.

---

With the above information at hand, there are some sets of telemetry logs available as generated by VS Code, which are used to help understand how a user interacts with the product. They are mainly aimed at improving VS Code and fixing any bugs that it may have. However, there are only very limited sets of information generated and collected which helps in feature prioritization and bug fixing. You can review the details of telemetry generated and collected from the link here - Telemetry in VS Code

This of course does not include tracking of keystrokes, mouse movements etc., but is limited to running specific commands or interaction in VS Code. For the complete set of telemetry which is being generated at the highest level of trace level (=trace), you can follow the steps as mentioned here - Output channel for telemetry events However, after doing some basic operations in VS code after enabling traces you would notice that the texts/code/file names etc. are not something that gets collected. Note that collection of these information also would pose security and compliance issues as it contains sensitive Intellectual properties, private/secure information etc.

There are no default ways to get this done, however you may choose to develop VS Code extension which can help you with this requirement (in this case, you would be able to use the Azure Monitor capability described in the first half of this answer). However, any such application/extension can pose serious securtity and compliance concersn, therefore it will become extremely importnant to review the implications.

Hope this helps.