How to query Databricks audit logs?

Question

How to query Databricks audit logs?

Mohammad Saber 591

Hi,

I would like to ask where the Databricks Audit Log files are stored on the DBFS.

And is there any way that I can query log files?

Thanks.

PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-02-16T13:38:25.9266667+00:00

Hello @Mohammad Saber

Glad to know that the below response was helpful. Thank you for marking this answer as accepted. This will help others find useful answers faster.

Please feel free to take a survey on the relevant answer. You can help us improve by leaving feedback verbatim.

Have a good day!

Regards,
PRADEEPCHEEKATLA-MSFT

Answer accepted by question author

1 additional answer

Your answer

PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-02-16T13:38:25.9266667+00:00

Hello @Mohammad Saber

Glad to know that the below response was helpful. Thank you for marking this answer as accepted. This will help others find useful answers faster.

Please feel free to take a survey on the relevant answer. You can help us improve by leaving feedback verbatim.

Have a good day!

Regards,
PRADEEPCHEEKATLA-MSFT

Answer 1

Umang Middha 555

Hello Mohammad,

Hope you are doing well today. To query Databricks audit logs, you can use the Databricks REST API, which allows you to programmatically retrieve audit log events.

Here are the general steps to follow:

Generate an access token for your Databricks workspace. You can do this from the Databricks UI by going to the "User Settings" page, selecting the "Access Tokens" tab, and clicking "Generate New Token". Make sure to save the token somewhere safe, as it will be used to authenticate your API requests.
Use the Databricks API to query the audit logs. The endpoint for querying audit logs is /api/2.0/audit/get. You can specify a range of time to query by setting the start_time and end_time parameters in ISO 8601 format. You can also filter by user, event type, resource type, and other parameters. For example, to get all audit events for a particular user, you can set the user_name parameter to the user's email address.
Analyze the results of the API query. The audit log events will be returned in JSON format. You can use a tool like Jupyter Notebook or Apache Spark to load the JSON data and analyze it using Python or SQL.

Regards,

Umang Middha

******@gmail.com

Mohammad Saber 591 Reputation points

2023-02-15T21:51:39.6933333+00:00

Many thanks for your reply.

Is there any way to get Logs as a Databricks table?

I mean, saving the Logs as a table (let's say delta table).

Also, I want it to be continuously working, saving the logs in a table when a new event happens (not just one time).

I am not sure if I can use SQL language for this purpose or not (instead of Rest API).
Mohammad Saber 591 Reputation points

2023-02-15T21:52:00.09+00:00

Many thanks for your reply.

Is there any way to get Logs as a Databricks table?

I mean, saving the Logs as a table (let's say delta table).

Also, I want it to be continuously working, saving the logs in a table when a new event happens (not just one time).

I am not sure if I can use SQL language for this purpose or not (instead of Rest API).
PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-02-16T13:36:30.4066667+00:00

Hello @Mohammad Saber

You question is answered here: https://learn.microsoft.com/en-us/answers/questions/1181136/databricks-audit-logs-where-the-log-files-are-stor

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Maddipatla Madhura 1

Hi @Umang Middha Can you please share official documentation link to support this

Use the Databricks API to query the audit logs. The endpoint for querying audit logs is /api/2.0/audit/get.

Share via

How to query Databricks audit logs?

1 additional answer

Your answer