Get Spark Livy Logs using Spark monitoring APIs

Retrieve Spark Livy Logs.

Permissions

The caller must have "read" permission on the item.

Required delegated scopes

Item.Read.All or Item.ReadWrite.All or one of the following three groups (according to the item which triggered the Spark application).

• Notebook.Read.All or Notebook.ReadWrite.All
• SparkJobDefinition.Read.All or SparkJobDefinition.ReadWrite.All
• Lakehouse.Read.All or Lakehouse.ReadWrite.All

Microsoft Entra supported identities

This API supports the Microsoft identities listed in this section.

Identity	Support
User	Yes
Service principal and Managed identities	Yes

Get Livy log metadata

Get metadata of Livy log.

Interface

Without attemptId

https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/notebooks|sparkJobDefinitions|lakehouses/{itemId}/livySessions/{livyId}/applications/none/logs?type=livy&meta=true

URI parameters

None

Responses

Name	Type	Description
200 OK	FileMeta	Request completed successfully

Examples

Sample request

GET https://api.fabric.microsoft.com/v1/workspaces/aaaabbbb-0000-cccc-1111-dddd2222eeee/notebooks/bbbbcccc-1111-dddd-2222-eeee3333ffff/livySessions/ccccdddd-2222-eeee-3333-ffff4444aaaa/application/none/logs?type=livy&meta=true

Status code: 200

{ 
   "fileName": "livy.log",
   "length": 34723, 
   "lastModified": "2025-03-05T12:11:17.000GMT", 
   "creationTime": "2025-03-05T12:10:42.000GMT", 
   "metaData": {} 
} 

Get Livy log content

Get file content of Livy log.

Interface

Without attemptId

https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/notebooks|sparkJobDefinitions|lakehouses/{itemId}/livySessions/{livyId}/applications/none/logs?type=livy

With optional parameters:

Without attemptId

https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/notebooks|sparkJobDefinitions|lakehouses/{itemId}/livySessions/{livyId}/applications/none/logs?type=livy&isDownload={isDownload}&isPartial={isPartial}&offset={offset}&size={size}

URI parameters

Name	In	Required	Type	Description
workspaceId	path	True	string uuid	The workspace ID.
itemId	path	True	string uuid	The item ID of the notebook or Spark job definition or Lakehouse.
livyId	path	True	string uuid	The Livy session ID.
appId	path	True	string	The Spark application ID, like application_1704417105000_0001.
isDownload	query	False	bool	True to download the log file as a stream. Default as false.
isPartial	query	False	bool	Only take effect when isDownload is true. True to download a part of file content according to the given offset and size. Default as false to download the whole file
offset, size	query	False	Long	The starting offset (in byte) and the size (in byte) to read the file content. Only take effect when isDownload = true and isPartial = true - For offset, the default value is 0, meaning reading from the beginning of the file - For size, the default value is 1M (1024*1024) bytes

Responses

Name	Type	Description
200 OK	FileMeta	Request completed successfully

Examples

Sample request

GET https://api.fabric.microsoft.com/v1/workspaces/aaaabbbb-0000-cccc-1111-dddd2222eeee/notebooks/bbbbcccc-1111-dddd-2222-eeee3333ffff/livySessions/ccccdddd-2222-eeee-3333-ffff4444aaaa/application/none/logs?type=livy

Definitions

FileMeta

Object

Metadata of a log file

Name	Type	Description
fileName	string	File name
length	long	The size of the file, in byte
lastModified	string	The last time when the file was modified
creationTime	string	The time when the file was created
metaData	map of string to string	Auxiliary data if there is any