AzureDLFile Class

Open ADL key as a file. Data is only loaded and cached on demand.

Inheritance
builtins.object
AzureDLFile

Constructor

AzureDLFile(azure, path, mode='rb', blocksize=33554432, delimiter=None)

Parameters

azure
<xref:<xref:azure.datalake.store.core.azure connection>>
Required
path
AzureDLPath
Required

location of file

mode
default value: rb
blocksize
int
default value: 33554432

Size of the write or read-ahead buffer. For writing(and appending, will be truncated to 4MB (2**22).

delimiter
bytes or None
default value: None

If specified and in write mode, each flush will send data terminating on this bytestring, potentially leaving some data in the buffer.

Examples


>>> adl = AzureDLFileSystem()  
>>> with adl.open('my-dir/my-file.txt', mode='rb') as f:  
...     f.read(10)  

Methods

close

Close file

If in write mode, causes flush of any unwritten data.

flush

Write buffered data to ADL.

Without delimiter: Uploads the current buffer.

With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block.

If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.

info

File information about this path

next
read

Return data from cache, or fetch pieces as necessary

read1

Return data from cache, or fetch pieces as necessary

readable

Return whether the AzureDLFile was opened for reading

readinto

Reads data into buffer b

readline

Read and return a line from the stream.

If length is specified, at most size bytes will be read.

readlines

Return all lines in a file as a list

seek

Set current file location

seekable

Return whether the AzureDLFile is seekable (only in read mode)

tell

Current file location

writable

Return whether the AzureDLFile was opened for writing

write

Write data to buffer.

Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

close

Close file

If in write mode, causes flush of any unwritten data.

close()

flush

Write buffered data to ADL.

Without delimiter: Uploads the current buffer.

With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block.

If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.

flush(syncFlag='METADATA', force=False)

Parameters

syncFlag
default value: METADATA
force
default value: False

info

File information about this path

info()

next

next()

read

Return data from cache, or fetch pieces as necessary

read(length=-1)

Parameters

length
int(<xref:-1>)
default value: -1

Number of bytes to read; if <0, all remaining bytes.

read1

Return data from cache, or fetch pieces as necessary

read1(length=-1)

Parameters

length
int(<xref:-1>)
default value: -1

Number of bytes to read; if <0, all remaining bytes.

readable

Return whether the AzureDLFile was opened for reading

readable()

readinto

Reads data into buffer b

readinto(b)

Parameters

b
bytearray
Required

Buffer to which bytes are read into

Return type

<xref:Returns> <xref:number> <xref:of> bytes <xref:read.>

readline

Read and return a line from the stream.

If length is specified, at most size bytes will be read.

readline(length=-1)

Parameters

length
default value: -1

readlines

Return all lines in a file as a list

readlines()

seek

Set current file location

seek(loc, whence=0)

Parameters

loc
int
Required

byte location

whence
default value: 0

from start of file, current location or end of file, resp.

seekable

Return whether the AzureDLFile is seekable (only in read mode)

seekable()

tell

Current file location

tell()

writable

Return whether the AzureDLFile was opened for writing

writable()

write

Write data to buffer.

Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

write(data)

Parameters

data
bytes
Required

Set of bytes to be written.