Struct object_store::buffered::BufReader

pub struct BufReader { /* private fields */ }

An async-buffered reader compatible with the tokio IO traits

Internally this maintains a buffer of the requested size, and uses ObjectStore::get_range to populate its internal buffer once depleted. This buffer is cleared on seek.

Whilst simple, this interface will typically be outperformed by the native ObjectStore methods that better map to the network APIs. This is because most object stores have very high first-byte latencies, on the order of 100-200ms, and so avoiding unnecessary round-trips is critical to throughput.

Systems looking to sequentially scan a file should instead consider using ObjectStore::get, or ObjectStore::get_opts, or ObjectStore::get_range to read a particular range.

Systems looking to read multiple ranges of a file should instead consider using ObjectStore::get_ranges, which will optimise the vectored IO.

Implementations

impl BufReader

pub fn new(store: Arc<dyn ObjectStore>, meta: &ObjectMeta) -> Self

Create a new BufReader from the provided ObjectMeta and ObjectStore

pub fn with_capacity( store: Arc<dyn ObjectStore>, meta: &ObjectMeta, capacity: usize, ) -> Self

Create a new BufReader from the provided ObjectMeta, ObjectStore, and capacity

Trait Implementations

impl AsyncBufRead for BufReader

fn poll_fill_buf( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Result<&[u8]>>

Attempts to return the contents of the internal buffer, filling it with more data from the inner reader if it is empty.

fn consume(self: Pin<&mut Self>, amt: usize)

Tells this buffer that amt bytes have been consumed from the buffer, so they should no longer be returned in calls to poll_read.

impl AsyncRead for BufReader

fn poll_read( self: Pin<&mut Self>, cx: &mut Context<'_>, out: &mut ReadBuf<'_>, ) -> Poll<Result<()>>

Attempts to read from the AsyncRead into buf.

impl AsyncSeek for BufReader

fn start_seek(self: Pin<&mut Self>, position: SeekFrom) -> Result<()>

Attempts to seek to an offset, in bytes, in a stream.

fn poll_complete( self: Pin<&mut Self>, _cx: &mut Context<'_>, ) -> Poll<Result<u64>>

Waits for a seek operation to complete.

impl Debug for BufReader

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.