dubfi.linalg.segments¶

Segments for distributing vector space to worker processes.

Added in version 0.1.0: (initial release)

Classes¶

`AbstractSegment`	Generic segments defining how a vector space is distributed to worker processes.
`ContiguousSegment`	Segment of contiguous data in the full vector space.
`IndexedSegment`	Index-based segment selecting parts of the full vector space.
`Segments`	Sequence of segments that shall jointly provide complete coverage of the full vector space.

Module Contents¶

class dubfi.linalg.segments.AbstractSegment¶

Bases: abc.ABC

Generic segments defining how a vector space is distributed to worker processes.

Segments define how a data is distributed to worker processes.

Consider data sharing a long dimension y and operations that are (approximately) local on y. Then y can be split and the data can be distributed to multiple processes, each containing one segment of y and some buffer. AbstractSegment contains the information required for selecting buffered and unbuffered data for one process.

property buffered_size: int¶

Abstractmethod:

Segment size including buffer.

property seg_size: int¶: Size of data excluding buffer.

abstractmethod selbuffered_impl(data: numpy.ndarray, axes: set[int]) → numpy.ndarray¶

Given full data (parent), return the buffered segment (child).

This method should be re-implemented by inheriting classes.

selbuffered(data: numpy.ndarray, axes=(-1,)) → numpy.ndarray¶: Given full data (parent), return the buffered segment (child).

abstractmethod selunbuffered_impl(data: numpy.ndarray, axes: set[int]) → numpy.ndarray¶

Given full data (parent), return the unbuffered segment (child).

This method should be re-implemented by inheriting classes.

selunbuffered(data: numpy.ndarray, axes=(-1,)) → numpy.ndarray¶: Given full data (parent), return the unbuffered segment (child).

abstractmethod into(target: numpy.ndarray, source: numpy.ndarray, axes=(-1,)) → None¶: Write segment from source (child) into target (parent).

abstractmethod addto(target: numpy.ndarray, source: numpy.ndarray, axes=(-1,)) → None¶: Add segment from source (child) to target (parent).

class dubfi.linalg.segments.ContiguousSegment(buffer_start: int, data_start: int, data_end: int, buffer_end: int, parent_size: int)¶

Bases: AbstractSegment

Segment of contiguous data in the full vector space.

Segment of contiguous data.

In this segment, buffered data form a contiguous slice of the parent array. The unbuffered data are a contiguous slice of the buffered data.

property buffered_size: int¶: Segment size including buffer.

selbuffered_impl(data, axes)¶: Given full data (parent), return the buffered segment (child).

selunbuffered_impl(data, axes)¶: Given full data (parent), return the unbuffered segment (child).

into(target, source, axes=(-1,))¶: Write segment from source (child) into target (parent).

addto(target, source, axes=(-1,))¶: Add segment from source (child) to target (parent).

class dubfi.linalg.segments.IndexedSegment(buffer_idcs: numpy.ndarray, start_idx: int, end_idx: int, parent_size: int)¶

Bases: AbstractSegment

Index-based segment selecting parts of the full vector space.

Segment of arbitrary indexed data.

This provides a generic segment of non-contiguous data. The buffered data are defined by an array of indices of the parent data. Unbuffered data are defined by a contiguous slice of these indices. This allows for arbitrary selection of buffered and unbuffered data.

property buffered_size: int¶: Segment size including buffer.

selbuffered_impl(data, axes)¶: Given full data (parent), return the buffered segment (child).

selunbuffered_impl(data, axes)¶: Given full data (parent), return the unbuffered segment (child).

into(target, source, axes=(-1,))¶: Write segment from source (child) into target (parent).

addto(target, source, axes=(-1,))¶: Add segment from source (child) to target (parent).

class dubfi.linalg.segments.Segments(size: int)¶

Sequence of segments that shall jointly provide complete coverage of the full vector space.

Sequence of AbstractSegments.

classmethod fromlist(segments: Sequence[AbstractSegment])¶

Create from a list of AbstractSegments.

Assert that all segments are compatible and jointly complete.

add(seg: AbstractSegment)¶: Add segment to segment list after checking that it is compatible.

property number: int¶: Number of segments in self.

__getitem__(idx: int) → AbstractSegment¶: Get segment by index.

__iter__() → Iterator[AbstractSegment]¶: Iterate over segments.

is_complete() → bool¶: Check whether segments are complete.