dubfi.linalg.segments

Segments for distributing vector space to worker processes.

Added in version 0.1.0: (initial release)

Classes

AbstractSegment

Generic segments defining how a vector space is distributed to worker processes.

ContiguousSegment

Segment of contiguous data in the full vector space.

IndexedSegment

Index-based segment selecting parts of the full vector space.

Segments

Sequence of segments that shall jointly provide complete coverage of the full vector space.

Module Contents

class dubfi.linalg.segments.AbstractSegment

Bases: abc.ABC

Generic segments defining how a vector space is distributed to worker processes.

Segments define how a data is distributed to worker processes.

Consider data sharing a long dimension y and operations that are (approximately) local on y. Then y can be split and the data can be distributed to multiple processes, each containing one segment of y and some buffer. AbstractSegment contains the information required for selecting buffered and unbuffered data for one process.

property buffered_size: int
Abstractmethod:

Return type:

int

Segment size including buffer.

property seg_size: int

Size of data excluding buffer.

Return type:

int

abstractmethod selbuffered_impl(data, axes)

Given full data (parent), return the buffered segment (child).

This method should be re-implemented by inheriting classes.

Parameters:

  • data (numpy.ndarray)

  • axes (set[int])

Return type:

numpy.ndarray

selbuffered(data, axes=(-1,))

Given full data (parent), return the buffered segment (child).

Parameters:

data (numpy.ndarray)

Return type:

numpy.ndarray

abstractmethod selunbuffered_impl(data, axes)

Given full data (parent), return the unbuffered segment (child).

This method should be re-implemented by inheriting classes.

Parameters:

  • data (numpy.ndarray)

  • axes (set[int])

Return type:

numpy.ndarray

selunbuffered(data, axes=(-1,))

Given full data (parent), return the unbuffered segment (child).

Parameters:

data (numpy.ndarray)

Return type:

numpy.ndarray

abstractmethod into(target, source, axes=(-1,))

Write segment from source (child) into target (parent).

Parameters:

  • target (numpy.ndarray)

  • source (numpy.ndarray)

Return type:

None

abstractmethod addto(target, source, axes=(-1,))

Add segment from source (child) to target (parent).

Parameters:

  • target (numpy.ndarray)

  • source (numpy.ndarray)

Return type:

None

class dubfi.linalg.segments.ContiguousSegment(buffer_start, data_start, data_end, buffer_end, parent_size)

Bases: AbstractSegment

Segment of contiguous data in the full vector space.

Segment of contiguous data.

In this segment, buffered data form a contiguous slice of the parent array. The unbuffered data are a contiguous slice of the buffered data.

Parameters:

  • buffer_start (int)

  • data_start (int)

  • data_end (int)

  • buffer_end (int)

  • parent_size (int)

property buffered_size: int

Segment size including buffer.

Return type:

int

selbuffered_impl(data, axes)

Given full data (parent), return the buffered segment (child).

selunbuffered_impl(data, axes)

Given full data (parent), return the unbuffered segment (child).

into(target, source, axes=(-1,))

Write segment from source (child) into target (parent).

addto(target, source, axes=(-1,))

Add segment from source (child) to target (parent).

class dubfi.linalg.segments.IndexedSegment(buffer_idcs, start_idx, end_idx, parent_size)

Bases: AbstractSegment

Index-based segment selecting parts of the full vector space.

Segment of arbitrary indexed data.

This provides a generic segment of non-contiguous data. The buffered data are defined by an array of indices of the parent data. Unbuffered data are defined by a contiguous slice of these indices. This allows for arbitrary selection of buffered and unbuffered data.

Parameters:

  • buffer_idcs (numpy.ndarray)

  • start_idx (int)

  • end_idx (int)

  • parent_size (int)

property buffered_size: int

Segment size including buffer.

Return type:

int

selbuffered_impl(data, axes)

Given full data (parent), return the buffered segment (child).

selunbuffered_impl(data, axes)

Given full data (parent), return the unbuffered segment (child).

into(target, source, axes=(-1,))

Write segment from source (child) into target (parent).

addto(target, source, axes=(-1,))

Add segment from source (child) to target (parent).

class dubfi.linalg.segments.Segments(size)

Sequence of segments that shall jointly provide complete coverage of the full vector space.

Sequence of AbstractSegments.

Parameters:

size (int)

classmethod fromlist(segments)

Create from a list of AbstractSegments.

Assert that all segments are compatible and jointly complete.

Parameters:

segments (Sequence[AbstractSegment])

add(seg)

Add segment to segment list after checking that it is compatible.

Parameters:

seg (AbstractSegment)

property number: int

Number of segments in self.

Return type:

int

__getitem__(idx)

Get segment by index.

Parameters:

idx (int)

Return type:

AbstractSegment

__iter__()

Iterate over segments.

Return type:

Iterator[AbstractSegment]

is_complete()

Check whether segments are complete.

Return type:

bool