cerebras.elf.init_dist_tensor.init_dist_tensor module

Cerebras Input Tensor Initializer.

Throughout this module we make use of the configuration map YAML. The YAML has the following fields. See the following example:

fabric_size: "{ FABRIC[10,11] }"
elf_idx: 13
elf_size: "{ PE[2,2] }"
array_file: <path to .npz file>
map: "{ X_TENSOR[i, j] -> [PE[x, y] -> x_local[ii, jj]] :
        0 <= i < 2 and
        0 <= j < 4 and
        0 <= x < 2 and
        y = j//2 and
        ii = i and
        jj = j%2 }"

where:

  • The above specifies a fabric of size 10 (width) by 11 (height).

  • The elf_idx of 13 says that the top-left corner of the ELF is at PE (1,3), as the PEs are counted in row-major manner.

  • The elf_size defines a kernel of size 2x3.

  • The array_file is a string path to a .npz file we want to load.

  • The map is ISL style map from tensor global indices to PEs and then to local indices.

  • The name "X_TENSOR" is important. This must be name of the symbol in the ELF file. So if we have a global tensor named “foo” we just replace “X_TENSOR” by “foo” in this map. It is important to have the numpy array in the .npz file as the same datatype as the global tensor in the ELF file to obtain consistently correct results.

cerebras.elf.init_dist_tensor.init_dist_tensor.InitTeardown(fab_w, fab_h, elf_info=None, result='teardown_tensor.elf', rpc_protocol: Optional[str] = 'http', rpc_hostname: Optional[str] = None, rpc_port: Optional[int] = None)

Creates teardown ELF.

If elf_info is given then makes for that specific set of PEs, else for the whole fabric.

Parameters

  • fab_w – Fabric width.

  • fab_h – Fabric height.

  • elf_info – The tuple(elf_w, elf_h, elf_idx), i.e., ELF parameters, or None.

  • result – The name of the resulting ELF file.

  • rpc_protocol (optional) – The protocol name as a string.

  • rpc_hostname (optional) – The hostname as a string.

  • rpc_port (optional) – The port number.

cerebras.elf.init_dist_tensor.init_dist_tensor.InitTensor(elf_file: Optional[str], map_config: Union[str, List[str]], result_elf: Optional[str] = None, symaddr: Union[None, int, List[int]] = None, local_size: Union[None, List[int], List[List[int]]] = None, rpc_protocol: Optional[str] = 'http', rpc_hostname: Optional[str] = None, rpc_port: Optional[int] = None) → List[List[Any]]

Initializes a list of distributed tensors.

Parameters

  • elf_file – The name of the ELF file with the symbol to be initialized.

  • map_config – The filename of the config YAML file, or a list of files.

  • result_elf – Name of the resulting ELF to be produced in the current working directory.

  • symaddr – Address of symbol if not using ELF, or a list of addresses.

  • local_size – The local size of the tensor, a list of int``s, needed to handle cases where the DTG-style map from global array to PE/``local_idx does not give PEs the same number of elements. Or a list of such sizes.

  • rpc_protocol (optional) – The protocol name as a string.

  • rpc_hostname (optional) – The hostname as a string.

  • rpc_port (optional) – The port number.

Returns

A list of per-PE parameter-lists. If multiple tensors or maps are given, then returns this for the last one in the list.

cerebras.elf.init_dist_tensor.init_dist_tensor.InitTensorBlock(elf_file_or_ctx: Union[str, Tuple[int, int, int, int, int, Union[int, List[int]]]], sym: Union[str, List[str]], np_arr: Union[numpy.ndarray, List[numpy.ndarray]], result_name: str, rpc_protocol: Optional[str] = 'http', rpc_hostname: Optional[str] = None, rpc_port: Optional[int] = None)

Initialize the tensor via automatic block mapping.

Parameters

  • elf_file_or_ctx – The filename of the ELF containing the global tensors, or a context that is a tuple(fab_w, fab_h, elf_w, elf_h, elf_idx, [sym_addr]). Here sym_addr is the int address (or a list).

  • sym – The symbol name of local tensor, or a list.

  • np_arr – The 2D numpy array to be block mapped, or a list.

  • result_name – The filename for the resulting ELF (default if absent).

  • rpc_protocol (optional) – The protocol name as a string.

  • rpc_hostname (optional) – The hostname as a string.

  • rpc_port (optional) – The port number.

cerebras.elf.init_dist_tensor.init_dist_tensor.MapTensor(elf_files: List[str], rects: List[Tuple[Tuple[int, int], Tuple[int, int]]], np_arrs: List[numpy.ndarray], maps: List[str], result_filename: str, local_sizes: Optional[List[List[int]]] = None, rpc_protocol: Optional[str] = 'http', rpc_hostname: Optional[str] = None, rpc_port: Optional[int] = None)

Map tensor(s) to a loadable ELF file.

Parameters

  • elf_files – A list of ELF files from which to find each symbol.

  • rects – A list of rectangles (pair of start coordinates and (width,height) pairs) for each tensor.

  • np_arrs – A list of numpy arrays to map to the symbols.

  • maps – A list of ISL maps as strings containing the mapping from numpy tensor to PE-local array.

  • result_filename – The filename of the resultant ELF.

  • local_sizes (optional) – A list of PE-local array sizes.

  • rpc_protocol (optional) – The protocol name as a string.

  • rpc_hostname (optional) – The hostname as a string.

  • rpc_port (optional) – The port number.

cerebras.elf.init_dist_tensor.init_dist_tensor.main()

Main method for Cerebras Input Tensor Initializer.

cerebras.elf.init_dist_tensor package cerebras.elf.cself module