PositionalEmbedding

class tfts.layers.position_layer.PositionalEmbedding(*args, **kwargs)

Bases: Layer

Positional embedding layer that adds positional information to input embeddings.

This layer implements the sinusoidal positional encoding as described in the paper “Attention Is All You Need” (Vaswani et al., 2017). It adds positional information to the input embeddings using sine and cosine functions of different frequencies.

Parameters:

• max_len (int, optional) – Maximum sequence length. Defaults to 5000.

• name (str, optional) – Layer name. Defaults to None.

Input shape:

• 3D tensor with shape (batch_size, sequence_length, embedding_dim)

Output shape:

• 3D tensor with shape (batch_size, sequence_length, embedding_dim)

Inherited-members:

Methods

add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric(*args, **kwargs)
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)	Build the layer by pre-computing the positional encodings.
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(x[, masking])	Applies positional encoding to the input tensor.
compute_mask(inputs, previous_mask)
compute_output_shape(input_shape)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Creates an operation from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Get layer configuration.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
load_own_variables(store)	Loads the state of the layer.
quantize(mode[, type_check])
quantized_build(input_shape, mode)
quantized_call(*args, **kwargs)
rematerialized_call(layer_call, *args, **kwargs)	Enable rematerialization dynamically for layer's call method.
save_own_variables(store)	Saves the state of the layer.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.

build(input_shape: Tuple[int | None, ...]) → None

Build the layer by pre-computing the positional encodings.

Parameters:

input_shape – Shape of the input tensor

call(x: Tensor, masking: bool = True) → Tensor

Applies positional encoding to the input tensor.

Parameters:

• x – Input tensor of shape (batch_size, sequence_length, embedding_dim)

• masking – If True, applies masking to the output tensor. Defaults to True.

Returns:

Output tensor of the same shape as the input tensor, after applying positional encoding.

get_config() → Dict[str, int | str]

Get layer configuration.

Returns:

Dictionary containing layer configuration.