AutoCorrelation#

class tfts.layers.autoformer_layer.AutoCorrelation(*args, **kwargs)[source]#

Bases: Layer

Self-Attention layer that computes time-delayed autocorrelation between queries and keys.

This layer implements a novel attention mechanism that uses Fast Fourier Transform (FFT) to compute autocorrelation between queries and keys in the frequency domain, which captures temporal dependencies more efficiently than traditional attention.

Parameters:

  • d_model (int) – The dimension of the model’s hidden states.

  • num_attention_heads (int) – Number of attention heads to use.

  • attention_probs_dropout_prob (float, optional) – Dropout probability for attention probabilities, by default 0.0.

Inherited-members:

Methods

add_loss(loss)

Can be called inside of the call() method to add a scalar loss.

add_metric(*args, **kwargs)

add_variable(shape, initializer[, dtype, ...])

Add a weight variable to the layer.

add_weight([shape, initializer, dtype, ...])

Add a weight variable to the layer.

build(input_shape)

Build the layer, creating the trainable weights.

build_from_config(config)

Builds the layer's states with the supplied config dict.

call(q, k, v[, dynamic])

Process inputs through the autocorrelation mechanism.

compute_mask(inputs, previous_mask)

compute_output_shape(*args, **kwargs)

compute_output_spec(inputs_spec)

Compute the output tensor spec from the input spec.

count_params()

Count the total number of scalars composing the weights.

from_config(config)

Creates an operation from its config.

get_build_config()

Returns a dictionary with the layer's input shape.

get_config()

Get the configuration of the layer.

get_weights()

Return the values of layer.weights as a list of NumPy arrays.

load_own_variables(store)

Loads the state of the layer.

quantize(mode[, type_check])

quantized_build(input_shape, mode)

quantized_call(*args, **kwargs)

rematerialized_call(layer_call, *args, **kwargs)

Enable rematerialization dynamically for layer's call method.

save_own_variables(store)

Saves the state of the layer.

set_weights(weights)

Sets the values of layer.weights from a list of NumPy arrays.

split_heads(x, batch_size)

Split the last dimension into (num_heads, depth).

stateless_call(trainable_variables, ...[, ...])

Call the layer without any side effects.

symbolic_call(*args, **kwargs)

time_delay_agg(q, k, v)

Compute time-delayed autocorrelation between queries and keys.

Attributes

compute_dtype

The dtype of the computations performed by the layer.

dtype

Alias of layer.variable_dtype.

dtype_policy

input

Retrieves the input tensor(s) of a symbolic operation.

input_dtype

The dtype layer inputs should be converted to.

input_spec

losses

List of scalar losses from add_loss, regularizers and sublayers.

metrics

List of all metrics.

metrics_variables

List of all metric variables.

non_trainable_variables

List of all non-trainable layer state.

non_trainable_weights

List of all non-trainable weight variables of the layer.

output

Retrieves the output tensor(s) of a layer.

path

The path of the layer.

quantization_mode

The quantization mode of this layer, None if not quantized.

supports_masking

Whether this layer supports computing a mask using compute_mask.

trainable

Settable boolean, whether this layer should be trainable or not.

trainable_variables

List of all trainable layer state.

trainable_weights

List of all trainable weight variables of the layer.

variable_dtype

The dtype of the state (weights) of the layer.

variables

List of all layer state, including random seeds.

weights

List of all weight variables of the layer.
build(input_shape: Tuple[int | None, ...])[source]#

Build the layer, creating the trainable weights.

Parameters:

input_shape (Tuple[Optional[int], ...]) – The shape of the input tensor.

call(q, k, v, dynamic=True)[source]#

Process inputs through the autocorrelation mechanism.

Parameters:

  • q (Tensor) – Query tensor of shape (batch_size, timesteps, d_model).

  • k (Tensor) – Key tensor of shape (batch_size, timesteps, d_model).

  • v (Tensor) – Value tensor of shape (batch_size, timesteps, d_model).

  • dynamic (bool, optional) – Not used in the current implementation, by default True.

Returns:

Output tensor of shape (batch_size, timesteps, d_model).

Return type:

Tensor

compute_output_spec(inputs_spec)[source]#

Compute the output tensor spec from the input spec.

This is needed for TensorFlow 2.x keras model API.

Parameters:

inputs_spec (tf.TensorSpec) – Input tensor specification.

Returns:

Output tensor specification.

Return type:

tf.TensorSpec

get_config()[source]#

Get the configuration of the layer.

Returns:

Configuration dictionary.

Return type:

dict

split_heads(x, batch_size)[source]#

Split the last dimension into (num_heads, depth).

Parameters:

  • x (Tensor) – Input tensor to split.

  • batch_size (int) – Batch size.

Returns:

Reshaped tensor with shape (batch_size, num_attention_heads, timesteps, depth)

Return type:

Tensor

time_delay_agg(q, k, v)[source]#

Compute time-delayed autocorrelation between queries and keys.

Parameters:

  • q (Tensor of shape (batch_size, num_attention_heads, time_steps, hidden_size)) – Queries.

  • k (Tensor of shape (batch_size, num_attention_heads, time_steps, hidden_size)) – Keys.

  • v (Tensor of shape (batch_size, num_attention_heads, time_steps, hidden_size)) – Values.

Returns:

Time-delayed autocorrelation between queries and keys.

Return type:

Tensor of shape (batch_size, num_attention_heads, hidden_size, time_steps)