single_feature_array_standard_scale

SingleFeatureArrayStandardScaleEstimator

SingleFeatureArrayStandardScaleEstimator(
    inputCol=None,
    outputCol=None,
    inputDtype=None,
    outputDtype=None,
    layerName=None,
    maskValue=None,
    sampleFraction=None,
)

Bases: BaseEstimator, SampleFractionParams, SingleInputSingleOutputParams, MaskValueParams

Single feature array standard scaler estimator for use in Spark pipelines. This estimator is used to calculate the mean and standard deviation of the input feature column when it is an array where all the elements represent the same feature. An example would be a sequence of trip durations or booking windows in a traveller's session. When fit is called it returns a StandardScaleTransformer which can be used to standardize/transform additional features, where the mean and standard deviation are calculated across all elements in all the arrays.

Initializes a SingleFeatureArrayStandardScaleEstimator estimator. Sets all parameters to given inputs.

Parameters:

Name	Type	Description	Default
inputCol	Optional[str]	Input column name to standardize.	None
outputCol	Optional[str]	Output column name.	None
inputDtype	Optional[str]	Input data type to cast input column to before transforming.	None
outputDtype	Optional[str]	Output data type to cast the output column to after transforming.	None
layerName	Optional[str]	Name of the layer. Used as the name of the tensorflow layer in the keras model. If not set, we use the uid of the Spark transformer.	None
sampleFraction	Optional[float]	Fraction of data to sample for statistics estimation (exclusive 0.0-1.0). Default None (no sampling).	None

Returns:

Type	Description
None	None - class instantiated.

Source code in src/kamae/spark/estimators/single_feature_array_standard_scale.py

@keyword_only
def __init__(
    self,
    inputCol: Optional[str] = None,
    outputCol: Optional[str] = None,
    inputDtype: Optional[str] = None,
    outputDtype: Optional[str] = None,
    layerName: Optional[str] = None,
    maskValue: Optional[float] = None,
    sampleFraction: Optional[float] = None,
) -> None:
    """
    Initializes a SingleFeatureArrayStandardScaleEstimator estimator.
    Sets all parameters to given inputs.

    :param inputCol: Input column name to standardize.
    :param outputCol: Output column name.
    :param inputDtype: Input data type to cast input column to before
    transforming.
    :param outputDtype: Output data type to cast the output column to after
    transforming.
    :param layerName: Name of the layer. Used as the name of the tensorflow layer
     in the keras model. If not set, we use the uid of the Spark transformer.
    :param sampleFraction: Fraction of data to sample for statistics
     estimation (exclusive 0.0-1.0). Default None (no sampling).
    :returns: None - class instantiated.
    """
    super().__init__()
    self._setDefault(maskValue=None, sampleFraction=None)
    kwargs = self._input_kwargs
    self.setParams(**kwargs)

compatible_dtypes property

compatible_dtypes

List of compatible data types for the layer. If the computation can be performed on any data type, return None.

Returns:

Type	Description
Optional[List[DataType]]	List of compatible data types for the layer.