Skip to content

Types

laktory.enums.DataFrameBackends ¤

Bases: str, Enum

METHOD DESCRIPTION
from_df

Instantiate DataFrameBackends object from a DataFrame.
from_nw_implementation

Instantiate DataFrameBackends object from a Narwhals implementation.
ATTRIBUTE DESCRIPTION
POLARS

Polars backend.
PYSPARK

PySpark backend.

POLARS = auto() class-attribute instance-attribute ¤

Polars backend.

PYSPARK = auto() class-attribute instance-attribute ¤

PySpark backend.

from_df(df) classmethod ¤

Instantiate DataFrameBackends object from a DataFrame.

PARAMETER DESCRIPTION
df

DataFrame

TYPE: Any

RETURNS DESCRIPTION
output

DataFrameBackend

TYPE: DataFrameBackends

Source code in laktory/enums.py 
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
@classmethod
def from_df(cls, df: Any) -> "DataFrameBackends":  # pragma: no cover
    """
    Instantiate DataFrameBackends object from a DataFrame.

    Parameters
    ----------
    df:
        DataFrame

    Returns
    -------
    output:
        DataFrameBackend
    """

    if not isinstance(df, (nw.DataFrame, nw.LazyFrame)):
        df = nw.from_native(df)

    return cls.from_nw_implementation(df.implementation)

from_nw_implementation(implementation) classmethod ¤

Instantiate DataFrameBackends object from a Narwhals implementation.

PARAMETER DESCRIPTION
implementation

Narwhals implementation

TYPE: Implementation

RETURNS DESCRIPTION
output

DataFrameBackend

TYPE: DataFrameBackends

Source code in laktory/enums.py 
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
@classmethod
def from_nw_implementation(
    cls, implementation: nw.Implementation
) -> "DataFrameBackends":  # pragma: no cover
    """
    Instantiate DataFrameBackends object from a Narwhals implementation.

    Parameters
    ----------
    implementation:
        Narwhals implementation

    Returns
    -------
    output:
        DataFrameBackend
    """
    mapping = {
        nw.Implementation.PYSPARK: DataFrameBackends.PYSPARK,
        nw.Implementation.PYSPARK_CONNECT: DataFrameBackends.PYSPARK,
        nw.Implementation.POLARS: DataFrameBackends.POLARS,
        # get_modin(): Implementation.MODIN,
        # get_cudf(): Implementation.CUDF,
        # get_pyarrow(): Implementation.PYARROW,
        # get_pyspark_sql(): Implementation.PYSPARK,
        # get_polars(): Implementation.POLARS,
        # get_dask_dataframe(): Implementation.DASK,
        # get_duckdb(): Implementation.DUCKDB,
        # get_ibis(): Implementation.IBIS,
        # get_sqlframe(): Implementation.SQLFRAME,
    }

    if implementation not in mapping:
        raise ValueError(f"Implementation {implementation} is not supported.")

    return mapping.get(implementation)

laktory.typing.VariableType ¤

Bases: str

A string placeholder that defers field resolution to inject_vars().

Any model field accepting VariableType in addition to its declared type, allows values to be specified as variable references or expressions in YAML configs and resolved at deploy time.

The string must start with ${ - arbitrary strings are not accepted. Two syntaxes are supported:

  • Simple substitution: ${vars.my_variable_name}
  • Python expression: ${{ 4 if vars.env == 'prod' else 1 }}

Examples:

num_workers: ${vars.workers}
size: ${{ 4 if vars.env == 'prod' else 1 }}
access_controls: ${vars.default_access_controls}
References

laktory.typing.AnyFrame = nw.LazyFrame | nw.DataFrame module-attribute ¤