pat2vec.util.post_processing_dataframe

Functions

aggregate_dataframe_mean(df[, group_by_column])

Aggregates a DataFrame by taking the mean of numeric columns.

collapse_df_to_mean(df[, output_filename, ...])

Collapses a DataFrame to means and saves/merges with an output file.

convert_true_to_float(df[, columns])

Converts 'True' strings to 1.0 and ensures columns are float.

drop_columns_with_all_nan(df)

Drops columns where all values are NaN.

extract_datetime_from_binary_columns(df)

Extracts datetime values from binary columns representing dates.

extract_datetime_from_binary_columns_chunk_reader(...)

Reads a CSV in chunks and extracts datetime from binary columns.

extract_datetime_to_column(df[, drop])

Extracts datetime information from binary columns and creates a new column.

impute_dataframe(df[, verbose, ...])

Imputes missing numeric values in a DataFrame based on patient ID and temporal order.

impute_datetime(df[, datetime_column, ...])

Imputes missing datetime values based on temporal order within patient groups.

missing_percentage_df(dataframe)

Calculates missing percentage per column.

save_missing_values_pickle(df, out_file_path)

Calculates missing percentage and saves as a pickle.
pat2vec.util.post_processing_dataframe.extract_datetime_to_column(df, drop=True)[source]

Extracts datetime information from binary columns and creates a new column.

Return type:

DataFrame

Parameters:

  • df (DataFrame)

  • drop (bool)

pat2vec.util.post_processing_dataframe.extract_datetime_from_binary_columns(df)[source]

Extracts datetime values from binary columns representing dates.

Return type:

DataFrame

Parameters:

df (DataFrame)

pat2vec.util.post_processing_dataframe.extract_datetime_from_binary_columns_chunk_reader(filepath, chunk_size=1000)[source]

Reads a CSV in chunks and extracts datetime from binary columns.

Return type:

DataFrame

Parameters:

  • filepath (str)

  • chunk_size (int)

pat2vec.util.post_processing_dataframe.drop_columns_with_all_nan(df)[source]

Drops columns where all values are NaN.

Return type:

tuple[DataFrame, Index]

Parameters:

df (DataFrame)

pat2vec.util.post_processing_dataframe.save_missing_values_pickle(df, out_file_path, overwrite=False)[source]

Calculates missing percentage and saves as a pickle.

Return type:

None

Parameters:

  • df (DataFrame)

  • out_file_path (str)

  • overwrite (bool)

pat2vec.util.post_processing_dataframe.convert_true_to_float(df, columns=None)[source]

Converts ‘True’ strings to 1.0 and ensures columns are float.

Return type:

DataFrame

Parameters:

  • df (DataFrame)

  • columns (List[str] | None)

pat2vec.util.post_processing_dataframe.impute_datetime(df, datetime_column='datetime', patient_column='client_idcode', forward=True, backward=True, mean_impute=True, verbose=False)[source]

Imputes missing datetime values based on temporal order within patient groups.

Return type:

DataFrame

Parameters:

  • df (DataFrame)

  • datetime_column (str)

  • patient_column (str)

  • forward (bool)

  • backward (bool)

  • mean_impute (bool)

  • verbose (bool)

pat2vec.util.post_processing_dataframe.impute_dataframe(df, verbose=True, datetime_column='datetime', patient_column='client_idcode', forward=True, backward=True, mean_impute=True)[source]

Imputes missing numeric values in a DataFrame based on patient ID and temporal order.

Return type:

DataFrame

Parameters:

  • df (DataFrame)

  • verbose (bool)

  • datetime_column (str)

  • patient_column (str)

  • forward (bool)

  • backward (bool)

  • mean_impute (bool)

pat2vec.util.post_processing_dataframe.missing_percentage_df(dataframe)[source]

Calculates missing percentage per column.

Return type:

DataFrame

Parameters:

dataframe (DataFrame)

pat2vec.util.post_processing_dataframe.aggregate_dataframe_mean(df, group_by_column='client_idcode')[source]

Aggregates a DataFrame by taking the mean of numeric columns.

Return type:

DataFrame

Parameters:

  • df (DataFrame)

  • group_by_column (str)

pat2vec.util.post_processing_dataframe.collapse_df_to_mean(df, output_filename='output.csv', client_idcode_string='client_idcode')[source]

Collapses a DataFrame to means and saves/merges with an output file.

Return type:

None

Parameters:

  • df (DataFrame)

  • output_filename (str)

  • client_idcode_string (str)