ml_grid.util.data_cache
=======================

.. py:module:: ml_grid.util.data_cache

.. autoapi-nested-parse::

   Data caching module for ml_grid pipeline.

   This module provides efficient data caching using joblib to cache preprocessed
   data, feature encoders/scalers, and transformed datasets for reuse between
   experiments. Implements smart cache invalidation based on file checksums/timestamps.



Exceptions
----------

.. autoapisummary::

   ml_grid.util.data_cache.CacheValidationError


Classes
-------

.. autoapisummary::

   ml_grid.util.data_cache.DataCache
   ml_grid.util.data_cache.CachedPipelineInstance
   ml_grid.util.data_cache.CachedDataPipeline


Functions
---------

.. autoapisummary::

   ml_grid.util.data_cache.with_cache_validation
   ml_grid.util.data_cache.get_cache_stats
   ml_grid.util.data_cache.clear_experiment_cache


Module Contents
---------------

.. py:exception:: CacheValidationError

   Bases: :py:obj:`Exception`


   Initialize self.  See help(type(self)) for accurate signature.


.. py:class:: DataCache(base_cache_dir: Optional[str] = None)

   Initialize the data cache system.


   .. py:attribute:: global_params


   .. py:attribute:: base_cache_dir


   .. py:property:: cache_dir
      :type: pathlib.Path


      Get the current experiment-specific cache directory.


   .. py:method:: get_cache_path(experiment_name: str, run_id: str) -> pathlib.Path

      Get full path for a cached item.

      :param experiment_name: Name of the experiment
      :param run_id: Unique identifier for this specific run

      :returns: Full path to cache file



   .. py:method:: save(experiment_name: str, run_id: str, data_dict: Dict[str, Any], metadata: Optional[Dict[str, Any]] = None) -> bool

      Save data dictionary to cache.

      :param experiment_name: Name of the experiment
      :param run_id: Unique identifier for this run
      :param data_dict: Dictionary of data to cache (X_train, X_test, scaler, etc.)
      :param metadata: Optional metadata to store

      :returns: True if save was successful, False otherwise



   .. py:method:: load(experiment_name: str, run_id: str) -> Optional[Dict[str, Any]]

      Load cached data.

      :param experiment_name: Name of the experiment
      :param run_id: Unique identifier for this run

      :returns: Cached data dictionary or None if not found/invalid



   .. py:method:: invalidate(experiment_name: str, run_id: str) -> bool

      Invalidate (delete) a specific cache entry.

      :param experiment_name: Name of the experiment
      :param run_id: Unique identifier for this run

      :returns: True if invalidated successfully



   .. py:method:: clear_experiment_cache(experiment_name: str) -> int

      Clear all cached data for an experiment.

      :param experiment_name: Name of the experiment

      :returns: Number of files removed



   .. py:method:: get_cache_stats() -> Dict[str, Any]

      Get statistics about the cache directory.



.. py:class:: CachedPipelineInstance(experiment_name: str, run_id: str, cache: DataCache, file_path: str, local_param_dict: Dict[str, Any], original_kwargs: Dict[str, Any], original_class: Any)

   Initialize cached pipeline instance.


   .. py:attribute:: experiment_name


   .. py:attribute:: run_id


   .. py:attribute:: cache


   .. py:attribute:: file_path


   .. py:attribute:: local_param_dict


   .. py:attribute:: original_kwargs


   .. py:attribute:: original_class


   .. py:method:: run_pipeline() -> Any

      Run the pipeline, using cached data when available.

      :returns: Pipeline instance with data populated



.. py:class:: CachedDataPipeline(original_pipeline_class: Any, cache: Optional[DataCache] = None)

   Initialize CachedDataPipeline wrapper.

   :param original_pipeline_class: The original pipe class to wrap
   :param cache: DataCache instance (creates new if None)


   .. py:attribute:: cache


   .. py:attribute:: cached_instances


   .. py:method:: create_cached_pipeline(experiment_name: str, run_id: str, file_path: str, local_param_dict: Dict[str, Any], original_kwargs: Dict[str, Any]) -> CachedPipelineInstance

      Create a cached pipeline instance.

      :param experiment_name: Name of the experiment
      :param run_id: Run identifier
      :param file_name: Path to input data
      :param local_param_dict: Pipeline parameters
      :param original_kwargs: Original __init__ arguments

      :returns: CachedPipelineInstance that uses cached data when available



.. py:function:: with_cache_validation(func: Callable) -> Callable

   Decorator to add cache validation before method execution.


.. py:function:: get_cache_stats(cache: DataCache) -> Dict[str, Any]

   Helper function to get cache statistics.


.. py:function:: clear_experiment_cache(cache: DataCache, experiment_name: str) -> int

   Helper function to clear cache for an experiment.