ml_grid.util.hierarchical_search
================================

.. py:module:: ml_grid.util.hierarchical_search

.. autoapi-nested-parse::

   Hierarchical Hyperparameter Search Module.

   This module implements advanced optimization strategies for hyperparameter search:
   - Two-stage coarse-to-fine parameter search
   - Dynamic space reduction based on early results
   - Early stopping for unpromising trials
   - Parameter importance analysis to focus search



Attributes
----------

.. autoapisummary::

   ml_grid.util.hierarchical_search.Result


Classes
-------

.. autoapisummary::

   ml_grid.util.hierarchical_search.ParameterGrid
   ml_grid.util.hierarchical_search.SearchResult
   ml_grid.util.hierarchical_search.ParameterImportanceAnalyzer
   ml_grid.util.hierarchical_search.DynamicSpaceReducer
   ml_grid.util.hierarchical_search.EarlyStoppingRule
   ml_grid.util.hierarchical_search.HierarchicalSearchOptimizer


Functions
---------

.. autoapisummary::

   ml_grid.util.hierarchical_search.optimize_hyperparameter_search


Module Contents
---------------

.. py:class:: ParameterGrid(*args, **kwargs)

.. py:class:: SearchResult

   Stores results from a single hyperparameter evaluation.


   .. py:attribute:: parameters
      :type:  Dict[str, Any]


   .. py:attribute:: score
      :type:  float


   .. py:attribute:: fit_time
      :type:  float
      :value: 0.0



   .. py:attribute:: trial_number
      :type:  int
      :value: 0



   .. py:attribute:: stage
      :type:  str
      :value: 'coarse'



   .. py:attribute:: confidence
      :type:  float
      :value: 1.0



.. py:data:: Result

.. py:class:: ParameterImportanceAnalyzer(logger: logging.Logger = None)

   Analyzes parameter importance using statistical methods.


   .. py:attribute:: logger


   .. py:method:: analyze_parameters(results: List[SearchResult], param_space: Dict[str, Any]) -> Dict[str, float]

      Analyze parameter importance based on cross-validation results.

      Uses correlation analysis to identify which parameters have the most
      significant impact on model performance.

      :param results: List of search results with parameters and scores
      :param param_space: Original parameter space for reference

      :returns: Dictionary mapping parameter names to importance scores (0-1)



.. py:class:: DynamicSpaceReducer(initial_space: Dict[str, Any], logger: logging.Logger = None)

   Dynamically reduces search space based on early results.


   .. py:attribute:: logger


   .. py:attribute:: initial_space


   .. py:attribute:: top_n_percentile
      :value: 25



   .. py:method:: get_reduced_space(results: List[SearchResult], reduction_factor: float = 0.3) -> Dict[str, Any]

      Reduce the search space based on top performing parameter values.

      :param results: Results from previous search stage
      :param reduction_factor: Fraction of original space to retain (0-1)

      :returns: Reduced parameter space for next stage



.. py:class:: EarlyStoppingRule(min_trials: int = 5, patience: int = 10, threshold_factor: float = 0.95)

   Early stopping rules based on trial performance.


   .. py:attribute:: min_trials
      :value: 5



   .. py:attribute:: patience
      :value: 10



   .. py:attribute:: threshold_factor
      :value: 0.95



   .. py:method:: should_stop(results: List[SearchResult]) -> Tuple[bool, str]

      Determine if search should stop based on early stopping rules.

      :param results: All results collected so far

      :returns: Tuple of (should_stop, reason)



.. py:class:: HierarchicalSearchOptimizer(initial_param_space: Dict[str, Any], max_total_trials: int = 100, coarse_ratio: float = 0.25, fine_ratio: float = 0.45, refinement_ratio: float = 0.3, logger: logging.Logger = None)

   Implements hierarchical hyperparameter search with following stages:

   1. Coarse Search: Broad exploration with minimal evaluations per parameter
   2. Fine Search: Focused exploitation on promising regions
   3. Refinement: Detailed optimization of top candidates

   Each stage uses dynamic space reduction and early stopping.


   .. py:attribute:: logger


   .. py:attribute:: initial_space


   .. py:attribute:: max_total_trials
      :value: 100



   .. py:attribute:: coarse_ratio
      :value: 0.25



   .. py:attribute:: fine_ratio
      :value: 0.45



   .. py:attribute:: refinement_ratio
      :value: 0.3



   .. py:attribute:: results
      :type:  List[SearchResult]
      :value: []



   .. py:method:: run_hierarchical_search(evaluate_fn: callable, max_trials_per_stage: Optional[Dict[str, int]] = None, verbose: bool = True) -> Tuple[SearchResult, Dict[str, List[SearchResult]]]

      Execute hierarchical hyperparameter search.

      :param evaluate_fn: Function to evaluate parameter set:
                          params -> score, fit_time
      :param max_trials_per_stage: Optional override for trial counts per stage
      :param verbose: Whether to log progress

      :returns: Tuple of (best_result, all_results_by_stage)



.. py:function:: optimize_hyperparameter_search(param_space: Dict[str, Any], max_total_evals: int = 100, verbose: bool = True) -> HierarchicalSearchOptimizer

   Create and configure a hierarchical search optimizer.

   :param param_space: Initial parameter space dictionary
   :param max_total_evals: Total number of evaluations to perform
   :param verbose: Whether to enable verbose logging

   :returns: Configured HierarchicalSearchOptimizer instance