ml_grid.util.hierarchical_search

Hierarchical Hyperparameter Search Module.

This module implements advanced optimization strategies for hyperparameter search: - Two-stage coarse-to-fine parameter search - Dynamic space reduction based on early results - Early stopping for unpromising trials - Parameter importance analysis to focus search

Attributes

Result

Classes

ParameterGrid
SearchResult	Stores results from a single hyperparameter evaluation.
ParameterImportanceAnalyzer	Analyzes parameter importance using statistical methods.
DynamicSpaceReducer	Dynamically reduces search space based on early results.
EarlyStoppingRule	Early stopping rules based on trial performance.
HierarchicalSearchOptimizer	Implements hierarchical hyperparameter search with following stages:

Functions

optimize_hyperparameter_search(...)

Create and configure a hierarchical search optimizer.

Module Contents

class ml_grid.util.hierarchical_search.ParameterGrid(*args, **kwargs)

class ml_grid.util.hierarchical_search.SearchResult

Stores results from a single hyperparameter evaluation.

parameters: Dict[str, Any]

score: float

fit_time: float = 0.0

trial_number: int = 0

stage: str = 'coarse'

confidence: float = 1.0

ml_grid.util.hierarchical_search.Result

class ml_grid.util.hierarchical_search.ParameterImportanceAnalyzer(logger: logging.Logger = None)

Analyzes parameter importance using statistical methods.

logger

analyze_parameters(results: List[SearchResult], param_space: Dict[str, Any]) → Dict[str, float]

Analyze parameter importance based on cross-validation results.

Uses correlation analysis to identify which parameters have the most significant impact on model performance.

Parameters:

• results – List of search results with parameters and scores

• param_space – Original parameter space for reference

Returns:

Dictionary mapping parameter names to importance scores (0-1)

class ml_grid.util.hierarchical_search.DynamicSpaceReducer(initial_space: Dict[str, Any], logger: logging.Logger = None)

Dynamically reduces search space based on early results.

logger

initial_space

top_n_percentile = 25

get_reduced_space(results: List[SearchResult], reduction_factor: float = 0.3) → Dict[str, Any]

Reduce the search space based on top performing parameter values.

Parameters:

• results – Results from previous search stage

• reduction_factor – Fraction of original space to retain (0-1)

Returns:

Reduced parameter space for next stage

class ml_grid.util.hierarchical_search.EarlyStoppingRule(min_trials: int = 5, patience: int = 10, threshold_factor: float = 0.95)

Early stopping rules based on trial performance.

min_trials = 5

patience = 10

threshold_factor = 0.95

should_stop(results: List[SearchResult]) → Tuple[bool, str]

Determine if search should stop based on early stopping rules.

Parameters:

results – All results collected so far

Returns:

Tuple of (should_stop, reason)

class ml_grid.util.hierarchical_search.HierarchicalSearchOptimizer(initial_param_space: Dict[str, Any], max_total_trials: int = 100, coarse_ratio: float = 0.25, fine_ratio: float = 0.45, refinement_ratio: float = 0.3, logger: logging.Logger = None)

Implements hierarchical hyperparameter search with following stages:

1. Coarse Search: Broad exploration with minimal evaluations per parameter

2. Fine Search: Focused exploitation on promising regions

3. Refinement: Detailed optimization of top candidates

Each stage uses dynamic space reduction and early stopping.

logger

initial_space

max_total_trials = 100

coarse_ratio = 0.25

fine_ratio = 0.45

refinement_ratio = 0.3

results: List[SearchResult] = []

run_hierarchical_search(evaluate_fn: callable, max_trials_per_stage: Dict[str, int] | None = None, verbose: bool = True) → Tuple[SearchResult, Dict[str, List[SearchResult]]]

Execute hierarchical hyperparameter search.

Parameters:

• evaluate_fn – Function to evaluate parameter set: params -> score, fit_time

• max_trials_per_stage – Optional override for trial counts per stage

• verbose – Whether to log progress

Returns:

Tuple of (best_result, all_results_by_stage)

ml_grid.util.hierarchical_search.optimize_hyperparameter_search(param_space: Dict[str, Any], max_total_evals: int = 100, verbose: bool = True) → HierarchicalSearchOptimizer

Create and configure a hierarchical search optimizer.

Parameters:

• param_space – Initial parameter space dictionary

• max_total_evals – Total number of evaluations to perform

• verbose – Whether to enable verbose logging

Returns:

Configured HierarchicalSearchOptimizer instance