ml_grid.util.hierarchical_param_space

Enhanced parameter space definitions with hierarchical search strategy.

This module provides advanced parameter space generation with: - Two-stage coarse-to-fine parameter search - Dynamic space reduction based on early results - Early stopping for unpromising trials - Parameter importance analysis to focus search

Key Classes:

HierarchicalParamSpace: Extends ParamSpace with hierarchical search capabilities

Classes

HierarchicalParamSpace	Initialize hierarchical parameter space.
AdaptiveParameterAnalyzer	Analyzes parameter importance and guides search focus.
HierarchicalSearchManager	Manages the hierarchical search process.

Functions

create_hierarchical_param_space(→ HierarchicalParamSpace)

Factory function to create hierarchical parameter space.

Module Contents

class ml_grid.util.hierarchical_param_space.HierarchicalParamSpace(size: str = 'medium', hierarchical_config: Dict[str, Any] = None, logger: logging.Logger = None)

Initialize hierarchical parameter space.

Parameters:

• size – Base parameter space size (“medium”, “xsmall”, “xwide”)

• hierarchical_config – Configuration for hierarchical search strategy - max_total_evals: Total evaluations budget (default: 100) - coarse_ratio: Proportion of trials for stage 1 (default: 0.25) - fine_ratio: Proportion of trials for stage 2 (default: 0.45) - refinement_ratio: Proportion of trials for stage 3 (default: 0.30) - reduction_factor: Space reduction factor per stage (default: 0.4)

• logger – Logger instance

base_size = 'medium'

logger

hierarchical_config

generate_hierarchical_space(base_param_dict: Dict[str, Any]) → Tuple[Dict[str, Any], Dict[str, Any], Dict[str, Any]]

Generate three-tiered parameter spaces for hierarchical search.

Parameters:

base_param_dict – Original parameter dictionary from ParamSpace

Returns:

Tuple of (coarse_space, fine_space, refinement_space)

class ml_grid.util.hierarchical_param_space.AdaptiveParameterAnalyzer(logger: logging.Logger = None)

Analyzes parameter importance and guides search focus.

logger

analyze_parameter_importance(results_list: List[Dict[str, Any]], param_space: Dict[str, Any]) → Dict[str, float]

Analyze which parameters most impact model performance.

Uses statistical methods to identify important parameters: - ANOVA for categorical parameters - Spearman correlation for continuous parameters

Parameters:

• results_list – List of result dictionaries with ‘score’ and parameter values

• param_space – Original parameter space

Returns:

Dictionary mapping parameter names to importance scores (0-1)

get_early_stopping_rule(min_trials: int = 5, patience: int = 10) → Dict[str, Any]

Create early stopping configuration.

Parameters:

• min_trials – Minimum trials before checking early stopping

• patience – Number of consecutive non-improving trials

Returns:

Early stopping configuration dict

class ml_grid.util.hierarchical_param_space.HierarchicalSearchManager(param_space: Dict[str, Any], max_total_evals: int = 100, reduction_factor: float = 0.4, logger: logging.Logger = None)

Manages the hierarchical search process.

param_space

max_total_evals = 100

reduction_factor = 0.4

logger

results_history: List[Dict[str, Any]] = []

get_staged_spaces(num_stages: int = 3) → Dict[int, Dict[str, Any]]

Generate parameter spaces for each hierarchical stage.

Parameters:

num_stages – Number of stages (default: 3 for coarse->fine->refine)

Returns:

Dictionary mapping stage number to reduced param space

update_with_results(trial_result: Dict[str, Any]) → None

Update history with new trial results.

should_early_stop() → Tuple[bool, str]

Check if search should stop based on early stopping rules.

Returns:

bool, reason: str)

Return type:

Tuple of (should_stop

ml_grid.util.hierarchical_param_space.create_hierarchical_param_space(size: str = 'medium', max_total_evals: int = 100, reduction_factor: float = 0.4, logger: logging.Logger = None) → HierarchicalParamSpace

Factory function to create hierarchical parameter space.

Parameters:

• size – Base parameter space size

• max_total_evals – Total evaluation budget

• reduction_factor – Space reduction per stage

• logger – Logger instance

Returns:

Configured HierarchicalParamSpace instance