plot_algorithms =============== .. py:module:: plot_algorithms .. autoapi-nested-parse:: Algorithm comparison plotting module for ML results analysis. Focuses on comparing algorithm performance with outcome variable stratification. Attributes ---------- .. autoapisummary:: plot_algorithms.MAX_OUTCOMES_FOR_STRATIFIED_PLOT Classes ------- .. autoapisummary:: plot_algorithms.AlgorithmComparisonPlotter Module Contents --------------- .. py:data:: MAX_OUTCOMES_FOR_STRATIFIED_PLOT :value: 20 .. py:class:: AlgorithmComparisonPlotter(data: pandas.DataFrame) Initializes the AlgorithmComparisonPlotter. :param data: A DataFrame containing the experiment results. :type data: pd.DataFrame .. py:attribute:: data .. py:attribute:: clean_data .. py:method:: plot_algorithm_boxplots(metric: str = 'auc', algorithms_to_plot: Optional[List[str]] = None, stratify_by_outcome: bool = False, outcomes_to_plot: Optional[List[str]] = None, figsize: Tuple[int, int] = (12, 6)) -> None Creates box plots comparing algorithm performance. :param metric: The performance metric to compare. Defaults to 'auc'. :type metric: str, optional :param algorithms_to_plot: A list of specific algorithms to include. If None, all are used. Defaults to None. :type algorithms_to_plot: Optional[List[str]], optional :param stratify_by_outcome: If True, creates separate plots for each outcome. Defaults to False. :type stratify_by_outcome: bool, optional :param outcomes_to_plot: A list of specific outcomes to plot. If None, all are used. Defaults to None. :type outcomes_to_plot: Optional[List[str]], optional :param figsize: The figure size. Defaults to (12, 6). :type figsize: Tuple[int, int], optional :raises ValueError: If the specified metric is not found in the data. .. py:method:: plot_algorithm_performance_heatmap(metric: str = 'auc', algorithms_to_plot: Optional[List[str]] = None, outcomes_to_plot: Optional[List[str]] = None, aggregation: str = 'mean', figsize: Tuple[int, int] = (12, 8)) -> pandas.DataFrame Creates a heatmap showing algorithm performance across outcomes. :param metric: The performance metric to visualize. Defaults to 'auc'. :type metric: str, optional :param algorithms_to_plot: A list of specific algorithms to include. Defaults to None. :type algorithms_to_plot: Optional[List[str]], optional :param outcomes_to_plot: A list of specific outcomes to include. Defaults to None. :type outcomes_to_plot: Optional[List[str]], optional :param aggregation: How to aggregate multiple runs ('mean', 'median', 'max'). Defaults to 'mean'. :type aggregation: str, optional :param figsize: The figure size. Defaults to (12, 8). :type figsize: Tuple[int, int], optional :raises ValueError: If 'outcome_variable' column is missing or an invalid aggregation method is provided. :returns: The pivot table data used for the heatmap. :rtype: pd.DataFrame .. py:method:: plot_algorithm_ranking(metric: str = 'auc', algorithms_to_plot: Optional[List[str]] = None, stratify_by_outcome: bool = False, outcomes_to_plot: Optional[List[str]] = None, top_n: int = 10, figsize: Tuple[int, int] = (10, 8)) -> None Plots a ranked bar chart of algorithm performance. :param metric: The performance metric to rank by. Defaults to 'auc'. :type metric: str, optional :param algorithms_to_plot: A list of specific algorithms to include. Defaults to None. :type algorithms_to_plot: Optional[List[str]], optional :param stratify_by_outcome: If True, creates separate plots for each outcome. Defaults to False. :type stratify_by_outcome: bool, optional :param outcomes_to_plot: A list of specific outcomes to plot when stratified. Defaults to None. :type outcomes_to_plot: Optional[List[str]], optional :param top_n: The number of top algorithms to display. Defaults to 10. :type top_n: int, optional :param figsize: The figure size. Defaults to (10, 8). :type figsize: Tuple[int, int], optional :raises ValueError: If the specified metric is not found, or if stratifying and 'outcome_variable' column is missing. .. py:method:: plot_algorithm_stability(metric: str = 'auc', top_n: int = 15, figsize: Tuple[int, int] = (10, 8)) -> None Plots the stability (standard deviation) of algorithm performance. A lower standard deviation indicates more stable and predictable performance across different runs and data subsets. :param metric: The performance metric to evaluate stability on. Defaults to 'auc'. :type metric: str, optional :param top_n: The number of algorithms to display, ranked by stability (lower is better). Defaults to 15. :type top_n: int, optional :param figsize: The figure size for the plot. Defaults to (10, 8). :type figsize: Tuple[int, int], optional :raises ValueError: If the specified metric is not found in the data. .. py:method:: plot_performance_tradeoff(metric_y: str = 'auc', metric_x: str = 'run_time', stratify_by_outcome: bool = False, top_n_algos: Optional[int] = 10, figsize: Tuple[int, int] = (12, 8)) -> None Plots a performance trade-off scatter plot between two metrics. This is useful for visualizing trade-offs like Performance vs. Speed. :param metric_y: The metric for the y-axis (e.g., 'auc'). Defaults to 'auc'. :type metric_y: str, optional :param metric_x: The metric for the x-axis (e.g., 'run_time'). Defaults to 'run_time'. :type metric_x: str, optional :param stratify_by_outcome: If True, creates a separate plot for each outcome. Defaults to False. :type stratify_by_outcome: bool, optional :param top_n_algos: If set, only shows the top N algorithms based on `metric_y`. Defaults to 10. :type top_n_algos: Optional[int], optional :param figsize: The figure size for the plot. Defaults to (12, 8). :type figsize: Tuple[int, int], optional :raises ValueError: If one or both specified metrics are not found. .. py:method:: plot_pareto_front(metric_y: str = 'auc', metric_x: str = 'run_time', lower_is_better_x: bool = True, figsize: Tuple[int, int] = (12, 8)) -> None Plots a Pareto front for two competing metrics. The Pareto front highlights the set of "optimal" algorithms where you cannot improve one metric without degrading the other. :param metric_y: The primary performance metric (higher is better). Defaults to 'auc'. :type metric_y: str, optional :param metric_x: The secondary metric, often a cost (e.g., 'run_time'). Defaults to 'run_time'. :type metric_x: str, optional :param lower_is_better_x: Set to True if a lower value of `metric_x` is better. Defaults to True. :type lower_is_better_x: bool, optional :param figsize: The figure size for the plot. Defaults to (12, 8). :type figsize: Tuple[int, int], optional .. py:method:: plot_statistical_significance_heatmap(metric: str = 'auc', outcome: Optional[str] = None, figsize: Tuple[int, int] = (14, 12)) -> None Performs pairwise t-tests and visualizes p-values in a heatmap. This helps determine if observed performance differences between algorithms are statistically significant. :param metric: The performance metric to compare. Defaults to 'auc'. :type metric: str, optional :param outcome: If specified, filters data for a single outcome. Otherwise, uses all data. Defaults to None. :type outcome: Optional[str], optional :param figsize: The figure size for the plot. Defaults to (14, 12). :type figsize: Tuple[int, int], optional :raises ValueError: If stratifying and 'outcome_variable' column is missing.