Skip to content

back

BuildIntersectionFromConverted

class arsa_ml.pipelines.pipelines_from_converters.BuildIntersectionFromConverted(converter_results_path, metrics, custom_weights, weighted_sum_method, delta)


This is a subclass of the Builder abctract class, which provides pipeline for creating and exploring the Rashomon Intersection created from parameters in a format propsed by the convert() method of converter classes.

Example usage of this pipeline can be found at demo_notebooks/converters_pipeline.ipynb.


Parameters


converter_results_path : Path
  Path to a folder where all necessary files are stored. Obligatory files to be included in this folder are:
  1. leaderboard.csv : file containing all trained models and their evaluation metrics scores.
  2. y_true.csv: target column from test dataset.
  3. predictions_dict.pkl : dictionary with model names as keys and their class prediction vectors as values.
  4. proba_predictions.pkl : dictionary with model names as keys and their probabilities predictions as values.
  5. (optional) feature_importance_dict.pkl : dictionary with model names as keys and their features sorted descending by their feature importance as values.

If you do not have the necessary files, please visit documentation of the pipelines accepting raw models as input:

H2O pipeline
AutoGluon pipeline

metrics : list
  Evaluation metrics to be used as the metrics list while creating the Rashomon Intersection.
custom_weights : list
  List containing two values of custom weights assigned by the user to every evaluation metric present in the metrics list. Can be set only while using the 'custom_weights' weighted sum method. Default is None.
weighted_sum_method : str
  Name of the weighted_sum_method to be used while selecting the base_model in the Rashomon Intersection. Possible options are 'entropy','critic', 'custom_weights'. Default is 'entropy'.
delta : float
  Delta parameter for probabilistic ambiguity and discrepancy (used only for binary task type).
  If not specified the default value of 0.1 will be used.

BuildRashomonFromConverted Pipeline – Key Steps


  1. Initialization (__init__) – Evaluation of the given arguments. If the provided folder contains all necessary files the Pipeline is initialized.
  2. Preview Rashomon (preview_rashomon()) – Visualize leaderboard and Rashomon Intersection sizes to guide epsilon selection.
  3. Set Epsilon (set_epsilon()) – Set epsilon parameter value to be used when creating the Rashomon Intersection.
  4. Build Pipeline (build()) – Create RashomonIntersection and IntersectionVisualizer objects and launch the Streamlit dashboard in the background.
  5. Interactive Analysis – Explore plots via returned IntersectionVisualizer object or the dashboard.
  6. Close Dashboard (dashboard_close()) – Stop Streamlit processes.
Methods


preview_rashomon()

  Method illustrating the leaderboard and the plot with all possible epsilon values and the Rashomon Intersection sizes for different epsilon values.
  Should be called to guide the selection of an appropriate epsilon threshold.

visualize_rashomon_set_volume()

  Method for visualising Rashomon Intersection size depending on different epsilon values.

set_epsilon(epsilon)

  Sets the epsilon parameter value to be used when constructing the Rashomon Intersection object.
  Epsilon value must be set before calling build() method.

  Parameters :
  epsilon : float

build(lanuch_dashboard)

 Builds the Rashomon Intersection pipeline from converted files found in the given directory and other provided parameters.
  Creates Rashomon Intersection object and IntersectionVisualizer object from user's input and launches a Streamlit dashboard in a subprocess for interactive visualization if launch_dashboard parameter is set to True.

  This method performs the following steps:

  • Validates that the epsilon threshold has been set using set_epsilon() method.

  • Creates the RashomonIntersection object based on the leaderboard, predictions, probability predictions, feature importances, metrics, methods to select base model, and epsilon threshold.
  • Initializes the IntersectionVisualizer object for interactive analysis of the Rashomon Intersection.
    Individual plots can later be generated directly from the IntersectionVisualizer object.

  • If launch_dashboard is set to True (default) it generates plots for analysis depending on the task type (binary or multiclass), and stores them temporarily with their descriptions for the Streamlit dashboard. It closes any previous Streamlit processes to avoid conflicts.

  • If launch_dashboard is set to True (default) it launches the Streamlit dashboard in a subprocess on the local machine (localhost), allowing interactive exploration of the Rashomon Intersection properties without blocking the main workflow.


  Returns :
  rashomon_intersection : RashomonIntersection
  visualizer : IntersectionVisualizer

dashboard_close()

  Method for stopping all Streamlit processes and closing the dashboard.
  Note: Always call this method after finishing the analysis to ensure the dashboard is properly closed.