Module Reference

CLI Entry Points

These top-level scripts parse arguments and re-export the library’s public API for backward-compatible imports; the :imported-members: option documents the re-exported names alongside anything each script defines directly (e.g. main).

CLI to download FAST CDF files from CDA Web since the web interface can have issues.

NOTE: can’t get orb ephemeris files to my knowledge :(

All downloading logic lives in configurable_spectrograms.download; this script only parses arguments and calls it.

FAST_CDF_download.FAST_ESA_CDF_download(base_url: str = 'https://cdaweb.gsfc.nasa.gov/pub/data/fast/esa/l2', year: int = 2000, data_folder: str = './FAST_data/', instruments: list[str] = ['eeb', 'ees', 'ieb', 'ies']) → None

Download one year of FAST ESA CDF files from CDA Web.

Scrapes each month/instrument listing page once, then calls download_single_day_cdf() for every calendar day of year against that cached listing, so every day is downloaded through the same single-day logic used for one-off single-day downloads elsewhere in this module, without re-requesting the same month page once per day.

Parameters:

• base_url (str, default FAST_ESA_BASE_URL) – Base CDA Web URL for FAST ESA level-2 data.

• year (int, default DEFAULT_YEAR) – Calendar year to download.

• data_folder (str, default DEFAULT_FOLDER) – Root output directory; files are saved under {data_folder}/{year}/{month}/.

• instruments (list of str, default DEFAULT_INSTRUMENT_LIST) – Instrument codes to download (e.g. ['eeb', 'ees']).

Notes

For downloading many years at once with thread-pool parallelism, see download_cdf_files_threaded().

FAST_CDF_download.main() → None

Parse CLI arguments and download one year of FAST ESA CDF files.

Provides batch spectrogram plotting utilities. Should work with CDFs like those from FAST (see batch_multi_plot_FAST_spectrograms.py) but should also be flexible with other data.

Assumed folder layout is::

{CDF_DATA_DIRECTORY}/year/month

Filenames in the month folders assumed to be in the following formats::

{??}_{??}_{??}_{instrument}_{timestamp}_{orbit}_v02.cdf (known “instruments” are ees, eeb, ies, or ieb) {??}_{??}_orb_{orbit}_{??}.cdf

Examples::

FAST_data/2000/01/fa_esa_l2_eeb_20000101001737_13312_v02.cdf FAST_data/2000/01/fa_k0_orb_13312_v01.cdf

All plotting/batch logic lives in the configurable_spectrograms package; this module re-exports the public functions/constants for backward compatibility with existing imports (from batch_multi_plot_spectrogram import make_spectrogram, etc).

batch_multi_plot_spectrogram.close_all_axes_and_clear(fig) → None

Close axes/subplots and clear a figure to free memory.

Parameters:

fig (matplotlib.figure.Figure) – Figure instance to clear and dispose.

Return type:

None

Notes

Ensures axes are deleted, the canvas is closed/detached, and removes the figure from the global Gcf registry when possible to mitigate memory growth during large batch operations.

batch_multi_plot_spectrogram.configure_log_batch(batch_size: int) → None

Configure buffered logging batch size.

Parameters:

batch_size (int) – Desired number of log records to accumulate before an automatic flush. Values less than 1 are coerced to 1.

batch_multi_plot_spectrogram.generic_batch_plot(items, output_dir: str, build_datasets_fn: Callable[[Any], list[dict]], zoom_center_fn: Callable[[Any], float | None] | None = None, zoom_window_seconds: float | None = None, vertical_lines_fn: Callable[[Any], list[float] | None] | None = None, y_scale: str = 'linear', z_scale: str = 'linear', colormap: str = 'viridis', cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None, max_workers: int = 2, progress_json_path: str = './batch_multi_plot_progress.json', ignore_progress_json: bool = False, flush_batch_size: int = 10, log_flush_batch_size: int | None = None, install_signal_handlers: bool = True) → list[tuple[Any, str]]

Generic batch runner for plotting datasets across many items in parallel.

Each item is rendered by calling configurable_spectrograms.plotting.generic_plot_spectrogram_set() exactly once, in a worker process managed by configurable_spectrograms.batch_runner.run_batch(), so a single item plotted through this batch driver produces the same output as calling the single-output function directly.

Parameters:

• items (iterable) – Iterable of item identifiers (any repr-able objects).

• output_dir (str) – Base output directory; plots saved under output_dir/<item>/generic.png.

• build_datasets_fn (callable) – Callable returning list[dict] describing datasets for an item.

• zoom_center_fn (callable, optional) – Callable mapping item -> center UNIX time (or None) for zoom.

• zoom_window_seconds (float, optional) – Duration of zoom window in seconds.

• vertical_lines_fn (callable, optional) – Callable mapping item -> list[float] UNIX timestamps (or None).

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling for all rows.

• z_scale ({'linear', 'log'}, default 'linear') – Color scaling for all rows.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to generic_plot_spectrogram_set.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

• max_workers (int, default 2) – Number of parallel worker processes.

• progress_json_path (str, default PLOTTING_PROGRESS_JSON_PATH) – Path to progress JSON (resumable state). Created/updated as needed.

• ignore_progress_json (bool, default False) – If True, skip reading existing progress prior to execution.

• flush_batch_size (int, default 10) – Progress/log batch size; values < 1 coerced to 1. Final partial batch flushed.

• log_flush_batch_size (int, optional) – Explicit log batch size; if None reuse flush_batch_size.

• install_signal_handlers (bool, default True) – When True, a temporary SIGINT handler is installed (restored on exit) to enable graceful interruption (progress & log flush).

Returns:

Sequence of (item, status) with status in {'ok', 'no_data', 'error'}.

Return type:

list of tuple

batch_multi_plot_spectrogram.generic_plot_multirow_optional_zoom(datasets, vertical_lines=None, zoom_duration_minutes=6.25, y_scale='linear', z_scale='linear', colormap='viridis', show=False, title=None, row_label_pad=50, row_label_rotation=90, y_min=None, y_max=None, z_min=None, z_max=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Render a multi-row spectrogram grid with an optional zoom column.

Parameters:

• datasets (list of dict) –

  Each dict must contain keys:

  • 'x' – 1D UNIX epoch seconds (float) array

  • 'y' – 1D energy (eV) array (unfiltered, 0-4000 typical)

  • 'data' – 3D ndarray that can be collapsed (time, pitch/angle, energy)

  Optional per-row keys (all honored when present):

  • 'label' – Row label placed on the left (rotated)

  • 'y_label' – Units label for y-axis (default: 'Energy (eV)')

  • 'z_label' – Color scale label (default: 'Counts')

  • 'y_min' / 'y_max' – Energy bounds (overrides global y_min / y_max args)

  • 'z_min' / 'z_max' – Color bounds (overrides global z_min / z_max args)

  • 'vmin' / 'vmax' – Precomputed percentile (or fixed) color bounds used when z_min / z_max not provided.

• vertical_lines (list of float, optional) – UNIX timestamps defining the cusp boundary and potential zoom window.

• zoom_duration_minutes (float, default 6.25) – Desired zoom window length in minutes (may auto-expand to include full marked span).

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• z_scale ({'linear', 'log'}, default 'linear') – Color (intensity) scale.

• colormap (str, default 'viridis') – Matplotlib colormap.

• show (bool, default False) – If True, display interactively.

• title (str, optional) – Figure suptitle.

• row_label_pad (int, default 50) – Padding for row labels.

• row_label_rotation (int, default 90) – Rotation angle (degrees) for row labels.

• y_min (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• y_max (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• z_min (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• z_max (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style forwarded to make_spectrogram().

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

(fig, canvas) or (None, None) if datasets is empty.

Return type:

tuple

Notes

Determines need for a zoom column dynamically: only rendered if at least one dataset contains non-NaN values inside the computed zoom window.

batch_multi_plot_spectrogram.generic_plot_spectrogram_set(datasets, collapse_axis=1, zoom_center=None, zoom_window_seconds=None, vertical_lines=None, x_is_unix=True, y_scale='linear', z_scale='linear', colormap='viridis', figure_title=None, show=False, y_min=None, y_max=None, z_min=None, z_max=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Plot a vertical stack of generic spectrograms.

Parameters:

• datasets (list of dict) – Each dict requires keys 'x', 'y', 'data' and may include optional keys: 'label', 'y_label', 'z_label', 'y_min', 'y_max', 'z_min', 'z_max'.

• collapse_axis (int, default 1) – Axis index of the 3D array collapsed prior to plotting.

• zoom_center (float, optional) – Center (UNIX time) for zoom column when used.

• zoom_window_seconds (float, optional) – Duration of zoom window (seconds) when zoom_center provided.

• vertical_lines (list of float, optional) – UNIX timestamps to annotate with a cusp-boundary marker.

• x_is_unix (bool, default True) – If True, x values are treated as UNIX seconds and formatted.

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling mode.

• z_scale ({'linear', 'log'}, default 'linear') – Color (intensity) scale mode.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• figure_title (str, optional) – Figure-level title (sup-title).

• show (bool, default False) – If True, display interactively (requires GUI backend).

• y_min (float, optional) – Global Y min fallback when per-row not supplied. Defaults to 0 if omitted and per-row missing.

• y_max (float, optional) – Global Y max fallback when per-row not supplied. If both global and per-row absent, inferred.

• z_min (float, optional) – Global colorbar lower bound fallback.

• z_max (float, optional) – Global colorbar upper bound fallback.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style forwarded to make_spectrogram().

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

(fig, canvas) or (None, None) if datasets is empty.

Return type:

tuple

batch_multi_plot_spectrogram.get_cdf_file_type(cdf_file_path: str) → str | None

Infer instrument type from a CDF file path.

Parameters:

cdf_file_path (str) – Path to the CDF file.

Returns:

Instrument type string (e.g. 'ees'), 'orb' for orbit files, or None if not recognized.

Return type:

str or None

Examples

>>> get_cdf_file_type("fa_esa_l2_eeb_20000101001737_13312_v02.cdf")
'eeb'
>>> get_cdf_file_type("fa_k0_orb_13312_v01.cdf")
'orb'

batch_multi_plot_spectrogram.get_cdf_var_shapes(cdf_folder_path: str = './FAST_data/', variable_names: list[str] = ['time_unix', 'data', 'energy', 'pitch_angle']) → dict[str, list[tuple[int, ...] | None]]

Collect shapes of variables across CDF files in a folder.

Parameters:

• cdf_folder_path (str, default CDF_DATA_DIRECTORY) – Directory containing CDF files.

• variable_names (list of str, default CDF_VARIABLE_NAMES) – Variable names to inspect.

Returns:

Mapping from variable name (str) to a list of shape tuples (or None) per file.

Return type:

dict

batch_multi_plot_spectrogram.get_timestamps_for_orbit(filtered_orbits_dataframe: pd.DataFrame | None, orbit_number: int, instrument_type: str | None, time_unix_array: np.ndarray | None) → list[float]

Compute orbit boundary UNIX timestamps from filtered indices.

Parameters:

• filtered_orbits_dataframe (pandas.DataFrame or None) – DataFrame containing filtered orbits and min/max indices per instrument.

• orbit_number (int) – Orbit number to look up.

• instrument_type (str or None) – Instrument type identifier (e.g. 'ees', 'ies').

• time_unix_array (numpy.ndarray or None) – 1D array of UNIX timestamps for the instrument.

Returns:

Boundary UNIX timestamps for the orbit: one value when the CSV row gives a degenerate (equal) min/max index, two values (start, end) otherwise. Returns an empty list when the orbit is not found or inputs are missing.

Return type:

list of float

Examples

>>> import pandas as pd
>>> import numpy as np
>>> orbits = pd.DataFrame({"orbit": [42], "ees min index": [1], "ees max index": [3]})
>>> times = np.array([100.0, 200.0, 300.0, 400.0])
>>> get_timestamps_for_orbit(orbits, 42, "ees", times)
[200.0, 400.0]
>>> get_timestamps_for_orbit(orbits, 99, "ees", times)
[]

batch_multi_plot_spectrogram.get_variable_shape(cdf_path: str, variable_name: str) → tuple[int, ...] | None

Return the shape of a variable in a CDF file.

Parameters:

• cdf_path (str) – Path to the CDF file.

• variable_name (str) – Variable name to inspect.

Returns:

Variable shape tuple, or None if the variable is absent, not an array, or an error occurs.

Return type:

tuple or None

batch_multi_plot_spectrogram.load_filtered_orbits(csv_path: str = './FAST_Cusp_Indices.csv') → pd.DataFrame | None

Load the filtered orbits CSV with a simple cache.

Parameters:

csv_path (str, default FILTERED_ORBITS_CSV_PATH) – Path to the filtered orbits TSV/CSV file.

Returns:

DataFrame of filtered orbits, or None if loading fails.

Return type:

pandas.DataFrame or None

Notes

A module-level dictionary caches previously loaded DataFrames keyed by the path string to avoid repeated disk I/O in batch routines.

batch_multi_plot_spectrogram.log_error(message: str, force_flush: bool = False) → None

Queue an error log message and echo it to the console immediately.

batch_multi_plot_spectrogram.log_message(message: str, force_flush: bool = False) → None

Queue an informational log message.

Messages are appended to an in-memory buffer; a flush occurs automatically once the configured batch size is reached or force_flush is True.

batch_multi_plot_spectrogram.make_spectrogram(x_axis_values, y_axis_values, data_array_3d, x_axis_min=None, x_axis_max=None, x_axis_is_unix=True, x_axis_label=None, center_timestamp=None, window_duration_seconds=None, y_axis_scale_function=None, y_axis_label=None, y_axis_min=0, y_axis_max=4000, z_axis_scale_function=None, z_axis_min=None, z_axis_max=None, z_axis_label=None, collapse_axis=1, colormap='viridis', axis_object=None, instrument_label=None, vertical_lines_unix=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Plot a spectrogram by collapsing a 3D data array along an axis.

Parameters:

• x_axis_values (array-like) – 1D array for x (horizontal) axis (e.g., time sequence).

• y_axis_values (array-like) – 1D array for y (vertical) axis (e.g., energy bins).

• data_array_3d (numpy.ndarray) – 3D data array, e.g. (time, angle/pitch, energy).

• x_axis_min (float, optional) – Explicit x-axis clipping bounds before plotting.

• x_axis_max (float, optional) – Explicit x-axis clipping bounds before plotting.

• x_axis_is_unix (bool, default True) – If True, x-axis treated as UNIX seconds and converted to dates.

• x_axis_label (str, optional) – Custom x-axis label (default depends on x_axis_is_unix).

• center_timestamp (float, optional) – Center of requested zoom window (UNIX seconds).

• window_duration_seconds (float, optional) – Duration of zoom window; both must be provided for zoom to apply.

• y_axis_scale_function ({'linear', 'log'}, optional) – Y-axis scaling; None behaves as 'linear'.

• y_axis_label (str, optional) – Y-axis label text.

• y_axis_min (float, default 0, 4000) – Y-axis clipping range applied before filtering / plotting.

• y_axis_max (float, default 0, 4000) – Y-axis clipping range applied before filtering / plotting.

• z_axis_scale_function ({'linear', 'log'}, optional) – Color scale mode; None behaves as 'linear'.

• z_axis_min (float, optional) – Optional color scale bounds (percentiles chosen if omitted).

• z_axis_max (float, optional) – Optional color scale bounds (percentiles chosen if omitted).

• z_axis_label (str, optional) – Colorbar label text.

• collapse_axis (int, default 1) – Axis index along which to collapse the 3D data array.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• axis_object (matplotlib.axes.Axes, optional) – Existing axes to draw into; if None a new figure/axes created.

• instrument_label (str, optional) – Title string applied to the axes.

• vertical_lines_unix (list of float, optional) – UNIX timestamps to annotate with a cusp-boundary marker.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style for vertical_lines_unix: 'line' reproduces the original double-line marker; 'bracket' draws a bracket spanning the boundary interval below the axis instead; 'both' draws both styles together.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the selected marker-drawing function (see configurable_spectrograms.cusp_marking).

Returns:

• axis_object (matplotlib.axes.Axes or None) – The axis object used for plotting (None if no data plotted).

• x_axis_plot (numpy.ndarray or None) – X values actually used (possibly filtered / converted), or None if skipped.

Plots a folder of FAST ESA data as spectrograms.

Assumed folder layout is::

{FAST_CDF_DATA_FOLDER_PATH}/year/month

Filenames in the month folders assumed to be in the following formats::

{??}_{??}_{??}_{instrument}_{timestamp}_{orbit}_v02.cdf (known “instruments” are ees, eeb, ies, or ieb) {??}_{??}_orb_{orbit}_{??}.cdf

Examples::

FAST_data/2000/01/fa_esa_l2_eeb_20000101001737_13312_v02.cdf FAST_data/2000/01/fa_k0_orb_13312_v01.cdf

All FAST-specific plotting/batch logic lives in configurable_spectrograms.fast; this module re-exports the public functions/constants for backward compatibility and provides the CLI entry point that runs every y/z scale combination in sequence.

batch_multi_plot_FAST_spectrograms.FAST_plot_instrument_grid(cdf_file_paths: dict[str, str], filtered_orbits_df=None, orbit_number: int | None = None, zoom_duration_minutes: float = 6.25, scale_function_y: str = 'linear', scale_function_z: str = 'linear', instrument_order: tuple[str, ...] = ('ees', 'eeb', 'ies', 'ieb'), show: bool = True, colormap: str = 'viridis', y_min: float | None = None, y_max: float | None = None, z_min: float | None = None, z_max: float | None = None, global_extrema: dict[str, int | float] | None = None, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → tuple[Any, Any]

Plot a multi-instrument ESA spectrogram grid for a single orbit.

Loads each instrument CDF, collapses across pitch-angle, and constructs datasets for generic_plot_multirow_optional_zoom. A zoom column is included when vertical lines are available for the orbit.

Parameters:

• cdf_file_paths (dict of {str: str}) – Mapping of instrument key ('ees', 'eeb', 'ies', 'ieb') to CDF file path. Missing instruments are skipped.

• filtered_orbits_df (pandas.DataFrame or None) – DataFrame for vertical line computation; None omits lines.

• orbit_number (int or None) – Orbit identifier used in titles and vertical lines.

• zoom_duration_minutes (float, default 6.25) – Zoom window length (minutes).

• scale_function_y ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• scale_function_z ({'linear', 'log'}, default 'linear') – Color scale for intensity.

• instrument_order (tuple of str, default DEFAULT_INSTRUMENT_ORDER) – Display order of instrument rows.

• show (bool, default True) – Whether to show the figure interactively.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• y_min (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• y_max (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• z_min (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• z_max (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• global_extrema (dict or None) – Precomputed extrema keyed as {instrument}_{y_scale}_{z_scale}_{axis}_{min|max} supplying per-instrument limits. Takes precedence over the direct y_min / y_max / z_min / z_max arguments.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style; see configurable_spectrograms.cusp_marking.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Figure and Canvas, or (None, None) when no datasets are produced.

Return type:

tuple[Figure or None, FigureCanvasBase or None]

Notes

• Files that fail to load are logged and skipped.

• Energy bins are restricted to [0, 4000] unless overridden.

• vmin/vmax per row use 1st/99th percentiles unless global_extrema provides per-instrument z_min / z_max.

batch_multi_plot_FAST_spectrograms.FAST_plot_pitch_angle_grid(cdf_file_path: str, filtered_orbits_df=None, orbit_number: int | None = None, zoom_duration_minutes: float = 6.25, scale_function_y: str = 'linear', scale_function_z: str = 'linear', pitch_angle_categories: dict[str, list[tuple[float, float]]] | None = None, show: bool = True, colormap: str = 'viridis', y_min: float | None = None, y_max: float | None = None, z_min: float | None = None, z_max: float | None = None, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → tuple[Any, Any]

Plot a grid of ESA spectrograms collapsed by pitch-angle categories.

Each row corresponds to a pitch-angle category (e.g. downgoing, upgoing, perpendicular, all). If orbit boundary timestamps are available a zoom column is added. Data are collapsed over pitch-angle via FAST_COLLAPSE_FUNCTION (np.nansum by default).

Parameters:

• cdf_file_path (str) – Path to the instrument CDF file.

• filtered_orbits_df (pandas.DataFrame or None) – DataFrame used to compute vertical lines; if None, lines are omitted.

• orbit_number (int or None) – Orbit number used to label vertical lines.

• zoom_duration_minutes (float, default 6.25) – Window length (minutes) for the optional zoom column.

• scale_function_y ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• scale_function_z ({'linear', 'log'}, default 'linear') – Color scale for intensity.

• pitch_angle_categories (dict or None) – Mapping of label -> list of (min_deg, max_deg) ranges; defaults to the four standard groups when None.

• show (bool, default True) – If True, display the figure interactively.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• y_min (float or None, optional) – Energy (y-axis) limits; defaults to [0, 4000] when None.

• y_max (float or None, optional) – Energy (y-axis) limits; defaults to [0, 4000] when None.

• z_min (float or None, optional) – Color scale limits; defaults to row-level 1st/99th percentiles when None.

• z_max (float or None, optional) – Color scale limits; defaults to row-level 1st/99th percentiles when None.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style; see configurable_spectrograms.cusp_marking.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Figure and Canvas, or (None, None) when no datasets are produced.

Return type:

tuple[Figure or None, FigureCanvasBase or None]

batch_multi_plot_FAST_spectrograms.FAST_plot_spectrograms_directory(directory_path: str = './FAST_data/', output_base: str = './FAST_plots/', y_scale: str = 'linear', z_scale: str = 'log', zoom_duration_minutes: float = 6, instrument_order: tuple[str, ...] = ('ees', 'eeb', 'ies', 'ieb'), verbose: bool = True, progress_json_path: str | None = './batch_multi_plot_FAST_progress.json', ignore_progress_json: bool = False, use_tqdm: bool | None = None, colormap: str = 'viridis', cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None, max_workers: int = 4, orbit_timeout_seconds: int | float = 60, instrument_timeout_seconds: int | float = 30, retry_timeouts: bool = True, flush_batch_size: int = 10, log_flush_batch_size: int | None = None, max_processing_percentile: float | None = None, override_plots: bool = True) → list[dict[str, Any]]

Batch process ESA spectrogram plots for all orbits in a directory.

Discovers instrument CDF files (excluding _orb_), groups them by orbit, and processes each orbit in parallel worker processes (safe for matplotlib). Progress is persisted to a JSON file to support resumable runs. When max_processing_percentile is not None, a global extrema pass runs first (configurable_spectrograms.fast.extrema.compute_global_extrema()) and both raw and given-extrema plots are saved; otherwise only raw plots are produced.

Parameters:

• directory_path (str, default FAST_CDF_DATA_FOLDER_PATH) – Root folder containing CDF files.

• output_base (str, default FAST_OUTPUT_BASE) – Base output directory; plots are saved under output_base/year/month/orbit.

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• z_scale ({'linear', 'log'}, default 'log') – Color scale for intensity.

• zoom_duration_minutes (float, default DEFAULT_ZOOM_WINDOW_MINUTES) – Zoom window length for zoom columns.

• instrument_order (tuple of str, default ('ees', 'eeb', 'ies', 'ieb')) – Display order for the instrument grid.

• verbose (bool, default True) – Print additional batch messages when True.

• progress_json_path (str or None, default FAST_PLOTTING_PROGRESS_JSON) – Path to persist progress across runs; None disables persistence.

• ignore_progress_json (bool, default False) – If True, do not read existing progress before starting.

• use_tqdm (bool or None, default None) – Show a tqdm progress bar when True; defaults to False when None.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to every orbit’s plots.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

• max_workers (int, default 4) – Max number of worker processes.

• orbit_timeout_seconds (int or float, default 60) – Total per-orbit timeout (seconds).

• instrument_timeout_seconds (int or float, default 30) – Per-instrument/grid timeout (seconds).

• retry_timeouts (bool, default True) – If True, retry timed-out orbits once with a smaller pool.

• flush_batch_size (int, default 10) – Orbit completions between progress/extrema JSON writes. Values < 1 become 1. Final partial batch always flushes.

• log_flush_batch_size (int or None, default None) – Logging buffer batch size; defaults to flush_batch_size when None.

• max_processing_percentile (float or None, default None) – Percentile (0-100] for pooled intensity (Z) maxima in compute_global_extrema. None skips the extrema pass and raw-only plots are produced. Energy (Y) maxima use a fixed 99% cumulative coverage rule regardless.

• override_plots (bool, default True) – If False, skip plots whose output file already exists.

Returns:

Result dictionaries from FAST_process_single_orbit (and retries).

Return type:

list of dict

Raises:

KeyboardInterrupt – Re-raised on SIGINT/SIGTERM so the caller can stop multi-combo loops.

Notes

• Progress JSON key f"progress_{y_scale}_{z_scale}_last_orbit" tracks the last completed orbit; error/timeout orbits are recorded under dedicated keys (including per-instrument).

• Signal handlers terminate child processes and raise KeyboardInterrupt to interrupt the main wait loop immediately.

batch_multi_plot_FAST_spectrograms.FAST_process_single_orbit(orbit_number: int, instrument_file_paths: dict[str, str], filtered_orbits_dataframe, zoom_duration_minutes: float, y_axis_scale: str, z_axis_scale: str, instrument_order: tuple[str, ...], colormap: str, output_base_directory: str, orbit_timeout_seconds: int | float = 60, instrument_timeout_seconds: int | float = 30, global_extrema: dict[str, int | float] | None = None, override_plots: bool = True, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → dict[str, Any]

Process and save all ESA spectrogram plots for a single orbit.

For each available instrument, generates two plot versions: 1. Using global_extrema (_given_extrema suffix) if provided. 2. Raw (per-file) extrema (_raw suffix). This applies to both pitch-angle and instrument-grid plots. Each figure is saved to disk and closed immediately after it’s rendered, so at most one or two figures are ever held in memory at once regardless of how many instruments and extrema variants an orbit produces.

Parameters:

• orbit_number (int) – The orbit identifier.

• instrument_file_paths (dict of {str: str}) – Mapping of instrument key to CDF file path.

• filtered_orbits_dataframe (pandas.DataFrame) – DataFrame used to compute orbit boundary timestamps.

• zoom_duration_minutes (float) – Zoom window length for zoomed plots.

• y_axis_scale ({'linear', 'log'}) – Y-axis scaling.

• z_axis_scale ({'linear', 'log'}) – Color scale for intensity.

• instrument_order (tuple of str) – Order used in the instrument grid.

• colormap (str) – Matplotlib colormap.

• output_base_directory (str) – Root folder for saving figures; year/month are inferred from the CDF path when possible, else 'unknown'.

• orbit_timeout_seconds (int or float, default 60) – Maximum wall-clock seconds for the entire orbit.

• instrument_timeout_seconds (int or float, default 30) – Per-instrument/grid timeout.

• global_extrema (dict or None) – Precomputed extrema mapping (from compute_global_extrema).

• override_plots (bool, default True) – If False, skip plotting when the output file already exists.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to the plotting functions.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Result with keys orbit (int), status ('ok', 'error', or 'timeout'), errors (list of str), and optionally timeout_type / timeout_instrument.

Return type:

dict

Notes

Each figure is saved and closed as soon as it’s rendered, so a mid-orbit timeout can leave some of an orbit’s PNGs already written to disk. This is safe given deterministic output filenames: a subsequent retry of a timed-out orbit simply overwrites the partial set.

batch_multi_plot_FAST_spectrograms.compute_global_extrema(directory_path: str, y_scale: str, z_scale: str, instrument_order: Iterable[str], extrema_json_path: str = './FAST_calculated_extrema.json', compute_mins: bool = False, max_percentile: float = 95.0, log_floor_cutoff: float = 0.1, log_floor_value: float = -1.0, flush_batch_size: int = 10) → dict[str, Any]

Compute (or incrementally update) cached axis extrema per instrument.

Performs a resumable pass over all instrument CDF files, flushing incremental progress to extrema_json_path after each flush_batch_size orbits.

Extrema logic

• Y (energy) minima are fixed to 0 unless compute_mins is True.

• Linear Y maxima: smallest energy whose cumulative positive finite count reaches 99% of total positive finite samples.

• Linear Z maxima: max_percentile-th percentile of pooled positive finite intensity samples.

• If the requested scale is log and linear_linear extrema already exist in the cache, they are log-transformed without re-scanning files. If the requested scale is linear and linear_linear extrema exist, they are copied directly.

• Log transform applies a floor: values <= log_floor_cutoff or non-finite are replaced by log_floor_value.

• Maxima are monotonically non-decreasing across incremental updates; energy maxima are capped at 4000.

param directory_path:

Root directory containing instrument CDF files.

type directory_path:

str

param y_scale:

Y scaling label (used for cache key names).

type y_scale:

{‘linear’, ‘log’}

param z_scale:

Z scaling label (used for cache key names).

type z_scale:

{‘linear’, ‘log’}

param instrument_order:

Instruments to process (e.g., ("ees", "eeb", "ies", "ieb")).

type instrument_order:

iterable of str

param extrema_json_path:

Path to the JSON cache file (created if absent).

type extrema_json_path:

str, default FAST_EXTREMA_JSON_PATH

param compute_mins:

If True, compute intensity minima; otherwise they are set to 0.

type compute_mins:

bool, default False

param max_percentile:

Percentile applied to pooled positive intensity for z_max.

type max_percentile:

float, default 95.0

param log_floor_cutoff:

Values at or below this threshold map to log_floor_value in log space.

type log_floor_cutoff:

float, default 0.1

param log_floor_value:

Floor value substituted for invalid log-domain extrema.

type log_floor_value:

float, default -1.0

param flush_batch_size:

Orbits with updates between JSON flushes; coerced to >= 1.

type flush_batch_size:

int, default 10

returns:

Updated extrema mapping containing values and progress entries.

rtype:

dict

batch_multi_plot_FAST_spectrograms.extract_orbit_and_instrument(cdf_path: str) → tuple[int, str, str] | None

Parse a CDF filename to (orbit_number, instrument_type, cdf_path).

Parameters:

cdf_path (str) – Path (or bare filename) of the CDF file.

Returns:

(orbit_number, instrument_type, cdf_path), or None when the filename does not match the expected pattern, the orbit number cannot be parsed, or the instrument type is None or 'orb'.

Return type:

tuple or None

Examples

>>> extract_orbit_and_instrument("fa_esa_l2_eeb_20000101001737_13312_v02.cdf")
(13312, 'eeb', 'fa_esa_l2_eeb_20000101001737_13312_v02.cdf')
>>> extract_orbit_and_instrument("fa_k0_orb_13312_v01.cdf") is None
True

batch_multi_plot_FAST_spectrograms.round_extrema(value: float | int, direction: str) → float

Round an extrema value to a clean significant-digit axis limit.

Rounds to the next significant digit in the specified direction so plot axis limits look consistent (e.g. 1234 -> 1300 for ‘up’).

Parameters:

• value (float or int) – Extrema value. Zero returns 0.0.

• direction ({'up', 'down'}) – Round up (for maxima) or down (for minima).

Return type:

float

Raises:

ValueError – If direction is not 'up' or 'down'.

Examples

>>> round_extrema(1234, 'up')
1300.0
>>> round_extrema(0.0123, 'down')
0.012

CLI to render a single generic spectrogram figure from one CDF file.

Companion to batch_multi_plot_spectrogram.py: where that script (via configurable_spectrograms.generic_batch) renders many items in parallel, this script renders exactly one item and exits – no ProcessPoolExecutor, no progress JSON. All rendering logic lives in configurable_spectrograms; this script only parses arguments, loads one CDF file, and calls the library.

single_plot_spectrogram.main() → int

Parse CLI arguments and render a single generic spectrogram figure.

single_plot_spectrogram.render_single_spectrogram(cdf_file_path: str, output_path: str, y_scale: str = 'linear', z_scale: str = 'linear', colormap: str = 'viridis', cusp_marker_style: str = 'both', vertical_lines: list[float] | None = None) → bool

Render a single generic spectrogram from one CDF file and save it.

Parameters:

• cdf_file_path (str) – Path to the CDF file (must contain time_unix, data, energy, and pitch_angle variables).

• output_path (str) – Destination PNG path.

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• z_scale ({'linear', 'log'}, default 'linear') – Color scale for intensity.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style; see configurable_spectrograms.cusp_marking.

• vertical_lines (list of float or None, optional) – UNIX timestamps to mark as a cusp boundary.

Returns:

True if a figure was produced and saved, False otherwise.

Return type:

bool

CLI to render a single FAST ESA spectrogram figure (one CDF file, or one orbit).

Companion to batch_multi_plot_FAST_spectrograms.py: where that script (via configurable_spectrograms.fast.batch_directory) processes every orbit in a directory in parallel, this script renders exactly one figure and exits. All rendering logic lives in configurable_spectrograms.fast.plotting; this script only parses arguments and calls it – the same function the GUI’s Single Plot page calls.

single_plot_FAST_spectrograms.main() → int

Parse CLI arguments and render a single FAST ESA spectrogram figure.

single_plot_FAST_spectrograms.render_single_instrument_grid(data_folder: str, orbit_number: int, output_path: str, y_scale: str = 'linear', z_scale: str = 'linear', colormap: str = 'viridis', cusp_marker_style: str = 'both') → bool

Render one orbit’s multi-instrument grid resolved from a data folder.

Parameters:

• data_folder (str) – Root folder to search for the orbit’s instrument CDF files.

• orbit_number (int) – Orbit number to resolve within data_folder.

• output_path (str) – Destination PNG path.

• y_scale ({'linear', 'log'}, default 'linear') – Axis scaling.

• z_scale ({'linear', 'log'}, default 'linear') – Axis scaling.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style.

Returns:

True if a figure was produced and saved, False otherwise.

Return type:

bool

single_plot_FAST_spectrograms.render_single_pitch_angle_grid(cdf_file_path: str, output_path: str, y_scale: str = 'linear', z_scale: str = 'linear', colormap: str = 'viridis', cusp_marker_style: str = 'both') → bool

Render one CDF file’s pitch-angle grid and save it to output_path.

The orbit number (used to look up the cusp boundary) is parsed automatically from the filename.

Parameters:

• cdf_file_path (str) – Path to the instrument CDF file.

• output_path (str) – Destination PNG path.

• y_scale ({'linear', 'log'}, default 'linear') – Axis scaling.

• z_scale ({'linear', 'log'}, default 'linear') – Axis scaling.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style.

Returns:

True if a figure was produced and saved, False otherwise.

Return type:

bool

Library: configurable_spectrograms

Shared constants for spectrogram plotting and batch processing.

configurable_spectrograms.constants.CDF_DATA_DIRECTORY = './FAST_data/'

Directory containing CDF data files.

configurable_spectrograms.constants.CDF_VARIABLE_NAMES = ['time_unix', 'data', 'energy', 'pitch_angle']

List of variable names expected in CDF files.

configurable_spectrograms.constants.COLLAPSE_FUNCTION

Function used to collapse a 3D data array down to 2D (e.g. sum over pitch angle).

configurable_spectrograms.constants.FILTERED_ORBITS_CSV_PATH = './FAST_Cusp_Indices.csv'

Path to the filtered cusp orbits CSV.

configurable_spectrograms.constants.OUTPUT_BASE_DIRECTORY = './plots/'

Parent directory for generic batch-plot output.

configurable_spectrograms.constants.PLOTTING_PROGRESS_JSON_PATH = './batch_multi_plot_progress.json'

Path to JSON tracking generic batch-plotting progress across sessions.

Buffered logging shared by the generic and FAST batch/plotting pipelines.

Log messages are queued in memory and flushed to disk in batches to avoid a disk write per message during large batch runs. The destination file is set explicitly via set_logfile_path() (typically once, from a CLI’s main()) rather than resolved as a side effect of importing this module, so importing the library never touches the filesystem.

configurable_spectrograms.logging_utils.configure_log_batch(batch_size: int) → None

Configure buffered logging batch size.

Parameters:

batch_size (int) – Desired number of log records to accumulate before an automatic flush. Values less than 1 are coerced to 1.

configurable_spectrograms.logging_utils.flush_log_buffer(force: bool = True) → None

Publicly flush any buffered log messages to disk (see log_message()).

configurable_spectrograms.logging_utils.get_logfile_path(prefix: str, datetime_marker_path: str) → str

Return a persistent per-run log file path derived from a marker file.

The marker file at datetime_marker_path holds a timestamp string that is created on first use and reused afterwards, so repeated runs of the same pipeline share one logfile instead of minting a new one on every call.

Parameters:

• prefix (str) – Filename prefix for the resulting log path (e.g. './batch_log').

• datetime_marker_path (str) – Path to a small text file used to persist the timestamp marker.

Returns:

Log file path of the form f"{prefix}_{timestamp}.log".

Return type:

str

configurable_spectrograms.logging_utils.log_error(message: str, force_flush: bool = False) → None

Queue an error log message and echo it to the console immediately.

configurable_spectrograms.logging_utils.log_exception(prefix: str, exception: BaseException | None = None, level: str = 'error', include_trace: bool = False, force_flush: bool = False) → None

Log a message, optionally with an exception and traceback.

Parameters:

• prefix (str) – Human-readable message prefix.

• exception (BaseException or None, optional) – Optional exception; if given, its class name and value are appended.

• level ({'error', 'message'}, default 'error') – 'error' routes to log_error(); anything else routes to log_message().

• include_trace (bool, default False) – If True and an exception is given, also log a formatted traceback.

• force_flush (bool, default False) – Force an immediate buffer flush after logging this message (and the traceback, if any).

configurable_spectrograms.logging_utils.log_message(message: str, force_flush: bool = False) → None

Queue an informational log message.

Messages are appended to an in-memory buffer; a flush occurs automatically once the configured batch size is reached or force_flush is True.

configurable_spectrograms.logging_utils.set_logfile_path(path: str | None) → None

Set the destination file that buffered log flushes are written to.

Process-tree management helpers used during batch shutdown handling.

configurable_spectrograms.process_utils.terminate_all_child_processes() → None

Best-effort terminate all child processes of the current process.

Uses psutil (imported lazily, since it’s only needed during shutdown) to enumerate child processes recursively and invoke terminate() on each. Exceptions are suppressed throughout because this function is used during best-effort shutdown handling where a single unkillable or already-dead child must not block the rest.

Return type:

None

CDF file discovery, metadata, and dataset-loading helpers.

Shared by both the generic and FAST-specific plotting/batch pipelines so that file-type detection, orbit-boundary lookup, and CDF loading logic exists in exactly one place.

configurable_spectrograms.cdf_utils.get_cdf_file_type(cdf_file_path: str) → str | None

Infer instrument type from a CDF file path.

Parameters:

cdf_file_path (str) – Path to the CDF file.

Returns:

Instrument type string (e.g. 'ees'), 'orb' for orbit files, or None if not recognized.

Return type:

str or None

Examples

>>> get_cdf_file_type("fa_esa_l2_eeb_20000101001737_13312_v02.cdf")
'eeb'
>>> get_cdf_file_type("fa_k0_orb_13312_v01.cdf")
'orb'

configurable_spectrograms.cdf_utils.get_cdf_var_shapes(cdf_folder_path: str = './FAST_data/', variable_names: list[str] = ['time_unix', 'data', 'energy', 'pitch_angle']) → dict[str, list[tuple[int, ...] | None]]

Collect shapes of variables across CDF files in a folder.

Parameters:

• cdf_folder_path (str, default CDF_DATA_DIRECTORY) – Directory containing CDF files.

• variable_names (list of str, default CDF_VARIABLE_NAMES) – Variable names to inspect.

Returns:

Mapping from variable name (str) to a list of shape tuples (or None) per file.

Return type:

dict

configurable_spectrograms.cdf_utils.get_timestamps_for_orbit(filtered_orbits_dataframe: pd.DataFrame | None, orbit_number: int, instrument_type: str | None, time_unix_array: np.ndarray | None) → list[float]

Compute orbit boundary UNIX timestamps from filtered indices.

Parameters:

• filtered_orbits_dataframe (pandas.DataFrame or None) – DataFrame containing filtered orbits and min/max indices per instrument.

• orbit_number (int) – Orbit number to look up.

• instrument_type (str or None) – Instrument type identifier (e.g. 'ees', 'ies').

• time_unix_array (numpy.ndarray or None) – 1D array of UNIX timestamps for the instrument.

Returns:

Boundary UNIX timestamps for the orbit: one value when the CSV row gives a degenerate (equal) min/max index, two values (start, end) otherwise. Returns an empty list when the orbit is not found or inputs are missing.

Return type:

list of float

Examples

>>> import pandas as pd
>>> import numpy as np
>>> orbits = pd.DataFrame({"orbit": [42], "ees min index": [1], "ees max index": [3]})
>>> times = np.array([100.0, 200.0, 300.0, 400.0])
>>> get_timestamps_for_orbit(orbits, 42, "ees", times)
[200.0, 400.0]
>>> get_timestamps_for_orbit(orbits, 99, "ees", times)
[]

configurable_spectrograms.cdf_utils.get_variable_shape(cdf_path: str, variable_name: str) → tuple[int, ...] | None

Return the shape of a variable in a CDF file.

Parameters:

• cdf_path (str) – Path to the CDF file.

• variable_name (str) – Variable name to inspect.

Returns:

Variable shape tuple, or None if the variable is absent, not an array, or an error occurs.

Return type:

tuple or None

configurable_spectrograms.cdf_utils.load_fast_cdf_dataset(cdf_path: str, variable_names: tuple[str, ...] = ('time_unix', 'data', 'energy', 'pitch_angle')) → dict[str, numpy.ndarray]

Load and reshape a FAST CDF file’s time/data/energy/pitch-angle arrays.

Energy and pitch-angle variables are collapsed from their raw (time, angle, energy) or (time, energy, angle) storage down to 1D bin arrays, and data is transposed to (time, pitch_angle, energy) order when needed, so the result is ready to collapse along pitch angle for a spectrogram.

Parameters:

• cdf_path (str) – Path to the instrument CDF file.

• variable_names (tuple of str, default CDF_VARIABLE_NAMES) – Names of the (time, data, energy, pitch_angle) variables, in that order.

Returns:

Mapping with keys 'times', 'data', 'energy', 'pitch_angle'.

Return type:

dict

configurable_spectrograms.cdf_utils.load_filtered_orbits(csv_path: str = './FAST_Cusp_Indices.csv') → pd.DataFrame | None

Load the filtered orbits CSV with a simple cache.

Parameters:

csv_path (str, default FILTERED_ORBITS_CSV_PATH) – Path to the filtered orbits TSV/CSV file.

Returns:

DataFrame of filtered orbits, or None if loading fails.

Return type:

pandas.DataFrame or None

Notes

A module-level dictionary caches previously loaded DataFrames keyed by the path string to avoid repeated disk I/O in batch routines.

Axis-extrema rounding and percentile-bound computation for color scales.

configurable_spectrograms.percentile_utils.compute_percentile_bounds(matrix: numpy.ndarray, low_percentile: float = 1, high_percentile: float = 99, z_min: float | None = None, z_max: float | None = None) → tuple[float, float]

Return (z_min, z_max) color-scale bounds for a data matrix.

Explicit z_min/z_max values are used as-is when given; otherwise each bound is computed independently via numpy.nanpercentile. This unifies the vmin/vmax percentile logic that plotting functions need when the caller hasn’t supplied fixed bounds.

Parameters:

• matrix (numpy.ndarray) – Data array (NaNs ignored).

• low_percentile (float, default 1) – Percentile used for the lower bound when z_min is None.

• high_percentile (float, default 99) – Percentile used for the upper bound when z_max is None.

• z_min (float or None, optional) – Explicit lower bound; overrides low_percentile when given.

• z_max (float or None, optional) – Explicit upper bound; overrides high_percentile when given.

Returns:

(z_min, z_max).

Return type:

tuple of float

Examples

>>> import numpy as np
>>> compute_percentile_bounds(np.array([[1.0, 2.0, 3.0, 100.0]]), 0, 100)
(1.0, 100.0)
>>> compute_percentile_bounds(np.array([1.0, 2.0, 3.0]), z_min=-5.0, z_max=5.0)
(-5.0, 5.0)

configurable_spectrograms.percentile_utils.round_extrema(value: float | int, direction: str) → float

Round an extrema value to a clean significant-digit axis limit.

Rounds to the next significant digit in the specified direction so plot axis limits look consistent (e.g. 1234 -> 1300 for ‘up’).

Parameters:

• value (float or int) – Extrema value. Zero returns 0.0.

• direction ({'up', 'down'}) – Round up (for maxima) or down (for minima).

Return type:

float

Raises:

ValueError – If direction is not 'up' or 'down'.

Examples

>>> round_extrema(1234, 'up')
1300.0
>>> round_extrema(0.0123, 'down')
0.012

Cusp-boundary markers drawn onto a spectrogram axis.

Three interchangeable styles are provided: the original double-line marker (draw_cusp_line_markers()), a bracket marker (draw_cusp_bracket_marker()) that spans the cusp interval below the axis instead of drawing lines through the data, and a combination of both (draw_cusp_both_markers()).

configurable_spectrograms.cusp_marking.draw_cusp_both_markers(axis_object, marker_positions_plot, **kwargs) → list

Draw both the line and bracket cusp-boundary markers at once.

Combines draw_cusp_line_markers() and draw_cusp_bracket_marker() at the same marker positions, so the boundary is both drawn through the data (visible without needing to look below the axis) and bracketed (showing the interval clearly even where the line style is hard to distinguish from the surrounding data).

Parameters:

• axis_object (matplotlib.axes.Axes) – Axes to draw onto.

• marker_positions_plot (list of float) – X positions, already converted to the axes’ plotting units.

• **kwargs – Forwarded to both draw_cusp_line_markers() and draw_cusp_bracket_marker(); each ignores keyword arguments it doesn’t recognize (e.g. line_color is used only by the line markers, color/bracket_y/etc. only by the bracket marker).

Returns:

The combined matplotlib artists from both drawing functions.

Return type:

list

configurable_spectrograms.cusp_marking.draw_cusp_bracket_marker(axis_object, marker_positions_plot, color: str = 'black', bracket_y: float = -0.08, bracket_tick_height: float = 0.02, caption: str | None = None, caption_offset: float = 0.04, caption_fontsize: float | None = None, linewidth: float = 1.5, **kwargs) → list

Draw a bracket spanning the cusp interval below the axis.

An alternative to draw_cusp_line_markers() that brackets the cusp region rather than drawing lines through the plotted data, using the axes’ x-data / y-axes-fraction transform so the bracket sits at a fixed relative offset below the axis regardless of the data’s y-range.

When two or more marker positions are given, the bracket spans the interval (min(marker_positions_plot), max(marker_positions_plot)). When exactly one position is given (no true interval to bracket), a single vertical tick is drawn at that position instead.

Parameters:

• axis_object (matplotlib.axes.Axes) – Axes to draw onto.

• marker_positions_plot (list of float) – X positions, already converted to the axes’ plotting units.

• color (str, default 'black') – Line color.

• bracket_y (float, default -0.08) – Y position of the bracket’s horizontal bar, in axes-fraction coordinates (negative values sit below the axis).

• bracket_tick_height (float, default 0.02) – Height of the vertical end-ticks, in axes-fraction coordinates.

• caption (str or None, optional) – Caption text centered below the bracket.

• caption_offset (float, default 0.04) – Additional axes-fraction offset below bracket_y for the caption.

• caption_fontsize (float or None, optional) – Caption font size; uses the matplotlib default when None.

• linewidth (float, default 1.5) – Bracket line width.

• **kwargs – Accepted and ignored, so callers can pass a single **style_kwargs dict regardless of which marker style is selected (e.g. the line_color used by draw_cusp_line_markers()).

Returns:

The matplotlib artists created: one Line2D, plus a Text when caption is given. Empty when marker_positions_plot is empty.

Return type:

list

Notes

The default offsets are deliberately small so the bracket clears the x-axis without colliding with the tick labels or x-axis label in this codebase’s default figure sizes, and so stacked multi-row grids don’t have one row’s bracket overlap the row below it. A caption, or a larger bracket_y magnitude, may need extra bottom margin reserved by the caller (e.g. via fig.subplots_adjust / fig.tight_layout(rect=...)) to avoid overlapping nearby text.

configurable_spectrograms.cusp_marking.draw_cusp_line_markers(axis_object, marker_positions_plot, line_color: str = 'red', **kwargs) → list

Draw a thick black line under a thinner coloured line at each marker position.

This is the original cusp-boundary marker style: for each position in marker_positions_plot a 4-pixel-wide black line is drawn first, followed by a 2-pixel-wide line of line_color on top, so the boundary remains visible against both light and dark spectrogram data.

Parameters:

• axis_object (matplotlib.axes.Axes) – Axes to draw onto.

• marker_positions_plot (list of float) – X positions, already converted to the axes’ plotting units, marking cusp boundaries.

• line_color (str, default 'red') – Color of the thinner top line; callers typically switch this to a colormap-appropriate color (e.g. white on top of 'turbo', whose high end is already red) so it stays visible.

• **kwargs – Accepted and ignored, so callers can pass a single **style_kwargs dict regardless of which marker style is selected.

Returns:

The matplotlib Line2D artists created (two per marker position).

Return type:

list

Single-output spectrogram rendering.

These functions render one figure (or one panel of a figure) for a single item – a single CDF, orbit, or caller-supplied dataset. Batch/loop callers (configurable_spectrograms.generic_batch, configurable_spectrograms.fast.process_orbit) call these same functions once per item rather than re-implementing rendering logic, so a single-plot CLI script and a batch driver always produce identical output for identical inputs.

configurable_spectrograms.plotting.close_all_axes_and_clear(fig) → None

Close axes/subplots and clear a figure to free memory.

Parameters:

fig (matplotlib.figure.Figure) – Figure instance to clear and dispose.

Return type:

None

Notes

Ensures axes are deleted, the canvas is closed/detached, and removes the figure from the global Gcf registry when possible to mitigate memory growth during large batch operations.

configurable_spectrograms.plotting.generic_plot_multirow_optional_zoom(datasets, vertical_lines=None, zoom_duration_minutes=6.25, y_scale='linear', z_scale='linear', colormap='viridis', show=False, title=None, row_label_pad=50, row_label_rotation=90, y_min=None, y_max=None, z_min=None, z_max=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Render a multi-row spectrogram grid with an optional zoom column.

Parameters:

• datasets (list of dict) –

  Each dict must contain keys:

  • 'x' – 1D UNIX epoch seconds (float) array

  • 'y' – 1D energy (eV) array (unfiltered, 0-4000 typical)

  • 'data' – 3D ndarray that can be collapsed (time, pitch/angle, energy)

  Optional per-row keys (all honored when present):

  • 'label' – Row label placed on the left (rotated)

  • 'y_label' – Units label for y-axis (default: 'Energy (eV)')

  • 'z_label' – Color scale label (default: 'Counts')

  • 'y_min' / 'y_max' – Energy bounds (overrides global y_min / y_max args)

  • 'z_min' / 'z_max' – Color bounds (overrides global z_min / z_max args)

  • 'vmin' / 'vmax' – Precomputed percentile (or fixed) color bounds used when z_min / z_max not provided.

• vertical_lines (list of float, optional) – UNIX timestamps defining the cusp boundary and potential zoom window.

• zoom_duration_minutes (float, default 6.25) – Desired zoom window length in minutes (may auto-expand to include full marked span).

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• z_scale ({'linear', 'log'}, default 'linear') – Color (intensity) scale.

• colormap (str, default 'viridis') – Matplotlib colormap.

• show (bool, default False) – If True, display interactively.

• title (str, optional) – Figure suptitle.

• row_label_pad (int, default 50) – Padding for row labels.

• row_label_rotation (int, default 90) – Rotation angle (degrees) for row labels.

• y_min (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• y_max (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• z_min (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• z_max (float, optional) – Global override bounds applied uniformly when provided. Any per-row y_min / y_max / z_min / z_max in a dataset dict take precedence.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style forwarded to make_spectrogram().

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

(fig, canvas) or (None, None) if datasets is empty.

Return type:

tuple

Notes

Determines need for a zoom column dynamically: only rendered if at least one dataset contains non-NaN values inside the computed zoom window.

configurable_spectrograms.plotting.generic_plot_spectrogram_set(datasets, collapse_axis=1, zoom_center=None, zoom_window_seconds=None, vertical_lines=None, x_is_unix=True, y_scale='linear', z_scale='linear', colormap='viridis', figure_title=None, show=False, y_min=None, y_max=None, z_min=None, z_max=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Plot a vertical stack of generic spectrograms.

Parameters:

• datasets (list of dict) – Each dict requires keys 'x', 'y', 'data' and may include optional keys: 'label', 'y_label', 'z_label', 'y_min', 'y_max', 'z_min', 'z_max'.

• collapse_axis (int, default 1) – Axis index of the 3D array collapsed prior to plotting.

• zoom_center (float, optional) – Center (UNIX time) for zoom column when used.

• zoom_window_seconds (float, optional) – Duration of zoom window (seconds) when zoom_center provided.

• vertical_lines (list of float, optional) – UNIX timestamps to annotate with a cusp-boundary marker.

• x_is_unix (bool, default True) – If True, x values are treated as UNIX seconds and formatted.

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling mode.

• z_scale ({'linear', 'log'}, default 'linear') – Color (intensity) scale mode.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• figure_title (str, optional) – Figure-level title (sup-title).

• show (bool, default False) – If True, display interactively (requires GUI backend).

• y_min (float, optional) – Global Y min fallback when per-row not supplied. Defaults to 0 if omitted and per-row missing.

• y_max (float, optional) – Global Y max fallback when per-row not supplied. If both global and per-row absent, inferred.

• z_min (float, optional) – Global colorbar lower bound fallback.

• z_max (float, optional) – Global colorbar upper bound fallback.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style forwarded to make_spectrogram().

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

(fig, canvas) or (None, None) if datasets is empty.

Return type:

tuple

configurable_spectrograms.plotting.make_spectrogram(x_axis_values, y_axis_values, data_array_3d, x_axis_min=None, x_axis_max=None, x_axis_is_unix=True, x_axis_label=None, center_timestamp=None, window_duration_seconds=None, y_axis_scale_function=None, y_axis_label=None, y_axis_min=0, y_axis_max=4000, z_axis_scale_function=None, z_axis_min=None, z_axis_max=None, z_axis_label=None, collapse_axis=1, colormap='viridis', axis_object=None, instrument_label=None, vertical_lines_unix=None, cusp_marker_style='both', cusp_marker_kwargs=None)

Plot a spectrogram by collapsing a 3D data array along an axis.

Parameters:

• x_axis_values (array-like) – 1D array for x (horizontal) axis (e.g., time sequence).

• y_axis_values (array-like) – 1D array for y (vertical) axis (e.g., energy bins).

• data_array_3d (numpy.ndarray) – 3D data array, e.g. (time, angle/pitch, energy).

• x_axis_min (float, optional) – Explicit x-axis clipping bounds before plotting.

• x_axis_max (float, optional) – Explicit x-axis clipping bounds before plotting.

• x_axis_is_unix (bool, default True) – If True, x-axis treated as UNIX seconds and converted to dates.

• x_axis_label (str, optional) – Custom x-axis label (default depends on x_axis_is_unix).

• center_timestamp (float, optional) – Center of requested zoom window (UNIX seconds).

• window_duration_seconds (float, optional) – Duration of zoom window; both must be provided for zoom to apply.

• y_axis_scale_function ({'linear', 'log'}, optional) – Y-axis scaling; None behaves as 'linear'.

• y_axis_label (str, optional) – Y-axis label text.

• y_axis_min (float, default 0, 4000) – Y-axis clipping range applied before filtering / plotting.

• y_axis_max (float, default 0, 4000) – Y-axis clipping range applied before filtering / plotting.

• z_axis_scale_function ({'linear', 'log'}, optional) – Color scale mode; None behaves as 'linear'.

• z_axis_min (float, optional) – Optional color scale bounds (percentiles chosen if omitted).

• z_axis_max (float, optional) – Optional color scale bounds (percentiles chosen if omitted).

• z_axis_label (str, optional) – Colorbar label text.

• collapse_axis (int, default 1) – Axis index along which to collapse the 3D data array.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• axis_object (matplotlib.axes.Axes, optional) – Existing axes to draw into; if None a new figure/axes created.

• instrument_label (str, optional) – Title string applied to the axes.

• vertical_lines_unix (list of float, optional) – UNIX timestamps to annotate with a cusp-boundary marker.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Marker style for vertical_lines_unix: 'line' reproduces the original double-line marker; 'bracket' draws a bracket spanning the boundary interval below the axis instead; 'both' draws both styles together.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the selected marker-drawing function (see configurable_spectrograms.cusp_marking).

Returns:

• axis_object (matplotlib.axes.Axes or None) – The axis object used for plotting (None if no data plotted).

• x_axis_plot (numpy.ndarray or None) – X values actually used (possibly filtered / converted), or None if skipped.

Executor-agnostic batch execution with resumable progress tracking.

run_batch() is the shared scaffolding (progress-JSON load/merge/flush, buffered-log flush cadence, as_completed loop, SIGINT handling) used by both the CPU-bound plotting batch driver (configurable_spectrograms.generic_batch.generic_batch_plot(), which supplies a ProcessPoolExecutor factory) and the I/O-bound download batch driver (configurable_spectrograms.download.download_cdf_files_threaded(), which supplies a ThreadPoolExecutor factory). Callers choose the concurrency primitive that matches their workload; this module only provides the orchestration around whichever executor they hand it.

configurable_spectrograms.batch_runner.run_batch(items: Iterable[Any], worker_fn: Callable[[Any], tuple[Any, str]], executor_factory: Callable[[], Executor], progress_json_path: str | None = None, ignore_progress_json: bool = False, flush_batch_size: int = 10, log_flush_batch_size: int | None = None, install_signal_handlers: bool = True) → list[tuple[Any, str]]

Run worker_fn over items in parallel with resumable progress tracking.

Parameters:

• items (iterable) – Iterable of item identifiers (any repr-able objects).

• worker_fn (callable) – Callable taking one item and returning (item, status), where status is a short label such as 'ok', 'no_data', or 'error'.

• executor_factory (callable) – Zero-argument callable returning a fresh concurrent.futures.Executor to use as a context manager (e.g. functools.partial(ProcessPoolExecutor, max_workers=4) for CPU-bound work, or functools.partial(ThreadPoolExecutor, max_workers=8) for I/O-bound work).

• progress_json_path (str or None, optional) – Path to a JSON file used for resumable progress tracking across runs. None disables persistence.

• ignore_progress_json (bool, default False) – If True, skip reading existing progress prior to execution.

• flush_batch_size (int, default 10) – Progress/log batch size; values less than 1 are coerced to 1. The final partial batch is always flushed.

• log_flush_batch_size (int or None, optional) – Explicit log batch size; if None, reuses flush_batch_size.

• install_signal_handlers (bool, default True) – When True, a temporary SIGINT handler is installed (and restored on exit) to enable graceful interruption with a final progress/log flush.

Returns:

Sequence of (item, status) results, one per submitted item.

Return type:

list of tuple

Notes

Items are identified via repr(item) for data-agnostic progress persistence, matching the pattern used across this codebase’s batch drivers.

Generic (data-agnostic) batch spectrogram plotting.

configurable_spectrograms.generic_batch.generic_batch_plot(items, output_dir: str, build_datasets_fn: Callable[[Any], list[dict]], zoom_center_fn: Callable[[Any], float | None] | None = None, zoom_window_seconds: float | None = None, vertical_lines_fn: Callable[[Any], list[float] | None] | None = None, y_scale: str = 'linear', z_scale: str = 'linear', colormap: str = 'viridis', cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None, max_workers: int = 2, progress_json_path: str = './batch_multi_plot_progress.json', ignore_progress_json: bool = False, flush_batch_size: int = 10, log_flush_batch_size: int | None = None, install_signal_handlers: bool = True) → list[tuple[Any, str]]

Generic batch runner for plotting datasets across many items in parallel.

Each item is rendered by calling configurable_spectrograms.plotting.generic_plot_spectrogram_set() exactly once, in a worker process managed by configurable_spectrograms.batch_runner.run_batch(), so a single item plotted through this batch driver produces the same output as calling the single-output function directly.

Parameters:

• items (iterable) – Iterable of item identifiers (any repr-able objects).

• output_dir (str) – Base output directory; plots saved under output_dir/<item>/generic.png.

• build_datasets_fn (callable) – Callable returning list[dict] describing datasets for an item.

• zoom_center_fn (callable, optional) – Callable mapping item -> center UNIX time (or None) for zoom.

• zoom_window_seconds (float, optional) – Duration of zoom window in seconds.

• vertical_lines_fn (callable, optional) – Callable mapping item -> list[float] UNIX timestamps (or None).

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling for all rows.

• z_scale ({'linear', 'log'}, default 'linear') – Color scaling for all rows.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to generic_plot_spectrogram_set.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

• max_workers (int, default 2) – Number of parallel worker processes.

• progress_json_path (str, default PLOTTING_PROGRESS_JSON_PATH) – Path to progress JSON (resumable state). Created/updated as needed.

• ignore_progress_json (bool, default False) – If True, skip reading existing progress prior to execution.

• flush_batch_size (int, default 10) – Progress/log batch size; values < 1 coerced to 1. Final partial batch flushed.

• log_flush_batch_size (int, optional) – Explicit log batch size; if None reuse flush_batch_size.

• install_signal_handlers (bool, default True) – When True, a temporary SIGINT handler is installed (restored on exit) to enable graceful interruption (progress & log flush).

Returns:

Sequence of (item, status) with status in {'ok', 'no_data', 'error'}.

Return type:

list of tuple

FAST ESA CDF file downloading from CDA Web: single-day, single-year, and threaded batch.

configurable_spectrograms.download.FAST_ESA_CDF_download(base_url: str = 'https://cdaweb.gsfc.nasa.gov/pub/data/fast/esa/l2', year: int = 2000, data_folder: str = './FAST_data/', instruments: list[str] = ['eeb', 'ees', 'ieb', 'ies']) → None

Download one year of FAST ESA CDF files from CDA Web.

Scrapes each month/instrument listing page once, then calls download_single_day_cdf() for every calendar day of year against that cached listing, so every day is downloaded through the same single-day logic used for one-off single-day downloads elsewhere in this module, without re-requesting the same month page once per day.

Parameters:

• base_url (str, default FAST_ESA_BASE_URL) – Base CDA Web URL for FAST ESA level-2 data.

• year (int, default DEFAULT_YEAR) – Calendar year to download.

• data_folder (str, default DEFAULT_FOLDER) – Root output directory; files are saved under {data_folder}/{year}/{month}/.

• instruments (list of str, default DEFAULT_INSTRUMENT_LIST) – Instrument codes to download (e.g. ['eeb', 'ees']).

Notes

For downloading many years at once with thread-pool parallelism, see download_cdf_files_threaded().

configurable_spectrograms.download.FAST_MIN_DATE: date = datetime.date(1996, 8, 21)

Earliest and latest calendar days with any FAST ESA CDF coverage on CDA Web.

configurable_spectrograms.download.download_cdf_files_threaded(base_url: str = 'https://cdaweb.gsfc.nasa.gov/pub/data/fast/esa/l2', years: list[int] | None = None, data_folder: str = './FAST_data/', instruments: set[str] = {'eeb', 'ees', 'ieb', 'ies'}, max_workers: int = 8, progress_json_path: str | None = None, ignore_progress_json: bool = False, flush_batch_size: int = 25) → list[tuple[tuple[str, str], str]]

Download many years of FAST ESA CDF files in parallel using a thread pool.

Listing pages are scraped sequentially first (cheap – one small HTML page per year/month/instrument combination), then every individual file download is dispatched to a ThreadPoolExecutor via configurable_spectrograms.batch_runner.run_batch(): downloading is I/O-bound, so thread-level concurrency is used here instead of the process-level concurrency the plotting batch drivers use for their CPU-bound rendering work.

Parameters:

• base_url (str, default FAST_ESA_BASE_URL) – Base CDA Web URL for FAST ESA level-2 data.

• years (list of int or None, optional) – Calendar years to download; defaults to [DEFAULT_YEAR] when None.

• data_folder (str, default DEFAULT_FOLDER) – Root output directory; files are saved under {data_folder}/{year}/{month}/.

• instruments (set of str, default INSTRUMENT_OPTIONS) – Instrument codes to download.

• max_workers (int, default 8) – Number of download threads.

• progress_json_path (str or None, optional) – Path to a JSON file used for resumable progress tracking. None disables persistence.

• ignore_progress_json (bool, default False) – If True, skip reading existing progress prior to execution.

• flush_batch_size (int, default 25) – Progress/log batch size passed through to run_batch.

Returns:

Sequence of ((download_link, output_file), status) results, where status is 'ok' or 'error'.

Return type:

list of tuple

configurable_spectrograms.download.download_single_day_cdf(date: date, instruments: list[str] = ['eeb', 'ees', 'ieb', 'ies'], base_url: str = 'https://cdaweb.gsfc.nasa.gov/pub/data/fast/esa/l2', data_folder: str = './FAST_data/', _page_file_names: dict[str, list[str]] | None = None) → dict[str, list[str]]

Download every FAST ESA CDF file for one calendar day, per instrument.

Parameters:

• date (datetime.date) – Calendar day to download. FAST ESA CDF coverage spans roughly FAST_MIN_DATE through FAST_MAX_DATE; a date outside that range simply returns empty lists.

• instruments (list of str, default DEFAULT_INSTRUMENT_LIST) – Instrument codes to download (e.g. ['eeb', 'ees']); the ones desired can be specified explicitly, e.g. when called from a CLI’s --instruments argument.

• base_url (str, default FAST_ESA_BASE_URL) – Base CDA Web URL for FAST ESA level-2 data.

• data_folder (str, default DEFAULT_FOLDER) – Root output directory; files are saved under {data_folder}/{year}/{month}/.

• _page_file_names (dict of {str: list of str} or None, optional) – Internal use only. Pre-scraped {instrument: [file_name, ...]} month listing, letting FAST_ESA_CDF_download() reuse one page fetch across every day of the month instead of re-requesting it for each day. None (the default) fetches a fresh listing here.

Returns:

dict of {str – Local CDF file paths for date, keyed by instrument – downloaded just now, or already present from an earlier run. A single day commonly spans several FAST orbits, so an instrument may map to more than one file; an instrument with no data that day maps to an empty list.

Return type:

list of str}

Library: configurable_spectrograms.fast

FAST-instrument-specific paths, variable names, and default colormaps.

configurable_spectrograms.fast.constants.DEFAULT_PITCH_ANGLE_CATEGORIES: dict[str, list[tuple[float, float]]] = {'all\n(0, 360)': [(0.0, 360.0)], 'downgoing\n(0, 30), (330, 360)': [(0.0, 30.0), (330.0, 360.0)], 'perpendicular\n(40, 140), (210, 330)': [(40.0, 140.0), (210.0, 330.0)], 'upgoing\n(150, 210)': [(150.0, 210.0)]}

Default pitch-angle category boundaries (degrees) used when a caller doesn’t supply their own mapping.

configurable_spectrograms.fast.constants.FAST_COLLAPSE_FUNCTION

Same collapse function as the generic pipeline (kept as a distinct name for readability at FAST call sites).

FAST orbit/instrument file discovery and progress-key bookkeeping.

configurable_spectrograms.fast.orbit_discovery.discover_orbit_files(directory_path: str, instrument_order: tuple[str, ...] = ('ees', 'eeb', 'ies', 'ieb')) → dict[int, dict[str, str]]

Discover FAST instrument CDF files and group them by orbit.

Walks directory_path recursively for non-orbit-ephemeris CDF files (paths containing _orb_ are excluded), parses each file’s orbit number and instrument type, and groups them into {orbit: {instrument: path}}.

Parameters:

• directory_path (str) – Root folder containing instrument CDF files.

• instrument_order (tuple of str, default DEFAULT_INSTRUMENT_ORDER) – Instrument codes to include; files for other instruments are skipped.

Returns:

dict of {int – Mapping of orbit number to {instrument: cdf_path}. When multiple files exist for the same orbit/instrument pair, the last one seen during the directory walk wins.

Return type:

dict of {str: str}}

configurable_spectrograms.fast.orbit_discovery.extract_orbit_and_instrument(cdf_path: str) → tuple[int, str, str] | None

Parse a CDF filename to (orbit_number, instrument_type, cdf_path).

Parameters:

cdf_path (str) – Path (or bare filename) of the CDF file.

Returns:

(orbit_number, instrument_type, cdf_path), or None when the filename does not match the expected pattern, the orbit number cannot be parsed, or the instrument type is None or 'orb'.

Return type:

tuple or None

Examples

>>> extract_orbit_and_instrument("fa_esa_l2_eeb_20000101001737_13312_v02.cdf")
(13312, 'eeb', 'fa_esa_l2_eeb_20000101001737_13312_v02.cdf')
>>> extract_orbit_and_instrument("fa_k0_orb_13312_v01.cdf") is None
True

configurable_spectrograms.fast.orbit_discovery.resolve_orbit_from_files(instrument_files: dict[str, str]) → int | None

Best-effort orbit number for a manually-assembled instrument file mapping.

Used for title/vertical-line labeling when a caller supplies its own {instrument: file_path} mapping directly rather than discovering one from a folder via discover_orbit_files(), so no orbit number is known up front.

Parameters:

instrument_files (dict of {str: str}) – Mapping of instrument key to CDF file path.

Returns:

The orbit number parsed from the first file in instrument_files whose name matches the expected FAST CDF naming pattern, or None if none do.

Return type:

int or None

Examples

>>> resolve_orbit_from_files({"eeb": "fa_esa_l2_eeb_20000101001737_13312_v02.cdf"})
13312
>>> resolve_orbit_from_files({"eeb": "not_a_fast_file.cdf"}) is None
True

configurable_spectrograms.fast.orbit_discovery.resolve_shared_orbit(instrument_day_files: dict[str, list[str]]) → tuple[int | None, dict[str, str]]

Pick one orbit’s worth of files out of a day’s downloaded/discovered CDFs.

A single FAST day commonly spans multiple orbits per instrument, each a separate CDF file. Callers that plot one orbit at a time (e.g. configurable_spectrograms.fast.plotting.FAST_plot_instrument_grid()) need exactly one file per instrument, so this resolves the day down to the orbit number shared by the most instruments, breaking ties by picking the lowest orbit number.

Parameters:

instrument_day_files (dict of {str: list of str}) – Mapping of instrument key to every CDF file path found for one day, as returned by configurable_spectrograms.download.download_single_day_cdf().

Returns:

tuple[int or None, dict of {str – The resolved orbit number (None if no file parsed an orbit number at all) and a mapping of instrument -> single file path for that orbit. Instruments with no file for the resolved orbit are omitted from the mapping.

Return type:

str}]

Examples

>>> resolve_shared_orbit({
...     "eeb": ["fa_esa_l2_eeb_20000101001737_100_v02.cdf",
...             "fa_esa_l2_eeb_20000101031737_101_v02.cdf"],
...     "ies": ["fa_esa_l2_ies_20000101001738_100_v02.cdf"],
... })
(100, {'eeb': 'fa_esa_l2_eeb_20000101001737_100_v02.cdf', 'ies': 'fa_esa_l2_ies_20000101001738_100_v02.cdf'})
>>> resolve_shared_orbit({"eeb": [], "ies": []})
(None, {})

Single-output FAST ESA spectrogram rendering.

Both functions here render one figure for a single CDF file or a single orbit’s worth of instrument files. Batch callers (configurable_spectrograms.fast.process_orbit) call these same functions once per orbit rather than duplicating the rendering logic.

configurable_spectrograms.fast.plotting.FAST_plot_instrument_grid(cdf_file_paths: dict[str, str], filtered_orbits_df=None, orbit_number: int | None = None, zoom_duration_minutes: float = 6.25, scale_function_y: str = 'linear', scale_function_z: str = 'linear', instrument_order: tuple[str, ...] = ('ees', 'eeb', 'ies', 'ieb'), show: bool = True, colormap: str = 'viridis', y_min: float | None = None, y_max: float | None = None, z_min: float | None = None, z_max: float | None = None, global_extrema: dict[str, int | float] | None = None, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → tuple[Any, Any]

Plot a multi-instrument ESA spectrogram grid for a single orbit.

Loads each instrument CDF, collapses across pitch-angle, and constructs datasets for generic_plot_multirow_optional_zoom. A zoom column is included when vertical lines are available for the orbit.

Parameters:

• cdf_file_paths (dict of {str: str}) – Mapping of instrument key ('ees', 'eeb', 'ies', 'ieb') to CDF file path. Missing instruments are skipped.

• filtered_orbits_df (pandas.DataFrame or None) – DataFrame for vertical line computation; None omits lines.

• orbit_number (int or None) – Orbit identifier used in titles and vertical lines.

• zoom_duration_minutes (float, default 6.25) – Zoom window length (minutes).

• scale_function_y ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• scale_function_z ({'linear', 'log'}, default 'linear') – Color scale for intensity.

• instrument_order (tuple of str, default DEFAULT_INSTRUMENT_ORDER) – Display order of instrument rows.

• show (bool, default True) – Whether to show the figure interactively.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• y_min (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• y_max (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• z_min (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• z_max (float or None, optional) – Global fallback axis/color limits used when global_extrema does not supply an instrument-specific key.

• global_extrema (dict or None) – Precomputed extrema keyed as {instrument}_{y_scale}_{z_scale}_{axis}_{min|max} supplying per-instrument limits. Takes precedence over the direct y_min / y_max / z_min / z_max arguments.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style; see configurable_spectrograms.cusp_marking.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Figure and Canvas, or (None, None) when no datasets are produced.

Return type:

tuple[Figure or None, FigureCanvasBase or None]

Notes

• Files that fail to load are logged and skipped.

• Energy bins are restricted to [0, 4000] unless overridden.

• vmin/vmax per row use 1st/99th percentiles unless global_extrema provides per-instrument z_min / z_max.

configurable_spectrograms.fast.plotting.FAST_plot_pitch_angle_grid(cdf_file_path: str, filtered_orbits_df=None, orbit_number: int | None = None, zoom_duration_minutes: float = 6.25, scale_function_y: str = 'linear', scale_function_z: str = 'linear', pitch_angle_categories: dict[str, list[tuple[float, float]]] | None = None, show: bool = True, colormap: str = 'viridis', y_min: float | None = None, y_max: float | None = None, z_min: float | None = None, z_max: float | None = None, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → tuple[Any, Any]

Plot a grid of ESA spectrograms collapsed by pitch-angle categories.

Each row corresponds to a pitch-angle category (e.g. downgoing, upgoing, perpendicular, all). If orbit boundary timestamps are available a zoom column is added. Data are collapsed over pitch-angle via FAST_COLLAPSE_FUNCTION (np.nansum by default).

Parameters:

• cdf_file_path (str) – Path to the instrument CDF file.

• filtered_orbits_df (pandas.DataFrame or None) – DataFrame used to compute vertical lines; if None, lines are omitted.

• orbit_number (int or None) – Orbit number used to label vertical lines.

• zoom_duration_minutes (float, default 6.25) – Window length (minutes) for the optional zoom column.

• scale_function_y ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• scale_function_z ({'linear', 'log'}, default 'linear') – Color scale for intensity.

• pitch_angle_categories (dict or None) – Mapping of label -> list of (min_deg, max_deg) ranges; defaults to the four standard groups when None.

• show (bool, default True) – If True, display the figure interactively.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• y_min (float or None, optional) – Energy (y-axis) limits; defaults to [0, 4000] when None.

• y_max (float or None, optional) – Energy (y-axis) limits; defaults to [0, 4000] when None.

• z_min (float or None, optional) – Color scale limits; defaults to row-level 1st/99th percentiles when None.

• z_max (float or None, optional) – Color scale limits; defaults to row-level 1st/99th percentiles when None.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style; see configurable_spectrograms.cusp_marking.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Figure and Canvas, or (None, None) when no datasets are produced.

Return type:

tuple[Figure or None, FigureCanvasBase or None]

Global axis-extrema computation for FAST batch plotting.

compute_global_extrema() performs a resumable pass over instrument CDF files to determine shared, sensible axis limits (so every orbit in a batch run uses the same energy/intensity scale) before the main plotting pass begins.

configurable_spectrograms.fast.extrema.compute_global_extrema(directory_path: str, y_scale: str, z_scale: str, instrument_order: Iterable[str], extrema_json_path: str = './FAST_calculated_extrema.json', compute_mins: bool = False, max_percentile: float = 95.0, log_floor_cutoff: float = 0.1, log_floor_value: float = -1.0, flush_batch_size: int = 10) → dict[str, Any]

Compute (or incrementally update) cached axis extrema per instrument.

Performs a resumable pass over all instrument CDF files, flushing incremental progress to extrema_json_path after each flush_batch_size orbits.

Extrema logic

• Y (energy) minima are fixed to 0 unless compute_mins is True.

• Linear Y maxima: smallest energy whose cumulative positive finite count reaches 99% of total positive finite samples.

• Linear Z maxima: max_percentile-th percentile of pooled positive finite intensity samples.

• If the requested scale is log and linear_linear extrema already exist in the cache, they are log-transformed without re-scanning files. If the requested scale is linear and linear_linear extrema exist, they are copied directly.

• Log transform applies a floor: values <= log_floor_cutoff or non-finite are replaced by log_floor_value.

• Maxima are monotonically non-decreasing across incremental updates; energy maxima are capped at 4000.

param directory_path:

Root directory containing instrument CDF files.

type directory_path:

str

param y_scale:

Y scaling label (used for cache key names).

type y_scale:

{‘linear’, ‘log’}

param z_scale:

Z scaling label (used for cache key names).

type z_scale:

{‘linear’, ‘log’}

param instrument_order:

Instruments to process (e.g., ("ees", "eeb", "ies", "ieb")).

type instrument_order:

iterable of str

param extrema_json_path:

Path to the JSON cache file (created if absent).

type extrema_json_path:

str, default FAST_EXTREMA_JSON_PATH

param compute_mins:

If True, compute intensity minima; otherwise they are set to 0.

type compute_mins:

bool, default False

param max_percentile:

Percentile applied to pooled positive intensity for z_max.

type max_percentile:

float, default 95.0

param log_floor_cutoff:

Values at or below this threshold map to log_floor_value in log space.

type log_floor_cutoff:

float, default 0.1

param log_floor_value:

Floor value substituted for invalid log-domain extrema.

type log_floor_value:

float, default -1.0

param flush_batch_size:

Orbits with updates between JSON flushes; coerced to >= 1.

type flush_batch_size:

int, default 10

returns:

Updated extrema mapping containing values and progress entries.

rtype:

dict

Per-orbit FAST spectrogram processing (the parallel batch worker unit).

configurable_spectrograms.fast.process_orbit.FAST_process_single_orbit(orbit_number: int, instrument_file_paths: dict[str, str], filtered_orbits_dataframe, zoom_duration_minutes: float, y_axis_scale: str, z_axis_scale: str, instrument_order: tuple[str, ...], colormap: str, output_base_directory: str, orbit_timeout_seconds: int | float = 60, instrument_timeout_seconds: int | float = 30, global_extrema: dict[str, int | float] | None = None, override_plots: bool = True, cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None) → dict[str, Any]

Process and save all ESA spectrogram plots for a single orbit.

For each available instrument, generates two plot versions: 1. Using global_extrema (_given_extrema suffix) if provided. 2. Raw (per-file) extrema (_raw suffix). This applies to both pitch-angle and instrument-grid plots. Each figure is saved to disk and closed immediately after it’s rendered, so at most one or two figures are ever held in memory at once regardless of how many instruments and extrema variants an orbit produces.

Parameters:

• orbit_number (int) – The orbit identifier.

• instrument_file_paths (dict of {str: str}) – Mapping of instrument key to CDF file path.

• filtered_orbits_dataframe (pandas.DataFrame) – DataFrame used to compute orbit boundary timestamps.

• zoom_duration_minutes (float) – Zoom window length for zoomed plots.

• y_axis_scale ({'linear', 'log'}) – Y-axis scaling.

• z_axis_scale ({'linear', 'log'}) – Color scale for intensity.

• instrument_order (tuple of str) – Order used in the instrument grid.

• colormap (str) – Matplotlib colormap.

• output_base_directory (str) – Root folder for saving figures; year/month are inferred from the CDF path when possible, else 'unknown'.

• orbit_timeout_seconds (int or float, default 60) – Maximum wall-clock seconds for the entire orbit.

• instrument_timeout_seconds (int or float, default 30) – Per-instrument/grid timeout.

• global_extrema (dict or None) – Precomputed extrema mapping (from compute_global_extrema).

• override_plots (bool, default True) – If False, skip plotting when the output file already exists.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to the plotting functions.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

Returns:

Result with keys orbit (int), status ('ok', 'error', or 'timeout'), errors (list of str), and optionally timeout_type / timeout_instrument.

Return type:

dict

Notes

Each figure is saved and closed as soon as it’s rendered, so a mid-orbit timeout can leave some of an orbit’s PNGs already written to disk. This is safe given deterministic output filenames: a subsequent retry of a timed-out orbit simply overwrites the partial set.

Batch driver: process every orbit in a FAST CDF directory in parallel.

configurable_spectrograms.fast.batch_directory.FAST_plot_spectrograms_directory(directory_path: str = './FAST_data/', output_base: str = './FAST_plots/', y_scale: str = 'linear', z_scale: str = 'log', zoom_duration_minutes: float = 6, instrument_order: tuple[str, ...] = ('ees', 'eeb', 'ies', 'ieb'), verbose: bool = True, progress_json_path: str | None = './batch_multi_plot_FAST_progress.json', ignore_progress_json: bool = False, use_tqdm: bool | None = None, colormap: str = 'viridis', cusp_marker_style: str = 'both', cusp_marker_kwargs: dict | None = None, max_workers: int = 4, orbit_timeout_seconds: int | float = 60, instrument_timeout_seconds: int | float = 30, retry_timeouts: bool = True, flush_batch_size: int = 10, log_flush_batch_size: int | None = None, max_processing_percentile: float | None = None, override_plots: bool = True) → list[dict[str, Any]]

Batch process ESA spectrogram plots for all orbits in a directory.

Discovers instrument CDF files (excluding _orb_), groups them by orbit, and processes each orbit in parallel worker processes (safe for matplotlib). Progress is persisted to a JSON file to support resumable runs. When max_processing_percentile is not None, a global extrema pass runs first (configurable_spectrograms.fast.extrema.compute_global_extrema()) and both raw and given-extrema plots are saved; otherwise only raw plots are produced.

Parameters:

• directory_path (str, default FAST_CDF_DATA_FOLDER_PATH) – Root folder containing CDF files.

• output_base (str, default FAST_OUTPUT_BASE) – Base output directory; plots are saved under output_base/year/month/orbit.

• y_scale ({'linear', 'log'}, default 'linear') – Y-axis scaling.

• z_scale ({'linear', 'log'}, default 'log') – Color scale for intensity.

• zoom_duration_minutes (float, default DEFAULT_ZOOM_WINDOW_MINUTES) – Zoom window length for zoom columns.

• instrument_order (tuple of str, default ('ees', 'eeb', 'ies', 'ieb')) – Display order for the instrument grid.

• verbose (bool, default True) – Print additional batch messages when True.

• progress_json_path (str or None, default FAST_PLOTTING_PROGRESS_JSON) – Path to persist progress across runs; None disables persistence.

• ignore_progress_json (bool, default False) – If True, do not read existing progress before starting.

• use_tqdm (bool or None, default None) – Show a tqdm progress bar when True; defaults to False when None.

• colormap (str, default 'viridis') – Matplotlib colormap name.

• cusp_marker_style ({'line', 'bracket', 'both'}, default 'both') – Cusp-boundary marker style forwarded to every orbit’s plots.

• cusp_marker_kwargs (dict or None, optional) – Extra keyword arguments forwarded to the marker-drawing function.

• max_workers (int, default 4) – Max number of worker processes.

• orbit_timeout_seconds (int or float, default 60) – Total per-orbit timeout (seconds).

• instrument_timeout_seconds (int or float, default 30) – Per-instrument/grid timeout (seconds).

• retry_timeouts (bool, default True) – If True, retry timed-out orbits once with a smaller pool.

• flush_batch_size (int, default 10) – Orbit completions between progress/extrema JSON writes. Values < 1 become 1. Final partial batch always flushes.

• log_flush_batch_size (int or None, default None) – Logging buffer batch size; defaults to flush_batch_size when None.

• max_processing_percentile (float or None, default None) – Percentile (0-100] for pooled intensity (Z) maxima in compute_global_extrema. None skips the extrema pass and raw-only plots are produced. Energy (Y) maxima use a fixed 99% cumulative coverage rule regardless.

• override_plots (bool, default True) – If False, skip plots whose output file already exists.

Returns:

Result dictionaries from FAST_process_single_orbit (and retries).

Return type:

list of dict

Raises:

KeyboardInterrupt – Re-raised on SIGINT/SIGTERM so the caller can stop multi-combo loops.

Notes

• Progress JSON key f"progress_{y_scale}_{z_scale}_last_orbit" tracks the last completed orbit; error/timeout orbits are recorded under dedicated keys (including per-instrument).

• Signal handlers terminate child processes and raise KeyboardInterrupt to interrupt the main wait loop immediately.