Generic Batch Spectrogram Plotting¶

Overview¶

The module batch_multi_plot_spectrogram.py provides a data-agnostic batch pipeline for generating multi-row spectrogram figures with optional per-row color scaling overrides, batched progress persistence, resumable execution, and buffered logging. It mirrors the FAST-specific implementation while removing mission-specific assumptions.

Key Public Functions¶

generic_batch_plot: High-level orchestration that iterates over item identifiers (any iterable) and invokes plotting for each data container while batching JSON progress writes and log flushing. Accepts install_signal_handlers to opt out of temporary SIGINT handler installation in embedded environments.
generic_plot_spectrogram_set: Render a multi-row spectrogram grid for a single logical item (e.g., orbit), applying per-row y/z min/max overrides and optional percentile-based color scaling.
make_spectrogram: Low-level helper to collapse a 3D data cube along one axis and render a single spectrogram panel with linear or logarithmic y and z scaling.

Batching Parameters¶

flush_batch_size: Number of successfully processed items buffered before progress JSON (and any extrema-like state) is persisted. Final flush always occurs at normal termination.
log_flush_batch_size: Size of the in-memory log message buffer; defaults to flush_batch_size when unspecified.

Progress JSON Schema (Generic)¶

The progress JSON managed by generic_batch_plot contains keys (illustrative):

{
    "completed_items": ["itemA", "itemB", ...],
    "errors": ["itemC"],
    "no_data": ["itemD"],
    "last_index": 42,
    "schema_version": 1
}

Notes:

errors is a list of item representations; detailed exception text lives
only in the log file to keep resumable state compact and schema-stable.
last_index is the zero-based index in the original items iterable
of the most recently processed entry (useful for sanity checks / debugging).

Resumability is supported by reloading this file unless ignore_progress_json=True.

Examples¶

Minimal Run¶

from batch_multi_plot_spectrogram import generic_batch_plot

# Suppose we have a pre-built list of items and a loader that yields datasets
items = ["orbit_001", "orbit_002", "orbit_003"]

def load_datasets_for_item(item_id):
    # Return list[dict] each with keys: x, y, data, label (and optional vmin/vmax)
    return build_datasets(item_id)  # user-defined

generic_batch_plot(
    items=items,
    output_dir="plots_generic",
    build_datasets_fn=load_datasets_for_item,
)

Custom Batching and Logging¶

generic_batch_plot(
    items=items,
    output_dir="plots_generic",
    build_datasets_fn=load_datasets_for_item,
    flush_batch_size=25,
    log_flush_batch_size=50,
)

Resuming After Interruption¶

generic_batch_plot(
    items=items,
    output_dir="plots_generic",
    build_datasets_fn=load_datasets_for_item,
    progress_json_path="generic_progress.json",  # default
)  # picks up where it left off

Force Fresh Processing (ignore prior progress):

generic_batch_plot(
    items=items,
    output_dir="plots_generic",
    build_datasets_fn=load_datasets_for_item,
    ignore_progress_json=True,
)

Per-Row Min/Max Overrides¶

Each dataset dict may include per-row overrides (all optional): y_min, y_max, z_min, z_max plus optional precomputed color scale bounds vmin / vmax. When z_min / z_max are absent but vmin / vmax are present, those act as the row’s native color limits (still clamped by any global z_min/z_max arguments passed to generic_plot_spectrogram_set).

def load_datasets_for_item(item_id):
    return [
        {"x": x1, "y": y1, "data": d1, "label": "EES", "y_min": 0, "y_max": 4000},
        {"x": x2, "y": y2, "data": d2, "label": "IES", "z_min": 1e-2, "z_max": 1e2},
    ]

generic_batch_plot(items, "plots_generic", load_datasets_for_item)

Zoom Column and Event Lines¶

Provide vertical_lines (UNIX timestamps) and a zoom_duration_minutes to add an optional right-hand zoom column when data exist inside the derived window:

generic_plot_spectrogram_set(
    datasets=my_datasets,
    vertical_lines=[t0, t1],
    zoom_duration_minutes=10,
    colormap="plasma",
    z_scale="log",
)

Low-Level Single Panel¶

from batch_multi_plot_spectrogram import make_spectrogram

ax, x_used = make_spectrogram(
    x_axis_values=time_sec,
    y_axis_values=energy_ev,
    data_array_3d=data_cube,
    collapse_axis=1,
    y_axis_scale_function="log",
    z_axis_scale_function="log",
    instrument_label="EES",
    vertical_lines_unix=[event_time],
)

Percentile-Based Color Scaling¶

If z_axis_min or z_axis_max are omitted, internal logic selects robust percentile-based bounds (see function docstring). Provide explicit bounds to fully control the color scale. Precomputing and supplying vmin/vmax when assembling datasets avoids recomputation in tight loops.

Error Handling¶

All exceptions are captured with descriptive variable names; errors are recorded in errors within the progress JSON while processing continues with remaining items.

Implementation Notes¶

Logging is buffered; call-level helpers flush automatically at batch boundaries.
Figures are explicitly closed via close_all_axes_and_clear to control memory use.
The module is intentionally stateless apart from in-memory log and progress batching.
A temporary SIGINT handler is installed (and restored) when install_signal_handlers=True.
Disable this in notebooks / embedded shells if global handler changes cause side-effects.

Axis Label Defaults¶

Unless overridden per dataset row, the following label defaults are applied:

Y-axis: Energy (eV)
Colorbar (Z): Counts

Provide y_label / z_label keys in each dataset dict to customize units or semantics (e.g., 'Differential Flux (eV cm^-2 s^-1 sr^-1)'). Global changes can be introduced at the dataset construction layer to keep the batch orchestration code data-agnostic.