New cross subject class for Transfer Learning

[toncho11]

General description

The code for: #1077
Please take the time to analyze this proposed contribution. A lot of thought/work has been put into it.

This PR proposes a target-aware cross-subject evaluation designed for transfer-learning BCI methods. The main motivation is that some transfer-learning methods need more than ordinary train/test indices: they need to know which training trials belong to which source subject, and in some cases they need controlled access to unlabeled or labeled data from the held-out target subject.

The implementation supports several explicit protocols, also called "modes" here. This makes it possible to evaluate methods such as Riemannian alignment or Riemannian Procrustes Alignment, where unlabeled target data and source-subject identity are needed.

At the same time, the implementation tries to stay as close as possible to MOABB’s existing machinery. It reuses the existing cross-subject structure, data loading, splitting, scoring, result construction, model saving, and pipeline handling wherever possible. The additional logic is limited to the parts that cannot be represented cleanly.

More information:

It provides a leave-one-subject-out (LOSO) evaluation with controlled access to the held-out subject. The held-out subject (HOS) is the target subject being evaluated.

Currently, it provides 6 modes:

HOS_SOURCE_ONLY_TRIALWISE - Source-only training; each held-out target
trial is predicted independently. This mode is
the closest to real-world BCI when we do not
have enough data.

HOS_SOURCE_ONLY_BLOCKWISE - Source-only training; held-out target trials
are predicted as a block, matching standard
MOABB behavior. It is a compatibility mode
with CrossSubjectEvaluation. Do not use it;
use one of the other modes, as they are more
clearly defined.

HOS_UNLABELED_20P - First 20% of held-out target trials are used unlabeled
for adaptation; the remaining 80% are evaluated.

HOS_UNLABELED_50P - First 50% of held-out target trials are used unlabeled
for adaptation; the remaining 50% are evaluated.

HOS_UNLABELED_100P - All held-out target trials are used unlabeled for
transductive adaptation and are also evaluated.

HOS_LABELED_20P - First 20% of held-out target trials are used with labels
for supervised calibration; the remaining 80% are
evaluated.

New fit method looks like this:

 def fit(
        self,
        X,
        y=None,
        subjects=None,
        cs_mode=None,
        X_target_unlabeled=None,
        X_target_labeled=None,
        y_target_labeled=None,
        **fit_params,
    ):
        """
        Input
        -----
        X : ndarray, shape (n_trials, n_channels, n_channels)

        Fit metadata
        ------------
        subjects : array-like
            Source subject ID for each source training trial.

        cs_mode : str, optional
            Cross-subject evaluation mode name, for example:
            "HOS_SOURCE_ONLY_BLOCKWISE", "HOS_UNLABELED_20P",
            "HOS_LABELED_20P".

        X_target_unlabeled : ndarray, optional
            Unlabeled covariance matrices from the held-out target subject.
            If provided, RPA uses them to estimate the target-domain alignment
            transform.

        X_target_labeled : ndarray, optional
            Labeled covariance matrices from the held-out target subject.
            If provided and no unlabeled target data is provided, RPA uses
            their covariance distribution for alignment. The labels themselves
            are not used.

        y_target_labeled : ndarray, optional
            Labels for X_target_labeled. RPA accepts this argument for
            evaluator compatibility but does not use the labels directly.
        """

Some extra notes:

The code also handles things like HOS_SOURCE_ONLY_TRIALWISE vs HOS_SOURCE_ONLY_BLOCKWISE. HOS_SOURCE_ONLY_BLOCKWISE is the one currently used in CrossSubjectEvaluation, but it is not strict on how benchmarking is performed. HOS_SOURCE_ONLY_BLOCKWISE allows you to both evaluate samples from the held-out subject sample by sample independently or use all the test samples to create for example a Riemannian mean on the test data or learn the distribution on the test data and use this to boost your classification performance. So it is better to use HOS_SOURCE_ONLY_TRIALWISE or HOS_UNLABELED_100P to be sure that the benchmark protocol is explicit.