Skip to content

Latest commit

 

History

History
138 lines (90 loc) · 17.6 KB

ModelGuide.md

File metadata and controls

138 lines (90 loc) · 17.6 KB

JadeML model and visualization guide

Back to the README

This guide documents the behavior and interpretation of the models available in JadeML 3.0.1. The README model tables give the recommended learning order.

Python setup

Every model except the TorchSharp MLP needs 64-bit Python 3.11 with NumPy and scikit-learn. PCA also needs Python, including when used before MLP training. The verified combination is:

Component Version
ScikitLearn C# NuGet wrapper 1.1.0
Python 3.11, x64
NumPy 1.26.4
scikit-learn Python package 1.5.1

The C# wrapper and Python package have separate version numbers. Run the test suite before upgrading Python packages, especially because the histogram-boosting explorer reads scikit-learn's internal tree arrays.

JadeML checks JADEML_PYTHON first, then the standard per-user Python 3.11 installation, then python.exe on PATH. Automatic discovery skips incompatible or broken candidates. An explicit JADEML_PYTHON selection must be compatible; JadeML reports an error instead of silently choosing another interpreter.

Use an interpreter or virtual environment with packages installed directly into it, without --user. The runtime gets import paths from an isolated Python configuration, which excludes user-site packages. Missing dependencies or an unsupported Python version produce a setup message.

One shared Python interpreter is initialized inside JadeML when needed. Training uses the wrapper; predictions use either managed snapshots of fitted parameters or a retained fitted Python estimator. Restart JadeML after changing the selected interpreter or its packages.

Data preparation and evaluation

Prediction tasks select numeric input columns and one target. Regression accepts one or multiple inputs and one numeric target. CSV numeric values use decimal points. Prediction files with headers are matched and reordered by selected input names; files without headers use positional order.

The processed-data tab initially previews preprocessing fitted to the full dataset. Creating the training/test split refits scaling, variance filtering, PCA, and output scaling on training rows only, then refreshes the datasets. A classification training split must contain every class. Changing data, columns, processing, or the split clears dependent models and results.

Standard scaling and variance filtering use the app's sample-variance convention. PCA centers inputs and whitens retained components using fitted statistics. Whitening changes distances as well as coordinates. Regression predictions are returned in original target units; evaluation uses the processed output units displayed in the test dataset. R-squared is undefined for a one-row evaluation and is reported as NaN.

MLP's optional validation split comes from the already processed training dataset. Its loss helps monitor training, but those validation rows have contributed to preprocessing statistics; use the separate test dataset for held-out evaluation.

Clustering uses no target or train/test split. Preprocessing is fitted to all rows being clustered. At least two rows and one usable feature are required. Selected features must contain finite numeric values in every row. Clustering CSV import preserves incomplete records; correct missing values or exclude an affected feature instead of silently dropping rows. Keep identifiers and known class labels out of selected inputs, including labels encoded as numbers.

Reading model visualizations

Supervised example explorers start with a training row when available. Inputs and coefficients use the model's processed units, after scaling or PCA. Edit an input and run the explanation again to refresh results. Curves vary one selected input while holding others at the edited example. These operations inspect the fitted model without changing its parameters.

Coefficient magnitudes alone do not establish feature importance when input scales differ. Distances and decision scores are not probabilities. Two-dimensional plots only show a projection of models that may use many inputs.

Baselines, linear models, and local methods

  • Dummy: predicts the most frequent training class, or the mean/median training target. Use the same split to compare other models with this baseline.
  • Linear, Ridge, Lasso, and Elastic Net regression: signed bars and an exact coefficient table separate the intercept from feature effects. Each effect holds other inputs fixed.
  • Nearest-centroid classification: compares an example with each class's mean vector. The smallest Euclidean distance wins, with equal prior weight for all classes. The explorer displays squared distances and a map in processed input units.
  • K-nearest neighbors: classification uses Euclidean distance and equal votes. Regression offers uniform or distance weighting; duplicate exact matches receive all weight in distance mode. Inspect neighbors and contributions, and scale inputs with different measurement units.
  • Logistic regression: inspect coefficients, signed score contributions, and the sigmoid converting the score into the probability of class 1. Both original class labels are shown.
  • Multinomial logistic regression: inspect each class's coefficients and score contributions. For three or more classes, all classes have learned scores, including class 0; softmax converts them to probabilities. For two classes, the equivalent display uses a zero reference score for the first class and a sigmoid score for the second.

Distribution-based classifiers

  • Naïve Bayes (Gaussian): fitted bell curves show means, standard deviations, and class priors. Variances include the learner's smoothing. Curve height is a probability density, not a class probability. Example evidence explains feature contributions to class probabilities.
  • Bernoulli Naive Bayes: requires processed input values of exactly 0 or 1; continuous values are not silently binarized.
  • Multinomial and Complement Naive Bayes: require nonnegative inputs, including fractional counts or weights. Standardization and PCA can violate this requirement. Complement weights are weights, not feature probabilities.
  • Linear discriminant analysis (LDA): compare linear class scores, coefficients, intercepts, and training class priors. The closest projected class center may differ from the predicted class because priors also affect the decision. Training needs at least two examples per class and enough within-class variation.
  • Quadratic discriminant analysis (QDA): learns a covariance structure for each class. Regularization helps stabilize small or correlated datasets. Ellipses show class spread, not decision boundaries; predictions use every selected input.

Regression extensions

  • Bayesian Ridge: displays coefficient standard deviations and predictive uncertainty, including estimated observation noise.
  • Huber and RANSAC: compare a robust fit with ordinary least squares. Huber reduces the influence of large residuals; RANSAC finds a consensus of inliers and refits on those rows. Outlier markings are the model's assessment.
  • Quantile regression: fits the selected conditional quantile and compares the 10th, 50th, and 90th percentiles. These independently fitted quantiles can cross; they are not confidence bounds.
  • Poisson, Gamma, and Tweedie: use exp(intercept + weighted input sum). Poisson and Tweedie require nonnegative targets with at least one positive value; Gamma requires strictly positive targets. Tweedie uses 1 < power < 2, the compound Poisson-Gamma range. These models require no output scaling; prepare the split again after changing processing. Input scaling is available.
  • Polynomial regression: accepts one or multiple inputs. Degree 2 with inputs x and y includes x, y, x², xy, and y², plus an intercept. Inspect powers, interactions, coefficients, and example contributions. Degrees 1–6 are supported, limited to 1,000 generated terms and 5,000,000 training expansion values. Higher degrees can overfit and may require input scaling for numerical stability.
  • Partial least squares (PLS): supports one numeric target. NIPALS components, centering, and optional input/target standardization are fitted on training rows. Start with one component; its count cannot exceed the number of independent centered input directions. Inspect rotation weights, loadings, means/scales, and final coefficients converted back to model input units. Component scores combine centered/scaled inputs with rotation weights; loadings are not final prediction coefficients.
  • Gaussian Process regression: uses a fixed, adjustable RBF kernel, configurable noise variance, and normalized training targets. Its band describes the latent function, excluding future observation noise and kernel-parameter uncertainty. Training is limited to 2,000 rows. Uncertainty bands are model-based and do not guarantee coverage.

Trees, kernels, ensembles, and neural networks

  • Decision trees: inspect decisions, leaves, depth, and complete rule paths. Run an example to highlight its path to a class-probability or numeric leaf.
  • Random forests: inspect individual trees and their contributions. Classification averages leaf probabilities before choosing the largest average; this can differ from a majority vote over trees' winning labels. Regression averages numeric tree predictions. Class probabilities are not guaranteed to be calibrated.
  • Extra Trees: use randomized candidate split thresholds and average tree outputs, including class probabilities for classification.
  • SVM: inspect support vectors, kernel values, signed contributions, and additive bias. Multiclass views compare a selected class pair with overall pairwise voting, including the fitted estimator's tie behavior. Classification uses SVC and regression uses SVR. Kernel settings include Gaussian width, polynomial degree/constant, and sigmoid parameters. Laplacian kernels use the app's distance-kernel matrix and are limited to 2,000 training rows. Regression explorer predictions use processed target units; SVM scores are not probabilities.
  • AdaBoost classification: binary SAMME uses depth-one trees. Inspect normalized learner weights and signed contributions: a vote contributes +2 × weight or −2 × weight, depending on its class. Settings are maximum rounds and learning rate; a perfect learner can stop training early.
  • AdaBoost regression: combines weak learners through a weighted median, rather than an arithmetic average. The explorer shows that calculation.
  • Gradient boosting: inspect each tree's correction to the running prediction or class scores. Classification also displays sigmoid or softmax probabilities.
  • Histogram Gradient Boosting: inspect learned bins, exact tree rules, and stage corrections. Displayed leaf corrections already include the learning rate. The app requires finite inputs. Its explorer checks snapshots of internal tree arrays against fitted-estimator predictions.
  • Voting: combines Ridge/logistic regression, a decision tree, and KNN with configurable nonnegative weights. Classification uses soft voting by averaging probabilities.
  • Stacking: uses the same three learners and a final Ridge/logistic model trained on out-of-fold predictions, with configurable folds. Linear and KNN learners scale inputs inside each fold. App-level preprocessing is already fitted on the training split before these folds, so use the separate test dataset for held-out evaluation. Base learners are refitted on all training rows. Combination controls expose this three-model recipe, rather than arbitrary saved models.
  • MLP: the layer diagram links to a TreeView outline and exact weights. Select a layer or neuron to inspect inputs, weights, bias, and activation. Large networks use a labeled diagram overview while the outline and weight table keep full details.

Solver and regularization settings

Logistic regression uses L2 regularization; smaller C means stronger shrinkage. Binary models offer L-BFGS and Newton-CG, and multinomial models also offer SAGA. Tolerance and maximum iterations control convergence.

Lasso and Elastic Net use coordinate descent. Smaller convergence tolerance requests a closer solution. Models that fail to converge report an error; increase the iteration limit or rescale inputs.

Ridge, Lasso, and Elastic Net use the app's lambda convention. With training squared-error sum SSE and training-row count n:

Model Objective, with unpenalized intercept scikit-learn mapping
Ridge SSE + lambda × sum(weight²) alpha = lambda
Lasso SSE + 2 × lambda × sum(abs(weight)) alpha = lambda / n
Elastic Net SSE + 2 × lambda × (1 − a) × sum(abs(weight)) + lambda × a × sum(weight²) alpha = lambda / n, l1_ratio = 1 − a

Here a is the app's L2 share, labeled alpha: 0 gives Lasso and 1 gives Ridge. It differs from scikit-learn's penalty parameter named alpha. Zero total penalty uses ordinary least squares; the pure L2 endpoint uses Ridge.

Exploring clusters

Cluster numbers start at 1 and are arbitrary group IDs, not known classes or rankings. DBSCAN noise is labeled separately. The result table retains original rows and columns, with an added assignment column, and can be exported to CSV.

The Results tab opens three groups of views:

  • Scatterplots: a display-only PCA map with explained variance and feature-pair plots. Display PCA is unwhitened and preserves fitted assignments. Large maps use labeled deterministic sampling.
  • Cluster charts: group-size bars, feature box plots showing quartiles and range, and mean profiles. These summaries use all fitted rows, including noise as a separate group. Profile standardization affects display only; group/feature pages keep large datasets readable.
  • Explore model: K-Means center distances, a hierarchical dendrogram of actual merge distances, DBSCAN neighborhoods, or Gaussian mixture membership and component weights.

Selectable-axis maps, profile tables, learned parameters, and the hierarchical TreeView are also available. Projected plots can hide separation in other dimensions. Gaussian covariance ellipses show spread, not decision boundaries. Box plots and group means describe assigned samples and need not equal learned mixture parameters. Views identify sampling and collapsed branches.

Model behavior

Model Settings and learned structure New samples
K-Means Choose the cluster count; inspect learned centers and distances. Standardize differently scaled features. Assign to fitted centers.
Hierarchical (agglomerative) clustering Choose the cluster count and linkage; inspect learned merges. Limited to 5,000 rows. Refit the complete dataset.
DBSCAN Choose epsilon in processed distance units and minimum samples, including the point itself. Inspect core/border points and noise. There may be no clusters. Limited to 10,000 rows. Refit the complete dataset.
Gaussian mixture Choose the component count; inspect means, weights, full covariance matrices, and membership probabilities. Assign using fitted distributions.

K-Means and Gaussian mixtures provide New samples for raw values or a CSV, automatically reusing fitted preprocessing. With headers, selected features match by exact name and extra columns remain in output. Without headers, supply only selected features in their original column order. Refitting hierarchical clustering or DBSCAN may change cluster IDs.

Cluster metrics

Silhouette is reported when defined, excluding DBSCAN noise. Large datasets use a reproducible sample of up to 1,000 non-noise rows; the explanation reports sampling or why no score is available. It describes separation in selected processed feature geometry, not classification accuracy.

K-Means also reports inertia and Gaussian mixtures report BIC. Compare those values only on the same processed data. The cluster count, noise treatment, and preprocessing choices affect interpretation.

Validation and practical limits

Training currently runs on the UI thread, so larger fits can temporarily make the window unresponsive. Model-specific row and expansion limits help bound computation and memory but do not guarantee a quick fit.

The xUnit tests cover supported task/model combinations, invalid inputs, numerical extremes, CSV row preservation, preprocessing reuse, train/test splits, stale-result clearing, failed-fit recovery, native model disposal, Python discovery, and File menu shortcuts. Model parity tests compare held-out predictions, probabilities, uncertainty, and fitted structures with independently configured scikit-learn estimators.

Visualization tests reconstruct displayed contributions, decision paths, tree probability averages, class scores, polynomial terms, PLS rotations, and SVM scores. Tree checks include split-threshold neighbors where scikit-learn's float32 conversion matters. Clustering tests cover all four estimators, silhouette, noise handling, and graphs. MLP checks include neuron selection, scheduler settings, finite training results, and checkpoint recovery.

Workflow tests verify processed feature names, original class labels, and example values passed to explorers. Rendering tests open offscreen WinForms windows at normal and minimum sizes; they need a Windows desktop session. Set JADEML_VISUAL_QA_DIRECTORY to save previews. Test classes run sequentially because native rendering and Python configuration are shared, while explicit cross-thread tests still exercise interpreter access from multiple threads. TorchSharp tests use CPU execution without requiring a CUDA GPU.