Genetic Programming for Non-Linear Feature Extraction & Dimensionality Reduction

An academic-grade Evolutionary Machine Learning Infrastructure engineered from scratch in Python to perform non-linear symbolical feature mapping.

This repository leverages Genetic Programming (GP) via the DEAP library to evolve mathematical expression trees. These trees map complex, non-linearly separable topological layouts (circular matrices and interleaved synthetic moons) into high-discrimination vector metrics optimized for downstream supervised classification and unsupervised density clustering.

---

🛠️ Technology Stack & Toolkits

• Evolutionary Framework: DEAP (Distributed Evolutionary Algorithms in Python)
• Predictive Suite: Scikit-Learn (Gaussian Naive Bayes, DBSCAN, PCA, StandardScaler)
• Data Engineering: NumPy, Pandas
• Visualization: Matplotlib, Seaborn

---

📐 Mathematical Framework & Fitness Strategies

1. Supervised Optimization (GP-Naive Bayes Pipeline)

To isolate highly complex concentric clusters where class boundary intersections ($C=2$) are non-linear, the evolutionary tree searches for an optimal function $\phi(x_1, x_2)$ using a protected division operator:

$$\text{protected_div}(a, b) = \begin{cases} \frac{a}{b} & \text{if } b \neq 0 \ 1 & \text{if } b = 0 \end{cases}$$

The fitness configuration maximizes the empirical classification accuracy score of a Gaussian Naive Bayes classifier trained inside the transformed mathematical space:

$$\text{Fitness}_{\text{Supervised}} = \text{Accuracy}\Big(\text{GNB}\big(\phi(X_{\text{train}})\big), y_{\text{test}}\Big)$$

2. Unsupervised Clustering Optimization (GP-DBSCAN Pipeline)

For density-based partition profiles where boundary shapes are intertwined (e.g., Make Moons), the GP architecture optimizes coordinate metrics to maximize the Fisher's Score Criterion, improving cluster density separations for DBSCAN allocation loops:

$$\text{Fitness}_{\text{Unsupervised}} = \text{Fisher's Score} = \frac{\sum_{k=1}^{K} n_k (\mu_k - \mu)^2}{\sum_{k=1}^{K} n_k \sigma_k^2}$$

---

🚀 Algorithmic Modules

• gp_naive_bayes_classifier.py: Initiates a 300-individual tree population evolved over 50 generations using a 70% crossover (cxOnePoint) and 30% uniform mutation probability layout. Maps nested coordinates into strict Gaussian structures.
• gp_dbscan_clustering.py: Implements unsupervised mapping utilizing a multi-operator mathematical primitive primitive set to maximize core cluster distance tracking metrics.

---

📈 Visual Analytics & Decoded Cluster Results

1. Concentric Decision Boundary Mappings

Evolving tree transformations enables standard linear and probabilistic density models to successfully map coordinates with peak validation limits.

2. Unsupervised Density Aggregations

Shows the optimized DBSCAN spatial partition after evolutionary feature alignment, removing high-frequency dimensional noise spikes.

---

💻 Installation & Local Verification

1. Clone Toolkit Infrastructure

git clone [https://github.com/mrhashx/genetic-programming-feature-extraction.git](https://github.com/mrhashx/genetic-programming-feature-extraction.git)
cd genetic-programming-feature-extraction

2. Execute Pipelines

python gp_naive_bayes_classifier.py
python gp_dbscan_clustering.py