Update deployment process to handle machine learning dependencies

Create a dedicated script to install machine learning dependencies in the correct order, ensuring Cython is installed before packages that require it for compilation, and update documentation accordingly.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: aa7dc534-7330-4bd4-b726-d6eeb29008af
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
This commit is contained in:
marco370 2025-11-24 16:38:41 +00:00
parent f5e212626a
commit b78f03392a
5 changed files with 122 additions and 11 deletions

View File

@ -14,6 +14,10 @@ run = ["npm", "run", "start"]
localPort = 5000 localPort = 5000
externalPort = 80 externalPort = 80
[[ports]]
localPort = 39403
externalPort = 3001
[[ports]] [[ports]]
localPort = 41303 localPort = 41303
externalPort = 3002 externalPort = 3002

View File

@ -0,0 +1,43 @@
📦 Aggiornamento dipendenze Python...
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: fastapi==0.104.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (0.104.1)
Requirement already satisfied: uvicorn==0.24.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.24.0)
Requirement already satisfied: pandas==2.1.3 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (2.1.3)
Requirement already satisfied: numpy==1.26.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.26.2)
Requirement already satisfied: scikit-learn==1.3.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (1.3.2)
Requirement already satisfied: psycopg2-binary==2.9.9 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.9.9)
Requirement already satisfied: python-dotenv==1.0.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (1.0.0)
Requirement already satisfied: pydantic==2.5.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (2.5.0)
Requirement already satisfied: httpx==0.25.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.25.1)
Collecting Cython==3.0.5
Downloading Cython-3.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 8.9 MB/s eta 0:00:00
Collecting xgboost==2.0.3
Using cached xgboost-2.0.3-py3-none-manylinux2014_x86_64.whl (297.1 MB)
Collecting joblib==1.3.2
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Collecting eif==2.0.2
Using cached eif-2.0.2.tar.gz (1.6 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-843eies2/eif_72b54a0861444b02867269ed1670c0ce/setup.py", line 4, in <module>
from Cython.Distutils import build_ext
ModuleNotFoundError: No module named 'Cython'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

View File

@ -14,26 +14,33 @@ Sistema ML avanzato per riduzione falsi positivi 80-90% con Extended Isolation F
## 🔧 Step 1: Installazione Dipendenze ## 🔧 Step 1: Installazione Dipendenze
⚠️ **IMPORTANTE**: Usare lo script dedicato per risolvere dipendenza Cython
```bash ```bash
# SSH al server # SSH al server
ssh user@ids.alfacom.it ssh user@ids.alfacom.it
# Attiva venv # Esegui script installazione ML dependencies
cd /opt/ids/python_ml cd /opt/ids
source venv/bin/activate chmod +x deployment/install_ml_deps.sh
./deployment/install_ml_deps.sh
# Installa nuove librerie # Output atteso:
pip install -r requirements.txt # ✅ Cython installato con successo
# ✅ Dipendenze ML installate con successo
# Verifica installazione # ✅ eif importato correttamente
python -c "import xgboost; import eif; import joblib; print('✅ Dipendenze OK')" # ✅ TUTTO OK! Hybrid ML Detector pronto per l'uso
``` ```
**Dipendenze nuove**: **Dipendenze nuove**:
- `Cython==3.0.5` - Build dependency per eif (installato per primo)
- `xgboost==2.0.3` - Gradient Boosting per ensemble classifier - `xgboost==2.0.3` - Gradient Boosting per ensemble classifier
- `eif==2.0.0` - Extended Isolation Forest - `eif==2.0.2` - Extended Isolation Forest
- `joblib==1.3.2` - Model persistence - `joblib==1.3.2` - Model persistence
**Perché lo script separato?**
`eif` richiede Cython per compilare il codice durante l'installazione. Lo script installa Cython PRIMA, poi le altre dipendenze.
--- ---
## 🧪 Step 2: Quick Test (Dataset Sintetico) ## 🧪 Step 2: Quick Test (Dataset Sintetico)

View File

@ -0,0 +1,55 @@
#!/bin/bash
# Script per installare dipendenze ML Hybrid Detector
# Risolve il problema di Cython richiesto come build dependency da eif
set -e
echo "╔═══════════════════════════════════════════════╗"
echo "║ INSTALLAZIONE DIPENDENZE ML HYBRID ║"
echo "╚═══════════════════════════════════════════════╝"
echo ""
# Vai alla directory python_ml
cd "$(dirname "$0")/../python_ml" || exit 1
echo "📍 Directory corrente: $(pwd)"
echo ""
# STEP 1: Installa Cython PRIMA (build dependency per eif)
echo "📦 Step 1/2: Installazione Cython (richiesto per compilare eif)..."
pip install --user Cython==3.0.5
if [ $? -eq 0 ]; then
echo "✅ Cython installato con successo"
else
echo "❌ Errore durante installazione Cython"
exit 1
fi
echo ""
# STEP 2: Installa tutte le altre dipendenze
echo "📦 Step 2/2: Installazione dipendenze ML (xgboost, joblib, eif)..."
pip install --user xgboost==2.0.3 joblib==1.3.2 eif==2.0.2
if [ $? -eq 0 ]; then
echo "✅ Dipendenze ML installate con successo"
else
echo "❌ Errore durante installazione dipendenze ML"
exit 1
fi
echo ""
echo "✅ INSTALLAZIONE COMPLETATA!"
echo ""
echo "🧪 Test import eif..."
python3 -c "from eif import iForest; print('✅ eif importato correttamente')"
if [ $? -eq 0 ]; then
echo ""
echo "✅ TUTTO OK! Hybrid ML Detector pronto per l'uso"
else
echo "❌ Errore durante test import eif"
exit 1
fi

View File

@ -103,6 +103,8 @@ The IDS employs a React-based frontend for real-time monitoring, detection visua
- **Backward Compatibility**: USE_HYBRID_DETECTOR env var (default=true) - **Backward Compatibility**: USE_HYBRID_DETECTOR env var (default=true)
- **Target Metrics**: Precision≥90%, Recall≥80%, FPR≤5%, F1≥85% - **Target Metrics**: Precision≥90%, Recall≥80%, FPR≤5%, F1≥85%
- **Deploy**: Vedere `deployment/CHECKLIST_ML_HYBRID.md` - **Deploy**: Vedere `deployment/CHECKLIST_ML_HYBRID.md`
- **Fix Deploy (24 Nov 2025 - 19:15)**: - **Fix Deploy (24 Nov 2025 - 19:30)**:
- Corretto `eif==2.0.0``eif==2.0.2` (versione 2.0.0 non disponibile) - Corretto `eif==2.0.0``eif==2.0.2` (versione 2.0.0 non disponibile)
- Aggiunto `Cython==3.0.5` come dipendenza (richiesto per compilare eif da source) - Aggiunto `Cython==3.0.5` come build dependency (eif richiede compilazione)
- Creato `deployment/install_ml_deps.sh` per installazione in 2 fasi (Cython → eif)
- **Soluzione**: pip non installa Cython in tempo per eif → script separa installazione