Update deployment process to handle machine learning dependencies
Create a dedicated script to install machine learning dependencies in the correct order, ensuring Cython is installed before packages that require it for compilation, and update documentation accordingly. Replit-Commit-Author: Agent Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528 Replit-Commit-Checkpoint-Type: full_checkpoint Replit-Commit-Event-Id: aa7dc534-7330-4bd4-b726-d6eeb29008af Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
This commit is contained in:
parent
f5e212626a
commit
b78f03392a
4
.replit
4
.replit
@ -14,6 +14,10 @@ run = ["npm", "run", "start"]
|
|||||||
localPort = 5000
|
localPort = 5000
|
||||||
externalPort = 80
|
externalPort = 80
|
||||||
|
|
||||||
|
[[ports]]
|
||||||
|
localPort = 39403
|
||||||
|
externalPort = 3001
|
||||||
|
|
||||||
[[ports]]
|
[[ports]]
|
||||||
localPort = 41303
|
localPort = 41303
|
||||||
externalPort = 3002
|
externalPort = 3002
|
||||||
|
|||||||
@ -0,0 +1,43 @@
|
|||||||
|
|
||||||
|
📦 Aggiornamento dipendenze Python...
|
||||||
|
Defaulting to user installation because normal site-packages is not writeable
|
||||||
|
Requirement already satisfied: fastapi==0.104.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (0.104.1)
|
||||||
|
Requirement already satisfied: uvicorn==0.24.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.24.0)
|
||||||
|
Requirement already satisfied: pandas==2.1.3 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (2.1.3)
|
||||||
|
Requirement already satisfied: numpy==1.26.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.26.2)
|
||||||
|
Requirement already satisfied: scikit-learn==1.3.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (1.3.2)
|
||||||
|
Requirement already satisfied: psycopg2-binary==2.9.9 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.9.9)
|
||||||
|
Requirement already satisfied: python-dotenv==1.0.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (1.0.0)
|
||||||
|
Requirement already satisfied: pydantic==2.5.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (2.5.0)
|
||||||
|
Requirement already satisfied: httpx==0.25.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.25.1)
|
||||||
|
Collecting Cython==3.0.5
|
||||||
|
Downloading Cython-3.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 8.9 MB/s eta 0:00:00
|
||||||
|
Collecting xgboost==2.0.3
|
||||||
|
Using cached xgboost-2.0.3-py3-none-manylinux2014_x86_64.whl (297.1 MB)
|
||||||
|
Collecting joblib==1.3.2
|
||||||
|
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
|
||||||
|
Collecting eif==2.0.2
|
||||||
|
Using cached eif-2.0.2.tar.gz (1.6 MB)
|
||||||
|
Preparing metadata (setup.py) ... error
|
||||||
|
error: subprocess-exited-with-error
|
||||||
|
|
||||||
|
× python setup.py egg_info did not run successfully.
|
||||||
|
│ exit code: 1
|
||||||
|
╰─> [6 lines of output]
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "<string>", line 2, in <module>
|
||||||
|
File "<pip-setuptools-caller>", line 34, in <module>
|
||||||
|
File "/tmp/pip-install-843eies2/eif_72b54a0861444b02867269ed1670c0ce/setup.py", line 4, in <module>
|
||||||
|
from Cython.Distutils import build_ext
|
||||||
|
ModuleNotFoundError: No module named 'Cython'
|
||||||
|
[end of output]
|
||||||
|
|
||||||
|
note: This error originates from a subprocess, and is likely not a problem with pip.
|
||||||
|
error: metadata-generation-failed
|
||||||
|
|
||||||
|
× Encountered error while generating package metadata.
|
||||||
|
╰─> See above for output.
|
||||||
|
|
||||||
|
note: This is an issue with the package mentioned above, not pip.
|
||||||
|
hint: See above for details.
|
||||||
@ -14,26 +14,33 @@ Sistema ML avanzato per riduzione falsi positivi 80-90% con Extended Isolation F
|
|||||||
|
|
||||||
## 🔧 Step 1: Installazione Dipendenze
|
## 🔧 Step 1: Installazione Dipendenze
|
||||||
|
|
||||||
|
⚠️ **IMPORTANTE**: Usare lo script dedicato per risolvere dipendenza Cython
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# SSH al server
|
# SSH al server
|
||||||
ssh user@ids.alfacom.it
|
ssh user@ids.alfacom.it
|
||||||
|
|
||||||
# Attiva venv
|
# Esegui script installazione ML dependencies
|
||||||
cd /opt/ids/python_ml
|
cd /opt/ids
|
||||||
source venv/bin/activate
|
chmod +x deployment/install_ml_deps.sh
|
||||||
|
./deployment/install_ml_deps.sh
|
||||||
|
|
||||||
# Installa nuove librerie
|
# Output atteso:
|
||||||
pip install -r requirements.txt
|
# ✅ Cython installato con successo
|
||||||
|
# ✅ Dipendenze ML installate con successo
|
||||||
# Verifica installazione
|
# ✅ eif importato correttamente
|
||||||
python -c "import xgboost; import eif; import joblib; print('✅ Dipendenze OK')"
|
# ✅ TUTTO OK! Hybrid ML Detector pronto per l'uso
|
||||||
```
|
```
|
||||||
|
|
||||||
**Dipendenze nuove**:
|
**Dipendenze nuove**:
|
||||||
|
- `Cython==3.0.5` - Build dependency per eif (installato per primo)
|
||||||
- `xgboost==2.0.3` - Gradient Boosting per ensemble classifier
|
- `xgboost==2.0.3` - Gradient Boosting per ensemble classifier
|
||||||
- `eif==2.0.0` - Extended Isolation Forest
|
- `eif==2.0.2` - Extended Isolation Forest
|
||||||
- `joblib==1.3.2` - Model persistence
|
- `joblib==1.3.2` - Model persistence
|
||||||
|
|
||||||
|
**Perché lo script separato?**
|
||||||
|
`eif` richiede Cython per compilare il codice durante l'installazione. Lo script installa Cython PRIMA, poi le altre dipendenze.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧪 Step 2: Quick Test (Dataset Sintetico)
|
## 🧪 Step 2: Quick Test (Dataset Sintetico)
|
||||||
|
|||||||
55
deployment/install_ml_deps.sh
Normal file
55
deployment/install_ml_deps.sh
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Script per installare dipendenze ML Hybrid Detector
|
||||||
|
# Risolve il problema di Cython richiesto come build dependency da eif
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "╔═══════════════════════════════════════════════╗"
|
||||||
|
echo "║ INSTALLAZIONE DIPENDENZE ML HYBRID ║"
|
||||||
|
echo "╚═══════════════════════════════════════════════╝"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Vai alla directory python_ml
|
||||||
|
cd "$(dirname "$0")/../python_ml" || exit 1
|
||||||
|
|
||||||
|
echo "📍 Directory corrente: $(pwd)"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# STEP 1: Installa Cython PRIMA (build dependency per eif)
|
||||||
|
echo "📦 Step 1/2: Installazione Cython (richiesto per compilare eif)..."
|
||||||
|
pip install --user Cython==3.0.5
|
||||||
|
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
echo "✅ Cython installato con successo"
|
||||||
|
else
|
||||||
|
echo "❌ Errore durante installazione Cython"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# STEP 2: Installa tutte le altre dipendenze
|
||||||
|
echo "📦 Step 2/2: Installazione dipendenze ML (xgboost, joblib, eif)..."
|
||||||
|
pip install --user xgboost==2.0.3 joblib==1.3.2 eif==2.0.2
|
||||||
|
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
echo "✅ Dipendenze ML installate con successo"
|
||||||
|
else
|
||||||
|
echo "❌ Errore durante installazione dipendenze ML"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "✅ INSTALLAZIONE COMPLETATA!"
|
||||||
|
echo ""
|
||||||
|
echo "🧪 Test import eif..."
|
||||||
|
python3 -c "from eif import iForest; print('✅ eif importato correttamente')"
|
||||||
|
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
echo ""
|
||||||
|
echo "✅ TUTTO OK! Hybrid ML Detector pronto per l'uso"
|
||||||
|
else
|
||||||
|
echo "❌ Errore durante test import eif"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
@ -103,6 +103,8 @@ The IDS employs a React-based frontend for real-time monitoring, detection visua
|
|||||||
- **Backward Compatibility**: USE_HYBRID_DETECTOR env var (default=true)
|
- **Backward Compatibility**: USE_HYBRID_DETECTOR env var (default=true)
|
||||||
- **Target Metrics**: Precision≥90%, Recall≥80%, FPR≤5%, F1≥85%
|
- **Target Metrics**: Precision≥90%, Recall≥80%, FPR≤5%, F1≥85%
|
||||||
- **Deploy**: Vedere `deployment/CHECKLIST_ML_HYBRID.md`
|
- **Deploy**: Vedere `deployment/CHECKLIST_ML_HYBRID.md`
|
||||||
- **Fix Deploy (24 Nov 2025 - 19:15)**:
|
- **Fix Deploy (24 Nov 2025 - 19:30)**:
|
||||||
- Corretto `eif==2.0.0` → `eif==2.0.2` (versione 2.0.0 non disponibile)
|
- Corretto `eif==2.0.0` → `eif==2.0.2` (versione 2.0.0 non disponibile)
|
||||||
- Aggiunto `Cython==3.0.5` come dipendenza (richiesto per compilare eif da source)
|
- Aggiunto `Cython==3.0.5` come build dependency (eif richiede compilazione)
|
||||||
|
- Creato `deployment/install_ml_deps.sh` per installazione in 2 fasi (Cython → eif)
|
||||||
|
- **Soluzione**: pip non installa Cython in tempo per eif → script separa installazione
|
||||||
Loading…
Reference in New Issue
Block a user