Go to file
marco370 955a2ee125 Fix backend startup issue by resolving port conflict
Resolves an "address already in use" error by killing existing processes on port 8000 before restarting the ML backend service.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 2c691790-1a58-44ba-94dd-f03a528d1174
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/jFtLBWL
2025-11-25 17:33:54 +00:00
attached_assets Fix backend startup issue by resolving port conflict 2025-11-25 17:33:54 +00:00
client Add visual indicators for the Hybrid ML model version 2025-11-25 17:24:29 +00:00
database-schema 🚀 Release v1.0.87 2025-11-25 17:26:07 +00:00
deployment Add automatic IP blocking system to enhance security 2025-11-25 11:52:13 +00:00
extracted_idf Saved progress at the end of the loop 2025-11-11 09:15:10 +00:00
python_ml Add automatic IP blocking system to enhance security 2025-11-25 11:52:13 +00:00
server Update Mikrotik router connection settings and remove redundant tests 2025-11-25 11:29:12 +00:00
shared Update Mikrotik router connection settings and remove redundant tests 2025-11-25 11:29:12 +00:00
.env.example Update application to use environment variables and force IPv4 2025-11-17 16:48:22 +00:00
.gitignore Add ability to export database schema structure to Git 2025-11-17 17:07:31 +00:00
.replit Saved progress at the end of the loop 2025-11-25 17:29:43 +00:00
components.json Extracted stack files 2025-11-11 09:12:50 +00:00
deploy-to-gitlab.sh Add ability to export database schema structure to Git 2025-11-17 17:07:31 +00:00
design_guidelines.md Extracted stack files 2025-11-11 09:12:50 +00:00
drizzle.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00
git.env.example Add comprehensive installation and configuration guides for the IDS system 2025-11-17 15:19:13 +00:00
GUIDA_INSTALLAZIONE.md Add comprehensive installation and configuration guides for the IDS system 2025-11-17 15:19:13 +00:00
package-lock.json Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
package.json Add file management and upload capabilities for IDF Mikrotik projects 2025-11-15 10:54:17 +00:00
postcss.config.js Extracted stack files 2025-11-11 09:12:50 +00:00
push-gitlab.sh 🚀 Release v1.0.1 2025-11-17 08:45:53 +00:00
README.md Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
replit.md Add automatic cleanup for old detections and IP blocks 2025-11-25 10:40:44 +00:00
RISPOSTA_DEPLOYMENT.md Add comprehensive deployment and configuration guides for the IDS system 2025-11-15 11:30:55 +00:00
tailwind.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00
tsconfig.json Extracted stack files 2025-11-11 09:12:50 +00:00
version.json 🚀 Release v1.0.87 2025-11-25 17:26:07 +00:00
vite.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00

🛡️ IDS - Intrusion Detection System

Sistema di rilevamento intrusioni moderno per router MikroTik, basato su Machine Learning.

🎯 Caratteristiche Principali

  • ML Efficiente: Solo 25 feature mirate (non 150+) per analisi veloce e accurata
  • Detection Real-time: Rilevamento anomalie in <2 secondi
  • Multi-Router: Gestione parallela di 10+ router MikroTik tramite API REST
  • Auto-Block: Blocco automatico IP anomali con timeout configurabile
  • Dashboard Web: Monitoring real-time completo
  • PostgreSQL: Database performante per analisi time-series

🏗️ Architettura

┌─────────────────┐
│ Router MikroTik │ ──(Syslog)──▶ ┌──────────────┐
│   (10+ router)   │               │  PostgreSQL  │
└─────────────────┘               │   Database   │
                                  └──────┬───────┘
                                         │
                    ┌────────────────────┼────────────────────┐
                    │                    │                    │
             ┌──────▼─────┐     ┌───────▼────┐      ┌───────▼────┐
             │  Python ML  │     │  FastAPI   │      │   React    │
             │   Analyzer  │     │  Backend   │      │  Dashboard │
             └─────────────┘     └────────────┘      └────────────┘
                    │                    │                    │
                    └────────────────────┼────────────────────┘
                                         │
                              ┌──────────▼──────────┐
                              │  MikroTik Manager   │
                              │   (API REST)        │
                              └─────────────────────┘
                                         │
                    ┌────────────────────┼────────────────────┐
                    ▼                    ▼                    ▼
            ┌───────────┐        ┌───────────┐        ┌───────────┐
            │  Router 1 │        │  Router 2 │        │  Router N │
            └───────────┘        └───────────┘        └───────────┘

🚀 Quick Start

1. Setup Backend Python

cd python_ml
pip install -r requirements.txt
python main.py

Il backend FastAPI partirà su http://0.0.0.0:8000

2. Setup Frontend (già configurato)

Il frontend React è già in esecuzione tramite il workflow "Start application". Accedi alla dashboard web all'URL del tuo Repl.

3. Configurazione Router MikroTik

Sul router MikroTik, abilita l'API REST:

/ip service
set api-ssl disabled=no
set www-ssl disabled=no

Poi aggiungi i router tramite la dashboard web oppure:

INSERT INTO routers (name, ip_address, username, password, api_port, enabled)
VALUES ('Router 1', '192.168.1.1', 'admin', 'password', 443, true);

📊 Come Funziona

1. Raccolta Dati

I log arrivano tramite Syslog dai router MikroTik e vengono salvati in PostgreSQL nella tabella network_logs.

2. Training ML

curl -X POST http://localhost:8000/train \
  -H "Content-Type: application/json" \
  -d '{
    "max_records": 10000,
    "hours_back": 24,
    "contamination": 0.01
  }'

Il sistema estrae 25 feature mirate:

  • Volume: bytes/sec, packets, connessioni
  • Temporali: burst, intervalli, pattern orari
  • Protocolli: diversità, entropia, TCP/UDP ratio
  • Port Scanning: porte uniche, sequenziali
  • Comportamentali: varianza dimensioni, azioni bloccate

3. Detection Real-time

curl -X POST http://localhost:8000/detect \
  -H "Content-Type: application/json" \
  -d '{
    "max_records": 5000,
    "hours_back": 1,
    "risk_threshold": 60.0,
    "auto_block": true
  }'

Il modello Isolation Forest assegna:

  • Risk Score (0-100): livello di pericolosità
  • Confidence (0-100): certezza del rilevamento
  • Anomaly Type: ddos, port_scan, brute_force, botnet, suspicious

4. Auto-Block

IP con risk_score >= 80 (CRITICO) vengono bloccati automaticamente su tutti i router via API REST con timeout 1h.

🎚️ Livelli di Rischio

Score Livello Azione
85-100 🔴 CRITICO Blocco immediato
70-84 🟠 ALTO Blocco + monitoring
60-69 🟡 MEDIO Monitoring
40-59 🔵 BASSO Logging
0-39 🟢 NORMALE Nessuna azione

📚 API Endpoints

  • GET /health - Health check
  • POST /train - Training modello ML
  • POST /detect - Detection anomalie
  • POST /block-ip - Blocco manuale IP
  • POST /unblock-ip - Sblocco IP
  • GET /stats - Statistiche sistema

Documentazione completa: http://localhost:8000/docs

🔧 Configurazione Automatica

Training Automatico (ogni 12h)

0 */12 * * * curl -X POST http://localhost:8000/train

Detection Continua (ogni 5 minuti)

*/5 * * * * curl -X POST http://localhost:8000/detect \
  -H "Content-Type: application/json" \
  -d '{"auto_block": true, "risk_threshold": 75}'

🆚 Vantaggi vs Sistema Precedente

Aspetto Sistema Vecchio Nuovo IDS
Feature ML 150+ 25 (mirate)
Velocità Training ~5 min ~10 sec
Velocità Detection Lento <2 sec
Comunicazione Router SSH (lento) API REST (veloce)
Falsi Negativi Alti Bassi
Multi-Router Sequenziale Parallelo
Database MySQL PostgreSQL

🔍 Troubleshooting

Troppi Falsi Positivi?

Aumenta risk_threshold (es. da 60 a 75)

Non Rileva Attacchi?

  • Diminuisci contamination nel training (es. da 0.01 a 0.02)
  • Abbassa risk_threshold (es. da 75 a 60)

Connessione Router Fallita?

  • Verifica API REST abilitata: /ip service print
  • Controlla firewall: porta 443 deve essere aperta
  • Test: curl -u admin:password https://ROUTER_IP/rest/system/identity

📁 Struttura Progetto

.
├── python_ml/              # Backend Python ML
│   ├── ml_analyzer.py      # Analisi ML (25 feature)
│   ├── mikrotik_manager.py # Gestione router API REST
│   ├── main.py             # FastAPI backend
│   └── requirements.txt    # Dipendenze Python
├── client/                 # Frontend React
│   └── src/
│       └── pages/          # Pagine dashboard
├── server/                 # Backend Node.js
│   ├── db.ts               # Database PostgreSQL
│   ├── routes.ts           # API routes
│   └── storage.ts          # Storage interface
└── shared/
    └── schema.ts           # Schema database Drizzle ORM

🔐 Sicurezza

  • Password router NON in chiaro nel codice
  • Timeout automatico sui blocchi (default 1h)
  • Whitelist per IP fidati
  • Logging completo di tutte le azioni
  • Database PostgreSQL con connessione sicura

📝 Note Importanti

  • Whitelist: IP in whitelist non vengono mai bloccati
  • Timeout: Blocchi hanno timeout (default 1h), poi scadono automaticamente
  • Parallelo: Sistema blocca su tutti i router simultaneamente (veloce)
  • Performance: Analizza 10K log in <2 secondi

📖 Documentazione

🤝 Supporto

Per problemi o domande:

  1. Controlla questa documentazione
  2. Verifica i log di debug (python_ml/main.py)
  3. Testa la connessione database e router
  4. Verifica i modelli addestrati (python_ml/models/)

IDS - Intrusion Detection System v1.0.0
Sistema moderno e performante per proteggere la tua rete MikroTik