Go to file
marco370 46ab780e60 Add detailed logging to track the syslog parser's execution and state
Enhance syslog_parser.py with debug, info, and error logging statements, including checks for log file existence and robust error handling for database connections and parsing.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 1663254c-86fe-4c1e-966a-168eb8cd8f97
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:46:37 +00:00
attached_assets Fix error when saving network logs to the database 2025-11-17 17:40:22 +00:00
client Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
database-backups 🚀 Release v1.0.11 2025-11-17 17:44:26 +00:00
database-schema Update deployment instructions and network log schema 2025-11-17 17:42:17 +00:00
deployment Provide initial setup instructions for deploying the application 2025-11-17 17:15:31 +00:00
extracted_idf Saved progress at the end of the loop 2025-11-11 09:15:10 +00:00
python_ml Add detailed logging to track the syslog parser's execution and state 2025-11-17 17:46:37 +00:00
server Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
shared Update deployment instructions and network log schema 2025-11-17 17:42:17 +00:00
.env.example Update application to use environment variables and force IPv4 2025-11-17 16:48:22 +00:00
.gitignore Add ability to export database schema structure to Git 2025-11-17 17:07:31 +00:00
.replit 🚀 Release v1.0.10 2025-11-17 17:38:30 +00:00
components.json Extracted stack files 2025-11-11 09:12:50 +00:00
deploy-to-gitlab.sh Add ability to export database schema structure to Git 2025-11-17 17:07:31 +00:00
design_guidelines.md Extracted stack files 2025-11-11 09:12:50 +00:00
drizzle.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00
git.env.example Add comprehensive installation and configuration guides for the IDS system 2025-11-17 15:19:13 +00:00
GUIDA_INSTALLAZIONE.md Add comprehensive installation and configuration guides for the IDS system 2025-11-17 15:19:13 +00:00
package-lock.json Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
package.json Add file management and upload capabilities for IDF Mikrotik projects 2025-11-15 10:54:17 +00:00
postcss.config.js Extracted stack files 2025-11-11 09:12:50 +00:00
push-gitlab.sh 🚀 Release v1.0.1 2025-11-17 08:45:53 +00:00
README.md Add a navigation sidebar and dashboard to the IDS system 2025-11-15 11:16:44 +00:00
replit.md Update deployment instructions and network log schema 2025-11-17 17:42:17 +00:00
RISPOSTA_DEPLOYMENT.md Add comprehensive deployment and configuration guides for the IDS system 2025-11-15 11:30:55 +00:00
tailwind.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00
tsconfig.json Extracted stack files 2025-11-11 09:12:50 +00:00
update_from_git.sh Update script to apply multiple database migration scripts 2025-11-17 17:44:11 +00:00
version.json 🚀 Release v1.0.11 2025-11-17 17:44:26 +00:00
vite.config.ts Extracted stack files 2025-11-11 09:12:50 +00:00

🛡️ IDS - Intrusion Detection System

Sistema di rilevamento intrusioni moderno per router MikroTik, basato su Machine Learning.

🎯 Caratteristiche Principali

  • ML Efficiente: Solo 25 feature mirate (non 150+) per analisi veloce e accurata
  • Detection Real-time: Rilevamento anomalie in <2 secondi
  • Multi-Router: Gestione parallela di 10+ router MikroTik tramite API REST
  • Auto-Block: Blocco automatico IP anomali con timeout configurabile
  • Dashboard Web: Monitoring real-time completo
  • PostgreSQL: Database performante per analisi time-series

🏗️ Architettura

┌─────────────────┐
│ Router MikroTik │ ──(Syslog)──▶ ┌──────────────┐
│   (10+ router)   │               │  PostgreSQL  │
└─────────────────┘               │   Database   │
                                  └──────┬───────┘
                                         │
                    ┌────────────────────┼────────────────────┐
                    │                    │                    │
             ┌──────▼─────┐     ┌───────▼────┐      ┌───────▼────┐
             │  Python ML  │     │  FastAPI   │      │   React    │
             │   Analyzer  │     │  Backend   │      │  Dashboard │
             └─────────────┘     └────────────┘      └────────────┘
                    │                    │                    │
                    └────────────────────┼────────────────────┘
                                         │
                              ┌──────────▼──────────┐
                              │  MikroTik Manager   │
                              │   (API REST)        │
                              └─────────────────────┘
                                         │
                    ┌────────────────────┼────────────────────┐
                    ▼                    ▼                    ▼
            ┌───────────┐        ┌───────────┐        ┌───────────┐
            │  Router 1 │        │  Router 2 │        │  Router N │
            └───────────┘        └───────────┘        └───────────┘

🚀 Quick Start

1. Setup Backend Python

cd python_ml
pip install -r requirements.txt
python main.py

Il backend FastAPI partirà su http://0.0.0.0:8000

2. Setup Frontend (già configurato)

Il frontend React è già in esecuzione tramite il workflow "Start application". Accedi alla dashboard web all'URL del tuo Repl.

3. Configurazione Router MikroTik

Sul router MikroTik, abilita l'API REST:

/ip service
set api-ssl disabled=no
set www-ssl disabled=no

Poi aggiungi i router tramite la dashboard web oppure:

INSERT INTO routers (name, ip_address, username, password, api_port, enabled)
VALUES ('Router 1', '192.168.1.1', 'admin', 'password', 443, true);

📊 Come Funziona

1. Raccolta Dati

I log arrivano tramite Syslog dai router MikroTik e vengono salvati in PostgreSQL nella tabella network_logs.

2. Training ML

curl -X POST http://localhost:8000/train \
  -H "Content-Type: application/json" \
  -d '{
    "max_records": 10000,
    "hours_back": 24,
    "contamination": 0.01
  }'

Il sistema estrae 25 feature mirate:

  • Volume: bytes/sec, packets, connessioni
  • Temporali: burst, intervalli, pattern orari
  • Protocolli: diversità, entropia, TCP/UDP ratio
  • Port Scanning: porte uniche, sequenziali
  • Comportamentali: varianza dimensioni, azioni bloccate

3. Detection Real-time

curl -X POST http://localhost:8000/detect \
  -H "Content-Type: application/json" \
  -d '{
    "max_records": 5000,
    "hours_back": 1,
    "risk_threshold": 60.0,
    "auto_block": true
  }'

Il modello Isolation Forest assegna:

  • Risk Score (0-100): livello di pericolosità
  • Confidence (0-100): certezza del rilevamento
  • Anomaly Type: ddos, port_scan, brute_force, botnet, suspicious

4. Auto-Block

IP con risk_score >= 80 (CRITICO) vengono bloccati automaticamente su tutti i router via API REST con timeout 1h.

🎚️ Livelli di Rischio

Score Livello Azione
85-100 🔴 CRITICO Blocco immediato
70-84 🟠 ALTO Blocco + monitoring
60-69 🟡 MEDIO Monitoring
40-59 🔵 BASSO Logging
0-39 🟢 NORMALE Nessuna azione

📚 API Endpoints

  • GET /health - Health check
  • POST /train - Training modello ML
  • POST /detect - Detection anomalie
  • POST /block-ip - Blocco manuale IP
  • POST /unblock-ip - Sblocco IP
  • GET /stats - Statistiche sistema

Documentazione completa: http://localhost:8000/docs

🔧 Configurazione Automatica

Training Automatico (ogni 12h)

0 */12 * * * curl -X POST http://localhost:8000/train

Detection Continua (ogni 5 minuti)

*/5 * * * * curl -X POST http://localhost:8000/detect \
  -H "Content-Type: application/json" \
  -d '{"auto_block": true, "risk_threshold": 75}'

🆚 Vantaggi vs Sistema Precedente

Aspetto Sistema Vecchio Nuovo IDS
Feature ML 150+ 25 (mirate)
Velocità Training ~5 min ~10 sec
Velocità Detection Lento <2 sec
Comunicazione Router SSH (lento) API REST (veloce)
Falsi Negativi Alti Bassi
Multi-Router Sequenziale Parallelo
Database MySQL PostgreSQL

🔍 Troubleshooting

Troppi Falsi Positivi?

Aumenta risk_threshold (es. da 60 a 75)

Non Rileva Attacchi?

  • Diminuisci contamination nel training (es. da 0.01 a 0.02)
  • Abbassa risk_threshold (es. da 75 a 60)

Connessione Router Fallita?

  • Verifica API REST abilitata: /ip service print
  • Controlla firewall: porta 443 deve essere aperta
  • Test: curl -u admin:password https://ROUTER_IP/rest/system/identity

📁 Struttura Progetto

.
├── python_ml/              # Backend Python ML
│   ├── ml_analyzer.py      # Analisi ML (25 feature)
│   ├── mikrotik_manager.py # Gestione router API REST
│   ├── main.py             # FastAPI backend
│   └── requirements.txt    # Dipendenze Python
├── client/                 # Frontend React
│   └── src/
│       └── pages/          # Pagine dashboard
├── server/                 # Backend Node.js
│   ├── db.ts               # Database PostgreSQL
│   ├── routes.ts           # API routes
│   └── storage.ts          # Storage interface
└── shared/
    └── schema.ts           # Schema database Drizzle ORM

🔐 Sicurezza

  • Password router NON in chiaro nel codice
  • Timeout automatico sui blocchi (default 1h)
  • Whitelist per IP fidati
  • Logging completo di tutte le azioni
  • Database PostgreSQL con connessione sicura

📝 Note Importanti

  • Whitelist: IP in whitelist non vengono mai bloccati
  • Timeout: Blocchi hanno timeout (default 1h), poi scadono automaticamente
  • Parallelo: Sistema blocca su tutti i router simultaneamente (veloce)
  • Performance: Analizza 10K log in <2 secondi

📖 Documentazione

🤝 Supporto

Per problemi o domande:

  1. Controlla questa documentazione
  2. Verifica i log di debug (python_ml/main.py)
  3. Testa la connessione database e router
  4. Verifica i modelli addestrati (python_ml/models/)

IDS - Intrusion Detection System v1.0.0
Sistema moderno e performante per proteggere la tua rete MikroTik