# IDS - Intrusion Detection System

## Overview
This project is a full-stack web application for an Intrusion Detection System (IDS) tailored for MikroTik routers, utilizing Machine Learning. Its core function is to monitor network traffic, identify anomalies indicative of intrusions, and automatically block malicious IP addresses across multiple routers. The system aims to provide real-time monitoring, efficient anomaly detection, and streamlined network security management for MikroTik environments, including advanced features like IP geolocation and robust service monitoring.

## User Preferences
### Operazioni Git e Deployment
- **IMPORTANTE**: L'agente NON deve usare comandi git (push-gitlab.sh) perché Replit blocca le operazioni git
- **Workflow corretto**:
  1. Utente riporta errori/problemi dal server AlmaLinux
  2. Agente risolve problemi e modifica file su Replit
  3. **Utente esegue manualmente**: `./push-gitlab.sh` per commit+push
  4. **Utente esegue sul server**: `./update_from_git.sh` o `./update_from_git.sh --db`
  5. Utente testa e riporta risultati all'agente
  6. Ripeti fino a funzionamento completo

### Linguaggio
- Tutte le risposte dell'agente devono essere in **italiano**
- Codice e documentazione tecnica: inglese
- Commit message: italiano

## System Architecture
The IDS employs a React-based frontend for real-time monitoring, detection visualization, and whitelist management, built with ShadCN UI and TanStack Query. The backend consists of a Python FastAPI service dedicated to ML analysis and a Node.js (Express) backend handling API requests, PostgreSQL database management, and service coordination.

**Key Architectural Decisions & Features:**
-   **Log Collection & Processing**: MikroTik syslog data (UDP:514) is parsed by `syslog_parser.py` and stored in PostgreSQL with a 3-day retention policy. The parser includes auto-reconnect and error recovery mechanisms.
-   **Machine Learning**: An Isolation Forest model (sklearn.IsolationForest) trained on 25 network log features performs real-time anomaly detection, assigning a risk score (0-100 across five risk levels). A hybrid ML detector (Isolation Forest + Ensemble Classifier with weighted voting) reduces false positives. The system supports weekly automatic retraining of models.
-   **Automated Blocking**: Critical IPs (score >= 80) are automatically blocked in parallel across configured MikroTik routers via their REST API.
-   **Automatic Cleanup**: An hourly systemd timer (`cleanup_detections.py`) removes old detections (48h) and auto-unblocks IPs (2h).
-   **Service Monitoring & Management**: A dashboard provides real-time status (ML Backend, Database, Syslog Parser). API endpoints, secured with API key authentication and Systemd integration, allow for service management (start/stop/restart) of Python services.
-   **IP Geolocation**: Integration with `ip-api.com` enriches detection data with geographical and AS information, utilizing intelligent caching.
-   **Database Management**: PostgreSQL is used for all persistent data. An intelligent database versioning system ensures efficient SQL migrations. Dual-mode database drivers (`@neondatabase/serverless` for Replit, `pg` for AlmaLinux) ensure environment compatibility.
-   **Microservices**: Clear separation of concerns between the Python ML backend and the Node.js API backend.
-   **UI/UX**: Utilizes ShadCN UI for a modern component library and `react-hook-form` with Zod for robust form validation. Analytics dashboards provide visualizations of normal and attack traffic, including real-time and historical data.

## External Dependencies
-   **React**: Frontend framework.
-   **FastAPI**: Python web framework for the ML backend.
-   **PostgreSQL**: Primary database for storing configurations, logs, detections, and whitelist entries.
-   **MikroTik API REST**: For router communication and IP blocking.
-   **ShadCN UI**: Frontend component library.
-   **TanStack Query**: Data fetching for the frontend.
-   **Isolation Forest (scikit-learn)**: Machine Learning algorithm for anomaly detection.
-   **xgboost, joblib**: ML libraries used in the hybrid detector.
-   **RSyslog**: Log collection daemon.
-   **Drizzle ORM**: For database schema definition in Node.js.
-   **Neon Database**: Cloud-native PostgreSQL service (used in Replit).
-   **pg (Node.js driver)**: Standard PostgreSQL driver for Node.js (used in AlmaLinux).
-   **psycopg2**: PostgreSQL adapter for Python.
-   **ip-api.com**: External API for IP geolocation data.
-   **Recharts**: Charting library for analytics visualization.