1b9df79d56
Integrates IP geolocation and Autonomous System (AS) information into detection records by modifying the frontend to display this data and updating the backend to perform asynchronous batch lookups for efficiency. This enhancement includes database schema updates and the creation of a new IP geolocation service. Replit-Commit-Author: Agent Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528 Replit-Commit-Checkpoint-Type: intermediate_checkpoint Replit-Commit-Event-Id: e81fd4a1-b7b0-48d2-ae38-f5905e278343 Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/SXFWABi
48 lines
4.2 KiB
Markdown
48 lines
4.2 KiB
Markdown
# IDS - Intrusion Detection System
|
|
|
|
## Overview
|
|
This project is a full-stack web application for an Intrusion Detection System (IDS) tailored for MikroTik routers, utilizing Machine Learning. Its core function is to monitor network traffic, identify anomalies indicative of intrusions, and automatically block malicious IP addresses across multiple routers. The system aims to provide real-time monitoring, efficient anomaly detection, and streamlined network security management for MikroTik environments, including advanced features like IP geolocation and robust service monitoring.
|
|
|
|
## User Preferences
|
|
### Operazioni Git e Deployment
|
|
- **IMPORTANTE**: L'agente NON deve usare comandi git (push-gitlab.sh) perché Replit blocca le operazioni git
|
|
- **Workflow corretto**:
|
|
1. Utente riporta errori/problemi dal server AlmaLinux
|
|
2. Agente risolve problemi e modifica file su Replit
|
|
3. **Utente esegue manualmente**: `./push-gitlab.sh` per commit+push
|
|
4. **Utente esegue sul server**: `./update_from_git.sh` o `./update_from_git.sh --db`
|
|
5. Utente testa e riporta risultati all'agente
|
|
6. Ripeti fino a funzionamento completo
|
|
|
|
### Linguaggio
|
|
- Tutte le risposte dell'agente devono essere in **italiano**
|
|
- Codice e documentazione tecnica: inglese
|
|
- Commit message: italiano
|
|
|
|
## System Architecture
|
|
The IDS employs a React-based frontend for real-time monitoring, detection visualization, and whitelist management, built with ShadCN UI and TanStack Query. The backend consists of a Python FastAPI service dedicated to ML analysis (Isolation Forest with 25 targeted features), MikroTik API management, and a detection engine that scores anomalies from 0-100 across five risk levels. A Node.js (Express) backend handles API requests from the frontend, manages the PostgreSQL database, and coordinates service operations.
|
|
|
|
**Key Architectural Decisions & Features:**
|
|
- **Log Collection & Processing**: MikroTik syslog data (UDP:514) is sent to RSyslog, parsed by `syslog_parser.py`, and stored in PostgreSQL. The parser includes auto-cleanup with a 3-day retention policy.
|
|
- **Machine Learning**: An Isolation Forest model trained on 25 network log features performs real-time anomaly detection, assigning a risk score.
|
|
- **Automated Blocking**: Critical IPs (score >= 80) are automatically blocked in parallel across all configured MikroTik routers via their REST API.
|
|
- **Service Monitoring & Management**: A dashboard provides real-time status (green/red indicators) for the ML Backend, Database, and Syslog Parser. Service management (start/stop/restart) for Python services is available via API endpoints, secured with API key authentication and Systemd integration for production-grade control and auto-restart capabilities.
|
|
- **IP Geolocation**: Integrated `ip-api.com` for enriching detection data with geographical and Autonomous System (AS) information, including intelligent caching.
|
|
- **Database Management**: PostgreSQL is used for all persistent data. An intelligent database versioning system ensures efficient SQL migrations, applying only new scripts. Dual-mode database drivers (`@neondatabase/serverless` for Replit, `pg` for AlmaLinux) ensure environment compatibility.
|
|
- **Microservices**: Clear separation of concerns between the Python ML backend and the Node.js API backend.
|
|
- **UI/UX**: Utilizes ShadCN UI for a modern component library and `react-hook-form` with Zod for robust form validation.
|
|
|
|
## External Dependencies
|
|
- **React**: Frontend framework.
|
|
- **FastAPI**: Python web framework for the ML backend.
|
|
- **PostgreSQL**: Primary database for storing configurations, logs, detections, and whitelist entries.
|
|
- **MikroTik API REST**: For router communication and IP blocking.
|
|
- **ShadCN UI**: Frontend component library.
|
|
- **TanStack Query**: Data fetching for the frontend.
|
|
- **Isolation Forest**: Machine Learning algorithm for anomaly detection.
|
|
- **RSyslog**: Log collection daemon.
|
|
- **Drizzle ORM**: For database schema definition in Node.js.
|
|
- **Neon Database**: Cloud-native PostgreSQL service (used in Replit).
|
|
- **pg (Node.js driver)**: Standard PostgreSQL driver for Node.js (used in AlmaLinux).
|
|
- **psycopg2**: PostgreSQL adapter for Python.
|
|
- **ip-api.com**: External API for IP geolocation data. |