marco/ids.alfacom.it
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
v1.0.100
ids.alfacom.it/replit.md
marco370 dee64495cd Add ability to manually unblock IPs and improve API key handling 
Add a "Unblock Router" button to the Detections page and integrate ML backend API key for authenticated requests.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 3f5fe7aa-6fa1-4aa6-a5b4-916f113bf5df
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C6BdLIt
2026-01-02 15:50:17 +00:00

5.4 KiB
Raw Permalink Blame History

IDS - Intrusion Detection System

Overview

This project is a full-stack web application for an Intrusion Detection System (IDS) tailored for MikroTik routers, utilizing Machine Learning. Its core function is to monitor network traffic, identify anomalies indicative of intrusions, and automatically block malicious IP addresses across multiple routers. The system aims to provide real-time monitoring, efficient anomaly detection, and streamlined network security management for MikroTik environments, including advanced features like IP geolocation and robust service monitoring.

User Preferences

Operazioni Git e Deployment

  • IMPORTANTE: L'agente NON deve usare comandi git (push-gitlab.sh) perché Replit blocca le operazioni git
  • Workflow corretto:
    1. Utente riporta errori/problemi dal server AlmaLinux
    2. Agente risolve problemi e modifica file su Replit
    3. Utente esegue manualmente: ./push-gitlab.sh per commit+push
    4. Utente esegue sul server: ./update_from_git.sh o ./update_from_git.sh --db
    5. Utente testa e riporta risultati all'agente
    6. Ripeti fino a funzionamento completo

Linguaggio

  • Tutte le risposte dell'agente devono essere in italiano
  • Codice e documentazione tecnica: inglese
  • Commit message: italiano

System Architecture

The IDS employs a React-based frontend for real-time monitoring, detection visualization, and whitelist management, built with ShadCN UI and TanStack Query. The backend consists of a Python FastAPI service dedicated to ML analysis and a Node.js (Express) backend handling API requests, PostgreSQL database management, and service coordination.

Key Architectural Decisions & Features:

  • Log Collection & Processing: MikroTik syslog data (UDP:514) is parsed by syslog_parser.py and stored in PostgreSQL with a 3-day retention policy. The parser includes auto-reconnect and error recovery mechanisms.
  • Machine Learning: An Isolation Forest model (sklearn.IsolectionForest) trained on 25 network log features performs real-time anomaly detection, assigning a risk score (0-100 across five risk levels). A hybrid ML detector (Isolation Forest + Ensemble Classifier with weighted voting) reduces false positives. The system supports weekly automatic retraining of models.
  • Automated Blocking: Critical IPs (score >= 80) are automatically blocked in parallel across configured MikroTik routers via their REST API. Auto-unblock on whitelist: When an IP is added to the whitelist, it is automatically removed from all router blocklists. Manual unblock button available in Detections page.
  • Public Lists Integration (v2.0.0 - CIDR Complete): Automatic fetcher syncs blacklist/whitelist feeds every 10 minutes (Spamhaus, Talos, AWS, GCP, Cloudflare, IANA, NTP Pool). Full CIDR support using PostgreSQL INET/CIDR types with <<= containment operators for network range matching. Priority-based merge logic: Manual whitelist > Public whitelist > Blacklist (CIDR-aware). Detections created for blacklisted IPs/ranges (excluding whitelisted ranges). CRUD API + UI for list management. See deployment/docs/PUBLIC_LISTS_V2_CIDR.md for implementation details.
  • Automatic Cleanup: An hourly systemd timer (cleanup_detections.py) removes old detections (48h) and auto-unblocks IPs (2h).
  • Service Monitoring & Management: A dashboard provides real-time status (ML Backend, Database, Syslog Parser). API endpoints, secured with API key authentication and Systemd integration, allow for service management (start/stop/restart) of Python services.
  • IP Geolocation: Integration with ip-api.com enriches detection data with geographical and AS information, utilizing intelligent caching.
  • Database Management: PostgreSQL is used for all persistent data. An intelligent database versioning system ensures efficient SQL migrations (v8 with forced INET/CIDR column types for network range matching). Migration 008 unconditionally recreates INET columns to fix type mismatches. Dual-mode database drivers (@neondatabase/serverless for Replit, pg for AlmaLinux) ensure environment compatibility.
  • Microservices: Clear separation of concerns between the Python ML backend and the Node.js API backend.
  • UI/UX: Utilizes ShadCN UI for a modern component library and react-hook-form with Zod for robust form validation. Analytics dashboards provide visualizations of normal and attack traffic, including real-time and historical data.

External Dependencies

  • React: Frontend framework.
  • FastAPI: Python web framework for the ML backend.
  • PostgreSQL: Primary database for storing configurations, logs, detections, and whitelist entries.
  • MikroTik API REST: For router communication and IP blocking.
  • ShadCN UI: Frontend component library.
  • TanStack Query: Data fetching for the frontend.
  • Isolation Forest (scikit-learn): Machine Learning algorithm for anomaly detection.
  • xgboost, joblib: ML libraries used in the hybrid detector.
  • RSyslog: Log collection daemon.
  • Drizzle ORM: For database schema definition in Node.js.
  • Neon Database: Cloud-native PostgreSQL service (used in Replit).
  • pg (Node.js driver): Standard PostgreSQL driver for Node.js (used in AlmaLinux).
  • psycopg2: PostgreSQL adapter for Python.
  • ip-api.com: External API for IP geolocation data.
  • Recharts: Charting library for analytics visualization.