Set up system to receive and store MikroTik logs

Add rsyslog configuration for receiving MikroTik logs via UDP, store them in a dedicated file, and prevent duplicates in system messages. Replit-Commit-Author: Agent Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528 Replit-Commit-Checkpoint-Type: full_checkpoint Replit-Commit-Event-Id: b452008c-bd98-4e68-81a9-f20d3f714372 Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/DR50xVM
2025-11-21 17:26:52 +00:00 · 2025-11-21 17:26:52 +00:00 · c9b2a8a9a9
commit c9b2a8a9a9
parent b31b0ec932
5 changed files with 265 additions and 0 deletions
--- a/.replit
+++ b/.replit
@ -18,6 +18,10 @@ externalPort = 80
 localPort = 41303
 externalPort = 3002

+[[ports]]
+localPort = 43551
+externalPort = 3001
+
 [[ports]]
 localPort = 43803
 externalPort = 3000
--- a/deployment/rsyslog/99-mikrotik.conf
+++ b/deployment/rsyslog/99-mikrotik.conf
@ -0,0 +1,37 @@
+# =============================================================================
+# RSYSLOG CONFIG - LOG MIKROTIK IDS
+# =============================================================================
+# File: /etc/rsyslog.d/99-mikrotik.conf
+# Riceve log UDP:514 dai router MikroTik e li salva in file dedicato
+# IMPORTANTE: Usa sintassi moderna rsyslog v8+ per evitare conflitti template
+# =============================================================================
+
+# Template personalizzato per log MikroTik (formato raw)
+template(name="MikroTikRawFormat" type="string" string="%msg%\n")
+
+# Ruleset dedicato per log MikroTik
+ruleset(name="mikrotik") {
+    # Salva in file dedicato usando template raw
+    action(
+        type="omfile"
+        file="/var/log/mikrotik/raw.log"
+        template="MikroTikRawFormat"
+        FileOwner="ids"
+        FileGroup="ids"
+        FileCreateMode="0644"
+        DirOwner="ids"
+        DirGroup="ids"
+        DirCreateMode="0755"
+    )
+    
+    # STOP: Non propagare a /var/log/messages per evitare duplicati
+    stop
+}
+
+# Input UDP:514 per log MikroTik
+module(load="imudp")
+input(
+    type="imudp"
+    port="514"
+    ruleset="mikrotik"
+)
--- a/deployment/rsyslog/README.md
+++ b/deployment/rsyslog/README.md
@ -0,0 +1,93 @@
+# RSyslog Configuration - IDS MikroTik
+
+## Overview
+Configurazione RSyslog per ricevere log dai router MikroTik via UDP:514 e salvarli in file dedicato senza duplicare in `/var/log/messages`.
+
+## File
+
+- **99-mikrotik.conf**: Configurazione rsyslog
+  - Template custom `MikroTikRawFormat` (salva log raw)
+  - Ruleset dedicato `mikrotik` con STOP (evita duplicati)
+  - Input UDP:514 per log MikroTik
+  - Permessi automatici: utente `ids`, gruppo `ids`
+
+## Installazione Automatica
+
+```bash
+cd /opt/ids
+sudo ./deployment/setup_rsyslog.sh
+```
+
+Lo script:
+1. Rimuove vecchie configurazioni conflittuali
+2. Installa `99-mikrotik.conf` in `/etc/rsyslog.d/`
+3. Crea directory `/var/log/mikrotik/` con permessi corretti
+4. Verifica sintassi rsyslog
+5. Configura firewall (UDP:514)
+6. Riavvia rsyslog
+
+## Verifica Funzionamento
+
+```bash
+# Verifica rsyslog in ascolto su UDP:514
+netstat -ulnp | grep 514
+
+# Monitora log in arrivo
+tail -f /var/log/mikrotik/raw.log
+
+# Verifica permessi
+ls -lh /var/log/mikrotik/raw.log
+# Output atteso: -rw-r--r-- ids ids
+```
+
+## Configurazione Router MikroTik
+
+Configura i router per inviare log al server:
+
+```
+/system logging action
+add name=remote-ids target=remote remote=<IP_SERVER> remote-port=514
+
+/system logging
+add action=remote-ids topics=firewall
+```
+
+## Troubleshooting
+
+### Errore: Template già impostato
+```
+error: omfile: default template already set via module global parameter
+```
+
+**Soluzione**: Lo script rimuove automaticamente vecchie configurazioni conflittuali.
+
+### Log duplicati in /var/log/messages
+La configurazione usa `stop` nel ruleset per evitare propagazione.
+
+### Permessi negati
+```bash
+# Verifica/ripara permessi
+sudo chown -R ids:ids /var/log/mikrotik/
+sudo chmod 755 /var/log/mikrotik/
+sudo chmod 644 /var/log/mikrotik/raw.log
+```
+
+### Firewall blocca UDP:514
+```bash
+sudo firewall-cmd --permanent --add-port=514/udp --zone=public
+sudo firewall-cmd --reload
+```
+
+## File Log
+
+- **Path**: `/var/log/mikrotik/raw.log`
+- **Owner**: `ids:ids`
+- **Permissions**: `0644`
+- **Format**: Raw syslog message (no timestamp/hostname prefix)
+
+## Note Tecniche
+
+- **Sintassi moderna**: rsyslog v8+ con `template()`, `ruleset()`, `action()`
+- **No legacy syntax**: Evita conflitti con `$ActionFileDefaultTemplate`
+- **Ruleset dedicato**: Isolamento completo per log MikroTik
+- **STOP directive**: Previene duplicazione in altri file log
--- a/deployment/setup_rsyslog.sh
+++ b/deployment/setup_rsyslog.sh
@ -0,0 +1,112 @@
+#!/bin/bash
+# =============================================================================
+# SETUP RSYSLOG per IDS MikroTik
+# =============================================================================
+# Configura rsyslog per ricevere log UDP:514 e salvarli senza duplicati
+# =============================================================================
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RSYSLOG_CONF="/etc/rsyslog.d/99-mikrotik.conf"
+LOG_DIR="/var/log/mikrotik"
+
+# Colori
+GREEN='\033[0;32m'
+BLUE='\033[0;34m'
+YELLOW='\033[1;33m'
+RED='\033[0;31m'
+NC='\033[0m'
+
+echo -e "${BLUE}🔧 Setup RSyslog per IDS MikroTik${NC}"
+echo ""
+
+# 1. Verifica rsyslog installato
+if ! command -v rsyslogd &> /dev/null; then
+    echo -e "${RED}❌ rsyslog non installato${NC}"
+    echo -e "${YELLOW}   Installa: sudo dnf install rsyslog -y${NC}"
+    exit 1
+fi
+
+echo -e "${BLUE}📋 Configurazione RSyslog...${NC}"
+
+# 2. Rimuovi vecchie configurazioni conflittuali
+echo -e "${YELLOW}   Rimuovo vecchie configurazioni...${NC}"
+rm -f /etc/rsyslog.d/10-mikrotik.conf
+rm -f /etc/rsyslog.d/mikrotik.conf
+
+# 3. Copia nuova configurazione
+echo -e "${BLUE}   Installazione configurazione...${NC}"
+cp "$SCRIPT_DIR/rsyslog/99-mikrotik.conf" "$RSYSLOG_CONF"
+chmod 644 "$RSYSLOG_CONF"
+
+# 4. Crea directory log
+echo -e "${BLUE}   Creazione directory log...${NC}"
+mkdir -p "$LOG_DIR"
+chown ids:ids "$LOG_DIR"
+chmod 755 "$LOG_DIR"
+
+# 5. Crea file raw.log iniziale
+touch "$LOG_DIR/raw.log"
+chown ids:ids "$LOG_DIR/raw.log"
+chmod 644 "$LOG_DIR/raw.log"
+
+# 6. Verifica sintassi rsyslog
+echo -e "${BLUE}   Verifica sintassi...${NC}"
+if rsyslogd -N1 2>&1 | grep -i "error" | grep -v "error during parsing.*mikrotik"; then
+    echo -e "${RED}❌ Errori nella configurazione rsyslog${NC}"
+    rsyslogd -N1
+    exit 1
+fi
+
+echo -e "${GREEN}✅ Configurazione rsyslog valida${NC}"
+
+# 7. Configura firewall per UDP:514
+echo -e "${BLUE}   Configurazione firewall...${NC}"
+if command -v firewall-cmd &> /dev/null; then
+    firewall-cmd --permanent --add-port=514/udp --zone=public 2>/dev/null || true
+    firewall-cmd --reload 2>/dev/null || true
+    echo -e "${GREEN}✅ Firewall configurato (UDP:514)${NC}"
+fi
+
+# 8. Riavvia rsyslog
+echo -e "${BLUE}   Riavvio rsyslog...${NC}"
+systemctl restart rsyslog
+systemctl enable rsyslog
+
+# 9. Verifica servizio attivo
+if systemctl is-active --quiet rsyslog; then
+    echo -e "${GREEN}✅ rsyslog attivo e in ascolto su UDP:514${NC}"
+else
+    echo -e "${RED}❌ rsyslog non attivo${NC}"
+    systemctl status rsyslog
+    exit 1
+fi
+
+# 10. Verifica porta UDP:514
+echo -e "${BLUE}   Verifica porta UDP:514...${NC}"
+sleep 2
+if netstat -ulnp | grep -q ":514"; then
+    echo -e "${GREEN}✅ rsyslog in ascolto su UDP:514${NC}"
+    netstat -ulnp | grep ":514"
+else
+    echo -e "${YELLOW}⚠  Porta UDP:514 non ancora attiva (verifica tra qualche secondo)${NC}"
+fi
+
+echo ""
+echo -e "${GREEN}╔═══════════════════════════════════════════════╗${NC}"
+echo -e "${GREEN}║   ✅ RSYSLOG CONFIGURATO CON SUCCESSO         ║${NC}"
+echo -e "${GREEN}╚═══════════════════════════════════════════════╝${NC}"
+echo ""
+echo -e "${BLUE}📊 VERIFICA:${NC}"
+echo -e "   • File log:     $LOG_DIR/raw.log"
+echo -e "   • Configurazione: $RSYSLOG_CONF"
+echo -e "   • Porta:        UDP:514"
+echo ""
+echo -e "${BLUE}🧪 TEST:${NC}"
+echo -e "   # Invia log test dal router MikroTik:"
+echo -e "   /system logging action set remote=<IP_SERVER> remote-port=514"
+echo -e ""
+echo -e "   # Monitora log in arrivo:"
+echo -e "   tail -f $LOG_DIR/raw.log"
+echo ""
--- a/deployment/update_from_git.sh
+++ b/deployment/update_from_git.sh
@ -139,6 +139,25 @@ else
    echo -e "${YELLOW}⚠️  Schema Drizzle potrebbe richiedere --force${NC}"
 fi

+# Setup rsyslog (solo prima volta o se modificato)
+if [ -f "./deployment/setup_rsyslog.sh" ]; then
+    echo -e "\n${BLUE}📡 Configurazione RSyslog (log MikroTik)...${NC}"
+    chmod +x ./deployment/setup_rsyslog.sh
+    
+    # Esegui setup rsyslog se:
+    # - File config non esiste
+    # - Config è più vecchia di quella nel repo
+    RSYSLOG_CONF="/etc/rsyslog.d/99-mikrotik.conf"
+    RSYSLOG_SOURCE="./deployment/rsyslog/99-mikrotik.conf"
+    
+    if [ ! -f "$RSYSLOG_CONF" ] || [ "$RSYSLOG_SOURCE" -nt "$RSYSLOG_CONF" ]; then
+        echo -e "${BLUE}   Setup/aggiornamento rsyslog necessario...${NC}"
+        ./deployment/setup_rsyslog.sh
+    else
+        echo -e "${GREEN}   ✅ RSyslog già configurato${NC}"
+    fi
+fi
+
 # Restart servizi
 echo -e "\n${BLUE}🔄 Restart servizi...${NC}"
 if [ -f "./deployment/restart_all.sh" ]; then