 ./deployment/update_from_git.sh

╔═══════════════════════════════════════════════╗
║    AGGIORNAMENTO SISTEMA IDS DA GIT         ║
╚═══════════════════════════════════════════════╝

 Verifica configurazione git...

 Backup configurazione locale...
✅ .env salvato in .env.backup

 Verifica modifiche locali...

 Download aggiornamenti da git.alfacom.it...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 16 (delta 13), reused 0 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (16/16), 2.36 KiB | 482.00 KiB/s, done.
From https://git.alfacom.it/marco/ids.alfacom.it
   07f1895..e957556  main       -> origin/main
 * [new tag]         v1.0.27    -> v1.0.27
From https://git.alfacom.it/marco/ids.alfacom.it
 * branch            main       -> FETCH_HEAD
Updating 07f1895..e957556
Fast-forward
 .replit                             |  4 ----
 database-schema/apply_migrations.sh |  9 +++++++++
 database-schema/schema.sql          |  4 ++--
 deployment/cleanup_database.sh      |  4 +++-
 deployment/debug_system.sh          | 10 +++++++++-
 version.json                        | 10 ++++++++--
 6 files changed, 31 insertions(+), 10 deletions(-)
✅ Aggiornamenti scaricati con successo

 Ripristino configurazione locale...
✅ .env ripristinato

 Aggiornamento dipendenze Node.js...

up to date, audited 492 packages in 2s

65 packages are looking for funding
  run `npm fund` for details

9 vulnerabilities (3 low, 5 moderate, 1 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
✅ Dipendenze Node.js aggiornate

 Aggiornamento dipendenze Python...
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: fastapi==0.104.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (0.104.1)
Requirement already satisfied: uvicorn==0.24.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.24.0)
Requirement already satisfied: pandas==2.1.3 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (2.1.3)
Requirement already satisfied: numpy==1.26.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.26.2)
Requirement already satisfied: scikit-learn==1.3.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (1.3.2)
Requirement already satisfied: psycopg2-binary==2.9.9 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.9.9)
Requirement already satisfied: python-dotenv==1.0.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (1.0.0)
Requirement already satisfied: pydantic==2.5.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (2.5.0)
Requirement already satisfied: httpx==0.25.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.25.1)
Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (3.7.1)
Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (0.27.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (4.15.0)
Requirement already satisfied: click>=7.0 in /home/ids/.local/lib/python3.11/site-packages (from uvicorn==0.24.0->-r requirements.txt (line 2)) (8.3.1)
Requirement already satisfied: h11>=0.8 in /home/ids/.local/lib/python3.11/site-packages (from uvicorn==0.24.0->-r requirements.txt (line 2)) (0.16.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2025.2)
Requirement already satisfied: tzdata>=2022.1 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2025.2)
Requirement already satisfied: scipy>=1.5.0 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (1.16.3)
Requirement already satisfied: joblib>=1.1.1 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (1.5.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (3.6.0)
Requirement already satisfied: annotated-types>=0.4.0 in /home/ids/.local/lib/python3.11/site-packages (from pydantic==2.5.0->-r requirements.txt (line 8)) (0.7.0)
Requirement already satisfied: pydantic-core==2.14.1 in /home/ids/.local/lib/python3.11/site-packages (from pydantic==2.5.0->-r requirements.txt (line 8)) (2.14.1)
Requirement already satisfied: certifi in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (2025.11.12)
Requirement already satisfied: httpcore in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (1.0.9)
Requirement already satisfied: idna in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (3.11)
Requirement already satisfied: sniffio in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (1.3.1)
Requirement already satisfied: six>=1.5 in /home/ids/.local/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas==2.1.3->-r requirements.txt (line 3)) (1.17.0)
✅ Dipendenze Python aggiornate

  Aggiornamento schema database...
   Applicando migrazioni SQL...
  Applicazione migrazioni database...
 Trovate 1 migrazioni
   Applicando: 001_add_missing_columns.sql
   ✅ 001_add_missing_columns.sql applicata
✅ Tutte le migrazioni applicate con successo
   ✅ Migrazioni SQL applicate
   Sincronizzando schema Drizzle...

> rest-express@1.0.0 db:push
> drizzle-kit push

No config path provided, using default 'drizzle.config.ts'
Reading config file '/opt/ids/drizzle.config.ts'
Using 'pg' driver for database querying
[✓] Pulling schema from database...
· You're about to add routers_ip_address_unique unique constraint to the table, which contains 1 items. If this statement fails, you will receive an error from the database. Do you want to truncate routers table?

 Warning  Found data-loss statements:
· You're about to delete last_check column in routers table with 1 items
· You're about to delete status column in routers table with 1 items

THIS ACTION WILL CAUSE DATA LOSS AND CANNOT BE REVERTED

Do you still want to push changes?
[x] All changes were aborted
✅ Schema database completamente sincronizzato

 Restart servizi...
✅ Servizi riavviati

╔═══════════════════════════════════════════════╗
║   ✅ AGGIORNAMENTO COMPLETATO                 ║
╚═══════════════════════════════════════════════╝

 VERIFICA SISTEMA:
   • Log backend:  tail -f /var/log/ids/backend.log
   • Log frontend: tail -f /var/log/ids/frontend.log
   • API backend:  curl http://localhost:8000/health
   • Frontend:     curl http://localhost:5000

 STATO SERVIZI:
ids         5038  0.2  2.0 1894024 331912 ?      Sl   09:20   1:17 /usr/bin/python3.11 main.py
root       12022  0.0  0.0   3088  1536 pts/3    S+   17:51   0:00 tail -f /var/log/ids/syslog_parser.log
root       12832  0.0  0.1 730448 32068 pts/5    Rl+  18:02   0:00 /usr/bin/node /usr/bin/npm run dev

[root@ids ids]# sudo -u ids /opt/ids/database-schema/apply_migrations.sh
  Applicazione migrazioni database...
 Trovate 1 migrazioni
   Applicando: 001_add_missing_columns.sql
   ✅ 001_add_missing_columns.sql applicata
✅ Tutte le migrazioni applicate con successo
[root@ids ids]# psql postgresql://ids_user:TestPassword123@127.0.0.1:5432/ids_database -c "\d routers"
                               Table "public.routers"
   Column   |            Type             | Collation | Nullable |      Default
------------+-----------------------------+-----------+----------+-------------------
 id         | character varying           |           | not null | gen_random_uuid()
 name       | text                        |           | not null |
 ip_address | text                        |           | not null |
 username   | text                        |           | not null |
 password   | text                        |           | not null |
 api_port   | integer                     |           | not null | 443
 enabled    | boolean                     |           | not null | true
 last_check | timestamp without time zone |           |          |
 status     | text                        |           |          |
 created_at | timestamp without time zone |           | not null | now()
 last_sync  | timestamp without time zone |           |          |
Indexes:
    "routers_pkey" PRIMARY KEY, btree (id)
    "routers_enabled_idx" btree (enabled)
    "routers_ip_address_key" UNIQUE CONSTRAINT, btree (ip_address)
    "routers_ip_idx" btree (ip_address)

[root@ids ids]# psql postgresql://ids_user:TestPassword123@127.0.0.1:5432/ids_database << 'EOF'
-- Conta log da eliminare
SELECT COUNT(*) as logs_da_eliminare FROM network_logs WHERE timestamp < NOW() - INTERVAL '7 days';

-- Elimina
DELETE FROM network_logs WHERE timestamp < NOW() - INTERVAL '7 days';

-- Libera spazio fisico
VACUUM FULL network_logs;

-- Verifica risultato
SELECT COUNT(*) as logs_rimasti FROM network_logs;
SELECT pg_size_pretty(pg_database_size(current_database())) as dimensione_db;
EOF
 logs_da_eliminare
-------------------
                 0
(1 row)

DELETE 0
VACUUM
 logs_rimasti
--------------
            0
(1 row)

 dimensione_db
---------------
 8853 kB
(1 row)

[root@ids ids]# sudo /opt/ids/deployment/setup_cron_cleanup.sh
 Configurazione cron job per pulizia database...
⚠  Cron job già configurato

 Cron jobs attuali per utente ids:
# ============================================
# SISTEMA IDS - CONFIGURAZIONE AUTOMATICA
# ============================================

# Training ML ogni 12 ore (alle 00:00 e 12:00)
0 */12 * * * /opt/ids/deployment/cron_train.sh

# Detection automatica ogni 5 minuti
*/5 * * * * /opt/ids/deployment/cron_detect.sh

# Verifica processo backend Python ogni 5 minuti (riavvia se non attivo)
*/5 * * * * /opt/ids/deployment/check_backend.sh >> /var/log/ids/cron.log 2>&1

# Verifica processo frontend ogni 5 minuti (riavvia se non attivo)
*/5 * * * * /opt/ids/deployment/check_frontend.sh >> /var/log/ids/cron.log 2>&1

# Pulizia log settimanale (ogni domenica alle 02:00)
0 2 * * 0 find /var/log/ids -name "*.log" -size +100M -exec truncate -s 50M {} \; >> /var/log/ids/cron.log 2>&1

# Restart completo del sistema ogni settimana (domenica alle 03:00)
0 3 * * 0 /opt/ids/deployment/restart_all.sh >> /var/log/ids/cron.log 2>&1

# Backup database giornaliero (alle 04:00)
0 4 * * * /opt/ids/deployment/backup_db.sh >> /var/log/ids/cron.log 2>&1
0 3 * * * /opt/ids/deployment/cleanup_database.sh >> /var/log/ids/cleanup.log 2>&1

離 Test manuale pulizia:
   sudo -u ids /opt/ids/deployment/cleanup_database.sh
[root@ids ids]# sudo -u ids /opt/ids/deployment/restart_all.sh
pkill: killing pid 12878 failed: Operation not permitted
pkill: killing pid 12832 failed: Operation not permitted
[root@ids ids]# /opt/ids/deployment/debug_system.sh

╔═══════════════════════════════════════════════╗
║    DEBUG SISTEMA IDS                        ║
╚═══════════════════════════════════════════════╝


═══ 1. VERIFICA DATABASE ═══
 Conta record per tabella:
     tabella      | record
------------------+--------
 detections       |      0
 network_logs     |      0
 routers          |      1
 training_history |      0
 whitelist        |      0
(5 rows)


 Schema tabella routers:
                               Table "public.routers"
   Column   |            Type             | Collation | Nullable |      Default
------------+-----------------------------+-----------+----------+-------------------
 id         | character varying           |           | not null | gen_random_uuid()
 name       | text                        |           | not null |
 ip_address | text                        |           | not null |
 username   | text                        |           | not null |
 password   | text                        |           | not null |
 api_port   | integer                     |           | not null | 443
 enabled    | boolean                     |           | not null | true
 last_check | timestamp without time zone |           |          |
 status     | text                        |           |          |
 created_at | timestamp without time zone |           | not null | now()
 last_sync  | timestamp without time zone |           |          |
Indexes:
    "routers_pkey" PRIMARY KEY, btree (id)
    "routers_enabled_idx" btree (enabled)
    "routers_ip_address_key" UNIQUE CONSTRAINT, btree (ip_address)
    "routers_ip_idx" btree (ip_address)


 Ultimi 5 network_logs:
 timestamp | router_name | source_ip | destination_ip | protocol | packet_length
-----------+-------------+-----------+----------------+----------+---------------
(0 rows)


 Training history:
 trained_at | model_version | records_processed | features_count | status | notes
------------+---------------+-------------------+----------------+--------+-------
(0 rows)


 Detections:
 detected_at | source_ip | risk_score | anomaly_type | blocked | log_count
-------------+-----------+------------+--------------+---------+-----------
(0 rows)


═══ 2. STATO SERVIZI ═══
 Processi attivi:
ids         5038  0.2  2.0 1894024 331912 ?      Sl   09:20   1:17 /usr/bin/python3.11 main.py
root       12022  0.0  0.0   3088  1536 pts/3    S+   17:51   0:00 tail -f /var/log/ids/syslog_parser.log
root       12832  0.2  0.3 1097848 59768 pts/5   Sl   18:02   0:00 npm run dev

═══ 3. BACKEND PYTHON ML ═══
✅ Backend Python attivo
 Statistiche ML:
{
  "logs": {
    "total": 0,
    "last_hour": 0
  },
  "detections": {
    "total": 0,
    "blocked": 0
  },
  "routers": {
    "active": 1
  },
  "latest_training": null
}

═══ 4. FRONTEND NODE.JS ═══
✅ Frontend Node attivo
 Test API:
{
  "routers": {
    "total": 1,
    "enabled": 1
  },
  "detections": {
    "total": 0,
    "blocked": 0,
    "critical": 0,
    "high": 0
  },
  "logs": {
    "recent": 0
  },
  "whitelist": {
    "total": 0
  }
}

═══ 5. SYSLOG PARSER ═══
❌ Syslog Parser NON attivo
   Avvia: cd /opt/ids/python_ml && nohup python syslog_parser.py > /var/log/ids/syslog_parser.log 2>&1 &

═══ 6. ERRORI RECENTI ═══
🔴 Errori backend Python:
ERROR:    [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
ERROR:    [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
ERROR:    [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
ERROR:    [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use

🔴 Errori frontend Node:
[DB ERROR] Failed to fetch routers: error: column "last_sync" does not exist

╔═══════════════════════════════════════════════╗
║   📋 RIEPILOGO                                ║
╚═══════════════════════════════════════════════╝
Database:
  • Network logs: 0
  • Detections: 0
  • Training history: 0

🔧 COMANDI UTILI:
  • Riavvia tutto: sudo -u ids /opt/ids/deployment/restart_all.sh
  • Test training: curl -X POST http://localhost:8000/train -H 'Content-Type: application/json' -d '{"max_records": 1000}'
  • Log frontend: tail -f /var/log/ids/frontend.log
  • Log backend: tail -f /var/log/ids/backend.log
