6f2d0da1c9
Applies recent code updates from the Git repository and executes necessary database schema migrations to ensure data integrity and system functionality. Replit-Commit-Author: Agent Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528 Replit-Commit-Checkpoint-Type: full_checkpoint Replit-Commit-Event-Id: 2bb36b2d-b880-414f-ad99-7f18bc18ee9e Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/4LjHWWz
361 lines
17 KiB
Plaintext
361 lines
17 KiB
Plaintext
./deployment/update_from_git.sh
|
|
|
|
╔═══════════════════════════════════════════════╗
|
|
║ AGGIORNAMENTO SISTEMA IDS DA GIT ║
|
|
╚═══════════════════════════════════════════════╝
|
|
|
|
Verifica configurazione git...
|
|
|
|
Backup configurazione locale...
|
|
✅ .env salvato in .env.backup
|
|
|
|
Verifica modifiche locali...
|
|
|
|
Download aggiornamenti da git.alfacom.it...
|
|
remote: Enumerating objects: 25, done.
|
|
remote: Counting objects: 100% (25/25), done.
|
|
remote: Compressing objects: 100% (16/16), done.
|
|
remote: Total 16 (delta 13), reused 0 (delta 0), pack-reused 0 (from 0)
|
|
Unpacking objects: 100% (16/16), 2.36 KiB | 482.00 KiB/s, done.
|
|
From https://git.alfacom.it/marco/ids.alfacom.it
|
|
07f1895..e957556 main -> origin/main
|
|
* [new tag] v1.0.27 -> v1.0.27
|
|
From https://git.alfacom.it/marco/ids.alfacom.it
|
|
* branch main -> FETCH_HEAD
|
|
Updating 07f1895..e957556
|
|
Fast-forward
|
|
.replit | 4 ----
|
|
database-schema/apply_migrations.sh | 9 +++++++++
|
|
database-schema/schema.sql | 4 ++--
|
|
deployment/cleanup_database.sh | 4 +++-
|
|
deployment/debug_system.sh | 10 +++++++++-
|
|
version.json | 10 ++++++++--
|
|
6 files changed, 31 insertions(+), 10 deletions(-)
|
|
✅ Aggiornamenti scaricati con successo
|
|
|
|
Ripristino configurazione locale...
|
|
✅ .env ripristinato
|
|
|
|
Aggiornamento dipendenze Node.js...
|
|
|
|
up to date, audited 492 packages in 2s
|
|
|
|
65 packages are looking for funding
|
|
run `npm fund` for details
|
|
|
|
9 vulnerabilities (3 low, 5 moderate, 1 high)
|
|
|
|
To address issues that do not require attention, run:
|
|
npm audit fix
|
|
|
|
To address all issues (including breaking changes), run:
|
|
npm audit fix --force
|
|
|
|
Run `npm audit` for details.
|
|
✅ Dipendenze Node.js aggiornate
|
|
|
|
Aggiornamento dipendenze Python...
|
|
Defaulting to user installation because normal site-packages is not writeable
|
|
Requirement already satisfied: fastapi==0.104.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (0.104.1)
|
|
Requirement already satisfied: uvicorn==0.24.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.24.0)
|
|
Requirement already satisfied: pandas==2.1.3 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (2.1.3)
|
|
Requirement already satisfied: numpy==1.26.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.26.2)
|
|
Requirement already satisfied: scikit-learn==1.3.2 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (1.3.2)
|
|
Requirement already satisfied: psycopg2-binary==2.9.9 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.9.9)
|
|
Requirement already satisfied: python-dotenv==1.0.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (1.0.0)
|
|
Requirement already satisfied: pydantic==2.5.0 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (2.5.0)
|
|
Requirement already satisfied: httpx==0.25.1 in /home/ids/.local/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.25.1)
|
|
Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (3.7.1)
|
|
Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (0.27.0)
|
|
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ids/.local/lib/python3.11/site-packages (from fastapi==0.104.1->-r requirements.txt (line 1)) (4.15.0)
|
|
Requirement already satisfied: click>=7.0 in /home/ids/.local/lib/python3.11/site-packages (from uvicorn==0.24.0->-r requirements.txt (line 2)) (8.3.1)
|
|
Requirement already satisfied: h11>=0.8 in /home/ids/.local/lib/python3.11/site-packages (from uvicorn==0.24.0->-r requirements.txt (line 2)) (0.16.0)
|
|
Requirement already satisfied: python-dateutil>=2.8.2 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2.9.0.post0)
|
|
Requirement already satisfied: pytz>=2020.1 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2025.2)
|
|
Requirement already satisfied: tzdata>=2022.1 in /home/ids/.local/lib/python3.11/site-packages (from pandas==2.1.3->-r requirements.txt (line 3)) (2025.2)
|
|
Requirement already satisfied: scipy>=1.5.0 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (1.16.3)
|
|
Requirement already satisfied: joblib>=1.1.1 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (1.5.2)
|
|
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/ids/.local/lib/python3.11/site-packages (from scikit-learn==1.3.2->-r requirements.txt (line 5)) (3.6.0)
|
|
Requirement already satisfied: annotated-types>=0.4.0 in /home/ids/.local/lib/python3.11/site-packages (from pydantic==2.5.0->-r requirements.txt (line 8)) (0.7.0)
|
|
Requirement already satisfied: pydantic-core==2.14.1 in /home/ids/.local/lib/python3.11/site-packages (from pydantic==2.5.0->-r requirements.txt (line 8)) (2.14.1)
|
|
Requirement already satisfied: certifi in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (2025.11.12)
|
|
Requirement already satisfied: httpcore in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (1.0.9)
|
|
Requirement already satisfied: idna in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (3.11)
|
|
Requirement already satisfied: sniffio in /home/ids/.local/lib/python3.11/site-packages (from httpx==0.25.1->-r requirements.txt (line 9)) (1.3.1)
|
|
Requirement already satisfied: six>=1.5 in /home/ids/.local/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas==2.1.3->-r requirements.txt (line 3)) (1.17.0)
|
|
✅ Dipendenze Python aggiornate
|
|
|
|
Aggiornamento schema database...
|
|
Applicando migrazioni SQL...
|
|
Applicazione migrazioni database...
|
|
Trovate 1 migrazioni
|
|
Applicando: 001_add_missing_columns.sql
|
|
✅ 001_add_missing_columns.sql applicata
|
|
✅ Tutte le migrazioni applicate con successo
|
|
✅ Migrazioni SQL applicate
|
|
Sincronizzando schema Drizzle...
|
|
|
|
> rest-express@1.0.0 db:push
|
|
> drizzle-kit push
|
|
|
|
No config path provided, using default 'drizzle.config.ts'
|
|
Reading config file '/opt/ids/drizzle.config.ts'
|
|
Using 'pg' driver for database querying
|
|
[✓] Pulling schema from database...
|
|
· You're about to add routers_ip_address_unique unique constraint to the table, which contains 1 items. If this statement fails, you will receive an error from the database. Do you want to truncate routers table?
|
|
|
|
Warning Found data-loss statements:
|
|
· You're about to delete last_check column in routers table with 1 items
|
|
· You're about to delete status column in routers table with 1 items
|
|
|
|
THIS ACTION WILL CAUSE DATA LOSS AND CANNOT BE REVERTED
|
|
|
|
Do you still want to push changes?
|
|
[x] All changes were aborted
|
|
✅ Schema database completamente sincronizzato
|
|
|
|
Restart servizi...
|
|
✅ Servizi riavviati
|
|
|
|
╔═══════════════════════════════════════════════╗
|
|
║ ✅ AGGIORNAMENTO COMPLETATO ║
|
|
╚═══════════════════════════════════════════════╝
|
|
|
|
VERIFICA SISTEMA:
|
|
• Log backend: tail -f /var/log/ids/backend.log
|
|
• Log frontend: tail -f /var/log/ids/frontend.log
|
|
• API backend: curl http://localhost:8000/health
|
|
• Frontend: curl http://localhost:5000
|
|
|
|
STATO SERVIZI:
|
|
ids 5038 0.2 2.0 1894024 331912 ? Sl 09:20 1:17 /usr/bin/python3.11 main.py
|
|
root 12022 0.0 0.0 3088 1536 pts/3 S+ 17:51 0:00 tail -f /var/log/ids/syslog_parser.log
|
|
root 12832 0.0 0.1 730448 32068 pts/5 Rl+ 18:02 0:00 /usr/bin/node /usr/bin/npm run dev
|
|
|
|
[root@ids ids]# sudo -u ids /opt/ids/database-schema/apply_migrations.sh
|
|
Applicazione migrazioni database...
|
|
Trovate 1 migrazioni
|
|
Applicando: 001_add_missing_columns.sql
|
|
✅ 001_add_missing_columns.sql applicata
|
|
✅ Tutte le migrazioni applicate con successo
|
|
[root@ids ids]# psql postgresql://ids_user:TestPassword123@127.0.0.1:5432/ids_database -c "\d routers"
|
|
Table "public.routers"
|
|
Column | Type | Collation | Nullable | Default
|
|
------------+-----------------------------+-----------+----------+-------------------
|
|
id | character varying | | not null | gen_random_uuid()
|
|
name | text | | not null |
|
|
ip_address | text | | not null |
|
|
username | text | | not null |
|
|
password | text | | not null |
|
|
api_port | integer | | not null | 443
|
|
enabled | boolean | | not null | true
|
|
last_check | timestamp without time zone | | |
|
|
status | text | | |
|
|
created_at | timestamp without time zone | | not null | now()
|
|
last_sync | timestamp without time zone | | |
|
|
Indexes:
|
|
"routers_pkey" PRIMARY KEY, btree (id)
|
|
"routers_enabled_idx" btree (enabled)
|
|
"routers_ip_address_key" UNIQUE CONSTRAINT, btree (ip_address)
|
|
"routers_ip_idx" btree (ip_address)
|
|
|
|
[root@ids ids]# psql postgresql://ids_user:TestPassword123@127.0.0.1:5432/ids_database << 'EOF'
|
|
-- Conta log da eliminare
|
|
SELECT COUNT(*) as logs_da_eliminare FROM network_logs WHERE timestamp < NOW() - INTERVAL '7 days';
|
|
|
|
-- Elimina
|
|
DELETE FROM network_logs WHERE timestamp < NOW() - INTERVAL '7 days';
|
|
|
|
-- Libera spazio fisico
|
|
VACUUM FULL network_logs;
|
|
|
|
-- Verifica risultato
|
|
SELECT COUNT(*) as logs_rimasti FROM network_logs;
|
|
SELECT pg_size_pretty(pg_database_size(current_database())) as dimensione_db;
|
|
EOF
|
|
logs_da_eliminare
|
|
-------------------
|
|
0
|
|
(1 row)
|
|
|
|
DELETE 0
|
|
VACUUM
|
|
logs_rimasti
|
|
--------------
|
|
0
|
|
(1 row)
|
|
|
|
dimensione_db
|
|
---------------
|
|
8853 kB
|
|
(1 row)
|
|
|
|
[root@ids ids]# sudo /opt/ids/deployment/setup_cron_cleanup.sh
|
|
Configurazione cron job per pulizia database...
|
|
⚠ Cron job già configurato
|
|
|
|
Cron jobs attuali per utente ids:
|
|
# ============================================
|
|
# SISTEMA IDS - CONFIGURAZIONE AUTOMATICA
|
|
# ============================================
|
|
|
|
# Training ML ogni 12 ore (alle 00:00 e 12:00)
|
|
0 */12 * * * /opt/ids/deployment/cron_train.sh
|
|
|
|
# Detection automatica ogni 5 minuti
|
|
*/5 * * * * /opt/ids/deployment/cron_detect.sh
|
|
|
|
# Verifica processo backend Python ogni 5 minuti (riavvia se non attivo)
|
|
*/5 * * * * /opt/ids/deployment/check_backend.sh >> /var/log/ids/cron.log 2>&1
|
|
|
|
# Verifica processo frontend ogni 5 minuti (riavvia se non attivo)
|
|
*/5 * * * * /opt/ids/deployment/check_frontend.sh >> /var/log/ids/cron.log 2>&1
|
|
|
|
# Pulizia log settimanale (ogni domenica alle 02:00)
|
|
0 2 * * 0 find /var/log/ids -name "*.log" -size +100M -exec truncate -s 50M {} \; >> /var/log/ids/cron.log 2>&1
|
|
|
|
# Restart completo del sistema ogni settimana (domenica alle 03:00)
|
|
0 3 * * 0 /opt/ids/deployment/restart_all.sh >> /var/log/ids/cron.log 2>&1
|
|
|
|
# Backup database giornaliero (alle 04:00)
|
|
0 4 * * * /opt/ids/deployment/backup_db.sh >> /var/log/ids/cron.log 2>&1
|
|
0 3 * * * /opt/ids/deployment/cleanup_database.sh >> /var/log/ids/cleanup.log 2>&1
|
|
|
|
離 Test manuale pulizia:
|
|
sudo -u ids /opt/ids/deployment/cleanup_database.sh
|
|
[root@ids ids]# sudo -u ids /opt/ids/deployment/restart_all.sh
|
|
pkill: killing pid 12878 failed: Operation not permitted
|
|
pkill: killing pid 12832 failed: Operation not permitted
|
|
[root@ids ids]# /opt/ids/deployment/debug_system.sh
|
|
|
|
╔═══════════════════════════════════════════════╗
|
|
║ DEBUG SISTEMA IDS ║
|
|
╚═══════════════════════════════════════════════╝
|
|
|
|
|
|
═══ 1. VERIFICA DATABASE ═══
|
|
Conta record per tabella:
|
|
tabella | record
|
|
------------------+--------
|
|
detections | 0
|
|
network_logs | 0
|
|
routers | 1
|
|
training_history | 0
|
|
whitelist | 0
|
|
(5 rows)
|
|
|
|
|
|
Schema tabella routers:
|
|
Table "public.routers"
|
|
Column | Type | Collation | Nullable | Default
|
|
------------+-----------------------------+-----------+----------+-------------------
|
|
id | character varying | | not null | gen_random_uuid()
|
|
name | text | | not null |
|
|
ip_address | text | | not null |
|
|
username | text | | not null |
|
|
password | text | | not null |
|
|
api_port | integer | | not null | 443
|
|
enabled | boolean | | not null | true
|
|
last_check | timestamp without time zone | | |
|
|
status | text | | |
|
|
created_at | timestamp without time zone | | not null | now()
|
|
last_sync | timestamp without time zone | | |
|
|
Indexes:
|
|
"routers_pkey" PRIMARY KEY, btree (id)
|
|
"routers_enabled_idx" btree (enabled)
|
|
"routers_ip_address_key" UNIQUE CONSTRAINT, btree (ip_address)
|
|
"routers_ip_idx" btree (ip_address)
|
|
|
|
|
|
Ultimi 5 network_logs:
|
|
timestamp | router_name | source_ip | destination_ip | protocol | packet_length
|
|
-----------+-------------+-----------+----------------+----------+---------------
|
|
(0 rows)
|
|
|
|
|
|
Training history:
|
|
trained_at | model_version | records_processed | features_count | status | notes
|
|
------------+---------------+-------------------+----------------+--------+-------
|
|
(0 rows)
|
|
|
|
|
|
Detections:
|
|
detected_at | source_ip | risk_score | anomaly_type | blocked | log_count
|
|
-------------+-----------+------------+--------------+---------+-----------
|
|
(0 rows)
|
|
|
|
|
|
═══ 2. STATO SERVIZI ═══
|
|
Processi attivi:
|
|
ids 5038 0.2 2.0 1894024 331912 ? Sl 09:20 1:17 /usr/bin/python3.11 main.py
|
|
root 12022 0.0 0.0 3088 1536 pts/3 S+ 17:51 0:00 tail -f /var/log/ids/syslog_parser.log
|
|
root 12832 0.2 0.3 1097848 59768 pts/5 Sl 18:02 0:00 npm run dev
|
|
|
|
═══ 3. BACKEND PYTHON ML ═══
|
|
✅ Backend Python attivo
|
|
Statistiche ML:
|
|
{
|
|
"logs": {
|
|
"total": 0,
|
|
"last_hour": 0
|
|
},
|
|
"detections": {
|
|
"total": 0,
|
|
"blocked": 0
|
|
},
|
|
"routers": {
|
|
"active": 1
|
|
},
|
|
"latest_training": null
|
|
}
|
|
|
|
═══ 4. FRONTEND NODE.JS ═══
|
|
✅ Frontend Node attivo
|
|
Test API:
|
|
{
|
|
"routers": {
|
|
"total": 1,
|
|
"enabled": 1
|
|
},
|
|
"detections": {
|
|
"total": 0,
|
|
"blocked": 0,
|
|
"critical": 0,
|
|
"high": 0
|
|
},
|
|
"logs": {
|
|
"recent": 0
|
|
},
|
|
"whitelist": {
|
|
"total": 0
|
|
}
|
|
}
|
|
|
|
═══ 5. SYSLOG PARSER ═══
|
|
❌ Syslog Parser NON attivo
|
|
Avvia: cd /opt/ids/python_ml && nohup python syslog_parser.py > /var/log/ids/syslog_parser.log 2>&1 &
|
|
|
|
═══ 6. ERRORI RECENTI ═══
|
|
🔴 Errori backend Python:
|
|
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
|
|
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
|
|
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
|
|
ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
|
|
|
|
🔴 Errori frontend Node:
|
|
[DB ERROR] Failed to fetch routers: error: column "last_sync" does not exist
|
|
|
|
╔═══════════════════════════════════════════════╗
|
|
║ 📋 RIEPILOGO ║
|
|
╚═══════════════════════════════════════════════╝
|
|
Database:
|
|
• Network logs: 0
|
|
• Detections: 0
|
|
• Training history: 0
|
|
|
|
🔧 COMANDI UTILI:
|
|
• Riavvia tutto: sudo -u ids /opt/ids/deployment/restart_all.sh
|
|
• Test training: curl -X POST http://localhost:8000/train -H 'Content-Type: application/json' -d '{"max_records": 1000}'
|
|
• Log frontend: tail -f /var/log/ids/frontend.log
|
|
• Log backend: tail -f /var/log/ids/backend.log
|