marco370
661e945f57
Implement automatic database cleanup and schema updates
...
Adds scripts for automatic database log cleanup, schema migration application, and cron job setup. Modifies the update script to apply SQL migrations before pushing Drizzle schema.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 9a659f15-d68a-4b7d-99f8-3eccc59afebe
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/4LjHWWz
2025-11-21 16:49:13 +00:00
marco370
d10b470793
Fix database connection errors and schema issues
...
Resolve 500 errors across database API endpoints by implementing a dual-mode database driver and adding missing columns to the routers table.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: c333ed12-07c5-412a-aff0-524321acc652
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:41:55 +00:00
marco370
f4803a7451
Fix error when fetching router and stats information
...
Add missing `api_port` and `last_sync` columns to the `routers` table in the database to resolve `column "last_sync" does not exist` errors.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: ed8286cc-22d7-40bd-ae28-6bae4dd7f5ea
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:35:52 +00:00
marco370
e8270da285
Fix database connection errors and improve logging
...
Update database connection handling to resolve errors and log database type instead of connection strings.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 81036932-56b6-46dc-a2eb-153586dfd77d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:25:32 +00:00
marco370
42354d5087
Adapt database connection for local and cloud environments
...
Update `server/db.ts` to support both Neon serverless and standard PostgreSQL drivers, add database health checks, and improve error logging for database operations.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 55ee188b-6bb4-49b0-8966-1795106363b1
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:09:48 +00:00
marco370
e5d038a2b4
Improve how the application starts to ensure environment variables are loaded correctly
...
Modify deployment/check_frontend.sh to use `env $(cat .env | grep -v '^#' | xargs) npm run dev` to inject environment variables into the npm process, resolving issues where `nohup` did not inherit exported variables. Update replit.md to reflect the corrected fix details.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 602cd9af-422f-4b1d-ab37-5fd3e0bd016f
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/dI1I79r
2025-11-21 15:58:17 +00:00
marco370
aec912cf16
Fix frontend startup to correctly load environment variables
...
Update `deployment/check_frontend.sh` to load `.env` variables before running `npm run dev`, resolving the `DATABASE_URL must be set` error and improving deployment on non-Replit environments.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8cf499b5-f15d-4b0d-b459-f48510ac6539
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/dI1I79r
2025-11-21 15:52:15 +00:00
marco370
b3bbcb69d3
Update system to apply database changes and synchronize dependencies
...
Apply SQL scripts to update database schema and install/update Node.js and Python dependencies.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: bf319927-85dd-43f9-9df2-348d41742705
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/Aqah4U9
2025-11-21 14:31:40 +00:00
marco370
203eaa9a54
Improve system stability by fixing cron job and process monitoring
...
Refactor cron job execution to use separate shell scripts and update process monitoring scripts to correctly handle PID files and log directories.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f6d4ec61-7130-41dd-aef9-87b4bc73d0e8
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/3R8dKMX
2025-11-18 08:33:05 +00:00
marco370
62ffbe499a
Add automated training and detection jobs to the system
...
Adds new cron jobs and shell scripts for automated ML model training and anomaly detection, along with configurations for logging and system checks.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 534244cd-b293-403e-a6d6-43cd9fbb30fb
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/1P26v7M
2025-11-18 08:24:09 +00:00
marco370
94724ff580
Add ability to detect and block malicious network traffic based on risk
...
Adds functionality to detect network anomalies and block suspicious IP addresses.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: e23fa694-b094-4b45-9ce0-0dbf8fd7a9e2
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/1P26v7M
2025-11-18 07:31:39 +00:00
marco370
dc91096d9d
Improve log processing and add automated tasks
...
Update log processing to use the correct timestamp field and introduce cron jobs for automated model training and anomaly detection.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f0653fd5-fc94-4fcb-8d7e-2a0e90fc81bf
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 18:11:49 +00:00
marco370
d345a24572
Improve intrusion detection system with functional updates and database fixes
...
Update `replit.md` to reflect recent system improvements including a fully functional syslog parser, PostgreSQL database, updated regex patterns, DDoS detection, and automated deployment workflows. Addresses issues with the `network_logs` table schema and incorrect regex matching. Includes SQL query outputs for verification.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 70827608-8ca8-471f-a794-336056b4ce88
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:54:18 +00:00
marco370
e5f307af27
Fix error when saving network logs to the database
...
Correct SQL query to insert data into the network_logs table, resolving "relation 'network_logs' does not exist" errors.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: e8e7edaf-8ac8-493c-b890-465ff617b0ce
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:40:22 +00:00
marco370
0d34bf7d3c
Update log parsing to better identify network traffic and DDoS events
...
Refactors the `SyslogParser` class in `python_ml/syslog_parser.py` to use a new, more comprehensive regex pattern (`main_pattern`) for parsing MikroTik logs. This includes improved identification of 'forward' and 'detected-ddos forward' actions, protocol details (UDP, TCP, ICMP), and associated IP addresses, ports, and lengths. The changes aim to accurately capture network traffic and potential DDoS events from MikroTik logs.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: b7377ada-e722-475a-86d2-07f21299ec70
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:35:37 +00:00
marco370
51aa026aae
Update logs to show processing progress
...
Adds new log entries to the syslog parser output, indicating incremental progress in row processing without saving any logs.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: b96cf229-a1bd-4e23-8aee-d92bd339420b
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:34:04 +00:00
marco370
7b6fc148a5
Update system to correctly process log files with proper permissions
...
Correctly sets file permissions and ownership for the `.env` file and demonstrates successful database connection and log file processing by the syslog parser script.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 91f9e939-55de-4fd8-9f48-6360540b2bfe
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:42:42 +00:00
marco370
1936a29ae4
Improve database connection security and restart log parser
...
Modify pg_hba.conf to use SCRAM-SHA-256 for database authentication and restart the syslog parser service.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 2c44c1cf-5725-4dd8-8305-3fa9b2a5136c
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:39:09 +00:00
marco370
08c2373aa5
Configure database user and password for secure access
...
Updates PostgreSQL configuration to set password encryption to SCRAM-SHA-256, creates a new user 'ids_user' with the specified password, grants necessary privileges on the 'ids_database', and configures default privileges for future objects. Includes troubleshooting steps and log excerpts related to authentication failures.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f8be77ab-2269-4666-9e56-9309e455e81c
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:28:10 +00:00
marco370
7c36dc039b
Fix PostgreSQL authentication issues for user access
...
Introduces a shell script to modify pg_hba.conf and reset PostgreSQL user passwords, resolving authentication failures.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 39b3e0c3-d6b2-4c6f-afb4-e32fe7f09b02
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:25:32 +00:00
marco370
0b9d0cf302
Initial commit
2025-11-11 09:12:13 +00:00
