marco370
b88377e2d5
Adapt ML model to new database schema and automate training
...
Adjusts SQL queries and feature extraction to accommodate changes in the network_logs database schema, enabling automatic weekly retraining of the ML hybrid detector.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: f4fdd53b-f433-44d9-9f0f-63616a9eeec1
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 18:14:43 +00:00
marco370
7ba039a547
Fix index out of bounds error during synthetic data testing
...
Corrected an indexing error in `train_hybrid.py` by using `enumerate` to ensure accurate mapping of detections to the test dataset, resolving an `IndexError` when processing synthetic data.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: d05c3dd2-6349-426d-be9c-ec80a07ea78f
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 17:57:22 +00:00
marco370
9fe2532217
Add timestamp to synthetic data for accurate model testing
...
Add a 'timestamp' column to the synthetic dataset generation in `python_ml/dataset_loader.py` to resolve a `KeyError` during model training and testing.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 276a3bd4-aaee-40c9-acb7-027f23274a9f
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 17:52:16 +00:00
marco370
75d3bd56a1
Simplify ML dependency to use standard Isolation Forest
...
Remove problematic Extended Isolation Forest dependency and leverage existing scikit-learn fallback for Python 3.11 compatibility.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 89ea874d-b572-40ad-9ac7-0c77d2b7d08d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 17:44:11 +00:00
marco370
051c838840
Add ability to install ML dependencies and resolve build issues
...
Update install_ml_deps.sh to use --no-build-isolation when installing eif to resolve ModuleNotFoundError during build.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 219383e3-8935-415d-8c84-77e7d6f76af8
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 17:06:43 +00:00
marco370
102113e950
Improve ML dependency installation script for robust deployment
...
Update deployment script to correctly activate virtual environment, install Cython and numpy as build dependencies before eif, and ensure sequential installation for the ML hybrid detector.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8b4c76c7-3a42-4713-8396-40f5db530225
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 17:02:15 +00:00
marco370
b78f03392a
Update deployment process to handle machine learning dependencies
...
Create a dedicated script to install machine learning dependencies in the correct order, ensuring Cython is installed before packages that require it for compilation, and update documentation accordingly.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: aa7dc534-7330-4bd4-b726-d6eeb29008af
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 16:38:41 +00:00
marco370
b4aaa5456f
Add Cython to Python dependencies for model compilation
...
Add Cython==3.0.5 to python_ml/requirements.txt and update replit.md to reflect this change, resolving a compilation issue with the eif library.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f24578fc-6be7-42c0-9a9c-5ffe13dacdbe
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 16:35:15 +00:00
marco370
043690f829
Update dependency version for improved compatibility
...
Update `eif` dependency from version 2.0.0 to 2.0.2 in `requirements.txt` and documentation to resolve a deployment issue, as version 2.0.0 is not available.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 104a0ac7-f020-4d00-9e3e-5a37b74bbc93
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/2lUhxO2
2025-11-24 16:32:31 +00:00
marco370
3d7a0ce424
Improve attack data accuracy and add validation checks
...
Update analytics aggregation logic to accurately count attack packets by type and country, including fallbacks for missing data, and add validation to ensure breakdown totals match reported attack packets.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: cafbc828-3e12-4d4f-8a02-5127b485612d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/F6DiMv4
2025-11-24 15:27:17 +00:00
marco370
e629bf4ed3
Improve analytics data fetching and server restart process
...
Fixes the analytics API call by correctly formatting the query parameters in the `AnalyticsHistory.tsx` component. Enhances the `restart_frontend.sh` script for more aggressive process killing and port cleanup to prevent 'address already in use' errors. Also, adds a check for the `country` column existence in the database schema, addressing a potential mismatch between development and production environments.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 23dd17a9-47b9-4533-bf4c-8b5cfdb426b4
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/x5P9dcJ
2025-11-24 09:59:06 +00:00
marco370
0bf61dc69d
Improve model training and file saving capabilities
...
Fixes permission errors for model saving and enhances training logging, ensuring proper storage of ML models and historical data.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 2afb7ddf-484b-4d07-8d99-8c1ca39c0be5
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/VDRknFA
2025-11-22 10:27:30 +00:00
marco370
9d5ecf99c4
Add branding and content assets for the application interface
...
Add various branding assets, content files, and screenshots to enhance the application's user interface and visual presentation.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 5db5df9a-80de-4fbe-96fb-67a573fc567e
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/VDRknFA
2025-11-22 10:12:00 +00:00
marco370
26f3589a7e
Improve systemd service setup to aggressively kill manual processes
...
Updates `setup_systemd_services.sh` to forcefully stop existing systemd services, kill all manual Python processes owned by the `ids` user, and verify that port 8000 is free before starting the services.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: ae567421-923d-4371-a127-7bdeca91b824
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/VDRknFA
2025-11-22 10:03:21 +00:00
marco370
e9e74f9944
Add missing Python libraries for backend functionality
...
Update the Python dependency installation script to include `httpx` and `joblib`, and enhance verification checks for installed modules.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: f0a704c9-cac4-4144-8f26-c6066459f615
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/n4Q2eeE
2025-11-22 09:57:47 +00:00
marco370
5b350ff95f
Add Python dependency installation and virtual environment support
...
Introduce a new script to install Python dependencies in a virtual environment, update systemd services to utilize this environment, and modify the setup script to automate dependency checks and installation.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: ea2a08f4-46e1-463d-9c58-16219914ad23
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/n4Q2eeE
2025-11-22 09:40:57 +00:00
marco370
d187aa533a
Fix ML backend service startup failures
...
Update the systemd service unit for the ML backend to correctly log errors to journalctl, enabling easier debugging of startup failures.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: adfa3c2c-2d2f-40c7-8113-83a526fb3a96
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/n4Q2eeE
2025-11-22 09:38:27 +00:00
marco370
015770609a
Update database schema and restart services
...
Apply SQL migrations and synchronize the database schema using Drizzle Kit, then restart systemd services.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 5dd94895-686f-45b1-9689-fcec435e180e
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/n4Q2eeE
2025-11-22 09:37:13 +00:00
marco370
4a2d7f9c5c
Add service monitoring and status indicators to the dashboard
...
Introduce a new services page, integrate real-time status monitoring for ML backend, database, and syslog parser, and update the dashboard to display service health indicators.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: cde95c60-908b-48a0-b7b9-38e5e924b3b3
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/n4Q2eeE
2025-11-22 09:24:10 +00:00
marco370
24b907e17b
Fix log parsing by adding missing timestamps to incoming data
...
The attached log file and agent reasoning indicate that the `ids-syslog-parser` service failed to restart and the `/var/log/mikrotik/raw.log` file lacks timestamps, leading to the parser saving 0 logs. The provided solution involves manually updating the `/etc/rsyslog.d/99-mikrotik.conf` file to include the `%TIMESTAMP%` directive, restarting the `rsyslog` service, clearing and restarting the log file, and then restarting the `ids-syslog-parser` service to process logs correctly.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 9fa8bbb2-1781-4d01-b6d3-3b872fb304a3
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/6ZTQSoP
2025-11-22 09:02:34 +00:00
marco370
9448d54156
Add timestamp to log entries to ensure proper parsing
...
Fixes an issue where log entries were missing timestamps, preventing the Python parser from correctly processing the data. This change modifies the rsyslog template to include the timestamp.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: d449ad65-7340-48be-804d-a473dce5d0d8
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/6ZTQSoP
2025-11-22 08:57:19 +00:00
marco370
495e845a79
Update log format to include timestamps and filter incoming connections
...
Correct the rsyslog template to include timestamps in logs, ensuring compatibility with the Python parser. This change also refactors the log filtering to capture only incoming connections, significantly reducing log volume.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: c2f849f9-105f-452a-bdc3-a956d102c54b
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/6ZTQSoP
2025-11-22 08:54:21 +00:00
marco370
c9a0ba1b66
Fix issue where logs are not saved to the database
...
Resolve a critical bug in the log processing system where millions of log entries were processed but not committed to the database, leading to zero records in the `network_logs` table. This involved stopping the parser, truncating/rotating the log file, and restarting the parser to process only new entries.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 6976842b-2cfa-4e4f-aea4-7e7206988693
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/DR50xVM
2025-11-21 17:37:31 +00:00
marco370
b31b0ec932
Add log file detailing network traffic and detected DDOS activity
...
Added a log file containing network traffic data, including packet forwarding information, connection states, and detected denial-of-service (DDOS) activities.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 43992744-515b-49bf-8459-fe8c14ec1bde
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/4LjHWWz
2025-11-21 17:10:36 +00:00
marco370
6f2d0da1c9
Update system with latest code and database changes
...
Applies recent code updates from the Git repository and executes necessary database schema migrations to ensure data integrity and system functionality.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 2bb36b2d-b880-414f-ad99-7f18bc18ee9e
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/4LjHWWz
2025-11-21 17:06:35 +00:00
marco370
661e945f57
Implement automatic database cleanup and schema updates
...
Adds scripts for automatic database log cleanup, schema migration application, and cron job setup. Modifies the update script to apply SQL migrations before pushing Drizzle schema.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 9a659f15-d68a-4b7d-99f8-3eccc59afebe
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/4LjHWWz
2025-11-21 16:49:13 +00:00
marco370
d10b470793
Fix database connection errors and schema issues
...
Resolve 500 errors across database API endpoints by implementing a dual-mode database driver and adding missing columns to the routers table.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: c333ed12-07c5-412a-aff0-524321acc652
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:41:55 +00:00
marco370
f4803a7451
Fix error when fetching router and stats information
...
Add missing `api_port` and `last_sync` columns to the `routers` table in the database to resolve `column "last_sync" does not exist` errors.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: ed8286cc-22d7-40bd-ae28-6bae4dd7f5ea
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:35:52 +00:00
marco370
e8270da285
Fix database connection errors and improve logging
...
Update database connection handling to resolve errors and log database type instead of connection strings.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 81036932-56b6-46dc-a2eb-153586dfd77d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:25:32 +00:00
marco370
42354d5087
Adapt database connection for local and cloud environments
...
Update `server/db.ts` to support both Neon serverless and standard PostgreSQL drivers, add database health checks, and improve error logging for database operations.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 55ee188b-6bb4-49b0-8966-1795106363b1
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/C4ZJnmQ
2025-11-21 16:09:48 +00:00
marco370
e5d038a2b4
Improve how the application starts to ensure environment variables are loaded correctly
...
Modify deployment/check_frontend.sh to use `env $(cat .env | grep -v '^#' | xargs) npm run dev` to inject environment variables into the npm process, resolving issues where `nohup` did not inherit exported variables. Update replit.md to reflect the corrected fix details.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 602cd9af-422f-4b1d-ab37-5fd3e0bd016f
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/dI1I79r
2025-11-21 15:58:17 +00:00
marco370
aec912cf16
Fix frontend startup to correctly load environment variables
...
Update `deployment/check_frontend.sh` to load `.env` variables before running `npm run dev`, resolving the `DATABASE_URL must be set` error and improving deployment on non-Replit environments.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8cf499b5-f15d-4b0d-b459-f48510ac6539
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/dI1I79r
2025-11-21 15:52:15 +00:00
marco370
b3bbcb69d3
Update system to apply database changes and synchronize dependencies
...
Apply SQL scripts to update database schema and install/update Node.js and Python dependencies.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: bf319927-85dd-43f9-9df2-348d41742705
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/Aqah4U9
2025-11-21 14:31:40 +00:00
marco370
203eaa9a54
Improve system stability by fixing cron job and process monitoring
...
Refactor cron job execution to use separate shell scripts and update process monitoring scripts to correctly handle PID files and log directories.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f6d4ec61-7130-41dd-aef9-87b4bc73d0e8
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/3R8dKMX
2025-11-18 08:33:05 +00:00
marco370
62ffbe499a
Add automated training and detection jobs to the system
...
Adds new cron jobs and shell scripts for automated ML model training and anomaly detection, along with configurations for logging and system checks.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 534244cd-b293-403e-a6d6-43cd9fbb30fb
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/1P26v7M
2025-11-18 08:24:09 +00:00
marco370
94724ff580
Add ability to detect and block malicious network traffic based on risk
...
Adds functionality to detect network anomalies and block suspicious IP addresses.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: e23fa694-b094-4b45-9ce0-0dbf8fd7a9e2
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/1P26v7M
2025-11-18 07:31:39 +00:00
marco370
dc91096d9d
Improve log processing and add automated tasks
...
Update log processing to use the correct timestamp field and introduce cron jobs for automated model training and anomaly detection.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f0653fd5-fc94-4fcb-8d7e-2a0e90fc81bf
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 18:11:49 +00:00
marco370
d345a24572
Improve intrusion detection system with functional updates and database fixes
...
Update `replit.md` to reflect recent system improvements including a fully functional syslog parser, PostgreSQL database, updated regex patterns, DDoS detection, and automated deployment workflows. Addresses issues with the `network_logs` table schema and incorrect regex matching. Includes SQL query outputs for verification.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 70827608-8ca8-471f-a794-336056b4ce88
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:54:18 +00:00
marco370
e5f307af27
Fix error when saving network logs to the database
...
Correct SQL query to insert data into the network_logs table, resolving "relation 'network_logs' does not exist" errors.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: e8e7edaf-8ac8-493c-b890-465ff617b0ce
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:40:22 +00:00
marco370
0d34bf7d3c
Update log parsing to better identify network traffic and DDoS events
...
Refactors the `SyslogParser` class in `python_ml/syslog_parser.py` to use a new, more comprehensive regex pattern (`main_pattern`) for parsing MikroTik logs. This includes improved identification of 'forward' and 'detected-ddos forward' actions, protocol details (UDP, TCP, ICMP), and associated IP addresses, ports, and lengths. The changes aim to accurately capture network traffic and potential DDoS events from MikroTik logs.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: b7377ada-e722-475a-86d2-07f21299ec70
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:35:37 +00:00
marco370
51aa026aae
Update logs to show processing progress
...
Adds new log entries to the syslog parser output, indicating incremental progress in row processing without saving any logs.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: b96cf229-a1bd-4e23-8aee-d92bd339420b
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/MkBJZ0L
2025-11-17 17:34:04 +00:00
marco370
7b6fc148a5
Update system to correctly process log files with proper permissions
...
Correctly sets file permissions and ownership for the `.env` file and demonstrates successful database connection and log file processing by the syslog parser script.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 91f9e939-55de-4fd8-9f48-6360540b2bfe
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:42:42 +00:00
marco370
1936a29ae4
Improve database connection security and restart log parser
...
Modify pg_hba.conf to use SCRAM-SHA-256 for database authentication and restart the syslog parser service.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 2c44c1cf-5725-4dd8-8305-3fa9b2a5136c
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:39:09 +00:00
marco370
08c2373aa5
Configure database user and password for secure access
...
Updates PostgreSQL configuration to set password encryption to SCRAM-SHA-256, creates a new user 'ids_user' with the specified password, grants necessary privileges on the 'ids_database', and configures default privileges for future objects. Includes troubleshooting steps and log excerpts related to authentication failures.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: f8be77ab-2269-4666-9e56-9309e455e81c
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:28:10 +00:00
marco370
7c36dc039b
Fix PostgreSQL authentication issues for user access
...
Introduces a shell script to modify pg_hba.conf and reset PostgreSQL user passwords, resolving authentication failures.
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 39b3e0c3-d6b2-4c6f-afb4-e32fe7f09b02
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/c9ITWqD
2025-11-17 16:25:32 +00:00
marco370
0b9d0cf302
Initial commit
2025-11-11 09:12:13 +00:00
