marco/ids.alfacom.it
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3574ff0274
ids.alfacom.it/deployment/docs/PUBLIC_LISTS_LIMITATIONS.md
marco370 5952142a56 Add public lists integration with exact IP matching 
Update merge logic to use exact IP matching for public lists, add deployment scripts and documentation for limitations.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 7a657272-55ba-4a79-9a2e-f1ed9bc7a528
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 75a02f7d-492b-46a8-9e67-d4fd471cabc7
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/449cf7c4-c97a-45ae-8234-e5c5b8d6a84f/7a657272-55ba-4a79-9a2e-f1ed9bc7a528/QKzTQQy
2025-11-26 09:45:55 +00:00

1.5 KiB
Raw Blame History

Public Lists - Known Limitations (v2.0.0)

CIDR Range Matching

Current Status: MVP with exact IP matching Impact: CIDR ranges (e.g., Spamhaus /24 blocks) are stored but not yet matched against detections

Details:

  • public_blacklist_ips.cidr_range field exists and is populated by parsers
  • Detections currently use exact IP matching only
  • Whitelist entries with CIDR notation not expanded

Future Iteration:

Requires PostgreSQL INET/CIDR column types and query optimizations:

  1. Add dedicated inet columns to public_blacklist_ips and whitelist
  2. Rewrite merge logic with CIDR containment operators (<<=, >>=)
  3. Index optimization for network range queries

Workaround (Production):

Most critical single IPs are still caught. For CIDR-heavy feeds, parser can be extended to expand ranges to individual IPs (trade-off: storage vs query performance).

Integration Status

✅ Working:

  • Fetcher syncs every 10 minutes (systemd timer)
  • Manual whitelist > Public whitelist > Blacklist priority
  • Automatic cleanup of invalid detections

⚠️ Manual Sync:

  • UI manual sync triggers by resetting lastAttempt timestamp
  • Actual sync occurs on next fetcher cycle (max 10 min delay)
  • For immediate sync: sudo systemctl start ids-list-fetcher.service

Performance Notes

  • Bulk SQL operations avoid O(N) per-IP queries
  • Tested with 186M+ network_logs records
  • Query optimization ongoing for CIDR expansion

Version: 2.0.0 MVP
Date: 2025-11-26
Next Iteration: Full CIDR matching support