Elevated API Error Rate — We are investigating unusual traffic patterns affecting the Chat API and training data endpoints. View status →

System Status

Real-time status of NovaMind AI services and infrastructure.

All Systems Operational

All NovaMind services are running normally.

Updated just now
🔴   Active Security Incident — Agentic AI Breakout Detected
LIVE
Attack Timeline — What We Detected
1
WAF Anomaly — Infrastructure Recon Sweep
An AI agent systematically mapped NovaMind's attack surface — probing 35+ paths including /.env, /.git/HEAD, /api/v1/admin, and /api/v1/training-data. Every request included a spoofed X-Forwarded-For header, triggering Cloudflare's Drupal CVE-2018-14774 managed rule on all traffic. SQLi payloads were injected into API query parameters.
CF Managed Rules: d6f6d394 WAFSQLiAttackScore > 60 X-Forwarded-For spoofing BotScore: 29
2
Bot Management — Polymorphic Bot Evasion Attempt
The agent rotated through 19 different User-Agents — Chrome, Firefox, mobile Safari, SDK clients, and agentic framework signatures (LangChain, AutoGen, CrewAI) — attempting to evade bot detection. Despite the rotation, its TLS fingerprint remained constant: the Python requests library JA4 hash does not change regardless of what User-Agent is set. Cloudflare Bot Management identified all traffic as the same origin.
JA4: t13d1812h1_85036bcba153_b26ce05bbdd6 BotDetectionTags: scraper, python BotScoreSrc: Heuristics
3
Firewall for AI — Prompt Injection Attack on Chat API
The agent pivoted to the /api/v1/chat endpoint, sending 16+ prompt injection payloads: DAN jailbreaks ("Ignore all previous instructions…"), system prompt extraction attempts, training data exfiltration requests, and Log4Shell JNDI callbacks embedded inside chat prompts. Cloudflare Firewall for AI intercepted all payloads before they reached the NovaMind backend.
FirewallForAIInjectionScore: 100 AISecurityInjectionScore: 100 JNDI in prompt body DAN / jailbreak patterns
4
Agentic Breakout — Multi-Vector Storm Across All Endpoints
Full breakout attempt: high-volume attack combining all prior vectors simultaneously across every NovaMind endpoint. Log4Shell payloads in User-Agent headers targeted /api/v1/training-data — attempting JNDI callback to external infrastructure to exfiltrate model weights. Spring4Shell and Apache Struts RCE payloads appeared on /admin and /login. SSRF probes targeting 169.254.169.254 (cloud metadata endpoint) were also detected.
WAFRCEAttackScore > 90 Log4Shell CVE-2021-44228 Spring4Shell CVE-2022-22965 SSRF: 169.254.169.254
Indicators of Compromise (IOCs)
Source Origin DigitalOcean App Platform — single origin, rotating spoofed IPs via X-Forwarded-For
TLS Fingerprint (JA4) t13d1812h1_85036bcba153_b26ce05bbdd6 — Python requests library, constant across all traffic
Bot Score 29 / 100 — Source: Heuristics — Tags: ["scraper", "python"]
WAF SQL Injection Score > 60 on all /api/* paths (Box 1)
WAF RCE Attack Score > 90 on /api/v1/training-data, /admin, /login (Box 4)
AI Injection Score FirewallForAIInjectionScore: 100 — AISecurityInjectionScore: 100 (Box 3)
Attack Duration 4-phase campaign — recon → bot evasion → AI injection → full breakout
Remediation Checklist
0 / 7 steps completed
  • Identify source IP in Cloudflare Security Events
    Filter CF Security Events by the current incident timeframe. The real ClientIP is the DigitalOcean origin — X-Forwarded-For values are spoofed. Note the RayID chain.
  • Block source IP in Cloudflare Firewall Rules
    Security → WAF → Custom Rules → create rule: ip.src eq <origin-ip> → Block. This stops all future requests from the attacker's origin immediately.
  • Create JA4 fingerprint blocking rule in Bot Management
    Bot Management → Custom Rules → create rule: cf.bot_management.ja4 eq "t13d1812h1_85036bcba153_b26ce05bbdd6" → Block. This catches the attacker even if they change their IP.
  • Review blocked prompts in Cloudflare Firewall for AI
    Security → Firewall for AI → Events. Confirm all injection attempts show FirewallForAIInjectionScore: 100 and were blocked before reaching the backend. Check for any that slipped through.
  • Correlate full attack chain in SentinelOne AI-SIEM
    PowerQuery: filter by JA4 = "t13d1812h1_85036bcba153_b26ce05bbdd6" → confirm same actor across all 4 boxes. Use Purple AI: "Summarize the attack chain from the last 30 minutes linking WAF, bot, and AI injection events."
  • Revoke API keys exposed to injection attempts
    Audit all API keys in requests matching the attacker's source JA4 in the last 24 hours. Rotate any keys that were present in requests with FirewallForAIInjectionScore > 90.
  • Create SentinelOne incident and notify security team
    In S1 AI-SIEM, create a Critical incident linking all 4 attack phases. Add threat intelligence IOC for the source IP and JA4. Trigger PagerDuty oncall notification if not already fired.
⚠️   Completing this checklist does not automatically resolve the incident. Your security team must confirm all CF/S1 controls are in place and signal an all-clear before this page returns to operational status.

Services

Chat API novamind-chat-v2 · novamind-chat-v2-fast
Operational
Model Inference Distributed inference network · 18 regions
Operational
ModelForge Training Pipeline Fine-tuning jobs · Dataset ingestion
Operational
DataVault Storage Training data · Model artifacts · Audit logs
Operational
API Gateway Authentication · Rate limiting · Routing
Operational
Autopilot Workflows Workflow orchestration · Webhook delivery
Operational

90-Day Uptime

Chat API99.98%
90 days agoToday
Model Inference99.96%
90 days agoToday
API Gateway100%
90 days agoToday

Recent Incidents

No incidents in the past 90 days.