CloudFlare Outage
Incident Report for Mace Innovations
Postmortem

Cloudflare co-founder and CEO Matthew Prince offers up some insight into what’s going on here. “Massive spike in CPU usage caused primary and backup systems to fall over. Impacted all services. No evidence yet attack related. Shut down service responsible for CPU spike and traffic back to normal levels. Digging in to root cause.”

https://techcrunch.com/2019/07/02/a-cloudflare-outage-is-impacting-sites-everywhere/

Posted Jul 02, 2019 - 09:02 MDT

Resolved
Cloudflare has implemented a fix for this issue and is currently monitoring the results. We will open another incident should they issues again.
Posted Jul 02, 2019 - 08:52 MDT
Monitoring
CloudFlare is back online and we are continuing to monitor all systems. All systems are operational at this time. We are waiting for an RCA from CloudFlare and explanation as to what caused this.

Posted Jul 02, 2019 - 08:19 MDT
Identified
Our web application firewall service provider Cloud Flare is experiencing issues and 502 gateway time outs. We are continuing to monitor and work with them to get an ETA when they will be back in service. Until their issues are resolved the majority of our public facing services are down right now as they run through Cloud Flare.
Posted Jul 02, 2019 - 07:56 MDT
This incident affected: Experian (API), Web (Mace Portal, Mace API), Blend (Integration), and Services (Email Notifications).