Live Status\Incident History

July 19, 2024

Incomplete Execution / Fiserv VPN Connection

Outage

Opened: Jul 19, 2024, 1:05 AM UTC

Duration: 49sec

  • Opened
    Jul 19, 2024, 1:05 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 19, 2024, 1:05 AM UTC

    Issue resolved on monitor recovery.

July 18, 2024

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 18, 2024, 11:43 PM UTC

Duration: 1hr 16min 24sec

  • Opened
    Jul 18, 2024, 11:43 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 18, 2024, 11:45 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 19, 2024, 12:12 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 19, 2024, 12:16 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 19, 2024, 12:42 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 19, 2024, 12:56 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 19, 2024, 12:59 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 19, 2024, 1:54 AM UTC

    Issue resolved on monitor recovery.

Missed Execution

Outage

Opened: Jul 18, 2024, 10:25 PM UTC

Duration: 2hrs 11min 16sec

  • Opened
    Jul 18, 2024, 10:25 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 18, 2024, 10:25 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 18, 2024, 11:15 PM UTC

    Issue re-opened on monitor failure.

  • Monitoring
    Jul 18, 2024, 11:22 PM UTC

    We are actively monitoring an outage with our cloud hosting provider, Azure, this is currently having a minimal impact on our services. This outage is affecting a large number of Azure customers. They are currently working to identify the cause and resolve it. Most of our APIs, transaction authorization and cardholder portal are still operating normally, however, our Push API is currently offline. We're trying to identify a possible resolution, but cannot access the server associated with this service due to the outage. You can monitor the Azure outage here: https://azure.status.microsoft/en-us/status

  • Resolved
    Jul 19, 2024, 1:26 AM UTC

    Issue resolved on monitor recovery.

Missed Execution / Auth Engine Status

Outage

Opened: Jul 18, 2024, 10:18 PM UTC

Duration: 2hrs 46min 26sec

  • Opened
    Jul 18, 2024, 10:18 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 18, 2024, 10:18 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 18, 2024, 10:43 PM UTC

    Issue re-opened on monitor failure.

  • Monitoring
    Jul 18, 2024, 10:46 PM UTC

    We are actively monitoring an outage with our cloud hosting provider, Azure, this is currently having a minimal impact on our services. This outage is affecting a large number of Azure customers. They are currently working to identify the cause and resolve it. Most of our APIs, transaction authorization and cardholder portal are still operating normally. This alert indicates our Auth Engine is experiencing an outage, however, it's actually the server that monitors our Auth Engine that is having an issue. Again, we are processing transactions normally at this time. You can monitor the Azure outage here: https://azure.status.microsoft/en-us/status

  • Monitoring
    Jul 18, 2024, 11:59 PM UTC

    Azure is having an outage, the server that hosts this monitor is experiencing issues, however Auth Engine is still online and functioning while being monitored. ~Kyle Simmons

  • Monitoring
    Jul 19, 2024, 1:07 AM UTC

    Azure outage has worsened. Transaction processing is officially offline as of 8:00 pm CT. ~Kyle Simmons

  • Resolved
    Jul 19, 2024, 1:25 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 19, 2024, 1:53 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 19, 2024, 1:57 AM UTC

    Issue resolved on monitor recovery.

  • Resolved
    Jul 19, 2024, 2:14 AM UTC

    Transactions started processing again at 8:54 pm CT. ~Kyle Simmons.

July 2, 2024

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Jul 2, 2024, 8:51 AM UTC

Duration: 1hr 25min 5sec

  • Opened
    Jul 2, 2024, 8:51 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 2, 2024, 10:16 AM UTC

    Issue resolved on monitor recovery.

June 14, 2024

Failed Execution / Auth Engine Status

Outage

Opened: Jun 14, 2024, 4:45 PM UTC

Duration: 11min 58sec

  • Opened
    Jun 14, 2024, 4:45 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 4:57 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Jun 14, 2024, 4:35 PM UTC

Duration: 22min 32sec

  • Opened
    Jun 14, 2024, 4:35 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 4:58 PM UTC

    Issue resolved on monitor recovery.

June 2, 2024

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Jun 2, 2024, 7:59 AM UTC

Duration: 13hrs 58min 58sec

  • Opened
    Jun 2, 2024, 7:59 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 2, 2024, 9:58 PM UTC

    Issue resolved on monitor recovery.

May 24, 2024

Site Unreachable / Akimbo Apps API

Outage

Opened: May 24, 2024, 9:34 PM UTC

Duration: 1min 45sec

  • Opened
    May 24, 2024, 9:34 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 24, 2024, 9:36 PM UTC

    Issue resolved on monitor recovery.

May 20, 2024

Missed Execution

Outage

Opened: May 20, 2024, 5:20 AM UTC

Duration: 31sec

  • Opened
    May 20, 2024, 5:20 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 20, 2024, 5:20 AM UTC

    Issue resolved on monitor recovery.

Missed Execution / Auth Engine Status

Outage

Opened: May 20, 2024, 5:18 AM UTC

Duration: 30sec

  • Opened
    May 20, 2024, 5:18 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 20, 2024, 5:18 AM UTC

    Issue resolved on monitor recovery.

Missed Execution / Fiserv VPN Connection

Outage

Opened: May 20, 2024, 5:11 AM UTC

Duration: 31sec

  • Opened
    May 20, 2024, 5:11 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 20, 2024, 5:11 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: May 20, 2024, 5:08 AM UTC

Duration: 37min 34sec

  • Opened
    May 20, 2024, 5:08 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 20, 2024, 5:45 AM UTC

    Issue resolved on monitor recovery.

April 12, 2024

Incomplete Execution / Card Funding

Outage

Opened: Apr 12, 2024, 9:43 PM UTC

Duration: 15hrs 43min 46sec

  • Opened
    Apr 12, 2024, 9:43 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Apr 13, 2024, 1:26 PM UTC

    Issue resolved on monitor recovery.

March 6, 2024

Missed Execution

Outage

Opened: Mar 6, 2024, 3:05 PM UTC

Duration: 54sec

  • Opened
    Mar 6, 2024, 3:05 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 6, 2024, 3:06 PM UTC

    Issue resolved on monitor recovery.

Missed Execution

Outage

Opened: Mar 6, 2024, 8:30 AM UTC

Duration: 33sec

  • Opened
    Mar 6, 2024, 8:30 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 6, 2024, 8:31 AM UTC

    Issue resolved on monitor recovery.

March 5, 2024

Missed Execution

Outage

Opened: Mar 5, 2024, 8:15 PM UTC

Duration: 30sec

  • Opened
    Mar 5, 2024, 8:15 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 5, 2024, 8:15 PM UTC

    Issue resolved on monitor recovery.

Missed Execution

Outage

Opened: Mar 5, 2024, 6:20 PM UTC

Duration: 53sec

  • Opened
    Mar 5, 2024, 6:20 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 5, 2024, 6:21 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Mar 5, 2024, 6:45 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Mar 5, 2024, 6:45 PM UTC

    Issue resolved on monitor recovery.

March 2, 2024

Incomplete Execution / Fiserv VPN Connection

Outage

Opened: Mar 2, 2024, 5:42 AM UTC

Duration: 30sec

  • Opened
    Mar 2, 2024, 5:42 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 2, 2024, 5:42 AM UTC

    Issue resolved on monitor recovery.

February 29, 2024

Missed Execution

Outage

Opened: Feb 29, 2024, 7:40 PM UTC

Duration: 30sec

  • Opened
    Feb 29, 2024, 7:40 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 29, 2024, 7:40 PM UTC

    Issue resolved on monitor recovery.

February 28, 2024

Missed Execution

Outage

Opened: Feb 28, 2024, 10:30 PM UTC

Duration: 33sec

  • Opened
    Feb 28, 2024, 10:30 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 28, 2024, 10:31 PM UTC

    Issue resolved on monitor recovery.

February 23, 2024

Missed Execution

Outage

Opened: Feb 23, 2024, 5:55 PM UTC

Duration: 33sec

  • Opened
    Feb 23, 2024, 5:55 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 23, 2024, 5:56 PM UTC

    Issue resolved on monitor recovery.

Missed Execution

Outage

Opened: Feb 23, 2024, 3:30 PM UTC

Duration: 54sec

  • Opened
    Feb 23, 2024, 3:30 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 23, 2024, 3:30 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 23, 2024, 3:50 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 23, 2024, 3:50 PM UTC

    Issue resolved on monitor recovery.

February 21, 2024

Missed Execution

Outage

Opened: Feb 21, 2024, 5:50 PM UTC

Duration: 34sec

  • Opened
    Feb 21, 2024, 5:50 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 21, 2024, 5:50 PM UTC

    Issue resolved on monitor recovery.

February 20, 2024

Missed Execution

Outage

Opened: Feb 20, 2024, 9:15 AM UTC

Duration: 32sec

  • Opened
    Feb 20, 2024, 9:15 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 20, 2024, 9:15 AM UTC

    Issue resolved on monitor recovery.

February 16, 2024

Missed Execution

Outage

Opened: Feb 16, 2024, 5:10 PM UTC

Duration: 36sec

  • Opened
    Feb 16, 2024, 5:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 16, 2024, 5:11 PM UTC

    Issue resolved on monitor recovery.

February 15, 2024

Missed Execution

Outage

Opened: Feb 15, 2024, 8:35 PM UTC

Duration: 32sec

  • Opened
    Feb 15, 2024, 8:35 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 15, 2024, 8:35 PM UTC

    Issue resolved on monitor recovery.

February 14, 2024

Job Not Completed / Fiserv VPN Connection

Outage

Opened: Feb 14, 2024, 8:06 AM UTC

Duration: 9hrs 18min 43sec

  • Opened
    Feb 14, 2024, 8:06 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 14, 2024, 5:25 PM UTC

    Issue resolved on monitor recovery.

Incomplete Execution / Fiserv VPN Connection

Outage

Opened: Feb 14, 2024, 3:38 AM UTC

Duration: 2min 43sec

  • Opened
    Feb 14, 2024, 3:38 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 14, 2024, 3:38 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 3:49 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 3:49 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 4:13 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 4:13 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 4:23 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 4:23 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 4:39 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 4:39 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 4:53 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 4:54 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Feb 14, 2024, 5:11 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Feb 14, 2024, 5:11 AM UTC

    Issue resolved on monitor recovery.

February 3, 2024

Site Unreachable / Akimbo Apps API

Outage

Opened: Feb 3, 2024, 6:09 AM UTC

Duration: 1min 43sec

  • Opened
    Feb 3, 2024, 6:09 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 3, 2024, 6:11 AM UTC

    Issue resolved on monitor recovery.

February 1, 2024

Card Operations / Akimbo Apps API

Minor Outage

Opened: Feb 1, 2024, 6:14 PM UTC

Duration: 1hr 8min 38sec

  • Investigating
    Feb 1, 2024, 6:14 PM UTC

    We're investigating an issue where one of our service APIs is experiencing connectivity problems.

  • Monitoring
    Feb 1, 2024, 7:12 PM UTC

    We're monitoring a fix for the API connectivity issues.

  • Resolved
    Feb 1, 2024, 7:22 PM UTC

Missed Execution

Outage

Opened: Feb 1, 2024, 6:10 PM UTC

Duration: 55sec

  • Opened
    Feb 1, 2024, 6:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 1, 2024, 6:10 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Feb 1, 2024, 6:02 PM UTC

Duration: 1hr 9min 25sec

  • Opened
    Feb 1, 2024, 6:02 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 1, 2024, 7:12 PM UTC

    Issue resolved on monitor recovery.

January 31, 2024

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Jan 31, 2024, 11:40 AM UTC

Duration: 1min 51sec

  • Opened
    Jan 31, 2024, 11:40 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jan 31, 2024, 11:41 AM UTC

    Issue resolved on monitor recovery.

January 12, 2024

Site Unreachable / Akimbo Apps API

Outage

Opened: Jan 12, 2024, 10:04 AM UTC

Duration: 1min 58sec

  • Opened
    Jan 12, 2024, 10:04 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jan 12, 2024, 10:06 AM UTC

    Issue resolved on monitor recovery.

December 8, 2023

Missed Execution

Outage

Opened: Dec 8, 2023, 4:10 PM UTC

Duration: 5min 22sec

  • Opened
    Dec 8, 2023, 4:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Dec 8, 2023, 4:16 PM UTC

    Issue resolved on monitor recovery.

November 5, 2023

Missed Execution / Fiserv VPN Connection

Outage

Opened: Nov 5, 2023, 10:57 PM UTC

Duration: 51sec

  • Opened
    Nov 5, 2023, 10:57 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 5, 2023, 10:58 PM UTC

    Issue resolved on monitor recovery.

Missed Execution / Auth Engine Status

Outage

Opened: Nov 5, 2023, 9:11 PM UTC

Duration: 30min 19sec

  • Opened
    Nov 5, 2023, 9:11 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 5, 2023, 9:15 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Nov 5, 2023, 9:39 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Nov 5, 2023, 10:06 PM UTC

    Issue resolved on monitor recovery.

Missed Execution

Outage

Opened: Nov 5, 2023, 8:15 PM UTC

Duration: 53sec

  • Opened
    Nov 5, 2023, 8:15 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 5, 2023, 8:16 PM UTC

    Issue resolved on monitor recovery.

November 1, 2023

Job Not Completed / Fiserv VPN Connection

Outage

Opened: Nov 1, 2023, 6:29 PM UTC

Duration: 19hrs 42min 42sec

  • Opened
    Nov 1, 2023, 6:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 2, 2023, 2:12 PM UTC

    Issue resolved on monitor recovery.

Incomplete Execution / Fiserv VPN Connection

Outage

Opened: Nov 1, 2023, 3:35 PM UTC

Duration: 30sec

  • Opened
    Nov 1, 2023, 3:35 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 1, 2023, 3:35 PM UTC

    Issue resolved on monitor recovery.

October 11, 2023

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Oct 11, 2023, 10:23 PM UTC

Duration: 6min 30sec

  • Opened
    Oct 11, 2023, 10:23 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 11, 2023, 10:29 PM UTC

    Issue resolved on monitor recovery.

September 18, 2023

Missed Execution / Auth Engine Status

Outage

Opened: Sep 18, 2023, 9:25 AM UTC

Duration: 1min 16sec

  • Opened
    Sep 18, 2023, 9:25 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 18, 2023, 9:26 AM UTC

    Issue resolved on monitor recovery.

August 2, 2023

Job Not Completed / Auth Engine Status

Outage

Opened: Aug 2, 2023, 6:49 AM UTC

Duration: 4min 52sec

  • Opened
    Aug 2, 2023, 6:49 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 2, 2023, 6:54 AM UTC

    Issue resolved on monitor recovery.

July 24, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 24, 2023, 10:14 AM UTC

Duration: 14min 36sec

  • Opened
    Jul 24, 2023, 10:14 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 24, 2023, 10:29 AM UTC

    An automatic failover with one of our database servers was successful, however, an issue with DNS prevented Akimbo Apps API from being directed to the correct failover database server. The underlying DNS issue has been resolved which will help ensure failover scenarios do not cause downtime.

  • Resolved
    Jul 24, 2023, 10:29 AM UTC

    Issue resolved on monitor recovery.

July 5, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 5, 2023, 12:38 PM UTC

Duration: 7min 16sec

  • Opened
    Jul 5, 2023, 12:38 PM UTC

    Issue opened on monitor failure.

  • Identified
    Jul 5, 2023, 12:45 PM UTC

    This issue was caused by an abnormal spike in traffic which slowed our systems temporarily with no actual downtime. Cardholders may have noticed slower than normal logins to our apps but still had access during this time. We are continuing to monitor.

  • Monitoring
    Jul 5, 2023, 12:45 PM UTC
  • Resolved
    Jul 5, 2023, 12:45 PM UTC

    Issue resolved on monitor recovery.

July 4, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 4, 2023, 9:10 AM UTC

Duration: 18min 34sec

  • Opened
    Jul 4, 2023, 9:10 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 4, 2023, 9:21 AM UTC

    Issue resolved on monitor recovery.

  • Identified
    Jul 4, 2023, 9:49 AM UTC

    Server upgrade performed.

  • Resolved
    Jul 4, 2023, 9:57 AM UTC

July 3, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 3, 2023, 11:27 PM UTC

Duration: 1min 29sec

  • Opened
    Jul 3, 2023, 11:27 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 3, 2023, 11:28 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 3, 2023, 9:01 PM UTC

Duration: 2min 26sec

  • Opened
    Jul 3, 2023, 9:01 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 3, 2023, 9:03 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 3, 2023, 1:07 PM UTC

Duration: 1hr 29min 14sec

  • Opened
    Jul 3, 2023, 1:07 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 3, 2023, 1:09 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 1:45 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 1:48 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 1:59 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 2:12 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 2:16 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 2:27 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 2:36 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 2:47 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 2:51 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 2:54 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 2:56 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 3:39 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 3, 2023, 3:41 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 3, 2023, 3:45 PM UTC

    Issue resolved on monitor recovery.

July 2, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 2, 2023, 4:20 PM UTC

Duration: 2min 18sec

  • Opened
    Jul 2, 2023, 4:20 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 2, 2023, 4:22 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 2, 2023, 4:29 AM UTC

Duration: 42min 13sec

  • Opened
    Jul 2, 2023, 4:29 AM UTC

    Issue opened on monitor failure.

  • Identified
    Jul 2, 2023, 4:45 AM UTC

    We received an abnormal spike in traffic which mimicked a denial of service flood. Upon further investigation we identified this traffic was originating from a vendor and temporarily blocked traffic to that vendor. The issue is resolved but are currently investigating why rate limiting didn't automatically slow this traffic.

  • Resolved
    Jul 2, 2023, 5:11 AM UTC

    Issue resolved on monitor recovery.

July 1, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jul 1, 2023, 1:17 PM UTC

Duration: 6min 37sec

  • Opened
    Jul 1, 2023, 1:17 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 1, 2023, 1:20 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 1, 2023, 1:22 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 1, 2023, 1:26 PM UTC

    Issue resolved on monitor recovery.

June 25, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jun 25, 2023, 7:41 PM UTC

Duration: 1min 54sec

  • Opened
    Jun 25, 2023, 7:41 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 25, 2023, 7:43 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable / Akimbo Apps API

Outage

Opened: Jun 25, 2023, 3:10 AM UTC

Duration: 2min 21sec

  • Opened
    Jun 25, 2023, 3:10 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 25, 2023, 3:13 AM UTC

    Issue resolved on monitor recovery.

June 22, 2023

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Jun 22, 2023, 5:14 PM UTC

Duration: 26min 47sec

  • Opened
    Jun 22, 2023, 5:14 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 22, 2023, 5:40 PM UTC

    Issue resolved on monitor recovery.

June 19, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jun 19, 2023, 10:03 AM UTC

Duration: 5min 25sec

  • Opened
    Jun 19, 2023, 10:03 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 19, 2023, 10:08 AM UTC

    Issue resolved on monitor recovery.

June 18, 2023

Failed Execution / Auth Engine Status

Outage

Opened: Jun 18, 2023, 6:18 AM UTC

Duration: 9min 1sec

  • Opened
    Jun 18, 2023, 6:18 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 18, 2023, 6:27 AM UTC

    Issue resolved on monitor recovery.

June 2, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Jun 2, 2023, 1:29 AM UTC

Duration: 12min 16sec

  • Opened
    Jun 2, 2023, 1:29 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 2, 2023, 1:38 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 2, 2023, 1:56 AM UTC

    Incident re-opened on monitor failure.

  • Resolved
    Jun 2, 2023, 2:00 AM UTC

    Issue resolved on monitor recovery.

May 24, 2023

Missed Execution / Auth Engine Status

Outage

Opened: May 24, 2023, 6:46 AM UTC

Duration: 26sec

  • Opened
    May 24, 2023, 6:46 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 24, 2023, 6:46 AM UTC

    Issue resolved on monitor recovery.

May 22, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: May 22, 2023, 10:26 AM UTC

Duration: 3min 7sec

  • Opened
    May 22, 2023, 10:26 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 22, 2023, 10:29 AM UTC

    Issue resolved on monitor recovery.

May 19, 2023

Missed Execution / Auth Engine Status

Outage

Opened: May 19, 2023, 6:39 AM UTC

Duration: 26sec

  • Opened
    May 19, 2023, 6:39 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 19, 2023, 6:39 AM UTC

    Issue resolved on monitor recovery.

May 10, 2023

Failed Execution / Auth Engine Status

Outage

Opened: May 10, 2023, 4:43 AM UTC

Duration: 49min 55sec

  • Opened
    May 10, 2023, 4:43 AM UTC

    Issue opened on monitor failure.

  • Resolved
    May 10, 2023, 5:33 AM UTC

    Issue resolved on monitor recovery.

April 14, 2023

Failed Execution / Card Funding

Outage

Opened: Apr 14, 2023, 2:24 PM UTC

Duration: 27min 3sec

  • Opened
    Apr 14, 2023, 2:24 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Apr 14, 2023, 2:51 PM UTC

    Issue resolved on monitor recovery.

April 13, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Apr 13, 2023, 8:28 PM UTC

Duration: 3min 45sec

  • Opened
    Apr 13, 2023, 8:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Apr 13, 2023, 8:32 PM UTC

    Issue resolved on monitor recovery.

April 8, 2023

Failed Execution / Auth Engine Status

Outage

Opened: Apr 8, 2023, 5:31 AM UTC

Duration: 40min 36sec

  • Opened
    Apr 8, 2023, 5:31 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Apr 8, 2023, 6:11 AM UTC

    Issue resolved on monitor recovery.

April 6, 2023

Network Maintenance Scheduled

Maintenance

Posted: Apr 6, 2023, 6:16 PM UTC

  • Monitoring
    Apr 5, 2023, 3:53 PM UTC

    Monday April 10th 2023 at 1 am CT (12 am ET) we will have approximately 10 mins of network down time for management of our Azure networking resources. This will likely impact the following services: Push API - Expect delays in sending out updates. Akimbo API - Connection Down. Akimbo Web App and Mobile App - Inability to connect to Akimbo API. Card Services API - Connection Down. External Auth - No outbound connection. Debit Card Portal (DCP) - No inbound connection. USIO SFTP - No inbound connection. Unaffected Services: Auth Engine - Should be unaffected Thank you, ~Kyle S.

  • Resolved
    Apr 10, 2023, 6:31 AM UTC

    Network Maintenance complete. All services online. Kyle S.

March 31, 2023

Failed Execution / Auth Engine Status

Outage

Opened: Mar 31, 2023, 2:29 PM UTC

Duration: 8min 19sec

  • Opened
    Mar 31, 2023, 2:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 31, 2023, 2:37 PM UTC

    Issue resolved on monitor recovery.

March 28, 2023

Missed Execution / Send Low Distributor Balance Emails

Outage

Opened: Mar 28, 2023, 8:01 PM UTC

Duration: 16hrs 59min 21sec

  • Opened
    Mar 28, 2023, 8:01 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 29, 2023, 1:00 PM UTC

    Issue resolved on monitor recovery.

Suspected Network Issues

Degraded Performance

Opened: Mar 28, 2023, 6:19 PM UTC

Duration: 20hrs 58min 21sec

  • Investigating
    Mar 28, 2023, 6:16 PM UTC

    An issue has arisen that appears to be directly impacting HTTPS traffic to Debit Card Portal and may have intermittent issues with Card Services API. Minimal impact to Akimbo API. Current investigation is underway. ~Kyle S.

  • Resolved
    Mar 29, 2023, 3:17 PM UTC

    As of 6:23pm CT March 28th, this issue has been resolved.

Missed Execution

Outage

Opened: Mar 28, 2023, 5:35 PM UTC

Duration: 4hrs 45min 41sec

  • Opened
    Mar 28, 2023, 5:35 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 28, 2023, 5:35 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Mar 28, 2023, 6:30 PM UTC

    Incident re-opened on monitor failure.

  • Resolved
    Mar 28, 2023, 11:15 PM UTC

    Issue resolved on monitor recovery.

Missed Execution / Auth Engine Status

Outage

Opened: Mar 28, 2023, 5:32 PM UTC

Duration: 5hrs 18min 40sec

  • Opened
    Mar 28, 2023, 5:32 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 28, 2023, 5:33 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Mar 28, 2023, 5:57 PM UTC

    Incident re-opened on monitor failure.

  • Resolved
    Mar 28, 2023, 11:15 PM UTC

    Issue resolved on monitor recovery.

March 20, 2023

Healthcheck Assertion Failed / Akimbo Apps API

Outage

Opened: Mar 20, 2023, 10:01 AM UTC

Duration: 2min 7sec

  • Opened
    Mar 20, 2023, 10:01 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Mar 20, 2023, 10:03 AM UTC

    Issue resolved on monitor recovery.

March 2, 2023

Failed Execution / Auth Engine Status

Outage

Opened: Mar 2, 2023, 7:03 PM UTC

Duration: 1hr 5sec

  • Opened
    Mar 2, 2023, 7:03 PM UTC

    Issue opened on monitor failure.

  • Investigating
    Mar 2, 2023, 7:52 PM UTC

    Authorization Engine is currently not processing transactions and we are in stand-in with our transaction vendor Fiserv. Updates to come, we are all hands investigating issue. ~Kyle S.

  • Resolved
    Mar 2, 2023, 8:03 PM UTC

    Issue resolved on monitor recovery.

  • Resolved
    Mar 2, 2023, 8:05 PM UTC

    Transactions are coming in and are now being process. External Authorizations are now being sent as well. ~Kyle S.

February 24, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Feb 24, 2023, 11:43 AM UTC

Duration: 29min 19sec

  • Opened
    Feb 24, 2023, 11:43 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 24, 2023, 12:13 PM UTC

    Issue resolved on monitor recovery.

February 23, 2023

Site Unreachable / Akimbo Apps API

Outage

Opened: Feb 23, 2023, 6:59 PM UTC

Duration: 14min 15sec

  • Opened
    Feb 23, 2023, 6:59 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Feb 23, 2023, 7:14 PM UTC

    Issue resolved on monitor recovery.

January 25, 2023

Emergency Maintenance / Akimbo Apps API

Maintenance

Posted: Jan 25, 2023, 10:37 AM UTC

  • Identified
    Jan 25, 2023, 10:35 AM UTC

    Database maintenance in progress. ~Kyle S.

  • Monitoring
    Jan 25, 2023, 11:12 AM UTC

    Database maintenance resolved. ~Kyle S.

  • Resolved
    Jan 30, 2023, 7:34 PM UTC

January 10, 2023

Healthcheck Assertion Failed

Outage

Opened: Jan 10, 2023, 4:36 PM UTC

Duration: 35min 55sec

  • Opened
    Jan 10, 2023, 4:36 PM UTC

    Issue opened on monitor failure.

  • Identified
    Jan 10, 2023, 5:07 PM UTC

    We have found the issue and are working to correct asap. ~Kyle S. SRE.

  • Resolved
    Jan 10, 2023, 5:12 PM UTC

    Issue resolved on monitor recovery.

November 26, 2022

Auth Engine Transaction Processing Offline

Degraded Performance

Opened: Nov 26, 2022, 2:31 AM UTC

Duration: 48min 41sec

  • Investigating
    Nov 26, 2022, 2:29 AM UTC

    SQL Connection Timeouts occurring intermittently during core services connection including Auth Engine. Unable to load resource in Azure portal to verify health checks. Contacted Azure Support. ~Kyle s.

  • Investigating
    Nov 26, 2022, 2:37 AM UTC

    Issue is affecting some transactions to be processed in stand in. Transactions are still being processed over all. ~Kyle S.

  • Resolved
    Nov 26, 2022, 3:19 AM UTC

    Issue was with resources being over utilized by a service. Corrective measure put in place to fix.

October 23, 2022

Failed Execution

Minor Outage

Opened: Oct 23, 2022, 3:27 PM UTC

Duration: 56min 17sec

  • Opened
    Oct 23, 2022, 3:27 PM UTC

    Incident opened on monitor failure.

  • Identified
    Oct 23, 2022, 3:31 PM UTC

    Issue with Azure VM detected. Working on resolving. ~Kyle S.

  • Resolved
    Oct 23, 2022, 4:23 PM UTC

    Incident resolved on monitor recovery.

September 14, 2022

Auth Engine Transaction Processing Offline

Minor Outage

Opened: Sep 14, 2022, 8:41 AM UTC

Duration: 1hr 13min 18sec

  • Identified
    Sep 14, 2022, 8:40 AM UTC

    Currently working on resolving issues with Auth Engine not responding. ~Kyle S. USIO

  • Resolved
    Sep 14, 2022, 9:54 AM UTC

    After rebooting host server, we were able to diagnose the issue and bring services back online. Thank you, ~Kyle S. USIO

July 16, 2022

Auth Engine Transaction Processing Offline

Minor Outage

Opened: Jul 16, 2022, 6:47 AM UTC

Duration: 7hrs 7min 1sec

  • Opened
    Jul 16, 2022, 6:47 AM UTC

    Incident opened on monitor failure.

  • Identified
    Jul 16, 2022, 1:24 PM UTC

    Our processor Fiserv is under a maintenance window that is preventing them from sending transactions to our Authorization Engine for approval or denial on our system. However, they are processing transactions in stand-in based on balance updates that send to Fiserv every 30 minutes, and previously defined stand-in rules. Debit cards will work and follow previously configured stand in instructions from Fiserv side. I have verified that there are active transactions being processed. I have also verified that there are multiple teams working on this issue from Fiserv's end. ~Kyle S.

  • Resolved
    Jul 16, 2022, 1:54 PM UTC

    It would appear that Fiserv has corrected the issues, we are now receiving transactions for processing again. ~Kyle S.

June 21, 2022

System Database Connectivity Issue

Outage

Opened: Jun 21, 2022, 7:22 PM UTC

Duration: 12hrs 43min

  • Opened
    Jun 21, 2022, 7:22 PM UTC

    Incident opened on monitor failure.

  • Monitoring
    Jun 21, 2022, 8:37 PM UTC

    Systems have been restored and are functioning normally again. We are monitoring all services manually as well as via automated means. Kyle S. USIO

  • Resolved
    Jun 22, 2022, 8:05 AM UTC

    Incident resolved on monitor recovery.

Response Assertion Failed

Outage

Opened: Jun 21, 2022, 6:16 PM UTC

Duration: 2hrs 9min 9sec

  • Opened
    Jun 21, 2022, 6:16 PM UTC

    Incident opened on monitor failure.

  • Investigating
    Jun 21, 2022, 7:07 PM UTC

    We are currently experiencing an outage have and have engaged Azure Support. Kyle S. USIO

  • Resolved
    Jun 21, 2022, 8:25 PM UTC

    Incident resolved on monitor recovery.

Site Unreachable

Outage

Opened: Jun 21, 2022, 6:30 AM UTC

Duration: 50min 51sec

  • Opened
    Jun 21, 2022, 6:30 AM UTC

    Incident opened on monitor failure.

  • Resolved
    Jun 21, 2022, 7:12 AM UTC

    Incident resolved on monitor recovery.

  • Unresolved
    Jun 21, 2022, 7:13 AM UTC

    Incident re-opened on monitor failure.

  • Monitoring
    Jun 21, 2022, 7:16 AM UTC

    Our 3rd-party DNS and firewall provider, CloudFlare, is currently experiencing an outage that is impacted our services. Because of this outage you may experience intermittent connectivity issues while accessing the following services: Akimbo Apps API, Card Services API, Debit Card Portal. CloudFlare is aware of the issue and has identified the root cause. They are currently implementing a fix for this issue. You can monitor their progress here: https://www.cloudflarestatus.com/

  • Resolved
    Jun 21, 2022, 7:22 AM UTC

    CloudFlare has resolved the issue and it does appear all of our services are accessible at this time. We will continue to monitor.

June 20, 2022

Failed Execution

Outage

Opened: Jun 20, 2022, 3:46 PM UTC

Duration: 5hrs 48min 4sec

  • Opened
    Jun 20, 2022, 3:46 PM UTC

    Incident opened on monitor failure.

  • Investigating
    Jun 20, 2022, 3:58 PM UTC
  • Identified
    Jun 20, 2022, 4:04 PM UTC

    VPN issue has come back up. Our MSP and Fiserv support have been engaged. Kyle S. USIO

  • Monitoring
    Jun 20, 2022, 4:22 PM UTC

    VPN is now back online and we are processing transactions. Following through with Fiserv and MSP to verify logs to see if we can determine cause. Kyle S. USIO.

  • Resolved
    Jun 20, 2022, 9:34 PM UTC

    Incident resolved on monitor recovery.

June 17, 2022

Fiserv VPN Connection Offline

Outage

Opened: Jun 17, 2022, 3:51 AM UTC

Duration: 10hrs 55min 27sec

  • Opened
    Jun 17, 2022, 3:51 AM UTC

    Incident opened on monitor failure.

  • Investigating
    Jun 17, 2022, 4:19 AM UTC

    Initial call has come in from Fiserv and Active Alert monitoring has alerted to Networking issue. Investigating currently. ~Kyle S. USIO

  • Identified
    Jun 17, 2022, 5:10 AM UTC

    Identified outage with third party processor. Investigation is occurring on their end, they are in the middle of restoring service to a large number of customers, we appear to be affected by this service outage. ~Kyle S.

  • Identified
    Jun 17, 2022, 5:55 AM UTC

    Update from Fiserv, we are not part of their global outage issues. Our VPN Client appears to be unresponsive. I have also engaged our MSP and will follow up with more updates soon. ~Kyle S. USIO.

  • Identified
    Jun 17, 2022, 9:28 AM UTC

    Fiserv Network Engineering has escalated this issue to both their advanced networking team and their firewall team. They are currently investigating from their end. ~Kyle S. USIO

  • Identified
    Jun 17, 2022, 12:34 PM UTC

    Usio and Fiserv network engineers are continuing to troubleshoot the VPN connection. - Jeff F.

  • Monitoring
    Jun 17, 2022, 2:36 PM UTC

    Usio and Fiserv engineers have restored the VPN connection and are monitoring the Auth Engine's processing of transactions. - Jeff F.

  • Resolved
    Jun 17, 2022, 2:46 PM UTC