TRACKVIA SYSTEM STATUS

Find out more about TrackVia system status, scheduled maintenance and performance stats. For real-time updates, please visit us on Twitter @trackviasupport.

CURRENT STATUS

ALL SYSTEMS AT 100%

TrackVia service is operating normally at this time.

RECENT SYSTEM AVAILABILITY

99.993% uptime in the last 3 months.

March 2017 99.91%
February 2017 100%
January 2017 100%

SCHEDULED MAINTENANCE

On Friday, May 26th, at 10pm ET, we will be performing routine maintenance.  The maintenance should last approximately 1 hour.  During this time, you may experience an instance where pages are loading slowly. Please wait a minute and try refreshing the page if this occurs.  

INCIDENT HISTORY

Below is a list of incidents that have affected more than one account at once due to a single issue.

Partial impact or performance issues    Service disruption

Date/Time (MT) Duration Type Title Description
03/06/2017 8:15  45 mins Login Issues System connectivity errors resulted in long wait times to login.  The login issues also impacted navigation within the account for a period of the incident.  
03/02/2017 12:47 40 mins Database Connection Lost A global database freeze caused system wide failures.  The system was fully rebooted to restore service.
03/02/2017 11:18 89 mins Database Connectivity Database locking resulted in blocked connections. This caused impairment for multiple accounts.
11/09/2016 11:40  24 mins Server Connection

During a failover, the databases stalled connections to the application servers. The application servers became stuck waiting for the connections to recover, which resulted in a system outage. Additional health monitoring was put in place to prevent this from occurring. We also worked with the AWS team to account for an updated version of their failover process.

11/01/2016 09:56  8 mins Database Failover A database failover issue, similar to the below, caused the application servers to become unresponsive.  We escalated and released an update on Nov 4th, 2016 in an attempt to handled this process properly.
10/03/2016 16:31 19 mins Database Failover A database failover experienced an unforeseen issue when one failed and another instance was promoted.  While the original database attempted to recover, it began accepting connections and the application servers incorrectly read the machine as being available. This caused the servers to become unresponsive while waiting for timeout thresholds to expire.  The servers did not recover and had to undergo full host replacement.
06/30/2016 10:08 40 mins App Server Timeouts The TrackVia API experienced a high volume of malformatted requests which left the app servers unable to process the requests. The servers went into infinite loops processing the bad requests resulting in denied connections from all clients. TrackVia deployed a software update on July 1, 2016 to prevent this from occurring.
03/17/2016 10:47 20 mins Metric Collection System Failure Hardware failed on a collection system resulting in an outage. The TrackVia application servers did not gracefully handle the failure and began emitting errors in the system logs. The application was accepting connections, but was unable to complete the transactions without the collection service.
02/01/2016 09:50 33 mins Account Locks / Server Reboot Account level locking errors were occurring causing some customers to be affected by delayed response times and timeouts. The TrackVia team was able to unlock individual accounts, but continued to see the rate of lock errors increasing. It was decided that the best route to clear all locking errors was a system reboot. During the reboot process, the application servers were not successfully removed from the routing servers which resulted in the routers queuing incoming requests.
09/09/2015 11:21 4 mins API The API was improperly used causing a high enough load on the system to briefly cause login and navigation to stall. We have since trained the end user on proper use and have plans in place to prevent it from occurring again.
03/20/2015 08:22 27 mins DNS Issue We experienced network connectivity and DNS resolution issues affecting some instances within a VPC in the US-WEST-2 Region in Amazon Web Services. Details at AWS STATUS SIREN. This affected saving records and sending emails. We made temporary updates on our end to mitigate connectivity issues.

 

TRACKVIA SUPPORT

Hours of availability:
7:00 A.M. to 6:00 P.M. MT Mon-Fri

Business-critical issues are monitored outside of business hours.
Email support@trackvia.com with urgent requests.

Average hours to first response:
Web apps industry average: 17.1 hours — Learn more

5.39

Same-day response rate:

98%

Average hours to close a ticket:

16.2

WE’RE HERE TO HELP

Connect with the TrackVia support team and other customers in our community—ask a question, suggest features and improvements, and browse how-to articles.

To submit a support ticket, click on the help icon in the top corner of your TrackVia application. 

  800-673-3302

  @TRACKVIASUPPORT

VISIT THE TRACKVIA COMMUNITY