Active Incident

Updated a few seconds ago

Incident Status

Degraded Performance

Components

Web Hosting

Services

Plesk, WAF - Web Application Firewall



April 24, 2024 8:47AM EDT
[Monitoring] OIT is continuing to monitor the situation, and there have been no significant updates yet. Support teams are still working with the vendor to get a more permanent fix to the issue.

April 16, 2024 3:35PM EDT
[Monitoring] While the workaround to address the WAF issue on the the Plesk Webhosting complex on 4/5 did restore service, it unfortunately also is causing an IP forwarding issue which is affecting the functionality of some of the sites. Support is continuing to work with the vendor to restore the WAF to service so that the webhosting complex can be brought back to its original state.

April 12, 2024 2:51PM EDT
[Monitoring] The workaround that was put in to restore connectivity to plesk hosted websites continues to be working. However, a permanent solution to the underlying causing is still being investigated and worked on with the vendor. Once a working solution is found, tested, and implemented, this issue will be cleared . Until then, we will continue to run off the workaround, and support is continuously monitoring the situation for any new issues that could spring up from the workaround.

April 6, 2024 12:34AM EDT
[Monitoring] The workaround was successful, and sites should be accessible once again. Support is continuing to work with the vendor to address the underlying issue for a more permanent fix.

April 5, 2024 11:33PM EDT
[Identified] Support is continuing to work with the vendor to try and fix the issue with the WAF. However, network team is currently getting ready to implement a workaround to restore connectivity to the affected sites as a temporary measure until a fix for the WAF issue can be found. Once the workaround is implemented, approximately 30 minutes of down time on the hosted sites is expected.

April 5, 2024 1:51PM EDT
[Identified] Staff are still working with the vendor to resolve the WAF issue. If students need to access Canvas, they can reach it by going to https://gatech.instructure.com/

April 5, 2024 11:06AM EDT
[Identified] The vendor's patch was unsuccessful in resolving the issue and will rollback their patch. Efforts are being made to prevent any downtime, but it is possible that there may be a brief outage of up to 10 minutes. Support is continuing to work with the vendor to troubleshoot the problem.

April 5, 2024 9:23AM EDT
[Identified] The vendor is applying a patch to address the communications issues with the Web Application Firewall. Efforts are being made to prevent any downtime, but it is possible that there may be a brief outage of up to 10 minutes.

April 5, 2024 8:33AM EDT
[Identified] Staff have discovered a communications issue between the Plesk hosts and the WAF (web application firewall) that is causing sites hosted on Plesk to be slow or unreachable. Staff are continuing to work with the vendor to resolve this issue. Sites affected include but are not limited to: buzzport.gatech.edu canvas.gatech.edu crc.gatech.edu ctl.gatech.edu dining.gatech.edu diversity.gatech.edu finance.gatech.edu honors.gatech.edu hosting.gatech.edu housing.gatech.edu ipat.gatech.edu

April 5, 2024 8:00AM EDT
[Investigating] Several sites hosted on the webplesk are unreachable due to issues with the WAF (web application firewall) including but not limited to : buzzport.gatech.edu canvas.gatech.edu crc.gatech.edu ctl.gatech.edu dining.gatech.edu diversity,gatech.edu housing.gatech.edu ipat.gatech.edu Staff are continuing to investigate with the vendor to resolve this issue.

April 4, 2024 4:58PM EDT
[Investigating] Several sites hosted on the webplesk are unreachable due to issues with the WAF (web application firewall) including but limited to : canvas.gatech.edu crc.gatech.edu ctl.gatech.edu dining.gatech.edu diversity,gatech.edu housing.gatech.edu ipat.gatech.edu

April 4, 2024 2:29PM EDT
[Monitoring] Service is currently restored and support is monitoring the situation. If you are continuing to experience issues, or if you have questions or concerns, please file a ticket via services.gatech.edu or contact the Administrative Services Center at 404/385-1111.

April 4, 2024 11:50AM EDT
[Identified] Related to the previous issue, web-plesk53 is now serving incorrect SSL certificates to sites hosted on the server. Support is working to resolve the issue.

April 4, 2024 9:49AM EDT
[Monitoring] Several sites hosted off of web-plesk53 were unreachable due to issues with the WAF (web application firewall) including: -Housing.gatech.edu, textbooks.math.gatech.edu, -undergradresearch.gatech.edu, -faset.gatech.edu, -transitionprograms.gatech.edu, -policylibrary.gatech.edu, -policies.gatech.edu, -alan.ece.gatech.edu -gtcad.gatech.edu -hasler.ece.gatech.edu Service is currently restored and support is monitoring the situation.

Academic Services

Operational

Campus Services

Operational

Campus Services - ITG

Operational

Campus Audio Visual Services

Operational

Email and Calendaring

Operational

Identity

Operational

Network

Operational

Web Hosting

Operational

Enterprise Data Services

Operational

IT Service Management

Operational

VideoConferencing

Operational

GTPE

Operational

Cloud Storage

Operational

External Services

Scheduled Maintenance

Schedule

May 7, 2024 6:00AM - May 10, 2024 11:59PM EDT

Components

Academic Services

Services

PACE

Description

ITEMS REQUIRING USER ACTION: [all] During the maintenance period, the PACE team will rename all POSIX user groups so that names will start with the “pace-” prefix. This will NOT affect numerical GIDs, but if your scripts or workflows rely on group names, they will need to be updated. If you don’t use POSIX user group names in your scripts or workflows, no action is required on your part. This is a step towards tighter integration of PACE systems with central IAM tools, which will lead to improvements across the board in the PACE user experience. NOTE: This item was originally planned for January, but was delayed to avoid integration issues with IAM services, which have now been resolved. [ICE] Migrate to the RHEL 9.3 operating system – if you need access to ICE for any summer courses, please let us know! Note – This WILL create new ssh host-keys on ICE login nodes – so please expect a message that “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!” when you (or your students) next access ICE after maintenance. ITEMS NOT REQUIRING USER ACTION: [datacenter] Databank maintenance: replace all components of cold loop water pump that had issues a couple of maintenance periods ago. [Hive] Upgrade the underlying GPFS filesystem to version 5.1 in preparation for RHEL9 [datacenter] Repairs to one InfiniBand switch and two DDN storage controllers with degraded BBUs (Battery Backup Unit) [datacenter] Upgrade storage controller firmware for DDN appliances to SFA 12.4.0. WHO IS AFFECTED? All users across all PACE clusters. WHO SHOULD YOU CONTACT FOR QUESTIONS? Please contact PACE at pace-support@oit.gatech.edu with questions or concerns.
Welcome to Georgia Tech's IT Service Status Page Don't see your issue posted here? Let us know! Administrative Services Center Email: support@oit.gatech.edu Phone: 404-385-1111 Location: Atlanta Campus, Clough Building Room 215