FORT DRUM, NY (November 4, 2024) — Public Works Department cybersecurity, information technology and water system operations professionals combined their technical expertise to work through a “what if” crisis scenario if” Oct. 24 at Fort Drum.

Facilitated by representatives of the Department of Homeland Security’s Critical Infrastructure Security Agency, the mass exercise tested what that team would do if the facility’s water supply were compromised — first in the hours following the attack and then in the days that followed until the crisis is resolved.

“The purpose of the exercise was to assess the Garrison’s ability to respond and recover from a cyber incident affecting the water telemetry control system,” said Pete Owen, CISA security advisor. “Our goals would be to identify any planning gaps, such as reporting requirements and service level agreements, and we want to identify capacity gaps in resources.”

Owen said the exercise was developed as a “free discussion” where they could inject details, such as the disclosure of an insider threat, while keeping the script authentic.

Tom Hudon, PW chief water operator, said the exercise helped identify where they could make improvements in communication and documentation.

“Our priority will always be providing clean and safe water to the community,” he said.

Hudon said the automatic water control system was built where daily checks can be done in a short period of time, with minimal manpower requirements and with redundancies in mind.

“If we had to shut down the system, we could do everything manually for a long time with just the people we have and there would be no disruption to service,” he said.

Robert Clements, Fort Drum information management officer, and April Eddy, Fort Drum information system security officer, coordinated the exercise for PW team members. A real incident response would require coordination and input from multiple agencies and organizations, but Clements said the mass exercise focused on the technical aspect of an emergency response.

“We wanted to bring the technical team together to walk the process through each member’s roles and how work flows between those roles,” he said. “What Tom Hudon’s team does to protect the water system itself is different than what system administrators and information system security officers do to protect the network. Putting it all together helps break down those silos between processes and simplifies our ability to respond to any type of IT incident.”

From what Eddy observed, she said the team was able to gather some lessons learned from the exercise, and now they can incorporate them into the response plan.

“This exercise takes all of their experience so it’s not just one person with all the knowledge,” she said. “Even though it didn’t always go well, I’d say it was a win because we identified some weaknesses that we can fix.”

Eddy said the Department of Emergency Services and the Department of Family and Morale, Welfare and Recreation recently had information technology assessments and that PW would need another for heating, ventilation and air conditioning (HVAC).

“There is a risk management framework requirement for each system to review the continuity of operations plan (COOP),” she said. “Every system within an organization – whether it’s CCTV, HVAC or water – has a COOP so that if the system crashes or something compromises it, then you have a plan to get it back up and running again.”

The 2023 Black Start exercise tested the facility’s ability to recover from a massive power outage. Not only were the lights turned off, but the automatic systems were also temporarily disabled. Hudon received the home systems alert and was able to get the water telemetry control system back to normal. People might have been disturbed by the lack of lights, but there were no interruptions in the water supply.

“Information technology literally touches everything and every organization on the job, whether people realize it or not,” Eddy said. “It needs to be protected, and part of that protection is a system recovery plan in the event of an outage or failure.”