What We Know of the Attack

One of the most disheartening things to learn about in the cyber security world is when threat actors take the liberty to use their talents to harm someone physically.  Don’t get me wrong, it’s not acceptable to do harm to disrupt lives and services, but it takes a special kind of person to want to do bodily harm to others.

Last week it was reported that a Florida metropolitan water plant was a victim of a cyber attack.  The threat actor reportedly accessed the water plant’s supervisory control and data acquisition or SCADA system to manipulate equipment that regulates the distribution of chemicals.

According to Pinellas County Sheriff Bob Gualtieri, the plant operator at the Oldsmar’s water treatment plant discovered anomalies in a computer system that was acting irregularly.  According to the plant operator, the threat actor used a popular remote monitoring software tool, typically used by IT admins and professionals to remotely monitor and access computer systems.  

The uninvited guest accessed the system at least twice that day, but the operator noticed on the second unauthorized access later that afternoon, that the user was able to increase the levels of sodium hydroxide commonly known as Lye into the water supply.  

Lye is a primary chemical used in liquid drain cleaners.  In small quantities to water systems, it is used to reduce the acidity in water and has other practical applications.  According to the Center of Disease Control (CDC), in high concentrations, Lye can cause irritation to skin and eyes as well as cause temporary hair loss.  It is recommended that if swallowed, individuals should seek immediate medical attention.

Despite how terrible an incident this could have become, it is reassuring to know that the plant had redundancy checks in place.  According to Sheriff Gualtieri, it would have taken nearly 24-hours for the increased amount of Lye to hit the water supply system and even at that point, there are testing and alerting systems in place near the end of the pipe that would allow for an emergency shutdown.  I shudder to think the outcome if those controls were bypassed and the operator had not seen the intrusion in real-time.

The Psychology of the Attack

It can’t be helped that my mind automatically goes to this being an insider attack.  The amount of time reportedly it took the threat actor to remotely log in and directly go into the right application used to distribute chemicals was a matter of minutes. I’m not familiar with the system, but to know which chemicals to elevate suggests a level of knowledge and familiarity with that particular plant’s infrastructure that would be akin to an employee or vendor familiar with the environment.   Simply put, it’s one thing to press random buttons and it’s another thing to press the right combination of buttons. 

Of course, it cannot be dismissed that this could be an outside threat as we do not know how long the reconnaissance was on Oldsmar.  It could certainly explain the familiarity with the systems.  It’s also plausible that a vendor could have been compromised and that their credentials could have been used as the initial attack vector but the method of attack rings hollow for anything more than this being an insider.  Why?  

Most technical savvy individuals know that remote software solutions like the one used, are extremely noisy and attribution is typically important to threat actors including nation-state. Among gaining access to a target’s machine, it is equally important for them to disguise their methods.  This actor was very boldent in their method. They logged in the earlier morning, quickly logged off then logged on approximately 5 hours later all visible by the plant’s employee.  If they wanted to gain access to a city of 15K people, there are less conspicuous options and methods to so.

Addressing The Flaws

Utility companies should be treated like banks.  It’s not enough to secure the front door and the physical vault, but you must take precautionary steps to secure the networks especially for systems that have the ability to cause significant bodily injury if malfunctioned and/or can be accessed through the internet.  Here are a few tips to better secure these assets:

• Users/Passwords
  • For all systems including remote access tools, each user should have their own username and password.  Passwords rules should be enforced and include lengths of 8 characters or more, uppercase, lowercase, special and/or numeric characters.
  • Passwords should expire automatically at minimum, once every 180 days.  For critical systems, it may be more appropriate to use 90-days.
  • Inactive accounts or accounts that have not been used in 60 days, should automatically be disabled.
  • Terminated employees or vendors access should be disabled immediately.
  • It may be tempting to have one password to rule them all for the sake of ease of use; however, this makes it easier for disgruntled or skilled adversaries to retaliate or cause harm the company.
• VPN with 2-Factor authentication (2FA)
  • We discussed the importance of 2FA in our previous post, but it cannot be stressed enough that if anyone is accessing a company network from the internet, a VPN should be in place with 2FA enabled.  The VPN will help ensure the connection is secure and can log a user for accountability purposes.
  • 2FA will also provide another layer of security in the event that the password is compromised.
• Intrusion Detection and Prevention Systems
  • Commonly known as IDS and IPS systems, these can be fine tuned to identify and notify administrators of anomalies for incoming and outgoing connections.  If a user is logging into a system at odd hours of the day/night or multiple connections (excessive logins) are made in a certain amount of time, action can be taken.
• Enabling Logging on Centralized Off-Site Server
  • One of the most common things overlooked when implementing security controls is logging.  Logging is the ability to track and provide attribution for users.  Coupled with a Security Information and Event Management (SIEM) tool can give an early advantage on unknown and suspicious activity on the network.
  • Having a seperate off-site server that pulls in all of the logging information will help prevent threat actors from covering their tracks and provide integrity to the logs.

The above suggested controls are not an end-all be-all, but should be considered the bare minimum technical controls put into place to help deter intrusion into critical infrastructures like the public utilities.

The Aftermath

As information develops about this event, I will update this section.