The unnamed water district had asked Verizon to assess its networks for indications of a security breach. It said there was no evidence of unauthorized access, and the assessment was a proactive measure as part of ongoing efforts to keep its systems and networks healthy.
Verizon examined the company’s IT systems, which supported end users and corporate functions, as well as Operational Technology (OT) systems, which were behind the distribution, control and metering of the regional water supply.
The assessment found several high-risk vulnerabilities on the Internet-facing perimeter and said that the OT end relied heavily on antiquated computer systems running operating systems from 10 or more years ago.
Many critical IT and OT functions ran on a single IBM AS/400 system which the company described as its SCADA (Supervisory Control and Data Acquisition) platform. This system ran the water district’s valve and flow control application that was responsible for manipulating hundreds of programmable logic controllers (PLCs), and housed customer and billing information, as well as the company’s financials.
Interviews with the IT network team uncovered concerns surrounding recent suspicious cyber activity and it emerged that an unexplained pattern of valve and duct movements had occurred over the previous 60 days. These movements consisted of manipulating the PLCs that managed the amount of chemicals used to treat the water to make it safe to drink, as well as affecting the water flow rate, causing disruptions with water distribution, Verizon reported.
An analysis of the company’s internet traffic showed that some IP addresses previously linked to hacktivist attacks had connected to its online payment application.
Verizon said that it “found a high probability that any unauthorized access on the payment application would also expose sensitive information housed on the AS/400 system.” The investigation later showed that the hackers had exploited an easily identified vulnerability in the payment application, leading to the compromise of customer data. No evidence of fraudulent activity on the stolen accounts could be confirmed.
However, customer information was not the full extent of the breach. The investigation revealed that, using the same credentials found on the payment app webserver, the hackers were able to interface with the water district’s valve and flow control application, also running on the AS/400 system.
During these connections, they managed to manipulate the system to alter the amount of chemicals that went into the water supply and thus interfere with water treatment and production so that the recovery time to replenish water supplies increased. Thanks to alerts, the company was able to quickly identify and reverse the chemical and flow changes, largely minimizing the impact on customers. No clear motive for the attack was found, Verizon noted.
The company has since taken remediation measures to protect its systems.
In its concluding remarks on the incident, Verizon said: “Many issues like outdated systems and missing patches contributed to the data breach — the lack of isolation of critical assets, weak authentication mechanisms and unsafe practices of protecting passwords also enabled the threat actors to gain far more access than should have been possible.”
Acknowledging that the company’s alert functionality played a key role in detecting the chemical and flow changes, Verizon said that implementation of a “layered defense-in-depth strategy” could have detected the attack earlier, limiting its success or preventing it altogether.
About the attack [UPDATED]
A “hacktivist” group with ties to Syria compromised Kemuri Water Company’s computers after exploiting unpatched web vulnerabilities in its internet-facing customer payment portal, it is reported.
The hack – which involved SQL injection and phishing – exposed KWC’s ageing AS/400-based operational control system because login credentials for the AS/400 were stored on the front-end web server. This system, which was connected to the internet, managed programmable logic controllers (PLCs) that regulated valves and ducts that controlled the flow of water and chemicals used to treat it through the system. Many critical IT and operational technology functions ran on a single AS400 system, a team of computer forensic experts from Verizon subsequently concluded.
Our endpoint forensic analysis revealed a linkage with the recent pattern of unauthorised crossover. Using the same credentials found on the payment app webserver, the threat actors were able to interface with the water district’s valve and flow control application, also running on the AS400 system. We also discovered four separate connections over a 60-day period, leading right up to our assessment.During these connections, the threat actors modified application settings with little apparent knowledge of how the flow control system worked. In at least two instances, they managed to manipulate the system to alter the amount of chemicals that went into the water supply and thus handicap water treatment and production capabilities so that the recovery time to replenish water supplies increased. Fortunately, based on alert functionality, KWC was able to quickly identify and reverse the chemical and flow changes, largely minimising the impact on customers. No clear motive for the attack was found.
Verizon’s RISK Team uncovered evidence that the hacktivists had manipulated the valves controlling the flow of chemicals twice – though fortunately to no particular effect. It seems the activists lacked either the knowledge of SCADA systems or the intent to do any harm.
The same hack also resulted in the exposure of personal information of the utility’s 2.5 million customers. There’s no evidence that this has been monetized or used to commit fraud.
Nonetheless, the whole incident highlights the weaknesses in securing critical infrastructure systems, which often rely on ageing or hopelessly insecure setups.
Monzy Merza, Splunk’s director of cyber research and chief security evangelist, commented: “Dedicated and opportunistic attackers will continue to exploit low-hanging fruit present in outdated or unpatched systems. We continue to see infrastructure systems being targeted because they are generally under-resourced or believed to be out of band or not connected to the internet.”
“Beyond the clear need to invest in intrusion detection, prevention, patch management and analytics-driven security measures, this breach underscores the importance of actionable intelligence. Reports like Verizon’s are important sources of insight. Organisations must leverage this information to collectively raise the bar in security to better detect, prevent and respond to advanced attacks. Working collectively is our best route to getting ahead of attackers,” he added.
Reports that hackers have breached water treatment plants are rare but not unprecedented. For example, computer screenshots posted online back in November 2011 purported to show the user interface used to monitor and control equipment at the Water and Sewer Department for the City of South Houston, Texas by hackers who claimed to have pwned its systems. The claim followed attempts by the US Department of Homeland Security to dismiss a separate water utility hack claim days earlier.
More recently hackers caused “serious damage” after breaching a German steel mill and wrecking one of its blast furnaces, according to a German government agency. Hackers got into production systems after tricking victims with spear phishing emails, said the agency.
Spear phishing also seems to have played a role in attacks lining the BlackEnergy malware against power utilities in the Ukraine and other targets last December. The malware was used to steal user credentials as part of a complex attack that resulted in power outages that ultimately left more than 200,000 people temporarily without power on 23 December.
Credit: watertechonline, theregister