Troy Upah thought his team had done everything right.
As CEO of Ag Partners in Albert City, Iowa, Upah took cybersecurity seriously. He had worked with the co-op’s IT team to guard against a ransomware attack, in which cybercriminals infiltrate a company’s computer network, encrypt data and files to make them inaccessible and demand a ransom to restore access.
The team had trained co-op employees to recognize malicious links, limited access to networks, worked with a vendor to find security weaknesses (known as penetration testing) and used industry-leading virus protection software. “We thought we had sufficient firewalls in place,” says Upah.
So when the FBI called on a Monday morning in August 2019, Jeff Emery, the co-op’s information systems analyst, thought it was a scam.
He rushed out of the training session he was attending and soon learned the call was legitimate: The FBI had confiscated a server in California that listed ransomware targets, and Ag Partners (now known as AgState after a merger with First Cooperative Association) was on that list.
“We immediately started to change passwords and lock everything down,” Emery says.
He didn’t realize that cybercriminals were already inside the co-op’s network, waiting for the right moment to attack. The lockdown activity triggered the ransomware group to encrypt three main servers, including a backup server, making them inaccessible. Employees who tried to open a file on those servers instead saw instructions on how to pay a ransom to restore access.
How cyberattacks interrupt business
At that point, the IT team “basically pulled the plug on all external networks,” Emery says, to limit further damage. As they worked with their cybersecurity vendor to assess the damage, they discovered one piece of good news: The accounting system was on a separate server that wasn’t affected, so the criminal group hadn’t gained access to sensitive data about employees and business partners.
The company’s feed, agronomy and energy businesses also used separate servers that the criminals weren’t able to infiltrate.
Despite that bright spot, being disconnected from networks caused a huge loss in efficiency and productivity, Upah says. For several weeks, teams had to use faxes and texts to communicate with customers, suppliers and each other. “You never realize how much you rely on email until it’s gone,” he says. “It made managing a $500 million company a lot more difficult.”
The IT team, meanwhile, was working long hours to field hundreds of help desk calls. “There were weeks in a row when our team was getting very few hours of sleep,” recalls Emery.
How do cyberattacks happen?
Like many co-ops, Ag Partners had gone through a series of mergers and acquisitions in the years leading up to the attack. By 2019, the business had grown to include 17 facilities and about 250 employees. One thing it didn’t intend to acquire was a compromised server.
“We found out that one of the facilities we’d recently acquired had a ransomware attack before, unbeknownst to us,” says Upah. Though there’s no way to be sure, he and Emery believe the web-facing server wasn’t thoroughly cleaned up before the acquisition, leaving an open door for criminals as soon as the server was connected to the co-op’s network.
Emery did have security in mind with the new systems and had scheduled penetration testing for several weeks after the acquisition – but that turned out to be too late. “We learned the hard way not to allow any systems on the network without testing them first,” he says.
New security measures following a cyberattack
Upah and Emery say there’s still a lot they don’t know. They never clicked on the link or paid the ransom, so they don’t know who the cybercriminals were or how much money they were demanding. The FBI and cybersecurity consultant told them the attack was likely random – the criminals scanned for security vulnerabilities, then pounced when they found one. “They picked the low-hanging fruit,” says Emery.
Since then, the co-op has significantly beefed up data security, with new security software, hardware and monitoring technology. Emery’s team rebuilt every server from the ground up, cleaned up laptops at all their locations and required all employees to change usernames and passwords and use dual factor authentication. They do constant penetration testing to look for vulnerabilities and have contracted with a 24/7 security monitoring service.
They’ve recovered about 70% of the affected files and data, says Emery, but they’re still rebuilding spreadsheets and reconstructing historical data.
His advice to other co-ops who may be feeling overwhelmed by the complexity and cost of cybersecurity measures: Start now.
“Prioritize the big items and start budgeting for the rest of it,” he says. “You don’t have to do it all at once, but you’re never going to get there unless you start planning.”
Related stories:
