SolarWinds CEO: Hackers Breached Our System Earlier than First Thought

The RSA Conference 2021 Virtual Experience is happening May 17-20 and Symantec, as a division of Broadcom, will be providing a summary of some of the leading stories from the conference to help you stay informed.

SolarWinds CEO Sudhakar Ramakrishna revised the timetable of a major breach of the company’s computer systems, suggesting that hackers believed to be working on behalf of the Russian government launched their attack against the company about half a year earlier than initially believed.

The narrative until now was that attackers first compromised SolarWinds systems in September of 2019 and remained undetected until December 2020.

“What we have found more recently is that the attackers may have been in an environment as early as January, 2019,” said Ramakrishna, who offered more details about the hack during an appearance at the RSA Conference 2021. “As we look back, they were doing very early recon activities in January of 2019, uh, which explains what they were able to do in September, October of 2019 as well.

“The tradecraft that the attackers used was extremely well done and extremely sophisticated, where they did everything possible to hide in plain sight, so to speak,” Ramakrishna added.

The damage is still being sorted out on what experts say rates as one of the biggest cyber attacks in American history. Hackers believed to be working on behalf of Russia penetrated SolarWinds' systems and then injected malicious code into the company's code.

That proved problematic because approximately 33,000 customers rely on SolarWinds' "Orion" system to manage their IT resources. As SolarWinds sent software updates to customers, the infected code created a backdoor that the attackers were able to use to spy or install more malware. SolarWinds’ customers include a variety of U.S. government agencies such as the Homeland Security Department and State Department as well as IT companies and non-governmental agencies throughout the world. 

“We were looking for all the usual clues,” according to Ramakrishna, who said that SolarWinds investigators examined “hundreds of terabytes of data and thousands of virtual build systems across the environment.”

“When you go through an investigation, you have a checklist. You have a set of hypotheses. You try to map things. And in this particular case, given the amount of time [the attackers] spent and given the deliberateness that they had in their efforts, they were able to cover their fingerprints, cover their tracks at every step of the way.”

He also said that SolarWinds found itself up against a nation-state, making it even more difficult to figure out what happened given the resource imbalance.

“It was a very difficult thing to uncover,” he said.

The disclosure may raise further concerns about how many organizations were actually breached and the full extent of the damage. Earlier this week, Rear Adm. William Chase, the deputy principal cyber advisor for the DOD, told the Senate Armed Services Cyber Security Subcommittee that 37 defense industrial base companies were attacked. SolarWinds critics have blamed the attack on its management, suggesting that it had shortchanged cyber security investment in its products in order to maximize profit and increase shareholder value. The company’s two primary owners are the private-equity firms Silver Lake and Thoma Bravo. 

Lessons Learned

Three days after his appointment as the company’s new CEO, Ramakrishna was celebrating his birthday with his family when the company’s chief legal officer called; FireEye was reporting the existence of a backdoor in SolarWinds’ Orion platform. The next day, he said, SolarWinds notified Orion customers about the compromise, urging them to upgrade immediately to a new software version that addressed the security vulnerability.

Since then, he said, the company is working “one customer at a time, essentially one day at a time” to repair the damage. 

The company has since put in place a program to assist customers that may not have the internal resources to upgrade or rebuild their systems by themselves.

“We felt it was our responsibility to help the customers get to a stable and secure environment,” he said. “This is going to be an ongoing thing as we move into the future.”

Ramakrishna also used the occasion of his public appearance at RSA to walk back earlier statements by the company that blamed an intern for the lapse in password security at SolarWinds during February testimony before Congress.

“So what happened at the congressional hearings where we attributed it to an intern was not appropriate and is not what we are about,” he said. “And so we have learned from that and I want to reset it here by saying that we are a very safe environment and we want to attract and retain the best talent.”

Rewinding the clock, Ramakrishna said SolarWinds was caught flat-footed after finding itself at the center of a media storm when the news broke.

“SolarWinds historically has kept to itself, focusing on customers, focusing on itself internally, and it was never trying to grab attention,” he said. “In this particular case, the attention was thrust upon us. And if I thought about one area where we were not fully prepared, unlike some companies that have armies of PR people just managing the message – and in many cases neutralizing it – we were not prepared.”