Security and Automation

I’ve recently been looking into some issues with a fleet of computers being infected with a ransomware threat. Fortunately, in this case, the Anti-Virus software on each computer was preventing the ransomware from actually encrypting files, but enough of it was active that the malware was spreading through the network, attempting to compromise more machines. It would only be a matter of time before it found a machine with malfunctioning (or missing!) Anti-Virus software.

This ransomware wasn’t new—it dates back to May, 2017—so it implied that perhaps something wasn’t right with the defences in place. Our investigation showed that these machines hadn’t been patched so as to be protected against the spreading part of this threat. Curious. The patch for this issue was released shortly after the ransomware was discovered, in 2017, so why are machines missing this patch in 2019?

There were many contributing factors behind this problem, but it turns out that the patching process for updating these machines was flawed. A component of the patching process required manual actions by an administrator at the locations where these computers were installed.

Any process relying on human action is prone to failure; unfortunately humans just aren’t that good at doing things without making mistakes. So why not automate this?

This story is more nuanced. We often apply automation to tasks which repeat often and need to be performed with accuracy. Indeed, in this process, a system was in place to automatically patch computers on the network. The administrator, having ‘approved’ a patch assumed that this automated scheme would distribute and apply the patches to the various computers on the network.

Unfortunately this automated patch distribution technology is flawed, and no process was in place to validate that it was in fact working as expected. As a consequence, we observed that patches weren’t always being distributed correctly, and as a result, this ransomware threat was starting to spread.

It’s important to realize that automation is not a silver bullet. Yes, it is far better to automate tasks which repeat often. Given a choice between automation and manual processing, automation is nearly always better. BUT (and it’s a big but) we must make sure that there is oversight of this automation. Technology will do what it’s designed to do, not necessarily what we think it should do—an important distinction.

In this particular case, the automation should be extended further to remove the human element from holding up the process. Instead, that human element could have been applied to monitor and verify the automation was working correctly. That’s not only a better use of resources, but it would have prevented this threat from expanding as it had.

Think about the processes you have in your organization, especially those which are maintaining the confidentiality, integrity and availability of your important information. Let me take an example: backups. Do you have to manually backup your data, or is it automated? If the former, consider automating it to reduce the chance you ‘forget’. And if the latter, then are those automated backups working correctly? You might want to check!