How to Connect to Remote Access & Beyond

Introduction

The problem that we’re seeking to solve is simply:

How do we provide access to our servers in a way that only authorized people have access and have it easily and reliably – i.e. when an IP address changes, authorized people don’t need to worry, admin staff don’t need to change anything and the system is still secure?

The solution is to use a bastion host as a SSH gateway to the devices we manage.

Basic access through gateway
Basic Access through Gateway Instance

By using a Virtual Private Cloud (VPC) on AWS we can access instances which are not exposed to the Internet on port 22 (Secure Shell) or any other port listening for SSH connections.  We still have to secure our Gateway Instance, but it becomes a bastion server with the sole purpose of providing SSH access to the other resources.

Protecting Keys

The bastion host is accessible and open to the Internet on port 22.  Fail2Ban is running on this device to minimize the potential for DDOS attacks.  The Security Group for this Instance allows only port 22 traffic.  But in order to connect securely to the other hosts on the private subnet of the VPC we need to still use certificates and we don’t want to store our certificates on the bastion host (from hereon we’ll refer to this gateway device by it’s DNS name: remoteaccess.counselkit.com or simply remoteaccess).

SSH is able to perform key forwarding if ssh-agent is installed locally on the client computer.  If the developer or system administrator is accessing remoteaccess via an Apple OS X device, ssh-agent is already installed.  Adding the key to ssh-agent is done via the command:

ssh-add -K myPrivateKey.pem
Enter passphrase for myPrivateKey.pem:
Passphrase stored in keychain: myPrivateKey.pem
Identity added: myPrivateKey.pem (myPrivateKey.pem)

You can list the keys that are included in your ssh-agent keychain by passing the -L argument:

ssh-add –L

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDHEXAMPLErl25NOrbhgIGQzyO+TYyqbbYEueiELcXtOQH
gEFpMAb1Nb8SSnlxMxiCXwTKd5/lVnmgcbDwBpe7ayQ6idzjHfvoxPsFrI3QSJVQgyNcx0RylX9IjcvJOyw
== myPrivateKey.pem

If you’re using a Windows workstation, PuTTY has ssh-agent functionality integrated.  A good explanation of using PuTTY and OS X with SSH key forwarding is here.

Connecting

Once you’ve added your key the SSH command requires only that you ssh to each successive instance:

kevinritchey ~ $ ssh -A kevinr@remoteaccess.counselkit.com
Last login: Sun Feb 15 21:38:36 2015 from 50-192-9-137-static.hfc.comcastbusiness.net

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2014.09-release-notes/
[kevinr@ip-172-30-3-194 ~]$ ssh -A kevinr@www.lincolnlaw.com
Last login: Sun Feb 15 21:42:33 2015 from ec2-54-152-199-138.compute-1.amazonaws.com

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2014.09-release-notes/
[kevinr@production ~]$

Note that I was able to connect from remoteaccess even though my private key is not in the ~/.ssh/ directory:

kevinritchey ~ $ ssh -A kevinr@remoteaccess.counselkit.com
Last login: Mon Feb 16 19:21:47 2015 from 101.sub-70-192-145.myvzw.com

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2014.09-release-notes/
[kevinr@ip-172-30-3-194 ~]$ ls .ssh/
authorized_keys  known_hosts
[kevinr@ip-172-30-3-194 ~]$ ssh -A kevinr@www.lincolnlaw.com
Last login: Mon Feb 16 19:11:31 2015 from ec2-54-152-199-138.compute-1.amazonaws.com

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2014.09-release-notes/
[kevinr@production ~]$

In this way we’re able to use SSH keys without compromising our private keys.

 

 

Ransomware Mitigation Steps

Do you have regular and versioned off-site backups on disconnected systems which rely upon third-party tools with inaccessible credentials?  If the answer to that question is no – please read on.

Much has been written in the past 12 months to raise the level of anxiety regarding so-called ransomware.  This new type of malware works by encrypting files with a key that is held on a command-and-control server.  After the files are encrypted – with a very good encryption algorithm, the user is notified and given a limited amount of time to either pay a ransom or lose access to their files forever due to the deletion of the decryption key on the command-and-control server.

This is scary but it gets worse.  Many IT service providers have incorrectly assumed that having a good backup is the best step to mitigate the damages caused by ransomware.  Some solution providers have incorrectly assumed that using measures such as a very good anti-virus program, a very good Unified Threat Management system or a very good DNS scanning tool can be used alone or in combination to thwart the criminals behind the ransomware schemes.  But each of these assumptions may leave the end user in a precarious position.

Data is the target of these threats.  In the wake of Suxnet we should anticipate that malware can and will evolve to anticipate threats.  DNS mitigation for example assumes that the malware component will need to call home using DNS mapped command-and-control servers.  But there are clever ways to avoid this including accessing IP addresses via whitespace text hidden in compromised but legitimate web sites; using P2P networks; temporary and short-lived DNS names generated by algorithm; and Tor/Onion routers.  All of these ways would defeat a DNS only approach where the IT service provider assumes that because the DNS addresses of C&C servers are redirected the network and hosts are protected because the malware can’t call home to get an encryption key or store an encryption key and therefore won’t start encrypting files.

Like DNS, anti-virus tools largely depend upon known intelligence – file signatures and known file activity.  In the case of prior ransomware tools the data directories are known, the file signatures are known and these are included in nightly updates.  Yet we’ve seen that anti-virus tools can suffer from the fate of too-late-to-the-party malware that simply defeats the anti-virus tool by shutting it down, hiding itself from the anti-virus tool or disguising the anti-virus tool altogether and preventing the user from knowing they are infected.  This is a good step to take – but insufficient to provide the best mitigation against ransomware.

Other methods, including anti-malware and Unified Threat Management tools suffer the same weaknesses and will always have these weaknesses.  There is no silver bullet for defending against a ransomware attack.  Why?  Because there is a huge amount of money to be made in ransomware.  Users pay the ransom in an alarming percentage of cases.  Ransomware authors are clever and there is a market for newer and better ransomware.  We are seeing the age of innocent disappear on the Internet.

But there are proper mitigation steps to be taken.

1. Proprietary Versioned Off-site Disconnected Backups
2. Anti-virus
3. Anti-malware
4. DNS screening tools such as OpenDNS
5. IDS/UTM devices such as a Fortigate UTM device
6. Diligent file management procedures

The first step is a proprietary and versioned off-site disconnected backup.  What this means is that the system isn’t connected to the backup store all the time.  This alone reduced the likelihood of infection.  Using a proprietary solution means that not just any program can access the file store either – e.g. using a program that requires credentials to access the file store (e.g. S3) and which stores those credentials securely means that the malware cannot piggy-back on.  The most important is, however, a versioned backup.  If – and more likely when – a file becomes encrypted, the IT service provider should be the knite in white armor who shows up to offer last night’s version – or last Tuesday’s version for that matter, with no encryption.  Imagine the accolades.  Imagine the good will.

Every step counts.  Cut off the infection by preventing drive-by-downloads with premium anti-virus tools such as Kaspersky, Bitdefender or Panda.  Cut off malware behavior by using heuristics and tools that watch suspicious system activity.  Cut off communication to the C&C servers.  Watch for suspicious traffic with UTM devices.  But most importantly make sure that any damage is limited.  Disk space is cheap.  It’s cheaper than the ransom, it’s cheaper than the other prevention tools and it’s the one thing that can save the day when all else fails.  Versioned backups made with proprietary tools which access off-site and disconnected storage is indispensable in the fight against data-encrypting malware.

ZenPan recommends using an IT service provider who is reluctant to rely on a single tool or method – and one who is aware of the weaknesses of the threat.  Ransomware’s achille’s heel, for now, is it’s need to access the file directly, either through a call over CIFs/SMB or via FAT/NTFS.  It will try to encrypt network shares first, then local files.  Stopping the threat before encryption is ideal.  But like Sun Tzu plans for victory before engaging in the battle, the IT service provider needs to plan for the recovery of data before allowing their client’s data to be exposed to new and unforeseen threats.