IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

- 8 mins read

Issue

When you access another device via SSH, there is always the possibility of falling victim to an On-Path (Man-in-the-Middle) attack. For example, you could accidentally connect to an attacker’s server, and they’d capture your credentials when you tried to authenticate.

We use PKI in the form of host keys to counter that vulnerability. When you SSH into a web server for the first time, you should see a message similar to this:

PuTTY Example

PuTTY GUI Example

Unix Example

Unix Shell Example

Windows PowerShell Example

PowerShell Example

I’m using Windows PowerShell, so the first time I connect to a web server, I see a message similar to this:

“The authenticity of host ‘[FQDN:Port] [IP Address:Port]’ can’t be established. RSA key fingerprint is [SHA256 Hash] This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])?”

How the public host key fingerprint is supposed to work

It took me a while to get that the RSA key fingerprint referenced in this warning message is NOT the same thing as an SSH key pair that you might create to sign into a server, as an alternative to username/password authentication.

Instead, this fingerprint is the SHA-256 hash of the host’s public key, which is established to identify the device (not to identify a user). Often that key is located at /etc/ssh/ssh_host_rsa_key.pub in the server’s directory.

If you manage that server, and have full access to it, you can check that file directly to see if the public key hash matches up. Even if you don’t personally have access, you can ask the server’s administrator to verify it. And attackers - with no access to the private key - can’t forge the public key.

Once you verify the host key, you can choose to trust it in the future so you don’t have to check every time. In a Windows environment, that means allowing it to be added to the known_hosts file, located at %USERPROFILE%\.ssh.

Unfortunately, this whole process gets more complicated in a shared hosting situation. If you only pay your hosting provider for a plan that lets you rent a slice of a web server, and they allow for remote “locked” SSH access into that slice, you still have to either verify the host key, or just cross your fingers and hope for the best.

Fingers Cross Warning Sign

SSH Host Keys in Shared Hosting Environment

You won’t be able to check the ssh_host_rsa_key.pub directly; if you could navigate out of your shared slice of the web server, you’d have access to everyone else’s slices, and root access. And, in most cases, you won’t be on a first-name basis with a friendly, neighborhood sysadmin who manages the server.

You could try using some alternative SSH commands to gather more information. Use ssh-keyscan and the port number to save the public key to a file. The port might be 22, or it might be a nonstandard port assigned by your particular hosting provider.

ssh-keyscan -p [Port Number] FQDN > temp_key

Then, use ssh-keygen to get the SHA-256 hash of the RSA key

ssh-keygen -lf temp_key

But that is just another way of displaying the same information the initial SSH command showed you. Which MIGHT just be the host key of the MiTM attacker’s server.

This would be the equivalent of a bartender checking that a teenager’s ID isn’t fake by also asking them if they are over 21 (which, by virtue of being ‘something they have’ and ‘something they know’, might technically qualify as MFA?).

Or, like deciding that the website “https://steal-my-data.shady” is trustworthy because it has an SSL certificate…which proves the site admins own the site. AND guarantees that any PII you enter into their web form will be transmitted to their database via secure ciphertext.

Solutions

Option 1: 🤷‍♂️

Option 2: Contact your hosting provider

Milage varies. I checked a handful of the larger hosting companies, and most of their SSH guides say something similar to “If you receive a message about the key fingerprint, type yes and forget about it. There is no war in Ba Sing Se.”

Search Terms: [hosting provider] verify public host key ssh

GoDaddy - Results: 🦗🦗🦗

HostGator - Results: “A ‘confirmation prompt to proceed’ [Italics and bold and quotation marks mine] will appear. Type in yes and hit Enter.”

Bluehost - Results: “Verify the domain name is correct.”

DreamHost - Results: A+ for nice documentation, F- for specifying how to verify the host key fingerprint

Namecheap - Results: “You will receive the Putty Security Alert. Press OK:”

So, just going by the hosting provider documentation I was able to find, I’d have to conclude that 1) Most hosting companies offer SSH access, even for shared hosting plans, and 2) Most advise a YOLO-based approach to potential MiTM attacks for clients that want to actually take them up on said SSH access.

I have experience with Namecheap, and I’ve had to convince them to verify a shared server’s public host key. At first, support told me that was impossible (🤨), that verifying the public key wasn’t secure (🥹), and that the internal policy was to keep the public key a secret (🤐). Then, later, after pestering support for a sufficient amount of time and in stages of mounting indignation, they relented.

Bonus: “Someone could be eavesdropping on you right now”

The first time I tackled this problem, I managed to wrangle the public key fingerprint from my hosting provider. I compared it to the hash displayed when I connected to the server, made sure it matched, and added it to my known_hosts file.

3 weeks later, the server’s OS was upgraded, and the host key changed. Maybe SSH had to be reinstalled, or the keys were rotated. So, SSH’ing back in displayed this rather disconcerting message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
[SHA256 Fingerprint]
Please contact your system administrator.
Add correct host key in [%UserProfile%/.ssh/known_hosts] to get rid of this message.

That warning does come across as a little over the top (right up to the begrudging caveat “…It is also p o s s i b l e…”). But it makes sense to me in context: I requested strict checking, and specifically added the (previous) verified fingerprint to my known_hosts file. If a MiTM attack was actually happening, this is exactly what it would look like.

This time around, after investing several hours over several days in a grueling back-and-forth with Namecheap’s helpdesk, they flatly refused to verify the host key fingerprint under any circumstances, and denied having ever verified it in the past (😿).

Questions and Takeaways

Illustration of Security and Convenience on a scale

Is there actually a valid reason for a shared hosting provider to simultaneously offer locked SSH access, but to withhold confirmation of the host key hash? Legal liability of some kind? Operational overhead?

If not, then I’d assume Namecheap’s reluctance goes back to the classic dilemma of Security vs. Convenience. I’d imagine it is a pain to tack that task onto a helpdesk already stretched thin by high-priority inquiries into “Why is my website down just because I cancelled my hosting package?”

To Be Fair…

I went down the rabbit hole on this topic as a learning exercise. But there probably aren’t that many IT professionals trying to maintain good OPSEC with SSH access into a cheapo shared hosting server running a tiny, static blog. It is, admittedly, a rare use case.

Namecheap recommended (ad nauseam) buying a VPS or dedicated package so I could maintain my own damn host key. And while I find that upsell to be irritating, and a way of dodging responsibility, it is true that most professional web developers wouldn’t run into this issue in the first place. And most dabblers with a beginner hosting package won’t try to use SSH.

All the same, IMO, none of these hosting providers should be offering SSH access on shared hosting plans while simultaneously withholding the means to utilize it securely. This oversight feels familiar.

It reminds me of SPF/DKIM/DMARC, and the weary sysadmins scrapping it all because the staff complain if 1 out of 1000 outbound emails may or may not have gone to the recipient’s spam folder.

Or, of competent and intelligent developers stuffing their apps with dependencies that they trust on the basis of 1) Everyone else uses them 2) This particular file convertor hasn’t ransomwared anyone, yet, and 3) The GitHub repo has lots of ✨✨✨

Sources