What a weekend.

I’m in the process of migrating servers, I’m moving from one hosting company to another.  I have a number of different sites, one is still on the old cloud-based server, while this one, among others are hosted on our new server.  I’m still doing a lot of tweaking, testing and securing before fully migrating everything over…and on the to-do-list is whole process for backing up, with a number of fail-safe’s in case anything goes wrong once all is migrated.  The problem?  It was still on the to-do-list.

So on Friday afternoon, I’m working on the server…I’m usually very cautious about working on it because I’m no system admin.  I have separate access for different domains, that way if anything is compromised or anything goes wrong, worst case scenario is that it should only affect that one domain.  During some of the work, I had to access multiple files from different domains, so I decided to login using root access, which can pretty much access anything and do everything.

To make a long story short, I was trying to delete the contents of a folder, and I used the following command:

rm -rf folder /*

I meant to do:

rm -rf folder/*

The difference of that one space?  Rather than deleting everything within the folder, it deleted the folder AND everything in /…which for Windows folks, is like saying delete C:\.  Seconds later and if you went to any of my sites you would get a quick error back saying permission denied.  On top of that, I hadn’t implemented my backup system yet, so I had no backups.  Talk about a nightmare!

The entire weekend was spent figuring out a variety of solutions to get everything restored to as close to the date as possible.  I got lucky with a couple things, like having a backup of some databases that I had downloaded while migrating servers (couple weeks old), and a recent backup in my recycle bin.  At the end of the day, after much wasted time, I was able to get everything fairly close to restored as possible (I did lose some stuff).  The first to-do after that?  Back everything up!  Had I not had certain files downloaded, or my host hadn’t had a disaster recovery file from a month prior, the situation could have been much worse.

This is a reminder to all site owners.  You never know what can happen.  Setup a backup plan, and implement it yesterday.

Yesterday I got an e-mail from somebody saying that they had gone to my site and their anti-virus software popped up a message saying their was a trojan. I didn’t see anything on the site, so I asked them to send me a screenshot of the error. Once I saw the screenshot, I saw a call being made to a data.js file, which is not something that I would make a call for or uploaded in that location. I checked the site in IE, and sure enough there was the call. Somebody had been able to get it on the server and create that file. I had been compromised.

The first thing I did was rename the file and check through the rest of the site to ensure nothing had gone into affect or been messed with. I seemed to have gotten lucky here. I called my host to see what steps can be taken. They told me to delete the file and follow all steps here which was extremely helpful.

You can go through the above document if you run into the same issue, but I’ll give a high-level overview of the two most important things to do.

Change ALL Passwords

I went through and changed all of my passwords. Think of everything you have passwords for, and change them. Make sure to use strong passwords (mix of upper and lowercase, numbers, and symbols). I changed everything from my host dashboard password, to database passwords, to username passwords (ex: WordPress), etc. You have to assume that if your server was compromised, they have your passwords. Changing them will lock them out.

Back Everything Up

I created an archive of my entire web folder and databases. Make sure you always have backups. This is a given, but the event was an eye opener of a reminder.

I got lucky that nothing worse occurred and that somebody brought this to my attention in time. To help prevent this, change your passwords and backup your data often.