Have to muck around with the server a bit, but most of these can't be done until the weekend... so that if the thing dies on me, nobody would likely notice while I get it back up :D
Moving SSH to a higher port, apparently stops quite a bit of attack since most script kiddies aren't likely to know how to edit or attack a higher range.
Going to cert based SSL, I'm concerned about this because I'd hate to break it totally. Or in the event when I need to access from a system without the cert :(
If these two work, then I can ignore the BFD/APF/F2B for now.
Monday, May 21, 2007
Hacking attempts
So some folks have been trying to break into the server via SSH and I realize I should had installed some kind of brute force detection. Now looking at the alternatives and found two. APF/BFD both from R-Fx Networks and the open source fail2ban.
Now I'm reluctant to install anything beyond the necessary. APF/BFD seems to be quite popular. There's even a third party DA script that installs it automatically. But I hate to be lock out of my own server again. And APF doesn't seem to have been updated for 3 years. Its readme file copyright notice stopped at 2004.
fail2ban seems more safe, although it's still a 0.8 product. But I'm still researching.
Now I'm reluctant to install anything beyond the necessary. APF/BFD seems to be quite popular. There's even a third party DA script that installs it automatically. But I hate to be lock out of my own server again. And APF doesn't seem to have been updated for 3 years. Its readme file copyright notice stopped at 2004.
fail2ban seems more safe, although it's still a 0.8 product. But I'm still researching.
Saturday, May 19, 2007
Server not responding at times
It was something I noticed during the days spent setting it up. The server will fail to respond every now and then for a while. The problem is finding out where the point of failure is. Is it the server itself having problems, or is the network?
Now frantically trying to find way to determine this.
Now frantically trying to find way to determine this.
Thursday, May 17, 2007
Locking yourself out of the server
Ok, this is one of the reason for the previous rant
Now one of the most dreaded things that could happen is doing something so stupid that you lock yourself out of the server... obviously I did.
The server's been running for a couple of days while I got things set up. But somewhere along the lines, don't ask me when where or why, one of the installation scripts must have edited the firewall/iptables settings. Or perhaps the control panel did for some reason.
After rebooting the server remotely for the first time, I found the server ignoring web connections. Panic!!!!
A quick check with ping and SSH revealed that the server did succeed in rebooting and was working. Just not accepting requests. Hmm apache's running. Then I discovered named/bind wasn't responding to DNS requests either. Essentially, the server's not going to listen to the world except through the primitives of ICMP and fortunately SSH.
On a hunch, from many a times my Windows browers just mysteriously could not access any websites, I did a /sbin/service iptables stop. Voila, webpages were served and pop3 access granted!
I turned it back on immediately however. The idea of leaving my server naked without a firewall is just scary.
So now I know, it's the iptables table settings. So briliantly, I googled for iptables firewall howtos on the web. See previous post for why I didn't bother with man pages on this, it just stare at me when I tried.
So wonderful, I found a forum posting on a site that says "Linux Firewall (iptables)
There it was the first glorious instruction "iptables -F" and the nice explanation that this flushes all current rules. Then "iptables -p INPUT DROP", this blocks all incoming connection. Nice since we don't want people trying to connect where they shouldn't.
See, what's so hard about writing documentation this way from a logical progression? People look at it, they know exactly what it does so they can figure out for themselves when they need to use it this.
But of course, after God knows how many hours without sleep, with bleary eyes and half braindead, I was so happy to find the guide that I started typing things in, step by step without reading ahead.
The astute readers will realize what that means when I hit the enter button on INPUT DROP.
SSH stopped responding. Basically the server was dead to the world because The Noob Admin has stupidly blocked all incoming connections to a server that wasn't physically next to him. Fucked.
It was a panicked half an hour trying to figure out if I could get access to the datacenter in the middle of the night. Finally cursing and swearing at the writer for not putting a BIG WARNING about not using them on a remote machine, I saw the small print that such command line changes are NOT saved to disk.
15 minutes later, after the wonderful IDC staff, God bless them, rebooted the machine, I was back in business.
Note to self: Never fuck with iptables in realtime. Always use the conf file and make sure the first line is to allow SSH so that whatever fucks up, you can always get back in.
Now one of the most dreaded things that could happen is doing something so stupid that you lock yourself out of the server... obviously I did.
The server's been running for a couple of days while I got things set up. But somewhere along the lines, don't ask me when where or why, one of the installation scripts must have edited the firewall/iptables settings. Or perhaps the control panel did for some reason.
After rebooting the server remotely for the first time, I found the server ignoring web connections. Panic!!!!
A quick check with ping and SSH revealed that the server did succeed in rebooting and was working. Just not accepting requests. Hmm apache's running. Then I discovered named/bind wasn't responding to DNS requests either. Essentially, the server's not going to listen to the world except through the primitives of ICMP and fortunately SSH.
On a hunch, from many a times my Windows browers just mysteriously could not access any websites, I did a /sbin/service iptables stop. Voila, webpages were served and pop3 access granted!
I turned it back on immediately however. The idea of leaving my server naked without a firewall is just scary.
So now I know, it's the iptables table settings. So briliantly, I googled for iptables firewall howtos on the web. See previous post for why I didn't bother with man pages on this, it just stare at me when I tried.
So wonderful, I found a forum posting on a site that says "Linux Firewall (iptables)
There it was the first glorious instruction "iptables -F" and the nice explanation that this flushes all current rules. Then "iptables -p INPUT DROP", this blocks all incoming connection. Nice since we don't want people trying to connect where they shouldn't.
See, what's so hard about writing documentation this way from a logical progression? People look at it, they know exactly what it does so they can figure out for themselves when they need to use it this.
But of course, after God knows how many hours without sleep, with bleary eyes and half braindead, I was so happy to find the guide that I started typing things in, step by step without reading ahead.
The astute readers will realize what that means when I hit the enter button on INPUT DROP.
SSH stopped responding. Basically the server was dead to the world because The Noob Admin has stupidly blocked all incoming connections to a server that wasn't physically next to him. Fucked.
It was a panicked half an hour trying to figure out if I could get access to the datacenter in the middle of the night. Finally cursing and swearing at the writer for not putting a BIG WARNING about not using them on a remote machine, I saw the small print that such command line changes are NOT saved to disk.
15 minutes later, after the wonderful IDC staff, God bless them, rebooted the machine, I was back in business.
Note to self: Never fuck with iptables in realtime. Always use the conf file and make sure the first line is to allow SSH so that whatever fucks up, you can always get back in.
Man Pages & Documentation
Hey, I'm a noob, I can't read man pages or documentation. They are like so fucking confusing without any examples most of the time. It's like, damnit, telling me what them various flags do doesn't tell me what to put into a conf file compared to a nicely commented conf file for various basic situations!
Exim documentation for example is a bloody nightmare. They have a tons of description on every little thing. Except showing you some examples of how it is put together. No, they throw you Cambridge's complete conf that even they themselves aren't too sure how it all works... -_-
I'm stupid, I need documentation that looks like this
Example 1 To do A
command -flagA value
This makes blah blah command does this
Example 2 To do A but with B exception
command -flagA value -flagB value
This makes the blah blah blah instead
and not some frigging command -aBcDeFgHiJkL [123456] -xYzRsTu [!@$^#]
then a whole textwall making you look back and fro, back & fro trying to match the description with the position and the big picture.
I just need the instructions on how to get the car moving first and will learn along the way why it moves or why I have to step on the clutch before changing gear. But documentation writers seem to want me to learn how to assemble a car from components first.
Exim documentation for example is a bloody nightmare. They have a tons of description on every little thing. Except showing you some examples of how it is put together. No, they throw you Cambridge's complete conf that even they themselves aren't too sure how it all works... -_-
I'm stupid, I need documentation that looks like this
Example 1 To do A
command -flagA value
This makes blah blah command does this
Example 2 To do A but with B exception
command -flagA value -flagB value
This makes the blah blah blah instead
and not some frigging command -aBcDeFgHiJkL [123456] -xYzRsTu [!@$^#]
then a whole textwall making you look back and fro, back & fro trying to match the description with the position and the big picture.
I just need the instructions on how to get the car moving first and will learn along the way why it moves or why I have to step on the clutch before changing gear. But documentation writers seem to want me to learn how to assemble a car from components first.
The Noob Admin begins
What this blog is about?
It's about the misadventures of a noob server admin who was thrown into the deep end of the pool and left to swim or drown.
As circumstances have it, I find myself the administrator of a webserver for paying customers. Given that I've never actually administered a Linux server before, with almost my entire experience limited to usage of the ever friendly CPanel, you can see where this is going.
Hence a few days into my life as a server administrator, I'm starting this blog to track all the dumb mistakes one could ever possibly make. Hopefully somebody else will benefit from it, even if just have a good laugh :)
It's about the misadventures of a noob server admin who was thrown into the deep end of the pool and left to swim or drown.
As circumstances have it, I find myself the administrator of a webserver for paying customers. Given that I've never actually administered a Linux server before, with almost my entire experience limited to usage of the ever friendly CPanel, you can see where this is going.
Hence a few days into my life as a server administrator, I'm starting this blog to track all the dumb mistakes one could ever possibly make. Hopefully somebody else will benefit from it, even if just have a good laugh :)
Subscribe to:
Posts (Atom)