Tuesday, November 08, 2011

Slow SSH and sudo Login (Ubuntu)

I have been annoyed by extremely slow SSH logins to an Ubuntu VM of mine for a while now. Both the login and password prompt displayed quickly enough and there was no delay in keyboard input, however after submitting my password I'd be faced with a delay of somewhere between 10 and 30 seconds. I had begun noticing similar delays when using sudo to switch to root (though these delays weren't quite as long).

My threshold of annoyance was finally crossed this evening and I decided I'd better figure out why this was occurring. Most resources I found online seemed to point in the direction of ensuring "UseDNS no" has been set in /etc/ssh/sshd_config, unfortunately this was not the problem in my case.

After digging around a bit longer, I bumped into a helpful thread on the Ubuntu forums that pointed out the actual problem was likely with PAM.

Making the recommended change by commenting out the following PAM connector in /etc/pam.d/common-session did the trick.

# session       optional                        pam_ck_connector.so nox11

Saturday, June 11, 2011

Perl Errors Installing Net::SSLeay or Crypt::SSLeay from CPAN

I've run into this same problem enough times now that I figure it's worth posting: Attempting to install Net::SSLeay or Crypt::SSLeay via CPAN on a clean system results in a number of unexpected errors despite having openssl already installed. Typicall errors include the following (plus a few thousand more of equal or lesser value):

SSLeay.xs:98:25: error: openssl/err.h: No such file or directory
SSLeay.xs:99:27: error: openssl/lhash.h: No such file or directory
SSLeay.xs:100:26: error: openssl/rand.h: No such file or directory
SSLeay.xs:101:28: error: openssl/buffer.h: No such file or directory
SSLeay.xs:102:25: error: openssl/ssl.h: No such file or directory
SSLeay.xs:103:74: error: openssl/comp.h: No such file or directory
SSLeay.xs:105:25: error: openssl/md2.h: No such file or directory
SSLeay.xs:107:25: error: openssl/md4.h: No such file or directory
SSLeay.xs:108:93: error: openssl/md5.h: No such file or directory
SSLeay.xs:112:26: error: openssl/x509.h: No such file or directory
SSLeay.xs:113:28: error: openssl/x509v3.h: No such file or directory
SSLeay.xs:114:28: error: openssl/engine.h: No such file or directory
SSLeay.xs:134: error: expected â;â, â,â or â)â before â*â token
SSLeay.xs:135: error: expected â=â, â,â, â;â, âasmâ or â__attribute__â before â*â token
SSLeay.xs:136: error: expected â=â, â,â, â;â, âasmâ or â__attribute__â before â*â token

The fix for this is always the same. These modules don't merely require an install of openssl, but instead need a full openssl-devel installation as well.

[root@whatever]# yum install openssl-devel

That is all... Breckenridge!

Thursday, June 09, 2011

Data Recovery From A Single RAID 1 Disk Image

Ran into a situation where there was an important need to recover some data from a long since defunct server. It just so happened that I had an archived bit-level image of one of the two physical drives in the server's RAID 1 array that had been taken sometime before the drives had been reprovisioned.

Not having a very good idea about how to proceed, I did a little Googling and started messing around. I found a lot of partial examples but very little in the way of a concrete guide on how to proceed. As such, I ran into a few hurdles. Hopefully my trials and tribulations in this area will prove useful to someone else who faces a similar task in the future.

[root@whatever]# mdadm --examine hd1.img
mdadm: No md superblock detected on hd1.img.

Clearly I wasn't using mdadm correctly. A glance at the man page and a few examples online got me started in the right direction... I needed to setup a loop device using losetup:

[root@whatever]# losetup /dev/loop0 hd1.img

[root@whatever]# fdisk -l /dev/loop0

Disk /dev/loop0: 80.0 GB, 80000000000 bytes
255 heads, 63 sectors/track, 9726 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xdd99b711

Device Boot Start End Blocks Id System
/dev/loop0p1 * 1 13 104391 fd Linux raid autodetect
/dev/loop0p2 14 144 1052257+ fd Linux raid autodetect
/dev/loop0p3 145 9726 76967415 fd Linux raid autodetect

Progress! Now that fdisk is able to see the partitions, I assumed I was pretty much finished. Back to mdadm to examine and ultimately assemble the RAID array... except mdadm couldn't find the partitions!

[root@whatever]# mdadm --examine /dev/loop0p3
mdadm: cannot open /dev/loop0p3: No such file or directory

This was unexpected. A quick file listing of /dev showed that indeed the /dev/loop0p* devices don't actually exist. A bit more Googling teaches me that I need to use kpartx to create device maps from the partition table for the loop device.

[root@whatever]# kpartx -a -v /dev/loop0
add map loop0p1 (253:2): 0 208782 linear /dev/loop0 63
add map loop0p2 (253:3): 0 2104515 linear /dev/loop0 208845
add map loop0p3 (253:4): 0 153934830 linear /dev/loop0 2313360

This step creates the loop0p* devices in /dev/mapper/. It is now a relatively simple task to use mdadm to assemble and run the array with a single disk:

[root@whatever]# mdadm -A --run /dev/md0 /dev/mapper/loop0p3 
mdadm: /dev/md0 has been started with 1 drive (out of 2).

[root@whatever]# mkdir /mnt/disk && mount /dev/md0 /mnt/disk

That's pretty much it. Unfortunately at the end of it all it turns out the files I wanted to recover had been purged from the disk before this image of the drive had been taken, so my efforts were largely for naught... But that's another story. ;)

Update 2011-06-10: It turns out I had another image of the same single RAID disk prior to the needed files being purged stored in another location and was able to apply the same technique to recover the files from this image. Success!

Thursday, March 03, 2011

Apache 2, mod_perl, HTML::Mason, "use lib" and "Can't Locate" issues with @INC

Long title, but accurate. This is a short tidbit for anyone who is experiencing errors with mod_perl relating to being unable to locate modules from a directory in @INC specified via a "use lib" statement.

My personal setup involved an Apache 2 directive within a VirtualHost utilizing HTML::Mason that referenced a "PerlRequire" module that in turn added a specific "use lib" path (/var/www/html/lib) to @INC for use of personal libraries within a new site I had been setting up.

Every single syshandler, autohandler, or page referencing any module within the specified path was crashing with an error that Perl "can't locate" the module in question within the known set of @INC paths (which visibly contained the correct path to the modules in question).

Many hours of Googling found numerous answers suggesting file/directory permission errors, however even setting 777 on every directory and file in question resulted in the same failed outcome.

Almost by accident I checked /var/log/messages and saw something similar to the following:

Mar  3 19:09:21 test_server1 kernel: type=1400 audit(1299197361.362:4):
avc: denied { getattr } for pid=1812 comm="httpd"
path="/var/www/html/lib"
dev=dm-0 ino=3342348 scontext=user_u:system_r:httpd_t:s0
tcontext=user_u:object_r:user_home_t:s0 tclass=dir

As soon as I saw this, the light bulb illuminated... This was a new CentOS 5 server and I had forgotten to disable SELinux.

For anyone else experiencing this issue, merely edit /etc/sysconfig/selinux and set "SELINUX=disabled" then reboot, you should be good to go.

The bigger question here is why SELinux would be enabled by default on most servers, but I'll leave that for the philosophers of the world, I've got better things to do.