null-o

There are some days where I wish I could be this guy. I’m going to have to plan a little hiking trip in the near future.

Click here

If I had the money, this would be the next PC that I build:

• Motherboard: Intel D945GCLF
• Hard drive: Seagate Momentus 7200.2 120GB 7200RPM
• Case: Morex 3688s
• RAM: 2GB 667MHz DDR2 Non-ECC CL5 DIMM

I can get all of this for around $300, but having just bought a house money is kind of tight right now. That and I just spent $100 for a new motherboard a few months ago. I suppose I’ll just have to wait till Christmas.

Because I cut my Linux teeth on Slackware I’m not that familiar with rpm. But now for work and pleasure, I am using rpm based distros, SuSE and CentOS, so it’s time to learn a bit more.

Here’s something simple to display installed packaged and their sizes:

rpm -qa --queryformat '%{NAME}-%{version}-%{release} %{SIZE}\n'

A few weeks back, I was just about to walk out the door for the day when I got an alarm message that our website had gone down. After feverishly try to ssh into the sever only to have it hang after I entered my password, I ran to the server room to get to the console. Now, the person who I replaced setup these SLES servers to boot into runlevel 5, which means that the console has a nifty graphical login prompt. In my opinion, that’s great for a laptop/desktop, but not for a server, especially a production server.

After trying to login only to have it hang, I killed the X session (ctrl-alt-backspace) and noticed a message about /tmp being out of space, and right away I knew that the hard drive had filled up for some reason or another. This leads me to another problem I have with these servers which is there was now partitioning done on the server at all, just 300B mounted as / which any *nix admin worth his paycheck will tell you is a bad idea. Normally, you partition hard drives to have multiple mounts for the file system to prevent trivial things like a partition filling up from bringing down the whole system.

But at times like these thank goodness for single user mode. I’ve never actually had a harddrive fill up on my before, so this was a good learning experience. To boot into single user mode in GRUB you type single at the prompt, and for LILO you type linux single. Then the system boots up, mounts the file system as read-only and you enter the root password. What I didn’t know at the time was that the filesystem is mounted read-only, so after a minute of googleing I found this command to remount as read-write: mount -o remount,rw /

Once the file system was remounted read-write, all that was left was the trivial task of freeing up enough space to bring up the server and go home. Luckily, I found an old 22GB tar file to delete.

After analysis the next day, I discovered over 90GB of heapdump/javacore files from WebSphere Commerce that was the cause of the hard drive filling up. Apparently, whenever WebSphere runs out of memory or has problems it dumps the contents of memory into a heapdump file on the hard drive, and there were close to 1000 heapdumps at around 120MB a pop.

At work, I recently took over the linux administration of some 30 servers. I’ve been using linux off and on since 2000, so to be able to do linux admin work in a corporate setting has been quite enjoyable and it’s a career path I’ve been wanting to head down for a long time. There’s been a little bit of a learning curve, since I’m used to Slackware and these servers are running various releases (7,9,10) of SuSE Linux Enterprise Server, but all in all it hasn’t been too bad.

Currently, we have two machines running under VMware, six Lintel boxes, and about 20 or so servers running under z/VM on our big mainframe. The one thing that I still haven’t wrapped my head around is how almost all of the hard disks on the z/VM guests are almost entirely filled up. For example, the other day, the websphere admin came to me and asked why the root partition of the big money making websphere server was 100% full. I wasn’t surprised, since this was probably the seventh or eight time in a month that I’ve run across this.

This prompted me to google a bash script that would search the file system for files bigger than a given value, and since google never does me wrong, I came across this site.

That was a nifty find, but for some reason the web software Jarrod is using to power his website wouldn’t display a much needed \ and the script wouldn’t work. That caused me to dig deep into some of my linux reference manuals to get it working, and I decided to improve it a little. This is what I came up with.

#!/bin/bash
# if nothing is passed to the script, show usage and exit
[[ -n "$1" && -n "$2" ]] || { echo “Usage: findlarge [PATHNAME] [SIZE in KB]“; exit 0 ; }
# simple using find, $1 is the first variable passed to the script
find $1 -type f -size +$2k -exec ls -lh {} \; | awk ‘{ print $8 “: ” $5 }’

Some will get the title. I read that site way too much.