refresherate.com

It doesn’t matter if your web site gets 10 hits a day or 10 million hits a day. It’s a good coder who plans for future growth and anticipates how a site will scale as traffic and load increases. Gregg Pollock of Rails Envy has an excellent (free!) 5 part series up at RailsLab. Definitely worth checking out. Gregg also has a (not free) Scaling Ruby screencast on the EnvyCasts web site. I haven’t watch it, but I have watched the Advanced ActiveRecord screencast and it was both informative and entertaining. Gregg and Jason make this stuff fun! So check them out.

If you work with web sites, DNS, etc. and have to make regular DNS changes, it can often take a while for DNS changes to propagate. So you can either wait the 12 – 24 that it takes for your computer to refresh, or you can try and force a refresh by clearing your local DNS cache. It doesn’t always work, mind you.

If you are using Mac OS X, you can try:

dscacheutil -flushcache

Or if you use Windows you can try:

winipcfg /flushdns

You may need to restart your browser or which ever application you are trying to access the site with.

I was installing the MySQL ruby gem this evening and ran into some errors. After some hunting around I found the solution:

gem install mysql -- --with-mysql-config=/usr/lib/mysql/mysql_config

Here’s the errors I was getting hit with. Hope this helps others who might encounter a similar problem.

Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
        ERROR: Failed to build gem native extension.

/opt/ruby-enterprise-1.8.6-20090201/bin/ruby extconf.rb install mysql
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... yes
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/opt/ruby-enterprise-1.8.6-20090201/bin/ruby
        --with-mysql-config
        --without-mysql-config
        --with-mysql-dir
        --without-mysql-dir
        --with-mysql-include
        --without-mysql-include=${mysql-dir}/include
        --with-mysql-lib
        --without-mysql-lib=${mysql-dir}/lib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-mlib
        --without-mlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-zlib
        --without-zlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-socketlib
        --without-socketlib
        --with-mysqlclientlib
        --without-mysqlclientlib
        --with-nsllib
        --without-nsllib
        --with-mysqlclientlib
        --without-mysqlclientlib

Gem files will remain installed in /opt/ruby-enterprise-1.8.6-20090201/lib/ruby/gems/1.8/gems/mysql-2.7 for inspection.
Results logged to /opt/ruby-enterprise-1.8.6-20090201/lib/ruby/gems/1.8/gems/mysql-2.7/gem_make.out

So you just installed a new kernel and updated your grub.conf file. But you’re a little nervous about rebooting the server in case the new kernel doesn’t work out. Here’s a simple way to grub to reboot your new kernel just once. That way, if it fails, the machine can just be power-cycled and will reboot into the known to work kernel.

echo "savedefault --default=0 --once" | grub --batch

Change default=0 to which ever kernel you wish to boot once. Once you are done reboot:

shutdown -r now

Now remember, this will only boot that new kernel once. Subsequent reboots will end up booting the old default. So if everything is working like it should make sure to edit the grub.conf file and set the new kernel to be the default.

If you’ve been around web design for any length of time you know about PNG transparency issues with Internet Explorer 6. PNG looks great in nearly every browser except IE6, and with plenty of folks still using IE6 despite its limitations, it’s a good idea to support these users as best as possible. Usually this means using Javascript hacks to exploit some proprietary features of IE6 to enable PNG transparency. But here is a an interesting screencast from Brenelz’s Web Solutions which looks pretty interesting. It’s basically a way of creating a PNG32 in Photoshop, then re-opening it in Fireworks and saving it as a PNG8 with alpha-transparency. The results are pretty impressive and the image file size is dramatically lower.

Edited: I tried this today using a PNG-32 which was previously saved in PhotoShop. The file was 16kb. The PNG-8 alpha transparency file was 4kb. 25% of the original size! In Firefox the image seemed just as good. It wasn’t as smooth as the PNG-32 given that PNG-8 has only a fraction of the color palette. The file in IE6 looked different, but was perfectly acceptable. It beat the usual method of Javascript hacks and conditional statements.

Lighttpd, affectionately known as Lighty is an extremely efficient and lightweight web server. I run Apache 2.2 on all my servers, but have been using Lighttpd for a couple years now to stream video files for a popular web site I run. Lighttpd has a flv streaming module which can handle more advanced streaming than the usual progressive http streaming. In fact, I hear YouTube uses Lighttpd for their video streaming, so it’s pretty rock solid. Lighttpd also excels at serving static content and setting up an assets server running Lighttpd is one way to take some of the burden off of Apache and dramatically increase the load time of a popular web site.

So, on to configuring Lighttpd. First we need to download and extract it.

cd /usr/local/src
wget http://www.lighttpd.net/download/lighttpd-1.4.20.tar.gz
tar -xzf lighttpd-1.4.20.tar.gz
cd lighttpd-1.4.20

Now, before we configure Lighttpd we need to install a few dependencies, otherwise it won’t configure.

yum install pcre-devel zlib-devel bzip2-devel

Now with those installed, we can configure, make and install Lighttpd

./configure
make
make install

If all goes according to plan, Lighttpd should be installed on your system. Now we need to edit the configuration file. In the doc directory is a default configuration file which we can edit for our needs. So lets make a copy of that and put it where it will reside on the server.

cd doc
mkdir /etc/lighttpd
cp -rfp lighttpd.conf /etc/lighttpd/.

So now we have a copy of the default configuration file in /etc/lighttpd/. Lets also move a copy of the startup script. There are a couple rc.* files in the doc directory. For my system I used the rc.lighttd.redhat version and moved it to my servers init.d directory.

cp -rfp rc.lighttpd.redhat /etc/rc.d/init.d/lighttpd

So now we have some editing to do. Open up the official copy of lighttpd.conf in your favorite editor. I prefer emacs.

emacs /etc/lighttpd/lighttpd.conf

Under server modules you will see all the default modules Lighttpd comes with. Most of these will be commented out using a #. For my assets and flvstreaming server I uncommented mod_status (provides server status information at domain.com/server-status), mod_compress (allows for compression of static files which can save bandwidth and speed up serving), and mod_expire (allows for setting expire headers of cachable assets to save bandwidth and speed up serving). By default, mod_access and mod_accesslog should be enabled.

Now under the server modules area I add my customizations. You may to add your own according to your server’s purpose and capabilities.

# extension of flv files
flv-streaming.extensions = ( ".flv" )

# aggressive timings for performance.
server.max-keep-alive-requests = 6
server.max-keep-alive-idle = 15
server.max-read-idle     = 15
server.max-write-idle    = 15

Next, we should set up the modules we configured. In order for mod_status to work we need to find this line and uncomment it like so:

status.status-url          = "/server-status"

Now you might not want to make this URL publicly accessible, in which case you can wrap it in a block like so, replacing the IP address with the IP of the computer you are accessing the page from:

$HTTP["remoteip"] == "192.168.0.1" {
  status.status-url = "/server-status"
}

Next, let’s configure mod_compress. Search for compress.cache-dir. Here are the mod_compress settings I am using:

compress.cache-dir         = "/var/cache/lighttpd/compress/"
compress.filetype          = ("text/plain", "text/html", "text/css", "image/png", "text/javascript", "image/jpg", "image/gif")

You will also need to create the cache directory for the mod_compress to do it’s business. So back in the shell you will want to type.

mkdir /var/cache/lighttpd
mkdir /var/cache/lighttpd/compress

Now, I am telling lighttpd to compress plain text, html, css, javascript, as well as various image formats. Performance gains compressing images is debatable since images are often compressed by default and the cpu cycles used to compress images often outweighs the bandwidth improvement. By the way, when we talk about compression under mod_compress, the actual files the browsers receive are not impacted. mod_compress simply compresses the file for transfer and the receiving browers uncompresses back to it’s original size. So degradation due to image compression is not an issue.

Next, let’s set up the expire module, mod_expire. This allows you to configure lighttpd to set an expire header on certain files or directories. Since I organize all my files in folders like /images /stylesheets, /javascripts it’s pretty easy to tell lighttpd which files I want to set expire headers. You can search for expire module and add this down there, but I prefer to add it higher up.

$HTTP["url"] =~ "^/images/" {
expire.url = ( "" => "access 240 hours" )
}

$HTTP["url"] =~ "^/javascripts/" {
expire.url = ( "" => "access 240 hours" )
}

$HTTP["url"] =~ "^/stylesheets/" {
expire.url = ( "" => "access 240 hours" )
}

This tells lighttpd to assign a expire header of 240 hours to files served from these directories. This will often save the browser from making additional requests for these files to check to see if they have changed. It can save a lot of overhead and bandwidth on a popular web site.

Lastly, we need to set up our host information. Again I like to place this in the upper part of the configuration file beneath my server.* configurations.

$HTTP["host"] == "flvstream.domain.com" {
server.document-root = "/home/lighttpd/flv_files/"
}

And if I want to create another host for assets, I’d just create it just below the first.

$HTTP["host"] == "assets.domain.com" {
server.document-root = "/home/lighttpd/flv_files/"
}

Now, with these two hosts set up, you can comment out this line since it’s not needed for our multi-host setup:

#server.document-root        = "/srv/www/htdocs/"

Now, we are just about ready to start up our server. So save this file and exit. Lighttpd will log access to the /var/log/lighttpd/directory. Let’s create that:

mkdir /var/log/lighttpd

Alright, so we are all set. If you have your files in place go ahead and start up lighttpd.

/etc/rc.d/init.d/lighttpd start

If all goes according to plan it should start up and you can call up your host in the browser. If you get an error message, Lighttpd is pretty clear about what went wrong and will give you some tips or guide you to what line in the conf file is out of whack.

Now if you are running lighttpd publicly, you will want to run it as an unprilegdged user unless you like to be hacked and cracked. So in the shell lets add a lighttpd user and group.

groupadd lighttpd
useradd -g lighttpd -d /home/lighttpd -s /sbin/nologin lighttpd

Now lets edit lighttpd.conf and tell lighttpd to run under this user. Uncomment and set the following two lines.

server.username            = "lighttpd"
server.groupname           = "lighttpd"

Any files that Lighttpd reads or writes to will need to be owned by Lighttpd. So your web files, log files, mod_compress directory, etc. Once you are set, you can start Lighttpd.

This concludes this quick and dirty guide to compiling and configuring Lighttpd for use as a high performance assets server. There is a lot more to Lighttpd, so I encourage you to read around. This is just a quick-start guide to configuring a relatively high performance assets and flv streaming server under Lighttd. I have noticed a 8 – 10X increase in performace under Lighttpd over Apache, while using only a fraction of the memory and CPU usage. So Lighttpd lives up to it’s name.

Some Lighttpd resource:

• Lighttpd Wiki
• mod_status
• mod_expire
• mod_compress

Sometimes you have a bunch of .html files which you need to rename .php for a project. You could rename them one at a time, but that gets a little old after about 5 files. This little shell script will rename the extension on a whole directory of files:

for f in *.html; do mv $f `basename $f .htm`.php; done;

Here’s a quick and painless way to change your server’s hostname in linux without having to reboot the server. This should work for most flavors of linux, although I have only tested it on CentOS and RedHat. So you may have to adjust things according to your server’s configuration.

Step 1: Edit your network file.

cd /etc/sysconfig
vi network

Look for your current hostname to the right of HOSTNAME= and edit it to reflect your new hostname. When done, save the document.

Step 2: Edit the hosts file.

cd /etc
vi hosts 

Again, you will want to edit out your old hostname and replace it with the new hostname.

Step 3: Use hostname program to set hostname.

hostname your-new-host-name

Step 4: Restart network services

service network restart

You should be all set. You can test your hostname by typing:

hostname

This should return the current hostname. You can do a further test by logging out of your current session and re-logging in. Your command prompt should reflect the new hostname:

[toor@rack10]

RailsConf 2009 registration is now open. You can save a boatload on registration if you register early. And you can blow the saved money on the tables at the casinos.

Here’s a quickie on how to determine the process specs in linux. I am always forgetting the command so this is as much for me as it is for others:

cat /proc/cpuinfo