TeraServer 2

I need an upgraded server. I want something that can handle replication of a SaaS database with on-disc encryption. Keeping with my habit of building my own machines, it would require more space than my current TeraServer could provide, and frankly the disk access times are a little slow now on the old one (I suspect the RAID backplane I sourced wasn’t capable of delivering the SATA-2 speeds the drives could). It’s being used for far more applications than I had planned too.

5 years is a long time in hardware and I’m amazed to see how storage costs have dropped further. This time I could get two disks that would provide me with full drive RAID redundancy at a capacity of 3TB at SATA-6 speeds. That means I could go for the 1U rack mount case at mini-itx.com (C2-RACK-V3). This case also supports a PCI-E card – so I can add a video capture card for some security monitoring using ZoneMinder too.

TerraServer2_Ext

The drives are fast and surprisingly quiet too (Seagate 3.5″ 3TB Barracuda 7200RPM). To ramp up the system performance of this one even further I used a Crucial M4 SSD for the Linux system partition. That was a new experience! The thing boots in seconds.

With the spinning drives mounted using rubber washers to reduce vibration and noise, the three drives fitted in comfortably, with room to spare.

TerraServer2_Int_2

For the mainboard, I wanted something with a bit more processor power (it would need to handle Linux software RAID and video capture analysis for ZoneMinder). It also needed to fit the case and I didn’t want any cooling fan that would add to the height and noise. Eventually, I found an Asus C60-M1 at a very good price from an on-line store in Germany. It was a compromise – the processor is the same clock speed as the previous server board, but anything higher wouldn’t come fanless. I settled for the same clock speed (but twice the cores helps!) It fitted ok in height, but there was a bit too much of a gap between the board and the chassis back-plate. I don’t think any of the provided blanking plates would cover the entire back-plate either. That wasn’t important for me anyway.

4GB of 1333MHz DDR3 RAM would be plenty to handle the extra Linux apps I have been (and will be) running on the server. With that amount of RAM, I could even run the Linux tmpfs in a ramdisk to make it even faster.

TerraServer2_Int_1

I installed Centos 6, configured the drives, and I’m amazed at how fast this little mini-itx powered beast flies compared to my previous creation. 5 years is a long time in hardware.

Configuring Linux Solid State Drive (SSD) Performance

SSDs are really fast (especially for reads) compared with spinning platter drives, but are relatively expensive and so generally have smaller capacity. This makes an SSD a good choice for the system partition of a Linux installation where all the frequently read system files are located – programs will boot up faster.

SSD drives also have a potentially shorter lifespan, especially if used in a write-delete-write fashion. At the time of writing, it’s estimated that system drives used this way will start to show degradation after about 1-2 years. To improve the performance and longevity of the SSD here are some of the best practices I’ve discovered for configuring Linux with an SSD.

  1. Check the SSD’s page size and erase block size (4K and 512K respectively in my case)
  2. Install the latest firmware available for the SSD
  3. Create the partition so as to align its start to the SSD erase block size (or multiple of it). This will ensure your SSD won’t have to do extra write operations.
    fdisk -S<sectors> -H<heads> -cu /dev/sda
    

    You do this by specifying a number of heads and sectors. Ultimately, you need to look this up because it depends on the erase block size of your drive. There’s some good advice here.

  4. Only format the partition with the ext4 filesystem – it has TRIM support
  5. Align the filesystem with the block size on the SSD when formatting the partition. If you aligned the partition (as above) then this further ensures minimum writes by choosing an ext4 block size that matches the page size on the drive.
    mkfs.ext4 -b <block size in bytes> /dev/sda2
    

    For me, I didn’t need to specify the block size because the default ext4 block size is 4K which already matches my drive’s 4K page size.

  6. Once Linux is install, edit fstab to enable TRIM on the SSD filesystem. This is done by adding the discard flag to the partition.
  7. A further tweak that can be done in fstab to minimize unnecessary writes on the SSD is to switch off updates to the file access times – remove atime and relatime flags on ssd partitions and replace with noatime and nodiratime
    UUID=XXXX    /   ext4  defaults,noatime,nodiratime,discard  1 1
    
  8. If you have a second, platter drive installed, move the swap partition to that device.
  9. You can minimize swap disk accesses by disabling swap which will keep swap space in memory (provided you have enough memory and it is not full). In /etc/rc.local file add the line:
    echo 0 > /proc/sys/vm/swappiness
    
  10. Use the deadline scheduler for writes. In /etc/rc.local file add the line:
    echo deadline > /sys/block/sda/queue/scheduler
    
  11. Move small temporary file directories away from the SSD storage and into ramdisk, by adding the following to fstab. (Note that this would could mean data loss if server crashes, but it’s an acceptable tradeoff for me).
    tmpfs   /tmp       tmpfs   defaults,noatime,mode=1777   0  0
    tmpfs   /var/spool tmpfs   defaults,noatime,mode=1777   0  0
    tmpfs   /var/tmp   tmpfs   defaults,noatime,mode=1777   0  0
    
  12. On a per application basis, repoint any temporary file write directories to platter partitions. The best example of such an application is a web browser, which maintains writes a large cache of temporary files. (In the end, to handle all these temporary application file scenarios, I decided to redirect the entire /var directory onto a second platter drive – this also covered Linux system /var/log files.)