Drupal as an office suite

So right now, ownCloud looks like it’s going to provide a platform for running Google Apps-style web apps (presumably with access to the files you have stored in ownCloud), but it doesn’t provide it yet. My next task, then, is to find a system that can provide me with “office productivity” functionality on my Microserver. There’s a few things that this functionality could include, but I’m looking for the “big three” functions of documents (primarily simple stuff for note-taking), spreadsheets and presentations. There’s a few solutions out there that aim to provide an open source could office suite, including Feng and Zimbra, but neither provide all three functions that I’m after, and both provide a lot of extra functionality (such as email) that I don’t need.

When I found a solution that gave me the three functions and was installable on my own server, I have to say it surprised me a bit – it was Drupal. The thing with Drupal is that while it’s a “content management system” (CMS), it’s extremely extensible and configurable, and can be made to handle pretty much any content you want in the way you want. Here are the steps I took to turn Drupal into my cloud office suite:

  1. I did a standard install of Drupal 6.2
  2. I disabled anonymous access, so even viewing the front page requires a login
  3. I installed Sheetnode. This is a Drupal module that’s based on SocialCalc, a Javascript-based spreadsheet system created by the co-author of VisiCalc, the original spreadsheet software. The SocialCalc code needs to be downloaded from GitHub seperately from the Sheetnode module. Once this is set up, you can create new Drupal nodes that contain a spreadsheet.
  4. I installed the Content Construction Kit (CCK) and s5 Presentational Player module. This allows you to view specially marked-up pages using Eric Myer’s Simple Standards-based Slideshow System (s5), which has been my preferred presentation format for a while. Getting it working required a fair bit of tweaking, and again some of the code (specifically the s5 engine) needs to be downloaded separately from the module. Luckily the author has a pretty good video explaining how to set it all up, although it appears to be based on a slightly older version of Drupal so I still had to tinker a bit.
  5. As part of the s5 setup, I installed the Markdown filter. This lets you really easily create formatted text with wiki-like syntax that’s human readable and can be rendered as HTML. Only a subset of HTML tags are supported, but it’s fine for about 90% of the documents I’ll write, and I’m happy enough to code tags for anything else I need.
  6. Now the main functions were taken care of, all that remained was to make it easier to get to my documents. I started this by enabling the Search module, meaning all my documents would be indexed and searchable.
  7. I added 3 Primary Links which appear at the top of every page. One to create a new document (or presentation, since they use the same page), one to create a new spreadsheet, and one to view the list of all my documents. I also set the site’s homepage to show the list of all my documents, which gives it a nice Google Docs feel.
  8. I enabled Tags to allow my to quickly categorise all my documents as I go
  9. For Mobile access, I enabled the Mobile Theme module with Browscap to automatically switch the theme for mobile browsers. The theme I found to work best was Nokia Mobile, with the following tweaks to the styles.css improve the editing form when viewed on my phone (an HTC Legend):
    #edit-body {height: 20em;}
    .collapsible.collapsed {display:none;}

This solution may not be spot on for everyone, although I see it as fairly optimal for the way I work. Alternatives (for the less text-oriented) might include using the s5 Book module included in the Presentational Player package, and using a WYSIWYG editor instead of Markdown.

One other things that is missing is real-time collaborative editing. There have been several projects started to implement these features in Drupal but not a lot of progress seems to have been made. The one promising-looking module is Etherpad Integration, although it appears to pull Etherpads from an existing installation rather than having the functionality in Drupal. Probably still worth a look if that’s what you’re after, especially if it can be made to work in conjunction with the new Node.js implementation of Etherpad.

ownCloud – It does exactly what it says on the tin

The first step of my self-hosted cloud is to find a replacement for DropBox, the habitual solution for storing files online and syncing them to your various machines. While there’s lots of alternatives to DropBox (there’s even one built in to Ubuntu), they all follow the same model of renting disk space on the provider’s server, with no scope for installing on your own hardware. So I looked for an open source solution, and while there’s about 3 in the works, none of them are a drop-in (pardon the pun) replacement for DropBox. These include SparkleShare, which seems more geared towards collaborating on documents than Dropbox-style functionality, not that that’s an undesirable feature in itself, Syncany, which allows you to use any storage you care to name, but doesn’t have a stable release yet, and ownCloud, which doesn’t yet sync your files.

This sounds like a bit of a non-starter, but on closer inspection ownCloud does something wonderfully simple and frightfully clever. You install it as a PHP web application, and it provides a web interface to upload and access your files, nothing too exciting there. It also acts as a WebDAV server, however, which allows you to access your files in most desktop file managers on Linux, Mac and Windows. I’ve even found a decent WebDAV client that gives me access via my Android phone, and provides functionality on par with the Android DropBox client (which doesn’t do syncing anyway). It does rely on you having an Internet connection at the moment, but a syncing client is in the works as part of Google’s Summer of Code, and it could even work in conjunction with Syncany since Syncany supports WebDAV as a storage engine. The other brilliant thing about the choice of WebDAV is that it operates over HTTP, which means that you can pretty much guarantee access through any firewalls you might be behind.

There are additional merits to ownCloud’s file storage alongside WebDAV access, some of which are already in the stable version (1.2) and others which are in the works for the next version (2.0). These include version control using Git, automated backups of all your files, and encryption.

Aside from the file storage, ownCloud has serious ambitions. It provides a plugin architecture, which it looks like will enable web applications to be written and run on top of the ownCloud platform. Once the API of the new version is stable and documented, I’m hoping it’ll be possible to build some seriously cool stuff on top of ownCloud (open source Google Docs anyone?). However, that’s for the future.

Self-hosted cloud

So now I’ve got my Microserver set up, I’ve got a machine in my own house, which is always on, and provides me with some peace of mind that data I put on it wont get lost as the result of a single hardware mishap.

A while ago I experimented with Dynamic DNS, a system which allows you to direct a hostname to a dynamic IP address used for your home connection. This means that, even though your IP address changes, you can always access it through the same human readable address.

I’m a big fan of the convenience that “cloud-hosted services” provide. I use Flickr to host photos, I sync my files through DropBox, I make notes in Google Docs. However, there are 2 things that really bug me about these services. You have a complete lack of control over the service, where it stores your data, if and when it’s encrypted, and who has access to it. Also, you can never use your own resources – for example DropBox gives you 2GB of data, and you have to pay for more. Never mind if you’ve got a 1TB drive in your desk drawer, or 100GB of web hosting you’re not using, you’re using their resources and that’s it. That’s not the way I think of computing.

So I’ve hatched a plan: I’ve now got a terabyte of storage connected to the Internet, and an address that allows me to access it from wherever I am – At home, at work, on public WiFi, on my phone, all the places I might use cloud services. My plan now is to replace all the hosted services I currently use with an equivalent, open source solution that runs on my own hardware, and I’ll cover it in a series of posts. Keep watching folks!

HP Proliant Microserver review

I’ve been looking for a while for a machine that can run a RAID array, giving me somewhere to put my media and backups that wont lose everything if a hard disk dies. My real problem when looking for something that fits my needs is price. There’s plenty of capable devices out there, but they tend to get expensive quickly – a Drobo or multi-disk NAS box can set you back £300 before you’ve bought any disks.

HP have a product that fits the bill in both features and price – the Proliant Microserver. The unit retails at £199 + VAT, and HP have been running some amazing cashback schemes netting you the box for about £140 all in.

Microserver on a desk
The form factor is similar to that of a shuttle case, but not as deep. It sits quite nicely on a desk, and is small enough to hide away anywhere with decent ventilation.

Microserver front panel
The front of the Microserver has 4 USB ports, the power button, and two indicator lights – 1 for disk activity and one for network activity. The power button glows orange when in standby and green when powered on. Additionally there’s a nice blue light that illuminates the HP logo when the machine’s on.

Microserver back panel
Things are kept simple round the back, with a couple of extra USB ports, the Gigabit Ethernet interface and an e-SATA interface. There’ also a VGA port for connecting a monitor during installation. It’s worth noting the lack of PS/2 interfaces – you’ll need a USB keyboard (or at least an adapter) for installation.

Microserver with front panel open
Opening up the Microserver is simple. The keys provided open the lock on the right of the front panel, which swings open like a door to reveal the innards. Here you have the 4 hard drive bays, with the power supply tucked away on the left, and the motherboard underneath.

Microserver Motherboard
The motherboard is easily accessibile by removing a couple of torx screws (using the supplied torx wrench) and unplugging a few cables, after which is slides out. Notable features here include an internal USB port (could be used to add WiFi, or even USB storage for the OS) and the passive heat sink on the processor at the back. All cooling is provided by the large fan at the back of the case – luckily this is big enough to be able to run reasonably quietly while still providing adequate cooling.
There’s 2 RAM slots available – the model I bought came with a single 1GB stick, but up to 8GB is supported. There’s also a couple of half-height PCI-e slots which can be used to expand the server’s functionality.

Microserver hard drive bays
There’s room in the server for 4 hard drives. Each needs to be screwed into one of the supplied mounts, after which is can be easily slotted in to the mounted SATA connectors – no faffing around with cables! The disks dont connect to a RAID controller, although Fake RAID is supported by the motherboard. However, I’ve been advised it’s safer to rely on software RAID in this case. My server was supplied with a 250GB drive – I’ve added 2 1TB drives in a RAID 1 (mirroring) configuration. There’s room for an optical drive in the bay above the hard drives, but this can also be converted to fit an extra hard drive. Of course, if that’s still not enough for you, the e-SATA port and PCI-e slots leave room for even more storage to be added on.

Installing the server was straightforward – I created a bootable USB drive and popped it in the front, then told the server to boot it (it defaults to PXE boot when there’s no OS installed). Once up and running, it’s nice and quiet thanks to the single large fan, although it does vibrate slightly. Placing the server on my wooden desk amplified this somewhat beyond acceptable levels, so I ended up sitting it on some of the plastic foam that came in the box (as shown in the first picture) which fixed the problem straight away. The 1GB of RAM is easily enough for my purposes (more on that in the next few posts), although the processor isn’t going to win you any races. You get a dual-core 1.3GHz Athlon, which spends a lot of its time dynamically throttled down to 0.8GHz. This can make some things (like installing updates) take longer than you might expect, but it does give the machine a big advantage – very low power consumption.

All in all, this server is one of those devices, like the O2 Joggler, that’s not amazing in itself but is great for the money you pay. At the time of writing, HP are still offering £100 cashback for up to 5 units, and we’re running a competition to win one in an upcoming episode of the Ubuntu UK Podcast.

I mentioned on the show that I’ve been using the server to replace some of the cloud services I use with self-hosted applications. My next few posts will talk about how I’ve done it!

5-line Random Password generator

I needed to generate a random password. There’s plenty of scripts out there for doing this, but most involve creating a dictionary of characters, then looping for a given number of times generating a random number and picking out a character based on the number. I wanted a more compact solution so here’s what I came up with:

$chars = str_split(sha1(rand()),2);
foreach ($chars as $k => $char) {
    $chars[$k] = chr((hexdec($char)/255*95)+31);
}
$password = implode($chars);

It’s PHP, but can probably be replicated easily in other languages. Here’s what it does:

  1. Generates an SHA1 hash of a random number, then splits the hash into groups of 2 characters (each is a hexadecimal number =<255).
  2. Converts each hex number into decimal, scales it to value between 32 and 127, then converts that number to an ASCII character
  3. Creates a string from the array of characters