Rick Hurst Web Developer in Bristol, UK

Menu

Category: software

Web developer tools of the trade

Posted on by Rick Hurst

I’ve had to change machines a few times recently, and each time I wish I had a decent checklist of what I need to be able to do my job. I thought it might actually be an interesting blog post, detailing not just what I use, but why.

Machine: macbook air

I blogged about using a 13 inch macbook as my main machine for web development before, so I won’t repeat it here – the main point I want to make here is that I don’t want to be using more than one computer and it needs to be a laptop that runs some flavour of unix natively and will run the latest versions of Adobe Creative Suite reliably, and a mac laptop fits the bill. Why does it need to be a laptop? It was essential when I was a freelancer – I worked on site at clients offices and not many of them would supply a computer. My current employer provides hot-desks with monitor/mouse keyboard etc. for people to plug their laptops into.

OSes: OSX, Ubuntu and Windows

In the past i’ve experimented with Linux on cheaper non-mac laptops, but I always get caught out by needing a native version of photoshop running – using wine or VM’s to do this has proved slow and unreliable. If Adobe really wanted to put a dent in Apple’s revenues, they should release a linux version – I know many people who would drop OSX if they weren’t dependent on Adobe products.

Why not windows as my main OS? Mainly because it doesn’t run unix natively, but I also find windows a pain to maintain and keep stable and secure.

I keep Virtual Machines (VM) for when I need to develop or test stuff in windows – Windows 7, plus other versions for testing on different versions of internet explorer. I still look after some legacy classic ASP stuff, so for this I have microsoft webmatrix installed on the Windows 7 VM, which as a cut-down version of IIS for local development. I also use Heidi SQL as a GUI for MySql on windows, and Sublime Text (mentioned later) for code editing.

I also keep an ubuntu VM for LAMP development, mainly so I can keep it separated from the Python/ Django/ Appengine orientated dev environment I use in my day-job at Potato. It also makes it transportable and easy to back up.

Graphics Editor: Adobe Photoshop latest version-ish

I’m not a designer so shouldn’t necessarily need Photoshop, but I receive layered photoshop files from different designers and agencies, and I need to do stuff with them. When i’ve tried switching to other tools in the past, as some will open layered photoshop files, but i’ve been caught out by one missing feature or another, usually from the latest, or almost latest version of photoshop. So it looks like I will be a slave to Adobe until such a point where I stop being a front-end developer, or start working with designers who don’t use Adobe products (do they exist?). Otherwise i’d happily jump to something like pixelmator on OSX, or Gimp.

Code Editor: Sublime Text 2

I actually began writing this blog post months ago, but left it as a draft, and this section used to read Textmate! I used textmate for several years, but always had stability issues with it when I needed to edit something from a mounted folder or over sFTP. Sublime does the former quite well, and the latter is possible using macfusion. I haven’t really explored sublime at all, so i’m probably only using a tiny percentage of its features, but it seems to do everything textmate did (or rather everything I did with textmate). The thing that really won me over is that there are versions for OSX, linux and windows, and the licence is transferable. This means that in the rare times I have to use other operating systems, I can use the same editor, which is great.

Code editing over FTP:Coda

I also bought a copy of coda, which I found invaluable when I was a freelancer, as some of my freelance clients were pretty old-skool with their hosting and I needed to use FTP, often including the necessity to edit stuff (local settings/ config files etc) directly on the live server without SSH access. Coda is great for this, and is also a nice text editor that I tried to start using full-time, but it was missing the “autocomplete strings already in the page” feature (i.e. if you added really_long_variable_name to a page, it appears as a suggestion when you start typing it again) that Textmate and sublime do really well. Actually, I haven’t even installed it on this machine, not sure I will now..

Code editing in the terminal: Emacs

I usually install Emacs (No X version) on whatever server i’m accessing via ssh, for editing files on the server. I only know all the basic operations needed to be productive with it.

Server-Side web development: Python, Django and PHP

Wait, did I say Python *and* PHP in the same sentence? My chosen technology for building websites these days is the Django framework which runs on Python, but for years I also built stuff with PHP. Unlike most of the Python community, I don’t have a problem with PHP, and will continue to use it for some things – the big advantage of PHP is that it runs on most shared web hosts, whereas Python/Django needs more specialist hosting environments.

Why not Rails? Basically i’ve been trying not to stretch myself too thinly, as I have a habit of doing. I wouldn’t rule out learning Rails in the future, but there’s only so many hours in the day! I am however learning Node.js, as I see it as a logical progression of my existing JavaScript knowledge, and think server-side JavaScript will be big in the future.

Source Control: GIT (with SourceTree as a GUI)

I’ve blogged before about how i’ve moved over to GIT from SVN. I do this mostly from the command line, but I also have a copy of SourceTree, which I find really useful for preparing large commits and searching logs etc.

General FTP and sFTP: Filezilla

I have to stress here that I only ever use FTP for website deployment when forced to – I don’t run it on my own server, and deploy all my sites via GIT, in pretty much the same way I used to with SVN. Also Appengine has it’s own deployment process, and doesn’t support FTP or anything close to it!

So, with that clear, if I ever need to do some general FTPing for whatever reason (the last time was to bulk upload about 200 video files to a client’s wordpress site on shared hosting), I use the excellent FileZilla. Just like Sublime Text 2, there are handily versions for OSX, Windows and Linux.

AppEngine python SDK/ GoogleAppEngineLauncher

To develop Django apps on AppEngine (or more specifically Python AppEngine apps that use parts of Django), you install Google App Engine Launcher. We actually deploy apps via the command line rather than using the launcher GUI, but it’s all part of the same install.

Databases: MySQL and PostGres

For PHP and non-appengine Django stuff I mainly use MySQL. It does what I need it to, and i’m confident the performance and scalability are more than enough for most projects. I also use PostgreSQL for GeoDjango projects, using the PostGIS extension.

General Django dev: virtualenv and pip

For any local sites running Django I install a copy of Django and specific packages in a virtualenv. This means that I can have multiple different versions of Django installed with different dependencies in a sandbox. I also use virtualenv when deploying a Django site on my server, allowing me to have different standalone django versions on the server too. I install dependencies using PIP.

Email: Gmail web interface

I don’t use a desktop mail client anymore, preferring to use the gmail web interface, for both my personal gmail account and google apps for domains email accounts. This is for two main reasons: my gmail account is so massive that the mail app on OSX struggles with it – it ends up contstantly syncing and you are never quite sure if it is up to date, and also in the past I have ended up switching machines so frequently that it was just a pain trying to get everything set up. I actually get on fine with the Gmail web interface and would be lost without its search!

Front end debugging: Firebug and chrome developer tools

I still think firebug is the most important tool in a front-end web developers toolkit. Chrome developer tools does exactly the same thing, but my familiarity with firebug stops me moving over to chrome full-time, along with possible caching issues I once experienced with JavaScript generated DOM elements.

So what is it that I do, exactly?

Posted on by Rick Hurst

different stuff that Rick Hurst does - freelance web developer, olivewood, kudos, lightplanet, foundry

If you ask any of my family or friends what it is that I do for a living, you will most likely get, at worst, a blank look, or, at best, maybe “web designer?”, “computer programmer?”, or maybe just “something to do with computers?”. It doesn’t help that my linked in profile has me claiming to be doing at least three different jobs at a time, and that I seemed to be involved with a number of different companies. I think possibly even those that work with me aren’t entirely sure what it is that I do most of the time! I thought i’d clarify what i’m up to at the moment, and how things have evolved over the last few years.

I am a freelance web developer

Primarily I make my living as a freelance web developer. This means that I spend most of my time building websites. By “building” a website I mean taking it from an idea to a finished website – planning, designing, templating, coding and uploading to a web server so that it is live on the internet. As well as web “sites”, I build web “applications”. These are computer programs that people interact with through a web browser. A web browser is the computer program that you use to look at websites e.g. internet explorer (the blue E), firefox (the fox wrapped around the world) or maybe safari (the compass). As well as looking at websites on the internet, you may be using a web browser to use web applications in your workplace.

But am I a designer?

I occasionally design websites, but it isn’t my specialism. If budget allows I prefer to hire a really good designer to come up with the design concepts. I know lots of good designers. I take their design concepts and run with it to put all the technical stuff in place to take it from an idea to an actual website. When I work for a design agency, I am hired always for my technical skills, never for my design skills – let’s face facts, I think there’s a reason for that! Most of the work on my “portfolio” wasn’t designed by me – I usually just did the technical bits.

Do I do any computer programming that isn’t related to web sites or web applications?

Increasingly more. I seem to be doing more and more data crunching these days, involving writing scripts that move files around on servers and extracting data from different places and putting things in databases. I would call that programming. Most of this type of programming is actually using skills I learned while building web sites and web applications, but some of the things i’ve built aren’t web sites or web applications at all.

So am I actually freelance, or do I work for a company?

By freelance, I mean that I work for myself, but companies and organisations employ me on a freelance basis. Therefore sometimes I may appear to be working for several companies at once. Sometimes I work directly for my own clients, other times transparently as an additional resource for a design or digital media agency. Either way, I invoice for the work I do on either a fixed price or a “time and materials” basis, and this is how I make a living.

What is Olivewood then?

In 2007 I co-formed a company called Olivewood Data Technologies Ltd, and all my freelance work is invoiced through Olivewood. Olivewood is co-owned by a client and friend, and we set it up primarily as a vehicle for consulting and development services, but also as a legal entity to own the IP for a number of niche eCommerce web applications that we plan to sell to other companies. At one point we thought Olivewood might be a digital media agency, but now we are pretty sure it is a software consultancy. [Update 2014 – Olivewood is now a supplier of high power LED lighting specialising in lighting for cold storage warehouses – how things change!]

And what about Foundry?

In May 2010 I sat on the watershed balcony with fellow freelancers Dan Fairs and Dan Hilton and talked about teaming up to be able to take on and pitch for projects bigger than we could handle as individual freelancers. We came up with the name Foundry, and shortly after collaborated on a successful project together. We all still work as freelancers, but hope to spend more time working together under the Foundry banner in 2011 and beyond. The challenge is moving the focus from looking after our own interests and incomes to working together, and to do that we need a big project that would keep us all too busy to take on other freelance work.

Will I fix your computer?

No I flippin’ wont! Have you tried switching it off and on again?

A few tips for running a web forum

Posted on by Rick Hurst

I recently stepped down from my voluntary role as one of the Administrators for a popular skateboarding web forum (for old gits), due to time commitments (i.e. I couldn’t commit any). I thought I would share a few tips I picked up from that and other forums i’ve implemented. The forum in question uses phpBB, but I think some of my observations will be useful for users of other forum software.

Choose your hosting carefully

A high volume web forum requires more availability than your average website, as regular users accessing it around the clock can get pretty upset when the forum is running slow or timing out. Physical location of the hosting can make a difference too – the host of this particular forum seemed to be having routing problems, with the forum unavailable for days at a time for some people while fine for others. Users complained a lot, and people complained that they couldn’t get to the forum to complain, and that sometimes the forum timed out while they were complaining, resulting in them posting their complaints multiple times!

Over xmas, the (windows) host was infected by a virus, which resulted in the server being rolled back with no warning several days to the point in time before it got infected. This meant that the forum lost all content in the meantime. I had my own nightly off-site backups (i’ll touch on this later), but because this happened over xmas and wasn’t keeping an eye on the forum, people came onto the forum and posted before I had a chance to restore it from one of my backups, so there wasn’t really much I could do without a lot of gruntwork, to merge it all back together, as there would have been problems with duplicate id’s.

I don’t want this to be a “linux vs windows” argument, because a linux server could also be hit by a virus and/or hacked, but I suspect – wrongly or rightly – that it’s much more likely to happen on a windows host. Apparently this one was infected by a customer with ftp access.

Server resources are also important for a forum, particularly if the server is on shared hosting. The phpBB based forum I administrated would frequently grind to a halt, with no real way of telling what the problem was without direct access to the server (as it is on shared hosting). The search index (mysql) table for the forum is massive (i.e. hundreds of thousands of rows) and this is searched everytime anyone hit the site, not just by people making searches, but to display things like “posts since last visit”. This also sais something about the efficiency of the phpBB software – it would be interesting to compare the efficiency with other systems once the search index has reached this sort of size.

I didn’t choose the hosting for the forum in question, but it was chosen by someone (non-techie) on the basis that they host several other (static) sites with the company and never had any problems. Running a web forum is a different ballgame to hosting static sites. People hardly ever complain when they can’t reach a static site, and a static site is less likely to go down because there are less failure points, and less resources needed.

Forums often have email notification functionality, i.e. people can choose to be notified when something is updated – this is also another point of failure on an inadequate host, such as this one.

I wasn’t going to name and shame the host, but they haven’t been too helpful in resolving the problems so I think in the interests of the public I would advise people not to use this company for a high volume web forum.

If your host turns out not to be suitable, you will need to move it somewhere else which can upset forum users even more, as it inevitably takes a few days for DNS servers to settle down.

If it’s not essential, block the forum from search spiders

This won’t be suitable for every forum, but if you don’t want or need the forum content to be indexed by search engines, use robots.txt to exclude the forum directory. I did this after being hit by the infamous phpBB virus which used google to search for phpBB based sites to take down. I reasoned that this alone would help hide it from repeat attacks. It has also proven fairly useful to hide from spambots that use search engines to find victims, and makes the forum less attractive to those seeking to improve their pagerank by having links on the forum.

remote backups

I mentioned before that the host server got hit by a virus resulting in the machine being rolled back several days – database backups stored on that machine would have been lost too. I used a mysql server client and cron job running on a linux machine to keep 30 days worth of remote backups in the form of date stamped sql dump files. If I had managed to disable posting on the forum as soon as it had been rolled back I could have restored remotely from one of the backups and only lost a few hours worth of content.

Don’t make people think

As the majority of the users of the forum are non-techies, they can have trouble with certain features such as posting images and links etc. The more features you give them, the more requests for help you will get. Only enable what you don’t mind supporting. A good example of this is enabling HTML in posts, then requiring people to select “disable HTML” for a post where having HTML could cause problems (e.g. the built in phpBB code that is usually used for formatting).

Avatar Size


One modification I did make was to use CSS to limit the area shown of an avatar. This was in response to people huge avatars and distorting the page, then either complaining, or not understanding when they were asked to limit the width of their linked (i.e not hosted on the server) avatars. Since I made this very simple hack, people tend to understand what they need to do to make their avatar display properly. On systems where avatars are uploaded to the host and resized, this isn’t a problem.

Modification hinders upgrades

The more you modify a piece of forum software, the more difficult it is to upgrade and apply security patches.

Single sign-on and spam handling

On a seperate forum where we (we = netsight) have integrated the sign-on (and sign-up) for an Invision Power Board (IPB) and a Plone site, we have just started having problems with spam, because the default plone sign-on allows people/spambots to bypass the anti-spam measures, and the user management features built into IPB. I should point out that anti spambot mods are available for Plone.

Beware banning IP addresses, and especially ranges of IP addresses


Some users share blocks of IP address, i.e people using a particular ISP. Banning an IP address or range of IP addresses can have the knock-on effect of blocking a whole load of innocent users. As people without fixed IP addresses get a different IP address virtually every time they log on, and spammers are well versed in spoofing and changing IP addresses, using IP addresses to block individuals is largely pointless, unless you can identify a persistent spammer who happens to have a fixed IP address.

Spam, spam, spam

Spam is the biggest challenge in running a public web forum, and it is getting more difficult. Multiple guards (Captchas, email verification, javascript foo, concealed weapons and lie detectors) should be used where possible. The more popular your web forum software is, the more it will be targeted (but also patches and mods will appear quicker)
Anyway, just a few noteson my experiences. I’m keep to hear other tips on this subject and to hear tips from people using a recent version of PloneBoard which I am about to start using for a commercial project.

archived comments

I would love to hear about your Ploneboard experiences. I have put it on a few websites for smaller needs but am really reluctant to migrate much bigger websites and boards to it as I cannot find a example of high traffic, large installation that is running well.

The guys over at livemodern.com seem to have the biggest Ploneboard installation but also seem to be going through huge growing pains (bugs, speed issues, etc).

Watching that website alone has kept me from using Ploneboard more.

BZ

BZ 2007-02-07 13:33:43

Excellent piece – ta. I facilitate online forums and have an entirely social aspect to them, but this is really handy.

Ed 2007-02-15 15:35:31

Hi Rick
Found this post of yours. We are getting ready to relaunch something that predates web forums – CIX Conferencing. Would be interested to get you to take a butchers if you are interested. This is web accessible but has always been a closed community. Drop me an email if you’d like a look.

Graham (MD)

Graham 2008-02-18 19:41:45