Watershed 2011 rebuild

screen grab of watershed.co.uk

Last night the new version of the watershed website was pushed live. I had the pleasure of being one of many people involved in this project, which involved combining several different sites representing different projects within the watershed brand. I did the “first cut” of the HTML/CSS, working from a PSD provided by the design agency Document, and also helped with some of the Drupal theme integration, working alongside some talented watershed staff and other freelancers (i’d name them all here, but would inevitably miss someone).

A few gotchas when your Drupal site is being deployed behind caching servers and proxy_pass

I recently launched a Drupal based site that forms part of the website for a global company (being a freelancer working through a design agency with an NDA, I can’t talk about it any more than that, or stick it on my portfolio unfortunately!). I though i’d make a few notes about some of the issues I had to overcome.

The site itself is hosted on a dedicated VPS, but served to the world through akamai caching servers, which means that everything is cached by default. Therefore in this set-up, the CMS is only available at a different URL, where caching is bypassed. Gotcha no.1 is that cookies set and read via PHP do not work in this scenario*. Fortunately Javascript cookies can still be used.

In addition to the caching, the site is served up as part of a much bigger site /deep/down/in/the/url/structure, so proxy_pass is used (before caching) to rewrite the paths. Gotcha no.2 is that base_path() in drupal picks up the path of the origin server, so I had to add a condition like this in my settings.php (excuse wrapping, really must sort this site out for code samples):-


if($_SERVER['HTTP_X_FORWARDED_HOST'] !=''){
$base_url = 'http://'.$_SERVER['HTTP_X_FORWARDED_HOST'].'/path/to/my/proxied/site';
}


The clue to gotcha number 3 is in that last example. $_SERVER[‘HTTP_HOST’] reports the host of the origin server, rather than the public host, so if it is used anywhere in your code, it may cause issues. I ended up adding a function getHost() that I use to return the appropriate host, depending on where the site is being viewed (once again excuse formatting):-


function getHost(){
$host = $_SERVER['HTTP_HOST'];
if(isset($_SERVER['HTTP_X_FORWARDED_HOST'])){
$host = $_SERVER['HTTP_X_FORWARDED_HOST'];
}
return $host;
}


Gotcha No.4 was image paths in optimised css – both the Drupal-provided CSS caching and some external stuff I had using minify – these both rewrote the image paths to use those from the origin server. This meant that CSS applied images worked in the non-cached editing environment but not on live. I haven’t come up with a solution for that one yet, other then to leave some of the stylesheets non-minified.

* maybe there is a solution to this, but assume in this case there is little or no scope to request server config changes.

Create a Drupal content type with no body field

It is possible to hide the body field of a drupal content type by doing the following:-

Administer-> Content Management -> content types
Click “Edit” against the content type for which you want to hide the body

Expand the “submission form settings” section

Remove the Text in the “Body field label” (it tells you to do this in the help text for this field)
Save.

These instructions are relevant to Drupal 6

Drupal 6, XAMPP and MAMP

I generally use XAMPP for local development, as I like to keep my dev environment portable and seperate to anything native on the machine. I discovered that Drupal 6 is not compatible with PHP 5.3.x, therefore I needed to either downgrade (not sure how/if I could do that with XAMPP) or use something else. I got round this by installing MAMP, which by default uses different ports to XAMPP, therefore they can be run simultaneously, which is really handy if you need to keep switching projects.

This also means that you end up with 2 mysql servers running, but because the mamp one is on a different port, you can still use the one controlled by xampp.

CMS strategies

When it comes to Content Management Systems (CMS’s), i’ve been “round the block a few times” so to speak, having spent the last ten years mostly working on CMS driven sites – starting with bespoke classic asp/ MS Access (shudder) back in 1999, before moving onto bespoke php/mysql systems, wordpress, drupal, zope/plone and dabbling in countless others. My conclusion: there is no holy grail, because the people developing and maintaining the code and content vary, and there are other important factors. Therefore I always take time to decide on the best approach for each project.
It is rare that I get a “greenfield” project where I am free to start from scratch. Without getting right down to weighing up the pros and cons of individual systems, here are some of the factors that help me (and my clients) make a decision:-

  • Will I be starting from scratch or building on something else?
  • Am I working alone, or with one or more other developers?
  • Does the client have a preferred technology i.e. if they already have an intranet written in a particular technology, it makes sense to use the same for their website. (Unless of course they hate their other systems – it might be a good bargaining point to go a different direction altogether!)
  • What’s the content like? – is there a large amount of text content, categories and pages or is it “snippets” of text that need to be editable, amongst otherwise highly graphical content?

Commercial Content Management Systems
I’m not going to talk about the “big” commercial CMS’s – the only people they are useful to are the people who’s job it is to go and buy a CMS license, companies who own or sell the software and companies who have made the investment to become specialists in such systems. That counts me out, and as i’ve never heard a developer, designer or content manager singing the praises of such a system (entirely the opposite), i’m going to presume they are all overpriced rubbish, and talk about what I know! I know that sounds flippant, but I refuse to believe that any of the vendors of the expensive, closed commercial systems have solved problems that a community of thousands of people working on competing open source systems haven’t. Also money saved on an expensive license can be spent on support, maintenance and customisation of a system with no license costs. Which brings me onto…

Off-the-Shelf open source CMS’s
One obvious route is to use one of the mature, all singing and dancing systems like Plone (zope/python) or Drupal (php/mysql). They are easy to install and have a comprehensive feature list, but usually one other thing in common – voodoo. By voodoo I mean a whole load of essential code deep down in the architecture that your average web developer (counting myself here) isn’t supposed to touch, and can spend hours/ days working out how to hook into it to make seemingly simple changes. For someone who becomes an expert in the system, they have full access to the code, so become familiar with the hooks and this isn’t a problem. For the charity/ company/ organisation where the development budget has run dry, it can be tricky finding anyone (available) who can work on it. If the content maintainer likes the admin interface these can work out really well, if they don’t it can be a ton of reverse engineering to make it “work” for the content manager.

Bespoke CMS’s
On the other extreme you have a bespoke CMS written from scratch giving the developer fine grained control over everything they need to do, and easily picked up by another developer, provided that it is sensibly written. However, as developers usually have their own way of doing things, it may take another developer time to learn how the system works before they can make changes, or they may want to re-write it altogether! The downside of any bespoke system is that it can take months to build features that you get for free in an off the shelf system. I often build a bespoke CMS where I need to bolt new functionality onto an existing system, where there isn’t enough budget to start from scratch.

Bespoke CMS built on a framework
In between are bespoke systems built on a framework (such as Cake PHP, Ruby on Rails, Django), so much of the repetitive coding work (such as user management and admin forms) is handled for you, but it’s still a bespoke system. The downside of these is that you need a developer who already knows, or is willing to take the time to learn the framework. The upside is that new functionality can be added quickly and in a consistent manner by someone familiar with the framework.

“In-House” CMS
By which I mean a bespoke CMS which isn’t written from scratch each time, but a mature “tried and tested” bespoke CMS built and maintained by an agency or developer, then reusued or repurposed for each new project. The advantage of this is that the developer/ agency have a high level of familiarity with the system, so are able to make fine-grained changes as required, that they may not be able to make with other systems. The disadvantage is that these can be the trickiest systems to be picked up by another agency or developer, and their may be IP issues (see below)

Intellectual Property / licensing
I’ve already said that i’m not going to talk about the commercial CMS’s, but one consideration when choosing which way to go, particularly with a bespoke CMS, is what happens if you were to move to another developer or agency? Will the source code and database files for your CMS be handed over along with the website or are you going to have to license it /buy it/ start again? Also what would happen if you wanted to sell the website and CMS on at a later date?

Standalone vs Retrofitting
This refers to the last of my bullet points above – not all websites are the same. Some (like news or university websites) are text-content heavy, with hundreds or maybe thousands of pages and categories, and some websites have highly designed (as in graphic design) content.

In the latter scenario it is often better to build the site as static HTML first and retrofit a simple CMS later. The bulk of the work in building such a site will be in getting it looking and behaving correctly, and their may need to be lots of fine tuning to the layout, which is easier if you aren’t reverse engineering a theming system. Moreover, the text content may be confined to small snippets of text, or a simple news blog, which can easily be plumbed in later.

The former scenario would suit an off-the-shelf system that works standalone – i.e. one that allows content editors to start populating content before the templates are finished, which are then applied as a skin/theme later – it may even be the case that the site can’t really be designed properly until the architecture and content are in place.

archived comments

Great post Rick, and it really rings true with me. Certainly the bit about taking ages to do a simple change.

I’m sure if you invest a lot of time with a framework/CMS system then it will pay dividends however, my concern is that, if the only tool you’ve got is a hammer then everything looks like a nail…if you catch my drift!

Joel

Joel Hughes 2009-09-19 18:43:28

Sitting on the fence – Why I sometimes choose not to use Plone in favour of Drupal or WordPress

As an experienced Plone front end developer, people are often surprised when I often decide not to use Plone, in favour of something like Drupal or WordPress. I thought it would be useful to explain why and how I make this decision. I know some of these points won’t be popular in the Plone community, but they are based on experience, and think this blog post will be useful to people deciding whether to use it or not.

Plone is complex
From a development point of view, Plone is a complex piece of software, at least compared to something like WordPress. I’m not going to go into specifics here, but for someone like myself who is primarily a front-end developer, sometimes I can lose days trying to add a simple custom feature, that I know I could create in hours on a php based site, probably just with a few lines of code in the template (a practice that is often frowned upon, but usually harmless and more importantly gets the job done). However much of this is down to my skillset and understanding (and probably my natural abilities to a certain extent) which brings me on to the next point.

Development resources – a chicken and egg situation
I have to hold my hand up and admit to being a bit of a “web monkey” when it comes to development – i’m self-taught, beginning with hacking around classic asp and php scripts and moving onto object orientated programming only in the last few years. With Zope and Plone the entry bar is higher – there is a whole framework to learn if you want to develop custom features, which is a good thing from a maintainability and system architecture point of view, but there is a significant time investment involved for someone wanting to transfer from something like php. Even an experienced Python developer will have some learning to do to get to know Zope and Plone.

This means that I struggle to find local freelance resources to work on a large Plone project with me. I also struggle to find sysadmins with zope/plone experience, and not being a sysadmin myself have found myself in a few hairy situations. I know of at least two ventures that have got into difficulties because of resource issues – I think partly because Plone can be a victim of it’s own success – there are lots of Plone developers, but they are all busy!

Hosting
Plone requires specialist hosting – or rather you can’t run it on the majority of locked-down budget shared hosting. Many of my clients already have hosting arranged and don’t want to move, so something like WordPress suits their needs better, for a simple site.

Overkill for simple sites
Sometimes a site only needs to be dynamic for the sake of a blog/ news section – I find plone is overkill for this, and often prefer to go with something more lightweight, even (gasp!) a bespoke CMS when I need to bolt extra functionality onto an existing site, and there is no budget for a rewrite.

So when would I use it?
Without a doubt Plone makes a fantastic intranet, out of the box. I eat my own dogfood here – I use an unmodified Plone 3 site for my own company intranet and document management system. Drupal/ WordPress do not even appear on the radar in comparison for this task. I would also use it for any site/ application that has a need for complex workflow, membership and groups.

Lastly, i’m happy to use it for any project where I can team up with an experienced zope/plone techie to help with the more low-level stuff. I was recently blown away by seeing how an experienced Plonista at Team Rubber quickly dealt with some temporary extremely high read/write traffic on a Plone site by firing up a whole set of zeo clients on amazon EC2 – a scenario that i’m sure would have been a nightmare to deal with on a drupal or wordpress site, and i’d be equally out of my depth without help.

In summary
I hope this is useful and doesn’t upset the Plone evangelists too much. Plone is a serious, well architected, secure system that leaves the competition standing in most cases. I think by “competition” I am talking about proprietry CMS and intranet systems that cost ten or hundreds of thousands of pounds, not the likes of Drupal or WordPress. If you don’t want to do much customisation it makes a good choice for smaller projects, but if you do you are going to need a developer (or invest in growing one of your own), who is a. available, b. you can afford, otherwise you might be better to go with something else, with a lower development entry level.

archived comments

This is a nice write up. Sounds completely reasonable to me.

Kai (a Plone developer)

Kai Diefenbach 2009-05-08 11:20:57

True words..

I use plone for my intranet/dms, too. Maybe u can share your concepts/ideas or how u use it and for what..would be cool.

Greets
Gomez

Gomez 2009-05-08 12:06:43

I agree, right on down the line. And if you have not already, you should consider blogging about how you’re using Plone as a DMS.

Rose Pruyne 2009-05-08 13:36:19

Great summary of everything I have been through since deciding on Plone at my day job about a year ago.

I don’t think even the most fervent Plone fan would advocate using Plone for everything. (Though I am sure there are people skilled enough to customize it to point will work for anything.)

Our major problem has been what to use when Plone IS overkill. We are a 3-person team and also learning Python at the same time. We are a little hesitant to take up another new framework (like Django) but at the same time it doesn’t seem to make sense to have some stuff on PHP and some stuff on Python. (And 1 of our team members would have to learn both.)

Anthony Bosio 2009-05-08 13:50:16

@Anthony yes, it doesn’t make a lot of sense to span php and python/zope/plone, unless like me you pick up different types of work from different clients and knowing both is quite useful. I guess Grok http://grok.zope.org/ would be a sensible choice in your situation because it keeps things python/zope 3

Rick 2009-05-08 13:57:45

Great, thoughtful, piece, Rick. If all you want is a few static pages and a blog, Plone is indeed overkill. The good news is that I think you’ll see the learning curve getting smoother over the next year as we simplify the content types, theming and page layout stories. I think this will put a lot more power into the hands of folks like me who haven’t mastered all of Plone’s innards.

The Plone community has learned a lot from both its successes and its mistakes over the past 8+ years, and we have a lot of great innovation in the pipeline.

Jon Stahl 2009-05-08 15:12:18

Why do you think Plone evangelists would be upset by this? I think most of them would (or at least, should) agree whole-heartedly with this analysis. In my opinion, Plone doesn’t even compete with WordPress and barely so with Drupal. As you said, – “overkill for simple sites”.

Martin

Martin Aspeli 2009-05-08 15:36:49

@Martin I guess the point I thought might be controversial is the bit about “getting into difficulties” with a plone site, by which I mean either a situation where a site stops altogether working one day (e.g. because of a problem with a corrupted ZODB or something like that), and you can’t get it back online without external help or a project becomes stale because a developer leaves and you can’t find anyone to replace them. I know this could potentially put people off using it.

Rick Hurst 2009-05-08 16:18:04

Fully agree, I use Django and CherryPy for simpler sites as my staffs will be still in Python environment. Switching different languages between projects can be counter-productive.

Michael Ang 2009-05-09 03:12:33

Good article. I agree with your opinion.
I quoted your point of view in my Japansese blog post.
http://www.shigeo.net/Computing/090509-not-always-plone

Maximum file upload size in Drupal

In a Drupal (5.something) site, there is a section in the admin for specifying how file uploads should be handled. As well as specifying the maximum file size allowed for a single upload it reports what the current php settings are e.g:-

“Your PHP settings limit the maximum file size per upload to 4 MB.”

This can be confusing! I just set the max_file_upload in my php.ini to 20MB and drupal was still reporting 4MB. After a bit of fishing around I found out there are two factors to this. First look for max_file_upload in your php.ini and check that that is set to something sensible, then search for post_max_size (also in php.ini) – it seems that drupal is reporting this value divided by 2

drupal permissions – remember input format

After banging my head against the desk for a while this morning, I worked out that the reason I could edit a page as the admin user, but not a “editor” user I had set up, was that I hadn’t allowed my editor user to use the “full HTML” input format, which had been used when the page was created (by the admin user). Enabling this input format for all users opened up the content to be editable how I needed it.

Drupal on a different port – access denied

(This is just for my own reference as i’ve hit this twice now – and can never find anything on google). if you run Drupal on a non standard port (e.g. on my windows machine I run apache on port 6060 so it doesn’t clash with port 80 hog IIS), you may find that you can’t log into a drupal site. This may be because of the way cookies are set up for domains. You can get round it by going to settings.php and looking for the section that is preceded by the comment:

/**
* We try to set the correct cookie domain. If you are experiencing problems
* try commenting out the code below or specifying the cookie domain by hand.
*/