Rick Hurst Web Developer in Bristol, UK

Menu

Category: plone

plone conference 2010 day 2

Posted on by Rick Hurst

Plone conf day 2

Some notes and thoughts from day of plone conference 2010. These are mostly in note form, but wanted to get these online anyway, while they are fresh in my mind.

fixing the ungoogleable by Elizabeth Leddy

What happens when something breaks, and a simple google search doesn’t offer any results?

First, warn people (so they don’t bother you while you fix it, and they don’t panic).

Work out how quickly can you access your backups? I think this is very important – personally for smaller sites i’m doing individual site backups as packages mainly to make it convenient to restore them in a local dev environment, but I know some people are relying on whole server backups on tape. This would be laborious to restore in an emergency.

Isolation by elimination -> network, hardware, software, data
map out your system!

Eliminate things by switching them off – write your code so that it can handle dependencies being switched off.

Set up a system so that you have an isolated instance (so you can look at logs of only your own activity, rather than mixed with everyone elses)

Create a dashboard (maintenance page)

Monitoring

Write Test case’s to diagnose faults

Start writing a “help” email, but don’t send it. This is apparently known as “rubber ducking”. I do this all the time, I often spot my mistake when I past in a traceback to an email message to send to someone!

“horizon of intervention” – At what point do you need external help?

Get to know people with specialisms and buy them a beer! Participate in the plone community (I really need to do more of that).

Create tickets and close them as you go

Document your processes.

The state of plone caching by Ricardo Newbery

cachefu – useful tool, but as far as “internet time” goes, this is a little long in the tooth now (started 2006). End of life – critical bugfixes only

plone.app.caching – will be in future plone releases, but available now

load testing with funkload (not accurate regarding CSS/ image discovery)

load testing with multi-mechanize

BrowserMob – not free

Building a custom app with plone with minimal development by Eric BREHAULT

This was extremely interesting – Eric is the project manage for Plomino – a plone add-on that provides application development toolkit for creating database applications within plone – i.e. the types of database applications that non-technical people might create on there own machines with Lotus Domino, Filemaker or Access. The data is still stored in the ZODB, but models and forms can views can be created through the web. Formulas are done in python.

I can see this being really useful – all too often I end up building custom applications based on complex access database or excel spreadsheets i’ve been provided, but if I can persuade clients to use Plomino, then not only will it help get that data in a format suiteble for building a web application, but data can be collaborated on across teams, rather then emailing around (and inevitably forking spreadsheets and databases). There were some nice examples of data visualisation – a great quote “almost useless, but very nice”” – you need these to impress your boss.

Themeing with XDV (Diazo) Laurence Rowe

I wrote some notes on this yesterday, connected to Nates Deliverance talk, so won’t go into this here, other than to quote Laurence: “we write XSLT, so you don’t have to”

collective.amberjack: chapter one. The interactive age. Massimo Azzolini

Amberjack is a Javascript library for creating a site tour/ tutorial. Collective amberjack wraps this up as an add-on for plone. Interactive tutrials can be created.

An example was given using the windmill testing framework (windmill in itself looks nice alternative to selenium).

The Art of Integrating Plone with Webservices with David Glick

Most of this was over my head, but one important note I made from this is about urlib (which I use in a django screen scraper app i’m developing) can have two possible error responses – URLError and HTTPError – two possible error responses.

external ecommerce and plone playing along with Sasha Vincic

There seems to be a bit of a theme going here – an acknowledgement that Plone works out best long term if you use it as a “black box” CMS, and don’t try to do everything with it. The upgrade path is easier if you don’t add on your own customisations – “clean plone”. The current plone ecommerce offerings are not as good as external systems, so it is better to integrate with an external system, which is also then kept clean to allow easier upgrades.

In the python world there are some Django Ecommerce stores LFS and Satchmo, but the store doesn’t have to be Python – other proven systems such as magento can also be integrated with Plone.

To integrate with plone you need to integrate search, linking, thumbnailing. Valentine achieved search compatibility by creating objects in plone via an RSS import – see valentine.rssobjects. A latecomer to the talk asked “but which plone ecommerce product would you use if you had to?”. Answer “we wouldn’t”.

collective.transcode.star (lightning talk)

manage online transcoding
http://plone.org/products/collective.transcode.star – looks interesting, and may bring plone back into the picture for a project i’m doing.

BlueDynamics bda.plone.finder

osx style finder widget for navigating/ organising content in plone – looks great!

Plone Conference 2010 Day 1

Posted on by Rick Hurst

quality schwag from Plone Conference 2010

Just back home after day one of Plone Conference 2010, with my mind buzzing so thought it would be a good time to write up some of my thoughts and notes. It was really difficult to choose between the talks on offer on the three different different tracks, but here are some thoughts on the ones I attended.

Keynote by Alexander Limi and Alan Runyan

Two main themes here – ubiquity/availaibility and designer friendliness.

To make Plone more mainstream it needs to be available to non-technical end users through the same means that other systems are already available – namely being able to deploy easily on cheap hosting, specifically the one-click installers on shared hosting in cPanel and similar. This would allow users to easily evaluate Plone for their needs in the same ay that they can already with wordpress, drupal and joomla – apparently there is a new joomla instance created about every two minutes. I must stop Alan Runyan and see if he has thought about microsoft web platform installer – nowadays this includes the option to install wordpress, drupal, modX, and load of other systems, including downloading and installing dependencies. It would be great if Plone was in that list.

One of the aims of Plone 5 is to make it more designer friendly. I think this is really important – even though since the release of plone 4 i’ve started using plone again for intranets and extranets (mainly straight out of the box with a few minor cosmetic tweaks), I currently still use something like wordpress, or a home-rolled CMS for website builds. That is now going to change – the theming story is being completely re-written by the introduction of Deliverance/XDV/Diazo (already available – more on that later), and Deco (TTW layout and content editing). The aim is to make Plone appeal to designers as something that helps, not hinders them.

Quote of the talk has to be from Limi – “Plone doesn’t suck, because the developers don’t hate the core technology” (or something like that) – in reference to the revelation that many drupal/wordpress/joomla developers admit they actually hate PHP, whereas Plone developers love python.

Deco: new editing interface for plone 5

The next talk I attended was Rob Gietema’s demo of Deco. This is looking really good, although i’m a little bit skeptical of drag and drop and in-place editing (I like front-end based editing, but prefer lightboxed modal editing to in-place), mainly because i’ve seem layouts explode and page elements disappear, or refuse to drop in the correct place on similar systems in the past. However, I haven’t actually tried this one yet, maybe i’m just clumsy! I think in general designers and content editors are going to love it.

LDAP and Active Directory integration

I attended Clayton Parker’s talk on LDAP and active directory integration – can’t say I absorbed much, but i’m sure i’ll be asked to do this one day, so it’s good to know that this is tried and tested and the tools are already there.

Easier and faster Plone theming with Deliverance and xdv

Nate Aune gave us an overview of Deliverance. I’ve known about Deliverance for ages, but the penny dropped for me today about how useful this is. The basic principle is this – deliverance acts as a proxy to transparently take HTML output from a website and merge it with HTML from a theme, according to a simple set of rules. In the case of plone, this means you can create a theme in static HTML and have content from a default theme Plone site displayed wrapped up in the static HTML. Simple rules can be applied e.g. “take the news portlet from the plone site, drop the header and footer and all the images and display in the element with and id of “recent-news” from my HTML theme. magic!

Nate quoted one example where the HTML theme is stored in a dropbox folder which the client has access to to make tweaks and changes. I can see front end developers and designers loving this.

There was much discussion at the end over which technology should be used for this – XDV is a fork of an earlier version of Deliverance, which has slightly different functionality. XDV, which is to be renamed Diazo, will be the theming engine for Plone 5. With that in mind, i’ll concentrate my efforts on Diazo. I’m excited by this for non-plone reasons – a majority of my works seems to involve integrating technologies that don’t belong together – this will really help.

Design and development with Dexterity and convention-over-configuration

Martin Aspelli gave a talk on dexterity – the (eventual) replacement for archetypes. This is already available, but not mature yet. The talk was mainly conceptual rather than code-led, focussing on best practice for designing your site or application – when it is suitable to create a content type, and when you might be better off creating a form, or using a relational database. Best quote “code is like a plastic bag” (reduce, reuse, recycle). Write less code.

Laying Pipe with Transmogrifier

Another talk from Clayton Parker – transmogrifier is a system to package up migrations of content from other systems. My thoughts on this were that it looked like hard work for a one off import (usually i’d write a one-off python script for something like this), but creating packages would benefit the plone community e.g. if there were packages available covering migrations from a standard wordpress, drupal or joomla, this would benefit plone. I suppose this could also be used to import content from older instances of Plone, where the upgrade path is broken.

Multilingual sites – caveats and tips

Sasha Vincic talkd about strategies and gotchas for multilingual site builds. Even though Plone has tools for this, there are common scenarios, such as the “missing page” scenario where a translation of a site may not have the same number of pages as the base translation. He also covered common issues such as escaped HTML being translated by third parties and being delivered content where HTML attributes have been translated, therefore breaking the HTML.

Guest Keynote: Challenging Business

This was an inspirational treat for the end of day one – Richard Noble is a fantastic speaker, and after a day of CMS talk it was great to hear his story of the challenges of his past world land speed record record achievements, and the current one – the Bloodhound SSC project. As well as building insane rocket powered cars (the current one has an F1 engine onboard just to drive the fuel pump!), his goal is to inspire children and young people to become engineers, as there is an impending massive shortage of engineers in training. I was also interest to hear that there will be no patents on the technology developed for the new car – the advancements will be made avaiolable for anyone in the engineering industry to build on – sound familiar?

archived comments

Thanks for the roundup Rick! Looking forward to the rest.

Mike 2010-10-27 21:29:18

Good, clear writeup. I could not make it to the conference this year. It’s nice to be able to follow it a bit this way. Thanks, Rick!

Maurits van Rees 2010-10-27 21:42:13

Thanks for the comments guys – videos for all the talks will eventually be online, so hopefully everyone who couldn’t make it won’t miss out 🙂

Rick 2010-10-27 21:58:26

Thanks for the post!

I agree with you re: transmogrifier which is (partially) why I wrote http://pypi.python.org/pypi/parse2plone

Keep the blogs coming 🙂

Alex Clark 2010-10-28 00:50:01

CMS strategies

Posted on by Rick Hurst

When it comes to Content Management Systems (CMS’s), i’ve been “round the block a few times” so to speak, having spent the last ten years mostly working on CMS driven sites – starting with bespoke classic asp/ MS Access (shudder) back in 1999, before moving onto bespoke php/mysql systems, wordpress, drupal, zope/plone and dabbling in countless others. My conclusion: there is no holy grail, because the people developing and maintaining the code and content vary, and there are other important factors. Therefore I always take time to decide on the best approach for each project.
It is rare that I get a “greenfield” project where I am free to start from scratch. Without getting right down to weighing up the pros and cons of individual systems, here are some of the factors that help me (and my clients) make a decision:-

  • Will I be starting from scratch or building on something else?
  • Am I working alone, or with one or more other developers?
  • Does the client have a preferred technology i.e. if they already have an intranet written in a particular technology, it makes sense to use the same for their website. (Unless of course they hate their other systems – it might be a good bargaining point to go a different direction altogether!)
  • What’s the content like? – is there a large amount of text content, categories and pages or is it “snippets” of text that need to be editable, amongst otherwise highly graphical content?

Commercial Content Management Systems
I’m not going to talk about the “big” commercial CMS’s – the only people they are useful to are the people who’s job it is to go and buy a CMS license, companies who own or sell the software and companies who have made the investment to become specialists in such systems. That counts me out, and as i’ve never heard a developer, designer or content manager singing the praises of such a system (entirely the opposite), i’m going to presume they are all overpriced rubbish, and talk about what I know! I know that sounds flippant, but I refuse to believe that any of the vendors of the expensive, closed commercial systems have solved problems that a community of thousands of people working on competing open source systems haven’t. Also money saved on an expensive license can be spent on support, maintenance and customisation of a system with no license costs. Which brings me onto…

Off-the-Shelf open source CMS’s
One obvious route is to use one of the mature, all singing and dancing systems like Plone (zope/python) or Drupal (php/mysql). They are easy to install and have a comprehensive feature list, but usually one other thing in common – voodoo. By voodoo I mean a whole load of essential code deep down in the architecture that your average web developer (counting myself here) isn’t supposed to touch, and can spend hours/ days working out how to hook into it to make seemingly simple changes. For someone who becomes an expert in the system, they have full access to the code, so become familiar with the hooks and this isn’t a problem. For the charity/ company/ organisation where the development budget has run dry, it can be tricky finding anyone (available) who can work on it. If the content maintainer likes the admin interface these can work out really well, if they don’t it can be a ton of reverse engineering to make it “work” for the content manager.

Bespoke CMS’s
On the other extreme you have a bespoke CMS written from scratch giving the developer fine grained control over everything they need to do, and easily picked up by another developer, provided that it is sensibly written. However, as developers usually have their own way of doing things, it may take another developer time to learn how the system works before they can make changes, or they may want to re-write it altogether! The downside of any bespoke system is that it can take months to build features that you get for free in an off the shelf system. I often build a bespoke CMS where I need to bolt new functionality onto an existing system, where there isn’t enough budget to start from scratch.

Bespoke CMS built on a framework
In between are bespoke systems built on a framework (such as Cake PHP, Ruby on Rails, Django), so much of the repetitive coding work (such as user management and admin forms) is handled for you, but it’s still a bespoke system. The downside of these is that you need a developer who already knows, or is willing to take the time to learn the framework. The upside is that new functionality can be added quickly and in a consistent manner by someone familiar with the framework.

“In-House” CMS
By which I mean a bespoke CMS which isn’t written from scratch each time, but a mature “tried and tested” bespoke CMS built and maintained by an agency or developer, then reusued or repurposed for each new project. The advantage of this is that the developer/ agency have a high level of familiarity with the system, so are able to make fine-grained changes as required, that they may not be able to make with other systems. The disadvantage is that these can be the trickiest systems to be picked up by another agency or developer, and their may be IP issues (see below)

Intellectual Property / licensing
I’ve already said that i’m not going to talk about the commercial CMS’s, but one consideration when choosing which way to go, particularly with a bespoke CMS, is what happens if you were to move to another developer or agency? Will the source code and database files for your CMS be handed over along with the website or are you going to have to license it /buy it/ start again? Also what would happen if you wanted to sell the website and CMS on at a later date?

Standalone vs Retrofitting
This refers to the last of my bullet points above – not all websites are the same. Some (like news or university websites) are text-content heavy, with hundreds or maybe thousands of pages and categories, and some websites have highly designed (as in graphic design) content.

In the latter scenario it is often better to build the site as static HTML first and retrofit a simple CMS later. The bulk of the work in building such a site will be in getting it looking and behaving correctly, and their may need to be lots of fine tuning to the layout, which is easier if you aren’t reverse engineering a theming system. Moreover, the text content may be confined to small snippets of text, or a simple news blog, which can easily be plumbed in later.

The former scenario would suit an off-the-shelf system that works standalone – i.e. one that allows content editors to start populating content before the templates are finished, which are then applied as a skin/theme later – it may even be the case that the site can’t really be designed properly until the architecture and content are in place.

archived comments

Great post Rick, and it really rings true with me. Certainly the bit about taking ages to do a simple change.

I’m sure if you invest a lot of time with a framework/CMS system then it will pay dividends however, my concern is that, if the only tool you’ve got is a hammer then everything looks like a nail…if you catch my drift!

Joel

Joel Hughes 2009-09-19 18:43:28

new sweetcron/ codeigniter based site Too Old To skate

Posted on by Rick Hurst

too old to skate

I’m pleased to say i’ve achieved another of my 2009 goals, by launching the (unfinished) new version of one of my personal projects Too Old To Skate. The main site is now running php/codeigniter based sweetcron, which is used to pull content in from the original wordpress, flickr, delicious, twitter, vimeo and a friends blogger based site. I have plenty more planned for the site, but little to no time to do it, so I thought i’d upload the site unfinished in the meantime to allow myself to do incremental updates, rather than be embarrased by a holding page.

Amongst the plans are:-

  • bring in the content (articles and photo galleries) from the now defunct plone(2) based DFR Skate zine – i’m thinking some sort of PHP/ codeigniter* front-end sucking in content (as XML/JSON or even plain HTML) from a Plone 3 site acting as a content server. Doing something like this with Plone has been on my mind for years! Maybe I should stick this in as a 2010 goal. Alternatively it may just end up as a skinned Plone 3 site on another subdomain – either would be good.
  • Skin the blog to fit in with the main site.
  • Other awesomeness – this is a non-commercial personal project and therefore my playground 😉

*Having now played with both codeigniter and cakephp, I think I prefer cake. However as sweetcron is codeigniter based it would be plain silly to be using cake on the same project.

Plans for 2009

Posted on by Rick Hurst

Inspired by Elliot Jay Stocks post “Goals, Old and New”, I thought i’d jot down a few goals* for 2009. In no particular order:-

  • Launch a web app – the original reason for starting Olivewood. We already have a working prototype (a web based trading/ procurement/communication hub – successfully in service for a distributor over a year), it just needs some refactoring to make it suitable to be released to additional clients with slightly different needs.
  • Relaunch Too Old To Skate – it’s a personal project, and it’s important for me to be working on at least one non-paying project, to remind me that i’m not just in this for the money. I’m already part of the way there with a sweetcron based mashup. I intend to launch it, unfinished or not in January.
  • Attend at least one web conference – last year was just too manic, and combined with bad timing I only managed WDC this year. On my possible list are FOWA dublin, bamboo juice in cornwall, plone conf in budapest, Europython in Birmingham and the list goes on.
  • Attend at least one barcamp. BathCamp this year was great.
  • Fix on a php framework and use it on a “from-scratch” commercial project, and get really, really good with it. Cake is the current fave, though will likely be using Zend Framework too on a magento based project.
  • Get to grips with plone 3 skinning – I want to consider myself a plone skinning expert again.
  • Get further than “hello world” with Django – ideally at least one commercial project. I’m not beating myself up too much over this, I thrive on examples, so i’m letting other people do all the early adopter stuff 😉
  • Better customer service – by managing my workload to avoid juggling projects, saying NO occasionally, and generally working in a more controlled and predictable manner.
  • Put up a proper portfolio on this site, maybe even a redesign – working on this website is always last on the list, it’s still using the default wordpress theme from way back.
  • Use and understand some technologies that i’ve been ignoring – JSON, WSGI, Zope 3
  • Do more on-site freelancing for agencies. I seem to thrive in that environment. Maybe it’s my agency background, or maybe it’s because I get to focus on the task in hand rather than the whole “business” side of being a freelancer. There’s also the whole co-worker thing.
  • Keep trading and survive the credit crunch recession (Touch wood) i’m already booked up for the first few months of 2009 – lets hope the work keeps coming!

* web/ career related – i’ll save the more personal goals for drunken new years eve conversations with friends!

archived comments

Nice one, Rick! I think that “saying no” one is a very wise tip indeed. I must learn to do more of that!

Elliot Jay Stocks 2008-12-22 16:19:42

Nice Rick. I wish you good luck and all the best for your 2009 projects.

Nicolas Alpi 2008-12-30 00:25:26

World Plone Day – Bristol, UK

Posted on by Rick Hurst

Today is World Plone Day.

From the website:

“The World Plone Day (WPD) is a worldwide event. Our goal is to promote and educate the worldwide public about of the benefits of using Plone in education, government, ngos, and in business.”

We held an event in Bristol, which I streamed and recorded via ustream – available here. I did a lightning talk on how to use plone out-of-the-box as a small company intranet and searchable knowledge base, other talks included “big plone”, contributing code back to the community and KSS.

The video was streamed straight from the webcam and built in mic on my macbook, so the sound quality isn’t great, especially when the fan kicked in, but it was a bit of a last minute effort. Next time it will be camera, mic and tripod!

Review: Professional Plone Development part 1

Posted on by Rick Hurst

Firstly an apology: I was sent Martin Aspeli’s Professional Plone Development for review back in October last year, and I still haven’t got round to reviewing it – not out of laziness, but extreme time shortage due to going freelance/ starting a company last year. Hopefully I can make amends now by reviewing it section by section as I jump into Plone 3.

A bit of background about me and Plone: I have worked predominantly with Plone for about 4 years, coming from a background in classic asp and php. My area of Plone expertise is front end templating and skinning, which was my main area of usefulness at Netsight for a couple of years. I have also presented a couple of Plone based workshops at Bristol Skillswap, attended two Plone conferences and a Snow Sprint. Despite this involvement with Plone, I have had a love/hate relationship with it, at times thoroughly lost deep down in the software stack, and fighting my tendency to think purely in terms of scripts and relational databases, other times singing it’s praises to anyone who will listen. Since leaving Netsight I have only worked on one Plone (2) project, as i’ve gone back to my old ways, favouring php for most projects, but just got stuck into an inherited Plone 3 project, and started using a vanilla Plone 3 site as a knowledge base/document repository/intranet/extranet for my own company, olivewood. (Hosted on a mac mini in-house, which I thoroughly recommend, but that’s another post!).

I’m currently feeling a little lost, with the Plone 3, which is why I decided to finally start reading/ reviewing the book.
Ok, enough about me, lets talk about the book: I opened the book last night and read through the foreword by Alexander Limi, and the first chapter. It left me with a fantastic feeling about Plone and the future of Plone – it summarised what is different about Plone, and how important (and fun) it is to participate in the Plone community to get the best out of it – the polar opposite to working with a proprietary product, where the vendor tries to cultivate a community around it, but it’s not the same. It also made me feel slightly guilty for moaning about aspects of Plone in the past – who exactly was I complaining to? I resolved to make sure that the next time something frustrates me, that I actively participate in making the situation better, i.e. to at least actively become part of the problem, if not part of the solution!

Chapter 1 covers Zope and Plone history, Plone as application vs Plone as framework, considerations for when deciding if Plone is the right tool for the job, and the importance of the community.
A couple of other things I learned from the foreword and chapter 1:-

  • The original release of Plone was built (predominantly*) by just two people, Alexander Limi and Alan Runyan who didn’t meet until Europython 2002 just after the first release
  • The Goldegg initiative is a sponsored effort to improve Plone’s framework stack, not a type of python egg!

Ok, I will try to deliver another installment as soon as possible, but i won’t necessarily be doing it in order – i’ll be skipping straight to chapter 8 (skinning)
* Building on top of Zope/ CMF – the work of many people!

archived comments

Hi!

Thanks for buying the book. I hope you enjoy it. 🙂

I would probably read the chapters in order, or at least skim them in order. You’ll find it a bit hard to grasp some of the things chapter 8 otherwise. At least, I’d read chapters 3 and 5 first.

Cheers,
Martin

Martin Aspeli 2008-09-18 22:22:29

Thanks for the compliments on the foreword. Interestingly enough, that foreword has generated more nice personal emails to me than anything else I can remember writing. 😉

Alexander Limi 2008-09-18 23:15:37

i’m joining the circus

Posted on by Rick Hurst

I once remember reading somewhere that leaving a steady job to set up your own business is the adult equivalent of running off to join the circus. But that’s what i’ve decided to do – as of June the 1st I will be my own boss, as after a lot of deliberation and heel dragging I have decided to bite the bullet and leave my contented job at Netsight to go into business with a friend of mine. I have always wanted to start my own business, but for various reasons I haven’t, and a business opportunity was put in front of me, that if I didn’t take then maybe it would never happen. At this point I wanted to link to the website of my new web application development company, but embarassingly I haven’t built it yet, so I’ll post about that when I have done it, and talk more about the new company, and the products and services we will be providing.

A few people have asked me if I will be continuing to work with Plone and the answer is basically this: The core business of my new company will be packaged web applications and these will not be built in plone, as the applications aren’t CMS-like, and plone wouldn’t be relevant. The applications will be built using some* scripting language/framework plus a RDMS. Zope 3 is a contender, but I don’t have the experience to hit the ground running with that, but i’ll certainly endeavour to get up to speed on it when time allows. However, the company will also be offering bespoke and ad-hoc development services and these will include Plone. For larger Plone projects and implementations I will refer enquiries to Netsight, as beyond the Plone skinning and customisation expertise that I have built up at Netsight over the last three years, the new company will not have the resources or expertise to support large plone projects. I will also still be contracting/freelancing for Netsight on Plone work – intially a couple of days a week, until they change the locks 😉

Anyway, new company website to follow, and more details about exactly what we will be doing.

* probably unpopular amongst plonistas – check out my previous commercial experience and fill in the blanks yourself!

archived comments

Good luck with your new venture Rick!

Andy Gale 2007-06-15 08:44:51

ah, that’ll be why they were advertising in the pest. Funnily I stumbled across your blog as the top result in googling Plone Confrence 2006. I’m looking for a good quality copy of the Eben Moglen speech.

sean 2007-07-11 22:12:25

Architen Landrell site launched

Posted on by Rick Hurst

arhciten landrell website screengrab

Netsight have been so busy recently that we haven’t updated our portfolio for a while, but I wanted to mention this site, as it gave us an excuse to experiment with some nice visual features such as scriptaculous effects, flash galleries etc. The site has a plone back end for Content Management, but the front end was built from the ground up, so is a nice example of a “non-ploney” plone-based site. It helps that Architen had some excellent photography to use on the site – all maintained by themselves via plone including image resizing and cropping for the portfolio pages.

For more details on the project see the write-up here

A few tips for running a web forum

Posted on by Rick Hurst

I recently stepped down from my voluntary role as one of the Administrators for a popular skateboarding web forum (for old gits), due to time commitments (i.e. I couldn’t commit any). I thought I would share a few tips I picked up from that and other forums i’ve implemented. The forum in question uses phpBB, but I think some of my observations will be useful for users of other forum software.

Choose your hosting carefully

A high volume web forum requires more availability than your average website, as regular users accessing it around the clock can get pretty upset when the forum is running slow or timing out. Physical location of the hosting can make a difference too – the host of this particular forum seemed to be having routing problems, with the forum unavailable for days at a time for some people while fine for others. Users complained a lot, and people complained that they couldn’t get to the forum to complain, and that sometimes the forum timed out while they were complaining, resulting in them posting their complaints multiple times!

Over xmas, the (windows) host was infected by a virus, which resulted in the server being rolled back with no warning several days to the point in time before it got infected. This meant that the forum lost all content in the meantime. I had my own nightly off-site backups (i’ll touch on this later), but because this happened over xmas and wasn’t keeping an eye on the forum, people came onto the forum and posted before I had a chance to restore it from one of my backups, so there wasn’t really much I could do without a lot of gruntwork, to merge it all back together, as there would have been problems with duplicate id’s.

I don’t want this to be a “linux vs windows” argument, because a linux server could also be hit by a virus and/or hacked, but I suspect – wrongly or rightly – that it’s much more likely to happen on a windows host. Apparently this one was infected by a customer with ftp access.

Server resources are also important for a forum, particularly if the server is on shared hosting. The phpBB based forum I administrated would frequently grind to a halt, with no real way of telling what the problem was without direct access to the server (as it is on shared hosting). The search index (mysql) table for the forum is massive (i.e. hundreds of thousands of rows) and this is searched everytime anyone hit the site, not just by people making searches, but to display things like “posts since last visit”. This also sais something about the efficiency of the phpBB software – it would be interesting to compare the efficiency with other systems once the search index has reached this sort of size.

I didn’t choose the hosting for the forum in question, but it was chosen by someone (non-techie) on the basis that they host several other (static) sites with the company and never had any problems. Running a web forum is a different ballgame to hosting static sites. People hardly ever complain when they can’t reach a static site, and a static site is less likely to go down because there are less failure points, and less resources needed.

Forums often have email notification functionality, i.e. people can choose to be notified when something is updated – this is also another point of failure on an inadequate host, such as this one.

I wasn’t going to name and shame the host, but they haven’t been too helpful in resolving the problems so I think in the interests of the public I would advise people not to use this company for a high volume web forum.

If your host turns out not to be suitable, you will need to move it somewhere else which can upset forum users even more, as it inevitably takes a few days for DNS servers to settle down.

If it’s not essential, block the forum from search spiders

This won’t be suitable for every forum, but if you don’t want or need the forum content to be indexed by search engines, use robots.txt to exclude the forum directory. I did this after being hit by the infamous phpBB virus which used google to search for phpBB based sites to take down. I reasoned that this alone would help hide it from repeat attacks. It has also proven fairly useful to hide from spambots that use search engines to find victims, and makes the forum less attractive to those seeking to improve their pagerank by having links on the forum.

remote backups

I mentioned before that the host server got hit by a virus resulting in the machine being rolled back several days – database backups stored on that machine would have been lost too. I used a mysql server client and cron job running on a linux machine to keep 30 days worth of remote backups in the form of date stamped sql dump files. If I had managed to disable posting on the forum as soon as it had been rolled back I could have restored remotely from one of the backups and only lost a few hours worth of content.

Don’t make people think

As the majority of the users of the forum are non-techies, they can have trouble with certain features such as posting images and links etc. The more features you give them, the more requests for help you will get. Only enable what you don’t mind supporting. A good example of this is enabling HTML in posts, then requiring people to select “disable HTML” for a post where having HTML could cause problems (e.g. the built in phpBB code that is usually used for formatting).

Avatar Size


One modification I did make was to use CSS to limit the area shown of an avatar. This was in response to people huge avatars and distorting the page, then either complaining, or not understanding when they were asked to limit the width of their linked (i.e not hosted on the server) avatars. Since I made this very simple hack, people tend to understand what they need to do to make their avatar display properly. On systems where avatars are uploaded to the host and resized, this isn’t a problem.

Modification hinders upgrades

The more you modify a piece of forum software, the more difficult it is to upgrade and apply security patches.

Single sign-on and spam handling

On a seperate forum where we (we = netsight) have integrated the sign-on (and sign-up) for an Invision Power Board (IPB) and a Plone site, we have just started having problems with spam, because the default plone sign-on allows people/spambots to bypass the anti-spam measures, and the user management features built into IPB. I should point out that anti spambot mods are available for Plone.

Beware banning IP addresses, and especially ranges of IP addresses


Some users share blocks of IP address, i.e people using a particular ISP. Banning an IP address or range of IP addresses can have the knock-on effect of blocking a whole load of innocent users. As people without fixed IP addresses get a different IP address virtually every time they log on, and spammers are well versed in spoofing and changing IP addresses, using IP addresses to block individuals is largely pointless, unless you can identify a persistent spammer who happens to have a fixed IP address.

Spam, spam, spam

Spam is the biggest challenge in running a public web forum, and it is getting more difficult. Multiple guards (Captchas, email verification, javascript foo, concealed weapons and lie detectors) should be used where possible. The more popular your web forum software is, the more it will be targeted (but also patches and mods will appear quicker)
Anyway, just a few noteson my experiences. I’m keep to hear other tips on this subject and to hear tips from people using a recent version of PloneBoard which I am about to start using for a commercial project.

archived comments

I would love to hear about your Ploneboard experiences. I have put it on a few websites for smaller needs but am really reluctant to migrate much bigger websites and boards to it as I cannot find a example of high traffic, large installation that is running well.

The guys over at livemodern.com seem to have the biggest Ploneboard installation but also seem to be going through huge growing pains (bugs, speed issues, etc).

Watching that website alone has kept me from using Ploneboard more.

BZ

BZ 2007-02-07 13:33:43

Excellent piece – ta. I facilitate online forums and have an entirely social aspect to them, but this is really handy.

Ed 2007-02-15 15:35:31

Hi Rick
Found this post of yours. We are getting ready to relaunch something that predates web forums – CIX Conferencing. Would be interested to get you to take a butchers if you are interested. This is web accessible but has always been a closed community. Drop me an email if you’d like a look.

Graham (MD)

Graham 2008-02-18 19:41:45