Rick Hurst Web Developer in Bristol, UK
Menu
Yesterday the www.timmytime.tv was was finally launched. I worked on this project along with the Aardman digital team building a bespoke php CMS and putting the site together to accommodate the mostly flash content with HTML alternative (where possible). XML was used to share the same data between flash and PHP, making it easier to maintain.
I use svn (subversion code version control) in a very simple way, but it has become an essential tool for how I build, deploy and manage websites. If you aren’t using some sort of source control, you should be. If you are, but you are using it only as a source code backup, you might be interested to see one of the ways it can be used as an integral part of the workflow for building, deploying and managing website updates. I’m not going to go into svn commands, this is more of an overview of the process.
This is my typical workflow, based on a php/mysql website build, but the process is relevant to most technologies:-
Create a new project in your svn repo
This basically consists of making a new directory in the root of the repo. At this stage, an experienced svn user might suggest you create sub-directories for “trunk” and “branches”, but I don’t bother (yet). I have my svn repo hosted on a web server so it can be accessed over https, by any machine with web access and an svn client installed.
Check it out to your local machine
I always develop on my local machine, I will jump through hoops to make sure I have a portable, standalone development environment for what i’m working on. It’s ok to develop on a remote server, as long as you have your own “sandbox” to work in, where you won’t be treading on anyone’s toes or vice versa. I always set up local hostnames and virtual hosts on my machine, e.g. myproject.macbook.local to allow me to easily work on multiple sites on the same port no. on my laptop. Check the project out and set the working copy folder as the root of your local website. When i’ve subcontracted work in the past, I have insisted that freelancers set up a local development environment and work with svn, even if we aren’t collaborating – there’s no way that i’m going to accept the work as a zip file over email! Working this way means that I can periodically check out the project to my machine to see how they are getting on, rather than having to ask them.
Build your site
As you add files to the site, add and commit them to svn. If you need to rename or delete a file, use svn to do it, to stop it getting out of sync. If you are collaborating with someone else, do an svn update and commit frequently (in that order). This way if you get any conflicts you can resolve them more easily than if you leave it till the end of the day. I find in that scenario it often becomes a “race” to commit stuff frequently, so that the other person is more likely to have to resolve conflicts if any turn up!
Contextual config file
I want to be able to deploy an identical code base to the live (and test) server environment, so I use some “if” statements that look at the host name to selectively load variables such as database config and anything else site/server specific, depending on which environment the site is running on.
Checkout the project on the live/ test server
Firstly, this scenario will only work for you if you have terminal/ ssh/ remote desktop access to your server. If you only have ftp or sftp available, this technique won’t work for you. If that is the case then you may have to resort to uploading the files individually or in bulk – which is a massive pain compared to being able to run an svn checkout/ export/ update on the live server.
There is also some debate about whether deploying or managing the live codebase as an svn working copy is a good idea. I think it is for the following reasons:-
One thing you will want to do if you manage the live site as a working copy is make sure your web server is configured not to serve up the .svn folders, or any other folders not part of the live site.
Database changes
I always keep a mysql dump in the svn repo. This may or may not include data, depending on the size of the database, or the need to keep config data in the database. Once again this should be in a directory that is not served up by your webserver. For a first deployment, I will use a checkout of the mysql on the live server to create and populate the database. Thereafter, I use numbered migration scripts to manage any (structural) changes to the database, such as adding or altering tables. These are added/ commited to svn, then checked out on the live server to run and apply the changes to the live site.
Getting started with svn
If you are reading this and thinking “sounds good, but i’ve never used svn”. A good place to start would be to sign up for a free hosted svn repo with someone like beanstalk, and then try one of the many GUI svn clients. Rapid SVN is a good free one for most platforms, but there are tons out there. It will help to learn the basic commands, and be essential if you only have ssh access on your server. I said I wouldn’t go into commands here, but to put things in perspective, I get by with very few: svn co (checkout a project), svn add (add file(s) to project), svn commit (commit changes to file(s)), svn update (update the working copy with latest changes), svn resolved (mark a conflict as resolved), svn mv (rename a file and tell svn to sync on next commit), svn rm (delete a file and tell svn to sync on next commit) and finally svn revert (go back to the last revision, or a specified revision).
There’s more (but I don’t know it)
I’m fully aware that there are more sophisticated ways to use svn to manage and deploy projects, involving branches, merging, switching, externals and a whole load of other clever stuff that I have yet to become familiar with. However a simple project with checkouts/ commits and updates on local and live server works well for me, and is a good starting point to get stuck in and start to feel comfortable with it. Also, apparently all the cool kids have abandoned svn for git, which works in a similar, but different way, so that might be worth a look too, if you aren’t already familiar with svn.
You’ll never go back
I’ve been working this way for a few years now, but occasionally i’ll do some work for a company that don’t or won’t work use version control, or don’t use it as an integral part of the workflow (i.e. they just use it as a backup). Without version control as an integral part of the workflow, I have often had to resort to pen and paper to remember what files have changed, so I know what to upload, i’ve accidentally overwritten new code with old versions and mysteriously lost days of work when someone else does an update from their non-versioned working copy. The learning curve is worth it, and so is getting your head round how to resolve conflicts (usually the first thing that scares people away from svn).
That’s the bit I’m a little lost on at the moment.. I use SVN a lot more now thanks to VersionsApp (http://versionsapp.com)
Nik 2009-06-04 10:20:09
I’m sure there is more that can be done with DB version control, but as you say there is a ton to learn.
Joel Hughes 2009-06-04 10:21:45
RedirectMatch 404 /\.svn(/|$)
(There should only be one backslash in front of .svn – need to update the blog software to do a stripslashes!
This stops svn stuff being served up. (I did write some other stuff to help with hiding other directories, but comment formatting messed it up, so removing!
@Joel +1 for tortoise svn on windows. I like the feature that you can commit a directory and it gives you the option to add any unadded files, and select which other files should/ shouldn’t be committed at the same time. A nice time saver
Rick 2009-06-04 10:58:16
One thing I would say, though, is as well as denying access to .svn (a must), also make sure that your SVN credentials (passwords, and so forth) are NOT stored on the webserver. When you do an update/checkout, your SVN username and password will probably be stored (by default) in ~/.subversion/auth.
Instead, you can set the “store-passwords = no” and “store-auth-creds = no” in your ~/.subversion/config file. This is a bit of a chore as you have to type the password each time you update.
The alternative is to make sure the SVN user is read-only, but even that gives a hacker access to historical (and future) release info.
Tom 2009-06-04 11:10:11
Not wanting to sound condescending, as I sometimes run my own personal projects as you describe i.e. just using the trunk, but when working on bigger, more complicated sites you should really considering creating a tag (just a svn cp of trunk) and deploying that so you can rollback in case of error.
Creating a SQL patch to the tag and one from the tag (i.e. one to add the new column and one to remove it helps when deploying sites that need to say up)
Andy Gale 2009-06-08 13:39:04
Rick 2009-06-08 13:55:35
Andy Gale 2009-06-10 11:44:17
As an experienced Plone front end developer, people are often surprised when I often decide not to use Plone, in favour of something like Drupal or WordPress. I thought it would be useful to explain why and how I make this decision. I know some of these points won’t be popular in the Plone community, but they are based on experience, and think this blog post will be useful to people deciding whether to use it or not.
Plone is complex
From a development point of view, Plone is a complex piece of software, at least compared to something like WordPress. I’m not going to go into specifics here, but for someone like myself who is primarily a front-end developer, sometimes I can lose days trying to add a simple custom feature, that I know I could create in hours on a php based site, probably just with a few lines of code in the template (a practice that is often frowned upon, but usually harmless and more importantly gets the job done). However much of this is down to my skillset and understanding (and probably my natural abilities to a certain extent) which brings me on to the next point.
Development resources – a chicken and egg situation
I have to hold my hand up and admit to being a bit of a “web monkey” when it comes to development – i’m self-taught, beginning with hacking around classic asp and php scripts and moving onto object orientated programming only in the last few years. With Zope and Plone the entry bar is higher – there is a whole framework to learn if you want to develop custom features, which is a good thing from a maintainability and system architecture point of view, but there is a significant time investment involved for someone wanting to transfer from something like php. Even an experienced Python developer will have some learning to do to get to know Zope and Plone.
This means that I struggle to find local freelance resources to work on a large Plone project with me. I also struggle to find sysadmins with zope/plone experience, and not being a sysadmin myself have found myself in a few hairy situations. I know of at least two ventures that have got into difficulties because of resource issues – I think partly because Plone can be a victim of it’s own success – there are lots of Plone developers, but they are all busy!
Hosting
Plone requires specialist hosting – or rather you can’t run it on the majority of locked-down budget shared hosting. Many of my clients already have hosting arranged and don’t want to move, so something like WordPress suits their needs better, for a simple site.
Overkill for simple sites
Sometimes a site only needs to be dynamic for the sake of a blog/ news section – I find plone is overkill for this, and often prefer to go with something more lightweight, even (gasp!) a bespoke CMS when I need to bolt extra functionality onto an existing site, and there is no budget for a rewrite.
So when would I use it?
Without a doubt Plone makes a fantastic intranet, out of the box. I eat my own dogfood here – I use an unmodified Plone 3 site for my own company intranet and document management system. Drupal/ WordPress do not even appear on the radar in comparison for this task. I would also use it for any site/ application that has a need for complex workflow, membership and groups.
Lastly, i’m happy to use it for any project where I can team up with an experienced zope/plone techie to help with the more low-level stuff. I was recently blown away by seeing how an experienced Plonista at Team Rubber quickly dealt with some temporary extremely high read/write traffic on a Plone site by firing up a whole set of zeo clients on amazon EC2 – a scenario that i’m sure would have been a nightmare to deal with on a drupal or wordpress site, and i’d be equally out of my depth without help.
In summary
I hope this is useful and doesn’t upset the Plone evangelists too much. Plone is a serious, well architected, secure system that leaves the competition standing in most cases. I think by “competition” I am talking about proprietry CMS and intranet systems that cost ten or hundreds of thousands of pounds, not the likes of Drupal or WordPress. If you don’t want to do much customisation it makes a good choice for smaller projects, but if you do you are going to need a developer (or invest in growing one of your own), who is a. available, b. you can afford, otherwise you might be better to go with something else, with a lower development entry level.
Kai (a Plone developer)
Kai Diefenbach 2009-05-08 11:20:57
I use plone for my intranet/dms, too. Maybe u can share your concepts/ideas or how u use it and for what..would be cool.
Greets
Gomez
Gomez 2009-05-08 12:06:43
Rose Pruyne 2009-05-08 13:36:19
I don’t think even the most fervent Plone fan would advocate using Plone for everything. (Though I am sure there are people skilled enough to customize it to point will work for anything.)
Our major problem has been what to use when Plone IS overkill. We are a 3-person team and also learning Python at the same time. We are a little hesitant to take up another new framework (like Django) but at the same time it doesn’t seem to make sense to have some stuff on PHP and some stuff on Python. (And 1 of our team members would have to learn both.)
Anthony Bosio 2009-05-08 13:50:16
Rick 2009-05-08 13:57:45
The Plone community has learned a lot from both its successes and its mistakes over the past 8+ years, and we have a lot of great innovation in the pipeline.
Jon Stahl 2009-05-08 15:12:18
Martin
Martin Aspeli 2009-05-08 15:36:49
Rick Hurst 2009-05-08 16:18:04
Michael Ang 2009-05-09 03:12:33
One of the projects I have worked on recently is the Holistic Community website – a directory site for Therapists, Training Courses and Treatment rooms. This was an interesting project, as I was originally approached to make a few amends, but the previous developer refused to hand over the php code running the site, therefore the only option was to rebuild it, based on the existing site and a database export. I have to confess that I went about this project the wrong way – the database structure was far from ideal, and also I subcontracted the initial build to an apprentice who was learning php, who then left halfway through the project! As a result I had to rewrite all of the code.
In retrospect this would have been an ideal project to build on a framework or CMS, but this would have been too steep a learning curve for the apprentice. By trying to initially replicate the old site (including numerous multi-page forms), and using the existing database structure (resulting in lots of convulted multiple table joins), and using a novice php developer, and taking it on at a time when I was too busy with other projects, the project dragged on a bit and I probably did more than double the amount of work that I needed to – not ideal on a fixed-price project! However, the bright side is that it is a nice example of a rebuild/ reskin, and I am full of ideas about how I could undertake similar projects in the future in a much more efficient way.
I’m pleased to say i’ve achieved another of my 2009 goals, by launching the (unfinished) new version of one of my personal projects Too Old To Skate. The main site is now running php/codeigniter based sweetcron, which is used to pull content in from the original wordpress, flickr, delicious, twitter, vimeo and a friends blogger based site. I have plenty more planned for the site, but little to no time to do it, so I thought i’d upload the site unfinished in the meantime to allow myself to do incremental updates, rather than be embarrased by a holding page.
Amongst the plans are:-
*Having now played with both codeigniter and cakephp, I think I prefer cake. However as sweetcron is codeigniter based it would be plain silly to be using cake on the same project.
In a Drupal (5.something) site, there is a section in the admin for specifying how file uploads should be handled. As well as specifying the maximum file size allowed for a single upload it reports what the current php settings are e.g:-
“Your PHP settings limit the maximum file size per upload to 4 MB.”
This can be confusing! I just set the max_file_upload in my php.ini to 20MB and drupal was still reporting 4MB. After a bit of fishing around I found out there are two factors to this. First look for max_file_upload in your php.ini and check that that is set to something sensible, then search for post_max_size (also in php.ini) – it seems that drupal is reporting this value divided by 2
Recently I decided to standardise on php/zend framework for future ground-up development (where I have a choice/ influence). As some of you may have gathered, i’m a bit of a “jack of all trades” when it comes to web dev and I have a legacy of projects using different technologies. One of these is a sprawling classic ASP/ mysql app for a distribution company which is a (very successfully) working prototype of something I want to rebuild in a more generic, modular and industrial way, as a flagship product for Olivewood to develop and sell.
To be honest the classic asp works fine for most purposes, but is becoming increasingly obsolete, along with availability of developers, and I have been trying to move on from it for years. The obvious choice for migrating a classic ASP app would usually be ASP.NET, but frankly the idea of spending the rest of my working life tied into windows-only development, and the fighting with software licences and scarce availability (it seems, at least in this town) of contractors makes me want to give up try a different career altogether. So I decided on php a while back (mainly because I wanted something open source, with an abundance of developers – I think if you throw a coin randomly in the watershed in bristol, chances are you would hit a freelance php developer).
So with php decided on, I then spent time evaluating a few frameworks. They all look good – this was a difficult decision – but I decided on Zend because Olivewood will be primarily concerned with eCommerce and eProcurement, and with magento being built on zend, and the “big industry” partnerships, it seemed the right fit. It also seemed to be useful and provide structure without being overly prescriptive. Also by writing this here i’m hoping to commit myself to at least something!
So, I also wanted something that would work cross platform, even though the obvious choice is to run open source web apps on linux/ apache, it was important to me that it would also play nicely with IIS, as many businesses have already invested in windows servers and already run other apps on IIS and wouldn’t be happy supporting anything else. This gets an indignant response from many “purist” developers/ sysadmins, but i’m basing this on real-life situations, and I hate the idea of a potential client ruling out open source software because it won’t run on their IIS server.
The other factor here was that I wanted something that would coexist with my classic ASP app until such a time that I have rewritten everything I need to make it a pure zend framework/php app. The classic asp scripts will handshake with the php code by dumping session data to the database and passing credentials via a cookie, so the session data can be shared between the two. This is vital to the plan, as it is a long term project – and much of the investment in migrating this will be my own time and money.
Installing Apache on the same server and setting up some kind of proxying would be another option, but not on the live server that this app runs on. I also needed to prove that it would work on just IIS before betting my future business plans on it, even if the first thing I would do is suggest that it is run on Apache.
I wasted a monumental amount of time getting this working, although it turned out to be fairly trivial once I had cracked it. The red herring is that there are two versions of isapi rewrite – version 2 and version 3. All the zend related documentation I found (hardly any) seems to be for version 2 (although this wasn’t mentioned!), which I just couldn’t get to work at all for my setup, but eventually I gathered that version 3 has been almost completely rewritten to work with apache mod_rewrite rules. So now I have a classic asp app, with a single folder (caled “zend”) containing my front controller from where all the php will be served the application folder (with all the models, views, and controllers) is outside the site root.
My httpd.conf file looks like this:-
RewriteEngine on
RewriteBase /zend/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule !.(js|ico|gif|jpg|png|css|asp)$ /zend/index.php
Remember this is for version 3 of Isapi rewrite. All quite simple really, when you know how. It’s a pity it took me about 9 hours to work out how!
My clients don’t allow ISAPI_Rewrite, so I’ve written a router that handles GET requests.
Regards,
Rob…
Rob… 2008-04-24 12:14:15
jenn 2008-05-31 15:01:45
Thank you.
Sam 2010-07-13 19:55:45
I’ve been sent a couple more books to review from Packt publishing – Codeigniter for Rapid PHP Development, and Object Oriented Programming with PHP5. I have immediate use for these as I am currently working on a few PHP5 projects and was looking around for a PHP framework to use for part of one of them, so expect reviews next month.
I’m also feeling a bit guilty as I still haven’t had time to review the Plone 3 book they sent me a while back – I need a nice Plone 3 project to get stuck into before I can do that justice, but will try to at least give it a read next month too, and report back my initial impressions.
Ian Wootten 2008-05-02 19:05:48
nicolas 2009-12-24 09:32:26