Rick Hurst Web Developer in Bristol, UK

Menu

Blog

Object storage and retrieval in PHP part 1 – JSON files

I mentioned in my post about eatStatic that I was using JSON files for storage of objects and arrays, but hoped to make it switchable to use mongdb. This is part one of a two-part post, demonstrating use of JSON files with json_encode() and json_decode().

Take the following simple class:-


class case_study {
var $id;
var $title;
var $body_text;
var $skills = array();
}


If we create an instance of this and add some data:-


$case_study = new case_study;
$case_study->id = 'my/case_study';
$case_study->title = 'My case study';
$case_study->body_text = 'Some text for the case study';
$case_study->skills['css'] = 'CSS';
$case_study->skills['sitebuild'] = 'Site Build';


and pass it to print_r(), we get this:-


case_study Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


If we now encode it as JSON:-


$json_str = json_encode($case_study);


At this point, we can save the file to the filesystem – I tend to create a unique ID based on the current date/time and a random string. I won’t detail it all here, but you can see some of the helper functions I use in eatStaticStorage.class.php and eatStatic.class.php. One thing worth noting is that sometimes when reading a .json file back in from the filesystem, I was experiencing a bug where the last three characters were omitted – i’m not sure what was causing this, but it was fixed by changing my read_file() method to use file_get_contents(), instead of fread().

Once you have retrieved your JSON string you can decode it again:-


$case_study = json_decode($json_str);


and we end up with this:-


stdClass Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => stdClass Object
(
[css] => CSS
[sitebuild] => Site Build
)

)


Notice that the array “skills” is now an object. We can set it back to an array using get_object_vars():-


$case_study->skills = get_object_vars($case_study->skills);


nb: this only happens for key => value arrays, if it was just a simple array e.g. array(‘css’,’sitebuild’), we wouldn’t need to pass it through get_object_vars(), as it would be maintained as an array.

This gets us back to where we started:-


stdClass Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


sort of – we now have an object instance with all the attributes of the original object instance, but it doesn’t know it is a case_study object. In fact it isn’t a case_study object instance at all – we would have to create a new instance of case_study and copy the attributes across if we needed the real thing, but if you just want the data, this can be used as it is in most cases.

The above example is very simple, but it can get quite complex when your object contains arrays of objects, which in turn may contain arrays (and arrays of objects). The initial cheap and convenient trick of encoding an object instance and saving it, then retrieving, decoding and using it can then get quite hairy, but still less effort than splitting it out into different objects and maintaining in several different relational database tables.

In part two i’ll talk about how to use mongoDB to save and retrieve object instances in PHP.

Site building workflow challenges – keeping HTML in a database

I was reminded to today of one of my pet hates – coordinating a site build, or a site rebuild when the CMS you are using keeps content, often containing HTML markup from use of an editor such as tiny mce, in the database.

Consider the following scenario:-

  • You have a staging site where the client has been using the CMS to input content
  • Meanwhile, you make some changes to the database on your local version and want to push them to staging
  • You can use a migration script to push your changes to the live database but you find yourself wanting to also copy the new content back to your local database, so you can work on CSS on real content. You then would probably drop your local database and restore from a backup from staging, losing any test content you put in locally

It’s basically a bit of a kerfuffle.

This is one of the scenarios that I hope could be avoided with a CMS based on eatStatic (if I ever develop it beyond a blog engine) – any content-types that contain bodies of text, whether thay are marked up with HTML or not, would be stored on the filesystem. This could be put under version control, so you can selectively synchronise your content with another instance of the site.

I can also see a case for some add-on for any existing CMS – an export function that routinely pushes text content from the db into text files to be kept under version control, and also allows import, allowing instances to selectively sync content.

Introducing eatStatic blog engine

creating a new blog post in textmate

Recently I ported this blog from an ancient version of wordpress to my own simple blog engine, which uses my PHP5 micro-framework, “eatStatic”. I use the phrase “blog engine” rather than blog software, as it isn’t really packaged up yet as something I would describe as software – its more just a collection of classes and templates that can be used to keep a blog.

The bulk of the code was written last year in the space of a couple of hours while sitting in a garage waiting for my car to be fixed – I was about to go on a long road trip and wanted a blogging solution that let me create blog posts and organise photos offline and then conveniently sync it to the live site when I had an internet connection. The result was my “on the road” blog about mobile working.

The thing that sets this apart from other blog engines (and the origin of the name “eatStatic”, along with a nod to a 90’s techno act), is that instead of using a relational database to store content, it uses simple text files for blog posts, and cached json files to store collections of data (e.g. post lists, tag references etc.). I have it set up to run with dropbox so that I compose my posts in textmate and they are synced to a dropbox folder on the webserver. You don’t have to use dropbox though – you can use any technique you like to upload the data files to the server – for “on the road” I use subversion, which means I also have versioning of blog content. Draft posts are composed in a drafts folder and moved into the main posts folder to push them live. There is currently no admin area on the site, though I might add one later.

The published date and URI for each post are taken from the text file name – i’ve adapted it for this blog to use the same url scheme as wordpress to avoid link rot on legacy content. Some people asked me why I don’t just use the title and created/ modified date of the text file to make it even simpler, and the answer is that I wanted finer control, and the option to specify the publish date – using created/ modified would have been a disaster for the content I imported from wordpress. Also by naming each file starting with YYYY-MM-DD, the post files are easier to sort/ find in the post folder, both visually/ manually and in code. You can use HTML in the blog post and additionally line breaks are converted to br tags, other then immediately after a closing tag. You can add tags and metadata at the end of the text file.

I’ve also got a simple thumbnail gallery which can be included in a post (see below) by uploading a folder full of full-size images with the same name as the post. The idea behind this is that a set of jpeg/ png images can be imported from a camera, and automatically pushed to the server by dropbox. A caching script creates the thumbnails and web-size version on demand, which are saved to the filesystem for efficiency during subsequent requests. I considered setting it up so that each post had it’s own folder, which could then contain images, but the blog engine was mostly written with the idea of quickly creating posts by opening textmate/ emacs, writing and saving rather then faffing around with creating folders.

I made the decision not to build in any commenting functionality – the anti-spam / moderation features needed are too much of a pain to deal with, so i’ve archived the old wordpress comments into the post body and integrated disqus instead.

As I mentioned before, I’ve been using a previous version of eatStatic successfully for my “on the road” blog, but I wanted to see how it coped with 100’s of posts rather then just a handful – it seems to be doing fine, coping with over 600 posts, but i’m sure there is room for improvement. I’ve also been investigating making the json read/write switchable to use mongodb so that it could potentially be very scaleable – i’ve encountered a few inconsistencies in the way that PHP json_decode() and mongodb object retrieval work, but nothing that can’t be worked around – expect a blog post on that later!

I don’t expect eatStatic blog to be a wordpress killer, but it may appeal to techie types who want a lightweight PHP5 blog engine, maybe to plug into an existing site and people who want to compose posts in textmate/ emacs (or any other code editor), rather than in a web form. If you are interested in trying it, keep an eye on the github repo, as i’ll commit an example of how this blog is formed, once i’ve ironed out the more embarrassing bugs! I may add a simple admin area at a later date, to allow publishing entirely via the web, and I think it would also benefit from a “post by email” feature, for convenient moblogging, but don’t hold your breath!

When I was importing content (I actually wrote a python script to parse a wordpress xml export file and create the text files), I found it quite fitting that the first ever post on this blog nearly ten years ago was made on a home-brewed ASP blog engine which used XML for data storage. I think before then I kept a static HTML blog of sorts, on a freeserve site, but unfortunately haven’t got a copy of that for completeness.

Lastly, whether or not you want to set up an eatStatic-based blog, if you aren’t already using dropbox, it really is excellent, so why not sign up for free 2GB account using my referral URL, so I can get some more free space? Even though I have a paid dropbox account, I use a second free account to mount on my server for automated site/ database backups and for this blog and it keeps filling up!

Watershed 2011 rebuild

screen grab of watershed.co.uk

Last night the new version of the watershed website was pushed live. I had the pleasure of being one of many people involved in this project, which involved combining several different sites representing different projects within the watershed brand. I did the “first cut” of the HTML/CSS, working from a PSD provided by the design agency Document, and also helped with some of the Drupal theme integration, working alongside some talented watershed staff and other freelancers (i’d name them all here, but would inevitably miss someone).

Worliday? I think I need a holiday

attempting another worliday in my favourite field

I read this article by Lucy Kellaway on the BBC news site – she invented the term “worliday” for a working holiday and is very enthusiastic about the idea, advocating it as the way forward. If you’ve read any of my previous articles about my experiences working throughout the summer last year on a six week road trip in france, you may have picked up that there are both good and bad points – I’ve written about the importance of “work/ loaf balance” here.

This summer we didn’t make it to france, but a couple of weeks ago I was working from borrowed dining room tables, cafe’s and the platform of a skateboard ramp in Ireland, and i’m writing this blog post from a field in Devon. So i’m still trying to make the nomadic working thing work, but this year I think fatigue is starting to creep in – the Ireland holiday was very social, staying with friends and house-sitting, but missing so much of what was going on around me bought home to me just how much of the actual holiday bit of a working holiday, is compromised by being obliged to work.

When I arrived at my favourite devon “secret campsite” this morning, my phone messages revealed that I had somehow missed not one but two client meetings in Bristol – my feelings of spontaneity were soon replaced by guilt. Then my wife and kid headed off to the beach, while I have ended up chain drinking tea in a camping chair, sending grovelling emails and grumpily punching keys on my laptop as I deal with technical issues that have cropped up on various websites.

One of the points that Lucy makes is that by working on holiday, you can justify going on holiday more often – I agree with this to an extent, but currently it feels more like I need longer and more holidays, so that I can scrape together enough actual holiday time. I think the long holiday worked last year because the time I spent not working equated to about the length of a normal holiday.

I start to relax a bit more as I get a bit of work done, but I know that when it’s time to down tools I will find it hard to switch off again, and tomorrow if anything delays me getting started with my work, anxiety will set in again – what I really need right now is a holiday!

(The pop-up tent in the photo above is a Quechua Base Seconds)

Combined messenger and pannier bag

Knog Franks Dog

I had been looking around for a messenger bag that can also be used as a pannier bag, as I tend to use a backpack when cycling with my laptop as I find a messenger bag moves around to much and I don’t like the unevenness of the weight distribution, but then would prefer a messenger bag when i’m back on foot.

I was surprised at the lack of available options on the market but eventually found this on from an Australian company called Knog. . This one (bought from chain reaction cycles) is suitable for up to 15″ laptops and has a showerproof canvas outer. There is a clever pull out silver reflective rain cover in a pocket at the back, if you get caught in heavy rain. There is also a waterproof version, but I wanted something that was primarily a comfy messenger bag, that also happens to work as a pannier, rather than a pannier bag that I can carry round.

As for the pannier attachment – it fits to a socket on the back of the bag, there are also attachments available (but bought seperately) to convert it to a backpack or handlebar mount, both of which I could see as being useful.

I’ve yet to use it in anger, but will update this blog post if I find any problems.

The importance of earphones for emergency use when cafe working

Mostly I can filter out the background noise in cafe’s so have no need for headphones. I don’t always want to cut myself off completely, I want to be able to hear the phone ringing or people trying to get my attention. But background noise is usually fine, I can filter it out and concentrate. However, sometimes a particular voice will cut through the cacophony, and whatever you do to try to ignore it you find yourself tuning in and listening to every word! In those situations you need earphones – they can make the difference between a wasted trip and a successful one!

Shedworking

Solar Gorilla on the roof

This blog has been a bit quiet recently, but contrary to what the title suggests, I haven’t been hiding in my shed. In fact since returning from our European road trip last summer I spent a while working from home, then on-site for Aardman for a few weeks, then I started renting some desk space in town. The desk space didn’t really work out – just like the previous time I did this, I found myself using it less and less, and eventually decided it wasn’t worth the expense. So now i’m back to home working, or more specifically right now, from the dining room table of a house we are house-sitting in Ireland!

The reason for the “shedworking” title is that I was given a book for my birthday “Shedworking – The alternative workplace revolution” by Alex Johnson, which showcases some lovely shed conversions for people to create a dedicated workspace in their garden. This is something i’ve considered doing, being the owner of a reasonable size breeze-block outbuilding in the garden of our victorian terraced house. While I was considering whether to give up the rented desk, it occurred to me that the money saved on rent could fund the work needed to create a work space.

I would need to replace the roof – currently blue asbestos, which although apparently safe if undisturbed, leaks a bit and i’d rather get rid of it. I’s also need to divide off some space to keep as a shed – storing bikes, tools etc. and create a well insulated space with some form of heating (a wood burner would be nice, but I suspect i’d need something else for shorter bursts of heat first thing on a winter morning!). Other than that, it would be very minimalist – I would need a chair and desk, from where I could work on my laptop. The wifi from the house reaches the garden so hopefully into the shed, and I could always use a range extender if necessary. Although I would likely gets mains electricity installed, I would like to keep this space “off the grid” – maybe I could invest in the big leisure batteries and larger solar panels that I would need to keep laptop/ monitor/ lights going. A trial run for whatever I would eventually fit into a camper van that i’m still fantasising about.

So walking a few metres down the garden to a shed could hardly be described as “on the road”, but it is very much in keeping with the theme of not working in an office, working with a minimal set of tools that can be packed into a bag and taken elsewhere at the drop of a hat.

Trying out working standing up

If any of my neighbours are spying on me today they might be wondering why I am working standing up by the window with my laptop on a chest of drawers. The reason is this article: “Sitting is killing you” (linked removed at content authors request, due to google backlink issues). I’m not getting all tin-foil hat after reading this, but I have been wondering what to do about my home “office” set up recently, and have been concerned about the amount of time I spend sitting (well slouching, if we are being honest) at a desk. I need to find the most comfortable height and see how I get on with it, so this blog post is being written standing up as the first of many experiments!

archived comments

Brilliant! I’ve started standing to work at home – it was a pain in the arse getting the desk to the right height (lots of piles of books) but well worth it. It’s hard work at first, and you still need to sit down occasionally, but I find it helps me focus better.

Nick Morgan 2011-05-19 10:05:02

There’s a long running thread on twitter from @andydiggle about moving away from sedentary work. His solution: the IKEA hack standing desk!

Expedit standing desk

I know the whole subject can seem quite trivial but to anyone who is a) deskbound at work and b) needs to focus for extended lengths of time in the same position I think it is pretty important to be able to move around. My experience is that sitting at a desk seems to make me want to stay sat down and I end up collecting all my desk oriented activities together. Anything that can force me to move around has got to be a win.

Robin Layfeild 2011-05-27 10:04:28