Rick Hurst Web Developer in Bristol, UK

Menu

Category: eatStatic

Another flat-file driven blog/ cms engine on the block

Posted on by Rick Hurst

I’ve just been looking at Statamic, which is a markdown-driven CMS and blog, self-hosted and written in PHP. It’s commercial rather than open-source, but at a reasonable price, like Perch. It looks reasonably mature and has support for add-ons and lots of cool stuff. I’ll probably buy a copy just to mess around with.

If I ever need to build another PHP CMS-driven site, i’ll certainly give this a go, as i’m still obsessed with the idea of having content as flat files under source control. I like the idea that you could set up a staging site using something like this, and the client/ project manager could be adding content, and you can periodically log into the server, commit the content to GIT and sync you local dev copy in both directions.

Also, with the end result still being a dynamic PHP site (unlike the static sites compiled by the likes of Jekyll), you can still develop dynamic features into the resulting site and integrate in other PHP modules, e.g. forums and shopping carts etc.

As for what this means for my own PHP text-file driven blog eatStatic, it re-confirms that I shouldn’t be putting any effort into building any CMS features in, and to concentrate only on maintaining as a blog engine for my own stuff, and only adding the features I need. I haven’t had a look at the blog functionality provided by Stamatic yet, but having built eatStatic to fit my own ideal travel blog workflow, it would probably need some work to make me happy!

I’ve also mentioned a Python port of eatStatic in the past, and would still like to do something in this area, but I can’t help but think i’d be better off concentrating on a module to plug into another blog engine / CMS, to avoid reinventing the wheel yet again. Before I do anything else I want to experiment with all the current static-site generators – to see if there is some way to build on this – e.g. a python app that takes content set up for a Jekyll site and syncs it with content in a mezzanine (or other Django-based CMS).

A quick google around for “python static site generator” has thrown up this: http://ringce.com/hyde

eatStatic demo site online

Posted on by Rick Hurst

eatstatic demo site screengrab

Shhh.. don’t tell anyone but i’ve quietly launched the eatStatic demo site, which now runs from the master branch of the github repo. This is still the PHP version, slightly tidied up, and now using twitter bootstrap for the default skin. I had previously said that I wouldn’t be maintaining the PHP version in favour of building a python/ django version, but as i’ve thought of a new personal project to build with django (which for once – STOP PRESS – is not a blog engine or CMS), I thought it would be good to get the PHP version tidied up into a usable state if anyone else wants to try it out. Also, as I happily run this blog from it, and am likely to do for years to come, it would be nice to get all the features built in that i’d like to use, and bugs ironed out.

eatStatic is Dead, long live eatStatic

Posted on by Rick Hurst

I was pleased to see people recently enthusing on twitter about text-file based blog and CMS tools – nesta CMS, Kirby and scriptogr.am, as it validated my theory that there is a demand for tools like these – I was met with fairly bemused looks when i’ve explained the concept to people in the past!.
I was also slightly annoyed with myself for not pushing my own text file based blogging system more and taking it further, as maybe people could have been getting excited about my project instead. It seems fairly pointless to develop it further now when these other tools exist, and i’d already decided to park it last year. My other reason for halting development on it was because it is written in PHP, and i’m determined to concentrate my efforts on django and python for personal projects.

Whilst at New Adventures in web design conference on thursday, I was feeling inspired to create something, so thought maybe I should build a django version of one of these systems. For a crazy second there I thought maybe I would be the first person to think of this, so I did about 30 seconds research and found that Luis Pedro has already created a django text file based blog/ CMS called Git CMS. It looks great, so my first reaction was that I should fork it and develop some of my other ideas around it, but then I noticed that he is still using a database, with a sync script to populate Django models from the text files. This is actually a very sensible way of doing it as it means that you can take advantage of all the stuff you get for free in django. However, my initial goal for eatStatic was to avoid the need for a database at all..

So i’ve decided to do a more or less straight port of eatStatic blog engine to python – a standalone text file blog engine implemented as a python library, that can be dropped into a django project, or any other python web framework, with little or no configuration. I’ve given myself an initial goal of moving this blog over to it as soon as it is stable, but then I want to take it further, and do some marketing around it – encourage people to try it, and to fork it, join me in my quest for a really fast, really simple, really portable, really powerful python powered database-less blog and cms. As soon as I have the initial code ready, and am convinced it isn’t too embarrassing i’ll get it up on gitHub alongside the PHP version 🙂

Test post from droptext

Posted on by Rick Hurst

Just experimenting to see if it is possible to create a blog post on my new eatStatic based blog using droptext on my iphone.

Under the current set-up, if I wanted to use the “drafts” folder, I’d then need to log into dropbox.com to move the file to the main folder, as there is no way of moving files in droptext (as far as I can see)

Inserting an image would also be tedious – I can upload a photo using the dropbox app, but I can’t rename it something suitable, without going to dropbox.com, and then writing HTML on an iPhone is never much fun

I think if I want to consider mobile blogging, I’ll need to build something more convenient, such as an email to blog post script, similar to Posterous and wordpress plugins I’ve seen.

Object storage and retrieval in PHP part 2 – MongoDb

Posted on by Rick Hurst

In part one, I talked about how to save and retrieve a PHP object instance using JSON files, and in this post I talk about the same operation using mongoDb, and some gotchas.

I’ve only tried this in very limited circumstances, mainly to see how feasible it would be to make eatStatic seamlessly switch between json files and mongo – I naively thought that you would just throw a json file at mongo and have it store it for you, but the examples i’ve found takes the php object and converts to JSON magically, and also passes back a PHP object instance rather than raw JSON.

This post doesn’t cover installing mongo, I skipped between several different examples/ tutorials before I got it working, so can’t remember exactly how I did it in the end. Once installed though you can connect to it from PHP like this:-


$m = new Mongo(); // connect
$db = $m->cms; // select a database


For comparison purposes we’ll create a simple case study object like in Part 1:-


class case_study {
var $id;
var $title;
var $body_text;
var $skills = array();
}

$case_study = new case_study;

$case_study->id = 'my/case_study';
$case_study->title = 'My case study';
$case_study->body_text = 'Some text for the case study';
$case_study->skills['css'] = 'CSS';
$case_study->skills['sitebuild'] = 'Site Build';


which gives us:-


case_study Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


To store this in mongo db, we simply specify a collection to use (“collection” is analogous to table in a relational database, but you can create them on demand, and don’t have to specify a schema) and them insert our object:-


$case_studies = $db->case_studies;
$case_studies->insert($case_study);


To get it back we use:-


$case_study = $case_studies->findOne(array('id' => 'my/case_study'));


Passing this to print_r() gives us:-


Array
(
[_id] => MongoId Object
(
[$id] => 4e6de720d2db288b0c000000
)

[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


Note a couple of things:-

  • It has given us back an Array, instead of an object
  • It has inserted it’s own unique ID [_id]

We don’t need to worry about the extra ID, as we’ll be using our own for lookups, so it can be ignored. To convert the array to an object, simply do:-


$case_study = (object) $case_study;


Which takes us back to:-


stdClass Object
(
[_id] => MongoId Object
(
[$id] => 4e6de720d2db288b0c000000
)

[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)

Object storage and retrieval in PHP part 1 – JSON files

Posted on by Rick Hurst

I mentioned in my post about eatStatic that I was using JSON files for storage of objects and arrays, but hoped to make it switchable to use mongdb. This is part one of a two-part post, demonstrating use of JSON files with json_encode() and json_decode().

Take the following simple class:-


class case_study {
var $id;
var $title;
var $body_text;
var $skills = array();
}


If we create an instance of this and add some data:-


$case_study = new case_study;
$case_study->id = 'my/case_study';
$case_study->title = 'My case study';
$case_study->body_text = 'Some text for the case study';
$case_study->skills['css'] = 'CSS';
$case_study->skills['sitebuild'] = 'Site Build';


and pass it to print_r(), we get this:-


case_study Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


If we now encode it as JSON:-


$json_str = json_encode($case_study);


At this point, we can save the file to the filesystem – I tend to create a unique ID based on the current date/time and a random string. I won’t detail it all here, but you can see some of the helper functions I use in eatStaticStorage.class.php and eatStatic.class.php. One thing worth noting is that sometimes when reading a .json file back in from the filesystem, I was experiencing a bug where the last three characters were omitted – i’m not sure what was causing this, but it was fixed by changing my read_file() method to use file_get_contents(), instead of fread().

Once you have retrieved your JSON string you can decode it again:-


$case_study = json_decode($json_str);


and we end up with this:-


stdClass Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => stdClass Object
(
[css] => CSS
[sitebuild] => Site Build
)

)


Notice that the array “skills” is now an object. We can set it back to an array using get_object_vars():-


$case_study->skills = get_object_vars($case_study->skills);


nb: this only happens for key => value arrays, if it was just a simple array e.g. array(‘css’,’sitebuild’), we wouldn’t need to pass it through get_object_vars(), as it would be maintained as an array.

This gets us back to where we started:-


stdClass Object
(
[id] => my/case_study
[title] => My case study
[body_text] => Some text for the case study
[skills] => Array
(
[css] => CSS
[sitebuild] => Site Build
)

)


sort of – we now have an object instance with all the attributes of the original object instance, but it doesn’t know it is a case_study object. In fact it isn’t a case_study object instance at all – we would have to create a new instance of case_study and copy the attributes across if we needed the real thing, but if you just want the data, this can be used as it is in most cases.

The above example is very simple, but it can get quite complex when your object contains arrays of objects, which in turn may contain arrays (and arrays of objects). The initial cheap and convenient trick of encoding an object instance and saving it, then retrieving, decoding and using it can then get quite hairy, but still less effort than splitting it out into different objects and maintaining in several different relational database tables.

In part two i’ll talk about how to use mongoDB to save and retrieve object instances in PHP.

Site building workflow challenges – keeping HTML in a database

Posted on by Rick Hurst

I was reminded to today of one of my pet hates – coordinating a site build, or a site rebuild when the CMS you are using keeps content, often containing HTML markup from use of an editor such as tiny mce, in the database.

Consider the following scenario:-

  • You have a staging site where the client has been using the CMS to input content
  • Meanwhile, you make some changes to the database on your local version and want to push them to staging
  • You can use a migration script to push your changes to the live database but you find yourself wanting to also copy the new content back to your local database, so you can work on CSS on real content. You then would probably drop your local database and restore from a backup from staging, losing any test content you put in locally

It’s basically a bit of a kerfuffle.

This is one of the scenarios that I hope could be avoided with a CMS based on eatStatic (if I ever develop it beyond a blog engine) – any content-types that contain bodies of text, whether thay are marked up with HTML or not, would be stored on the filesystem. This could be put under version control, so you can selectively synchronise your content with another instance of the site.

I can also see a case for some add-on for any existing CMS – an export function that routinely pushes text content from the db into text files to be kept under version control, and also allows import, allowing instances to selectively sync content.

Introducing eatStatic blog engine

Posted on by Rick Hurst

creating a new blog post in textmate

Recently I ported this blog from an ancient version of wordpress to my own simple blog engine, which uses my PHP5 micro-framework, “eatStatic”. I use the phrase “blog engine” rather than blog software, as it isn’t really packaged up yet as something I would describe as software – its more just a collection of classes and templates that can be used to keep a blog.

The bulk of the code was written last year in the space of a couple of hours while sitting in a garage waiting for my car to be fixed – I was about to go on a long road trip and wanted a blogging solution that let me create blog posts and organise photos offline and then conveniently sync it to the live site when I had an internet connection. The result was my “on the road” blog about mobile working.

The thing that sets this apart from other blog engines (and the origin of the name “eatStatic”, along with a nod to a 90’s techno act), is that instead of using a relational database to store content, it uses simple text files for blog posts, and cached json files to store collections of data (e.g. post lists, tag references etc.). I have it set up to run with dropbox so that I compose my posts in textmate and they are synced to a dropbox folder on the webserver. You don’t have to use dropbox though – you can use any technique you like to upload the data files to the server – for “on the road” I use subversion, which means I also have versioning of blog content. Draft posts are composed in a drafts folder and moved into the main posts folder to push them live. There is currently no admin area on the site, though I might add one later.

The published date and URI for each post are taken from the text file name – i’ve adapted it for this blog to use the same url scheme as wordpress to avoid link rot on legacy content. Some people asked me why I don’t just use the title and created/ modified date of the text file to make it even simpler, and the answer is that I wanted finer control, and the option to specify the publish date – using created/ modified would have been a disaster for the content I imported from wordpress. Also by naming each file starting with YYYY-MM-DD, the post files are easier to sort/ find in the post folder, both visually/ manually and in code. You can use HTML in the blog post and additionally line breaks are converted to br tags, other then immediately after a closing tag. You can add tags and metadata at the end of the text file.

I’ve also got a simple thumbnail gallery which can be included in a post (see below) by uploading a folder full of full-size images with the same name as the post. The idea behind this is that a set of jpeg/ png images can be imported from a camera, and automatically pushed to the server by dropbox. A caching script creates the thumbnails and web-size version on demand, which are saved to the filesystem for efficiency during subsequent requests. I considered setting it up so that each post had it’s own folder, which could then contain images, but the blog engine was mostly written with the idea of quickly creating posts by opening textmate/ emacs, writing and saving rather then faffing around with creating folders.

I made the decision not to build in any commenting functionality – the anti-spam / moderation features needed are too much of a pain to deal with, so i’ve archived the old wordpress comments into the post body and integrated disqus instead.

As I mentioned before, I’ve been using a previous version of eatStatic successfully for my “on the road” blog, but I wanted to see how it coped with 100’s of posts rather then just a handful – it seems to be doing fine, coping with over 600 posts, but i’m sure there is room for improvement. I’ve also been investigating making the json read/write switchable to use mongodb so that it could potentially be very scaleable – i’ve encountered a few inconsistencies in the way that PHP json_decode() and mongodb object retrieval work, but nothing that can’t be worked around – expect a blog post on that later!

I don’t expect eatStatic blog to be a wordpress killer, but it may appeal to techie types who want a lightweight PHP5 blog engine, maybe to plug into an existing site and people who want to compose posts in textmate/ emacs (or any other code editor), rather than in a web form. If you are interested in trying it, keep an eye on the github repo, as i’ll commit an example of how this blog is formed, once i’ve ironed out the more embarrassing bugs! I may add a simple admin area at a later date, to allow publishing entirely via the web, and I think it would also benefit from a “post by email” feature, for convenient moblogging, but don’t hold your breath!

When I was importing content (I actually wrote a python script to parse a wordpress xml export file and create the text files), I found it quite fitting that the first ever post on this blog nearly ten years ago was made on a home-brewed ASP blog engine which used XML for data storage. I think before then I kept a static HTML blog of sorts, on a freeserve site, but unfortunately haven’t got a copy of that for completeness.

Lastly, whether or not you want to set up an eatStatic-based blog, if you aren’t already using dropbox, it really is excellent, so why not sign up for free 2GB account using my referral URL, so I can get some more free space? Even though I have a paid dropbox account, I use a second free account to mount on my server for automated site/ database backups and for this blog and it keeps filling up!