Rick Hurst Web Developer in Bristol, UK

Menu

Category: python

My current tools and tech in 2019

Posted on by Rick Hurst

I try to avoid jumping on every new trend which comes along in the web world, especially over the last few years, where things seem to have moved very fast, with javascript frameworks becoming obsolete before i’ve even got round to trying them, but I also need to keep my skills and tech relevant. Here’s what i’m using as of late summer 2019.

Editor/IDE

I’ve been consistently using Sublime Text (Version 2 because I never got round to upgrading to 3) for years as my code editor, though i’ve never really taken full advantage of the extensions and customisations available. It just works and I saw no real need to change. One thing it has never done well is working over ssh, or rather from a mounted sshfs folder on a mac. I have occasional need to do this, when using the likes of nano in a terminal doesn’t feel good enough, and it’s one thing that I missed from Coda, when I used that for a while, years ago.

Then after spotting a tweet from an old colleague mentioning remote SSH editing in VS Code (via the Remote – SSH extension), I thought i’d take a look at it. Firstly, having used old-skool Microsoft Visual Studio in the past, I was surprised to see that Microsoft had built and released such a cool free editor, with builds for windows, mac and even linux. It’s already become my default editor – the intellisense and built-in terminal and git functionality is really good. There’s a lot there to explore and a ton of extensions. It also looks good.

Languages and frameworks

Most of my work is currently PHP. I no longer feel the need to apologise for that! PHP 7 is mature, stable and fast. I work on a couple of legacy PHP projects, one of which uses a combination of bespoke MVC code, an old fork of codeigniter (currently being refactored out) and Redbean ORM, and the other uses Zend framework, but is being re-built as a de-coupled app with Lumen (micro-framework by Laravel) on the back end and Vue.js on the front-end.

I haven’t done any web projects with Python recently (though i’m keen to try Pyramid), but I have been using it for a custom FTP server, using pyftpdlib (yes, FTP – for an old-skool EDI project – an old, but still very widely used technology where XML and FTP are used to exchange data – probably predating the invention of JSON and REST). I’ve also been using python for a raspberry pi datalogger project, reading serial data from microcontrollers via USB and then relaying it to a REST API.

JavaScript-wise, I still use bits and pieces of  jQuery to get stuff done while I learn Vue.js properly, which is happening as part of a current aforementioned web app rebuild. We settled on Vue.js instead of React because Vue seems to be a popular component in the Lumen/Laravel ecosystem.

For CSS, i’ve settled for now on SASS and Gulp. Sass is a keeper, i’m just using Gulp until I find something that I like better. I tend to use Susy for grid/layout.

Ansible is my go-to tool for devops – I use it to provision web servers and for formerly manual tasks such as migrations of websites, moving content between dev, live and staging on wordpress sites etc. For local development of any sort I almost always use Vagrant now, with an ansible provisioning script, which gives me a very portable set-up, provided I have enough RAM to keep multiple virtual machines running as I juggle projects.

During my last foray back into a day-job (back in 2016), working at an agency, I got a fair amount of experience using wordpress as a CMS, including for some fairly complex projects, relying heavily on the Advanced Custom Fields (ACF) plugin. Now, historically I haven’t always been a fan of WordPress, but it proved to be an excellent tool when used with ACF, making the creation of custom page types incredibly quick and easy compared to other Content Management Systems i’ve used, and even compared to using frameworks which would be my usual preference for custom content. Being able to create custom fields, included nested repeater fields via the WordPress admin, having the field definition saved to the filesystem and then versioned in GIT for seamless syncing to production is a game changer!

 

mysql_config not found osx

Posted on by Rick Hurst

This always gets me when I try to install mysql-python on osx using pip:-


$pip install mysql-python

(ugly traceback, ending in:-)

EnvironmentError: mysql_config not found


Googling round finds that most people suggest editing a line in MySQL-python/site.cfg (wherever that is..), but my preferred solution is add the following line to my .profile:-


export PATH=$PATH:/usr/local/mysql/bin


This works for me – for a mysql installed from a package on mysql.com

Running Google App Engine SDK on OSX lion

Posted on by Rick Hurst

OSX Lion comes with python 2.7 as default. Although App Engine will run on 2.7, it is safer to develop on the version of python that your live app engine instances are running on, which in my case is python 2.5 currently.

From a fresh install of Lion, I did the following:-

Installed xcode via app store

Installed Google App Engine SDK for Python, by installing google app engine launcher (This contains the SDK and a launcher app).

Installed mac ports

In a terminal:-


sudo port -v selfupdate
sudo port install python25
sudo port install py25-pil


This means I can start my django server on python 2.5 using:-


python2.5 manage.py runserver


However if you want to run dev_appserver.py it will still try to use python 2.7 (which, unless you’ve installed it into your default python, will be without PIL)

you can use macports to set the default version of python:-


sudo port select python python25


Finally, you might want to set Python Path to /opt/local/bin/python2.5 in your Google App Engine Launcher preferences

refs:-

http://stackoverflow.com/questions/4868185/how-to-install-pil-on-mac-osx-10-5-8-for-google-app-engine

http://stackoverflow.com/questions/6152765/macports-didnt-place-python-select-in-opt-local-bin

Update: after upgrading google app engine SDK to version 1.6, I was getting an import error “No module named _ctypes”. I have now upgraded to python2.6 and this fixed the issue.

plone conference 2010 day 2

Posted on by Rick Hurst

Plone conf day 2

Some notes and thoughts from day of plone conference 2010. These are mostly in note form, but wanted to get these online anyway, while they are fresh in my mind.

fixing the ungoogleable by Elizabeth Leddy

What happens when something breaks, and a simple google search doesn’t offer any results?

First, warn people (so they don’t bother you while you fix it, and they don’t panic).

Work out how quickly can you access your backups? I think this is very important – personally for smaller sites i’m doing individual site backups as packages mainly to make it convenient to restore them in a local dev environment, but I know some people are relying on whole server backups on tape. This would be laborious to restore in an emergency.

Isolation by elimination -> network, hardware, software, data
map out your system!

Eliminate things by switching them off – write your code so that it can handle dependencies being switched off.

Set up a system so that you have an isolated instance (so you can look at logs of only your own activity, rather than mixed with everyone elses)

Create a dashboard (maintenance page)

Monitoring

Write Test case’s to diagnose faults

Start writing a “help” email, but don’t send it. This is apparently known as “rubber ducking”. I do this all the time, I often spot my mistake when I past in a traceback to an email message to send to someone!

“horizon of intervention” – At what point do you need external help?

Get to know people with specialisms and buy them a beer! Participate in the plone community (I really need to do more of that).

Create tickets and close them as you go

Document your processes.

The state of plone caching by Ricardo Newbery

cachefu – useful tool, but as far as “internet time” goes, this is a little long in the tooth now (started 2006). End of life – critical bugfixes only

plone.app.caching – will be in future plone releases, but available now

load testing with funkload (not accurate regarding CSS/ image discovery)

load testing with multi-mechanize

BrowserMob – not free

Building a custom app with plone with minimal development by Eric BREHAULT

This was extremely interesting – Eric is the project manage for Plomino – a plone add-on that provides application development toolkit for creating database applications within plone – i.e. the types of database applications that non-technical people might create on there own machines with Lotus Domino, Filemaker or Access. The data is still stored in the ZODB, but models and forms can views can be created through the web. Formulas are done in python.

I can see this being really useful – all too often I end up building custom applications based on complex access database or excel spreadsheets i’ve been provided, but if I can persuade clients to use Plomino, then not only will it help get that data in a format suiteble for building a web application, but data can be collaborated on across teams, rather then emailing around (and inevitably forking spreadsheets and databases). There were some nice examples of data visualisation – a great quote “almost useless, but very nice”” – you need these to impress your boss.

Themeing with XDV (Diazo) Laurence Rowe

I wrote some notes on this yesterday, connected to Nates Deliverance talk, so won’t go into this here, other than to quote Laurence: “we write XSLT, so you don’t have to”

collective.amberjack: chapter one. The interactive age. Massimo Azzolini

Amberjack is a Javascript library for creating a site tour/ tutorial. Collective amberjack wraps this up as an add-on for plone. Interactive tutrials can be created.

An example was given using the windmill testing framework (windmill in itself looks nice alternative to selenium).

The Art of Integrating Plone with Webservices with David Glick

Most of this was over my head, but one important note I made from this is about urlib (which I use in a django screen scraper app i’m developing) can have two possible error responses – URLError and HTTPError – two possible error responses.

external ecommerce and plone playing along with Sasha Vincic

There seems to be a bit of a theme going here – an acknowledgement that Plone works out best long term if you use it as a “black box” CMS, and don’t try to do everything with it. The upgrade path is easier if you don’t add on your own customisations – “clean plone”. The current plone ecommerce offerings are not as good as external systems, so it is better to integrate with an external system, which is also then kept clean to allow easier upgrades.

In the python world there are some Django Ecommerce stores LFS and Satchmo, but the store doesn’t have to be Python – other proven systems such as magento can also be integrated with Plone.

To integrate with plone you need to integrate search, linking, thumbnailing. Valentine achieved search compatibility by creating objects in plone via an RSS import – see valentine.rssobjects. A latecomer to the talk asked “but which plone ecommerce product would you use if you had to?”. Answer “we wouldn’t”.

collective.transcode.star (lightning talk)

manage online transcoding
http://plone.org/products/collective.transcode.star – looks interesting, and may bring plone back into the picture for a project i’m doing.

BlueDynamics bda.plone.finder

osx style finder widget for navigating/ organising content in plone – looks great!

Plone Conference 2010 Day 1

Posted on by Rick Hurst

quality schwag from Plone Conference 2010

Just back home after day one of Plone Conference 2010, with my mind buzzing so thought it would be a good time to write up some of my thoughts and notes. It was really difficult to choose between the talks on offer on the three different different tracks, but here are some thoughts on the ones I attended.

Keynote by Alexander Limi and Alan Runyan

Two main themes here – ubiquity/availaibility and designer friendliness.

To make Plone more mainstream it needs to be available to non-technical end users through the same means that other systems are already available – namely being able to deploy easily on cheap hosting, specifically the one-click installers on shared hosting in cPanel and similar. This would allow users to easily evaluate Plone for their needs in the same ay that they can already with wordpress, drupal and joomla – apparently there is a new joomla instance created about every two minutes. I must stop Alan Runyan and see if he has thought about microsoft web platform installer – nowadays this includes the option to install wordpress, drupal, modX, and load of other systems, including downloading and installing dependencies. It would be great if Plone was in that list.

One of the aims of Plone 5 is to make it more designer friendly. I think this is really important – even though since the release of plone 4 i’ve started using plone again for intranets and extranets (mainly straight out of the box with a few minor cosmetic tweaks), I currently still use something like wordpress, or a home-rolled CMS for website builds. That is now going to change – the theming story is being completely re-written by the introduction of Deliverance/XDV/Diazo (already available – more on that later), and Deco (TTW layout and content editing). The aim is to make Plone appeal to designers as something that helps, not hinders them.

Quote of the talk has to be from Limi – “Plone doesn’t suck, because the developers don’t hate the core technology” (or something like that) – in reference to the revelation that many drupal/wordpress/joomla developers admit they actually hate PHP, whereas Plone developers love python.

Deco: new editing interface for plone 5

The next talk I attended was Rob Gietema’s demo of Deco. This is looking really good, although i’m a little bit skeptical of drag and drop and in-place editing (I like front-end based editing, but prefer lightboxed modal editing to in-place), mainly because i’ve seem layouts explode and page elements disappear, or refuse to drop in the correct place on similar systems in the past. However, I haven’t actually tried this one yet, maybe i’m just clumsy! I think in general designers and content editors are going to love it.

LDAP and Active Directory integration

I attended Clayton Parker’s talk on LDAP and active directory integration – can’t say I absorbed much, but i’m sure i’ll be asked to do this one day, so it’s good to know that this is tried and tested and the tools are already there.

Easier and faster Plone theming with Deliverance and xdv

Nate Aune gave us an overview of Deliverance. I’ve known about Deliverance for ages, but the penny dropped for me today about how useful this is. The basic principle is this – deliverance acts as a proxy to transparently take HTML output from a website and merge it with HTML from a theme, according to a simple set of rules. In the case of plone, this means you can create a theme in static HTML and have content from a default theme Plone site displayed wrapped up in the static HTML. Simple rules can be applied e.g. “take the news portlet from the plone site, drop the header and footer and all the images and display in the element with and id of “recent-news” from my HTML theme. magic!

Nate quoted one example where the HTML theme is stored in a dropbox folder which the client has access to to make tweaks and changes. I can see front end developers and designers loving this.

There was much discussion at the end over which technology should be used for this – XDV is a fork of an earlier version of Deliverance, which has slightly different functionality. XDV, which is to be renamed Diazo, will be the theming engine for Plone 5. With that in mind, i’ll concentrate my efforts on Diazo. I’m excited by this for non-plone reasons – a majority of my works seems to involve integrating technologies that don’t belong together – this will really help.

Design and development with Dexterity and convention-over-configuration

Martin Aspelli gave a talk on dexterity – the (eventual) replacement for archetypes. This is already available, but not mature yet. The talk was mainly conceptual rather than code-led, focussing on best practice for designing your site or application – when it is suitable to create a content type, and when you might be better off creating a form, or using a relational database. Best quote “code is like a plastic bag” (reduce, reuse, recycle). Write less code.

Laying Pipe with Transmogrifier

Another talk from Clayton Parker – transmogrifier is a system to package up migrations of content from other systems. My thoughts on this were that it looked like hard work for a one off import (usually i’d write a one-off python script for something like this), but creating packages would benefit the plone community e.g. if there were packages available covering migrations from a standard wordpress, drupal or joomla, this would benefit plone. I suppose this could also be used to import content from older instances of Plone, where the upgrade path is broken.

Multilingual sites – caveats and tips

Sasha Vincic talkd about strategies and gotchas for multilingual site builds. Even though Plone has tools for this, there are common scenarios, such as the “missing page” scenario where a translation of a site may not have the same number of pages as the base translation. He also covered common issues such as escaped HTML being translated by third parties and being delivered content where HTML attributes have been translated, therefore breaking the HTML.

Guest Keynote: Challenging Business

This was an inspirational treat for the end of day one – Richard Noble is a fantastic speaker, and after a day of CMS talk it was great to hear his story of the challenges of his past world land speed record record achievements, and the current one – the Bloodhound SSC project. As well as building insane rocket powered cars (the current one has an F1 engine onboard just to drive the fuel pump!), his goal is to inspire children and young people to become engineers, as there is an impending massive shortage of engineers in training. I was also interest to hear that there will be no patents on the technology developed for the new car – the advancements will be made avaiolable for anyone in the engineering industry to build on – sound familiar?

archived comments

Thanks for the roundup Rick! Looking forward to the rest.

Mike 2010-10-27 21:29:18

Good, clear writeup. I could not make it to the conference this year. It’s nice to be able to follow it a bit this way. Thanks, Rick!

Maurits van Rees 2010-10-27 21:42:13

Thanks for the comments guys – videos for all the talks will eventually be online, so hopefully everyone who couldn’t make it won’t miss out 🙂

Rick 2010-10-27 21:58:26

Thanks for the post!

I agree with you re: transmogrifier which is (partially) why I wrote http://pypi.python.org/pypi/parse2plone

Keep the blogs coming 🙂

Alex Clark 2010-10-28 00:50:01