Content Aware Scaling

CAS - Content Aware Scaling
Seam Carving, or Liquid Rescale is a new procedure for 2D digital image scaling. I say new, but it’s over a year old now. I mention it because it’s being shipped - right now as I type - with Photoshop CS4. It uses some snazzy algorithms to determine whereabouts a ’seam’ (or path) can be defined which reaches from one side of the image to the other (like the students used to attempt to do with the question-hexagons on ITV’s Blockbusters) i.e. top to bottom or left to right.
It actually supports diagonal seams too. But that didn’t fit with my Blockbusters metaphor.

“I’ll have a ‘low importance’ seam please, Bob”
The algorithm chooses a path ‘of low importance’. In other words a path right across the image which you’d be unlikely to miss if it wasn’t there (in the case of reducing the scale), or unlikely to notice if it was duplicated and blended (in the case of increasing the scale).

Squash/stretch
Traditional image scaling uses a simple algorithm to uniformly remove or duplicate straight lines of pixels, vertically or horizontally, from an image resulting in squashing or stretching of the image which we’re all familiar with. Seam Carving handles the scaling in a totally different way.

Classic shot of Hong Kong IslandLose that sky
For example, if you look at this classic daytime shot of Hong Kong Island’s Central District taken from Kowloon (the night shot is more classic, I know), with its bar-graph of skyscrapers filling the harbour front and the blue sky above… well, if you were to scale that down vertically using CAS, the process would recognise (as would most humans) that the sky was pretty much all blue with low variation in detail, whereas the waterfront buildings and bay contains much more detail and variation, therefore the sky section of the image would be percieved as being of ‘low importance’ and scaling down would result in areas of the sky being dropped and the new seams subtly blended.

“Squeeze up, you on the edge..”
Whereas traditional scaling results in the old squash/stretch issue, which almost always looks wrong if not intentional, the application of this new process gives us a higher-brow issue: the adjustment of the composition of the image i.e. the distance between objects being varied. Imagine taking a considered photograph, with nicely balanced and well-thought content, only to find that someone had shuffled everything closer together like a junior school assembly photograph.
Considering that CAS is now shipping as a feature in Adobe Photoshop CS4, I’m sure we’ll see all sorts of results like this from daft to clever, and awful to entertaining. Either way, it’s a great tool, and a clever approach to a problem that was generally taken for granted.

Certain types of images aren’t really suitable for CAS, such as high-density-detail images (a satellite photograph of a city, or a crowd scene), some are ideal as you’ll see on this video (Flash 8 req)

Seam Carving was first presented at Siggraph 2007 by Shai Avidan and Ariel Shamir