Expressionist painting?

IF YOU’VE PHONED a utility company recently you might wonder how long it will be before it’s impossible to tell if you are speaking to a human or robotic telephonist? Now, it seems, computers cannot only read our speech but also our facial expressions, and will be able to offer us electronic sympathy whenever we look down in the mouth.

As an intriguing proof of principle, computer scientists from Bath and Boston have developed a novel kind of electronic artwork known as ‘empathic painting’. This is described as “an interactive painterly rendering whose appearance adapts in real time to reflect the perceived emotional state of the viewer” – in other words, the painting looks back at you and alters its form to reflect the mood you appear to be in.

Using images collected through a webcam, special software recognises eight key facial features that characterise, broadly, the emotional state of the person viewing the artwork. The software then adapts the colours and brush strokes of the digital artwork to suit the changing mood of the viewer. When the viewer looks angry, for example, the colours are dark and appear to have been applied to the canvas with more violent brush strokes. If the viewer’s expression changes to happy, the artwork adapts so that the colours are vibrant and more subtly applied.

The project forms part of on-going research looking to develop a range of advanced artwork tools for use in the computer graphics industry. The researchers have previously developed software which produces highly detailed artistic versions of photographs, and allows designers to create animations directly from digital footage.

Dr John Collomosse, of the Department of Computer Science at the University of Bath, explained: “The programme analyses the image for eight facial expressions, such as the position and shape of the mouth, the openness of the eyes, and the angle of the brows, to work out the emotional state of the viewer. It does all of this in real time, meaning that as the viewer’s emotions change the artwork responds accordingly.”

The result is a digital canvas which smoothly varies its colours and style, and provides a “novel interactive artistic experience”.

He continued: “This kind of empathic painting only needs a desktop computer and a webcam to work, so once you have the programme and have calibrated it for the individual viewer, you are ready to start creating personalised art based on your mood.

Collomosse makes clear that the purpose of this is not to create a novel art form – though, doubtless, video and computer artists will be intrigued by the possibilities. The researchers are more focused on the technical possibilities of such subtle forms of interaction: “The empathic painting is really an experiment into the feasibility of using high-level control parameters, such as emotional state, to replace the many low-level tools that users currently have at their disposal to affect the output of artistic rendering.”

The empathic painting project was carried out with Maria Shugrina and Margrit Betke from the University of Boston. The images used in the project were created by the researchers using advanced artistic rendering techniques which give the computer-generated artwork the appearance of having been painted onto canvas.