Guest post by PhD Student Mark Allen from the University of Cambridge you can read more about me at my blog at https://naturalhouse.org.uk/
About me
I am a second year PhD student at the Department of Engineering, University of Cambridge with broad academic interests ranging from architecture to biomimetics. A civil engineer by training, these interests have now led me look at how buildings directly affect people.
About the Project
We spend over 90% of our time indoors, making these environment key to our health and well-being. But the built environment can often be poorly designed in relation to human well-being, causing health issues and other more serious physical illnesses. The title of this project, ‘Capturing Occupant Well-being Through Facial Expressions’ is purposefully optimistic, particularly considering that experts in the realm of psychology still cannot agree as to how exactly well-being should be measured. Capturing occupant well-being in buildings is rarely done, instead occupant ‘satisfaction’ is usually measured using surveys. However, there has been a rapid increase in interest in well-being in buildings as organisations have become more and more aware of its potential benefits.
The overarching idea of this research project is to create a method of capturing occupant well-being constantly, without interfering with people as they work or go about their daily lives. In the same way that there exist sensors to measure temperature or CO2, what if we could create a sensor to measure ‘well-being’ in the same way? This project will take advantage of recent improvements in emotion recognition software’s and compliment them with other software’s, such as speech and posture detection, to see how they could be used to gauge elements of the well-being of building occupants. These software’s are quite prevalent now and with the incorporation of artificial intelligence and machine learning, some can even recognise emotions with a greater accuracy than humans.
Emotion API
This is where Microsoft’s Cognitive Services fits with my research. Through securing one of the Microsoft Azure Research Awards , I have been able to do a small trial study. Six office occupants agreed to take part, in an office located in Cambridge, UK. In order to check that the software was working correctly, I used a video of the Trump and Hillary debate to be able to compare my results with the article by The Economist . With Trump looking a little sad but the software working well, I began recording the 6 occupants every day using webcams mounted on their monitors. This monitoring took place for two months, with occupants recorded at 480pi and 5fps resulting in 500GB of data. The post processing of this data is still in progress, but initial results are quite interesting.
The graph below shows how the average facial emotions of two occupants vary over several days in relation to office temperature.
Future Work and Applications
There is still a lot of data to be processed for this initial pilot study. In hindsight, post-processing is very labour intensive, with each video having to be cut up into 100mb pieces. Instead, future studies will use a different capture technique, taking small videos and processing on the fly, instead of storing large videos and chopping them up to process later. All this information can then be fed into a database which is updated every 15 minutes or so.
Using these software’s to gauge occupant well-being could allow the Building Management System (BMS) to proactively change environmental conditions within a building (such as temperature or CO2) with the objective of maximising occupant well-being, productivity, and creativity, rather than minimising energy use. Additionally, it could be used to create a database of evidence for ‘design for well-being’ which is currently lacking. Overall if successful, this project will provide a totally new way of measuring well-being in buildings, enabling a smarter and healthier method of controlling building elements and even helping to inform better design.
Resources
Microsoft Cognitive Services https://azure.microsoft.com/en-gb/services/cognitive-services/directory/
Microsoft IOT Edge https://blogs.microsoft.com/iot/2017/05/10/microsoft-azure-iot-edge-extending-cloud-intelligenc...