Exerscent
Employer
Stockholm University
Year
2018-2020
Research project
Our Unique Sense Of Smell
Location
Stockholm/Malmö
Exerscent is the product of my research in the Our Unique Sense Of Smell project. Exerscent is a platform for training your sense of smell.
I was approached by a team of researchers at Stockholm University and Malmö University to engage in a project researching smell training in relation to, preventive care in the early stages of dementia, smell and taste habituation among kids, and recovery of temporary loss of smell.
The result came to be called Exerscent and this is a brief story of the design process.
Exerscent is a shippable game for olfactory training outside of the laboratory.
This work formed basis for the paper “A Method for Computerized Olfactory Assessment and Training Outside of Laboratory or Clinical Settings” that was published in iPerception journal May 2021.
The work and the publication of the paper generated some press in Swedish media.
Ingenjören did an interview with me on the topic (in Swedish): Ingenjören link.
Dagens ETC did a piece (in Swedish): Dagens ETC link.
SVT did an interview with a colleague: SVT link.
Continue reading for how my process played out and how Exerscent came to be.
As mentioned above, the larger research project with base at SU centered around smell training in relation to, preventive care in the early stages of dementia, smell and taste habituation among kids (picky eaters), and recovery of temporary loss of smell.
Common for these cases are that recurring exposure of smells and training of the cognitive capabilities of identifying, describing, pairing, sorting etc of smells are seen as praxis.
The outline was then roughly that we needed to monitor olfactory training over time. Although such training and monitoring was already conducted, the process was cumbersome and commonly performed in a laboratory setting with lab personnel always present instructing and documenting each session. In the rarer cases of subjects training at home with Sniffing Sticks [link to product] (a product for just this) the process had issues with subjects dropping off the training program or completing but cheating with the training frequency. This since they had no recurring motivator or follow-ups during the training period. FYI: Sniffing sticks are essentially scented markers that the subjects are sent home and told to sniff.
In other words, we had identified the practice of smell training set in labs, as well as a need for more, and faster, quantifiable data from training outside of the lab. It needed to be easy enough for subjects to conduct on their own and the activity should be motivating over time. It also needed to inform the lab of potential dropouts still or any side-stepping of the training program.
As you can tell by now this was an exiting opportunity for research through design and research of smell interaction.
...an exiting opportunity for research through design and research of smell interaction.
You could say I was employed as designer by the lab at SU with expectations from MAU to produce research through design. >This meant that I could see the SU lab as main stakeholder with the soul objective of collecting data from smell training. They didn't really have any input or opinions on how that was done. Their interest in interaction design and UX was purely from a perspective of getting the data. From MAU's perspective the opportunity to conduct interesting interaction design research was out spoken and expected. In other words I needed to serve SU's needs to have test subject conduct training and generate data as well as conducting proper design research of the process.
I identified the overarching design opportunity as:
-
HMW: Conduct monitored smell training outside of the lab environment?
Then broken down to:
-
HMW: Monitor the practice of smell training remotely and save the data?
-
HMW: Motivate subjects to keep training over time?
A softer design problem to bite into was how to approach the fact that the three cases mentioned above (dementia care, adolescent picky eaters, and individuals who by any reason lost their sense of smell) are all very different and aim towards very different user groups. And whatever solution is catered for one will not suit the others.
It became clear that we needed to design for adaptability and customization. One general design wouldn’t do for all. But at the same time, we needed a general platform to build upon. To customize, you need something to customize. Otherwise, it’s just designing, I guess.
Technology research and conceptualization.
Now that we settled on creating a basic platform to later customize, I could go to work on concretizing the design tasks.
The design process for me wasn’t exactly linear (as if they ever are, right?). At the beginning three different threads of research and design areas became concrete under the bigger concept of remotely monitored smell training. Going forward these three threads were entertained simultaneously.
I sketched out a system for training, monitoring, and delivery of hardware.
1
2
3
This sketch was used for communicating the proposed simple system of communication between the touch points of the process.
This became the model for data and hardware movement we worked after.
–1. The screen interface and the screen-based user activities.
We knew we wanted the users to train in their own homes and that we needed to register the training results at minimum. We also knew that we needed to keep the users motivated during a rather mundane and boring activity. As anyone fairly acquainted to this industry can imagine; the concept of ‘gamification’ was soon on the table.
I am for one not particularly fond of referring to gamification in a professional context. Mostly because it just doesn’t work. One cannot simply gamify a process to make users ‘stick’. Gamification aside we could at this point investigate making a proper game. As you may realize we did end up with a game, a very simple one, but an actual game, nonetheless.
The activity of smell training can be as simple as spending time with different smells. Smells that differ greatly, or that are very close to each other. It is the recurring activity of actively smelling, as in using the olfactory sense in connection to cognition, memory, language etc, that is crucial to entertain.
I sketched and we discussed several different game concepts that as a secondary function enabled users to smell different smell materials.
This idea builds on the classic card game of Concentration, or Memory. The game objective is to find two matching scents. The screen interface could be as playful as depicted but more importantly it should guide the user through the process, giving feedback and feedforward in the practise of picking and matching physical smell materials.
Another playful approach to getting the user to spend time exposing them selves to scents and activating their cognitive capabilities of processing scent.
The objective of this game is to help the unicorn put matching scent and color back to the object magically robbed of their characteristics.
The objective here is to identify what you are smelling with as few hints as possible. When a scent is smelled the player is presented with a number of alternatives, here playfully depicted as "suspects". It is the player's task to dig through layer after layer of disguises until they are able to identify the scent.
The final decision to create a basic quiz game was due to how it would fit the criteria of being customizable and adaptable to specific target groups.
A super lo-fi and manual prototype setup to get proof of concept regarding the practice of training ones sense of smell was designed and conducted by my over the summer of 2019. The data was gathered manually and there was no real computer intelligence in place. You could read up on the details here if you'd like to. Keep in mind I wrote this documentation with myself as test subject and refer to myself in third person in the text.
The objective from the game play perspective is to identify what is smelled. The player smells something and is subjected to a number of alternatives. Again, the number of alternatives, and what kind of alternatives (easy to mix up or in high contrast to the correct alternative) can be customized to fit the target group. Simple, easy, and reliable.
To achieve the user experience of smelling something and having a computer system knowing what you are smelling is not the most common interaction design issue out there...
From there the first crude prototype in Unity was made. It didn’t store any data at this point and several parts of the interface was mocked up. [image]
This prototype served several purposes.
-
It allowed me to stress test the technology.
-
Different RFID-readers and tags was evaluated and decided on.
-
-
It allowed me to evaluate the flow and game play.
-
It allowed for user experience exploration.
The visuals in this mock-up is not made by me. But served as initial inspiration and kept in the first rounds of usability iterations.
My process forward with the digital quiz game was to work out the flow of actions and explore possible visual concepts
As mentioned the project had tied new competencies to itself at this point and I needed document the design decisions more strictly to be able to communicate broadly to everyone involved. From code developers and interns to the research team at Stockholm University.
To get going with developing a suitable visual concept I started sketching even if I really don't have the skills nor the interest really...
As it is hinted in some of the sketches above I entertained the idea of having the the user login/identification process to be physical and without text input. The idea was to include a personal token in the kits that was sent out to the subjects. The token could be preset with an identification number that was tracked in the database. By logging a training session the user would simply start the session with placing their token on the reader and get going. Not implementing this was due to nothing else than time running out and text input served the basic needs. I feel it's a bit of a missed opportunity. But as I took my hands of the projects and handed it over I hadn't had time to add it to the manufacturing and tag administration process nor have our code developers revise the software. I do believe such small change would make significant difference to the overall experience. Coherent actions of input and movement of physical parts.
Non of the visual proposals I sketched out made it through very far and we ended up with a visual concept that came from one of our interns.
Much better than mine!
–2. The physical components such as smell material and potentially other physical objects.
The ongoing work in process was presented with a poster at ECRO conference in Trieste, Italy 2019.
This concludes the story of how the game concept and software was developed and designed.
Moving on to focus on the physical aspects of this system I could remind of how the software and hardware developing happened in and out of each other simultaneously. I've merrily choose to tell this story with these threads somewhat separated.
So let's go back in time and get started with the story of the hardware!
So called electronic noses that can identify odours through their chemical compound are too expensive and unobtainable for this scenario. We needed to think differently. The solution landed in RFID. By tagging each scent with unique ID-tags we could have the software know what was smelled by the user.
Regarding the look and feel of the physical components I got to work on exploring potential designs. This process resembled the process with the digital interface and game mechanics. I went wide and explorative in my sketching to narrow down with an informed settlement on something simple and customizable for our first design.
The next question to investigate was the physical form of the RFID-reader.
From the user perspective it needed be intuitive to what and where to move the physical parts of the game.
The metaphor of a “computer nose” was explored. What if you smell something and then let the computer nose have a whiff? It seemed playful and intuitive enough for a younger target group. It could either stand-alone aside of the computer or hung on the screen.
Above you can see another direction explored was an integrated reader and storage and holder for the smell material. Easy to ship, intuitive enough for all target groups, neat and elegant. Harder to scale for larger smell material libraries. Over-all a narrower scope for customization.
Below you can see another direction explored with an integrated reader and storage and holder for the smell material. Easy to ship, intuitive enough for all target groups, neat and elegant. Harder to scale for larger smell material libraries. Over-all a narrower scope for customization.
Yet another direction is this honest design proposal in regards to the encased technology. A clear and intuitive surface for reading the RFID-tags. Geometries designed for fast 3d-printing manufacturing. Non-intrusive aesthetics. Adaptable and customizable. Neat and elegant.
This is the design direction I presented and that we went ahead with for the first launch and test runs.
The top left is the of-the-shelf apparatus box that served as encasing during the first hardware and software trials. Middle is a size and format prototype. Front right is the first printed version with detachable bottom for accessing the technology within. You can also see the Le Nez Du Vin vial in this picture. Here with a printed sleeve that covers any visual hints of the content (some scents had slightly different colour tints). The sleeve was printed with a wider brim at the bottom to fit the size of the RFID-tags, it also served as a physical feedforward in relation to where to place the vial on the reader.
Blow-up sketch of the casing with mounting points and USB port access
While exploring the physicality of the hardware interface I also researched alternatives regarding the technology. We ended up using off-the-shelf components like the Arduino UNO and an RFID RC522-module. Before settling on that one i looked in to what else was being used in similar settings and what was available in general.
One of the things that got me excited was the self service systems commonly used for self registering of books and other media in libraries.
–3. The technology to support it all.
Self-service desk in the University library in Malmö. It uses RFID identification of the media through the flat antenna embedded in the desk under the red surface. On this system the RFID reader and the antenna are separate pieces of the hardware and are precisely adjustable when it comes to range both in shape of the reading space as well as distance.
I managed to get my hands on the exact same technology and hardware used in the library desks and was able to test and evaluate it for our purposes.
With this understanding of the technology I did round of ideation and sketching.
I explored different formats and functions. The major functional differences with this antenna and reader is the size of the antenna and the intelligence of the chip in the reader. This reader can effortlessly identify up to 30 different tags placed within the antennas range. Compared to the single tag capability of the Arduino and RC522-module.
in this sketch bot the larger antenna and the single tag antenna is depicted. With the larger antenna the system could tell each of the vials being on the platform and thereby also know which one is removed and being smelled.
With a rack for Sniffing Sticks.
ABOVE.
A completely different direction of the game mechanic would be to have the scented material in tagged vials and have tagged cards in a deck with matching pictures and descriptors printed on them. This would enable a screen-less interaction with feedback and feedforward from indicators of light and sound from the reader.
TOP and BOTTOM RIGHT.
Lo-fi prototype of encasing layout over the antenna. with the previously mentioned personal identification token.
Something needs to be said about the choice of smell material. In the beginning I was equipped from the project with odd containers of smell material. Such as a bottle of rose water and a vial of distilled citrus. It was clear that we needed to scale up the variety of scents immediately and consider accessibility of the material since we aimed at shipping kits of this training equipment with scent included. We investigated what was at hand of-the-shelf and found the Le Nez Du Vin wine connoisseur kits to suit our purpose. A kit of 54 individual scents stored in glass vials. Perfect! Again, the idea of customization in a later stage was with us and the Le Nez Du Vin kit seemed like a perfect way to get going fast with a suitable number of scents to begin working with.
Something needs to be said about the choice of smell material...
From getting the right design
to getting the design right.
We’ve reached a point where I’ve told you how all the different threads of this project got started and I’m about to wrap it up and tell you how it ended.
The concept now forming was that of a quiz game with a screen interface for displaying game-play choices and game statistics to the user. Physical scent material stored in vials and a physical interface for reading the scents.
Through-out this project I’ve been working from Malmö together with a college at MAU in close communication with the lab and people in Stockholm. At this point in the project, we managed to employ an interaction design student at MAU to help with JavaScript programming in Unity. We also took on interns to help with further usability testing and further stress testing of the administrative elements of the training software, and further smell material research.
From here most efforts went in to hone in the design and stabilizing the technical parts and elements of the concept.
The Unity code was revised as well as the Arduino code for reading and communicating data from the RFID-tags. Up until this point my crude code constructions had served well for prototyping the experience and getting proof of concept. At his point we called in the experts for making the code stable enough for more excessive user testing and eventually launching.
At this point we went to the ECRO conference in Trieste, Italy to present our work in process with a poster presentation, seen here.
Usability testing in the IOIO lab.
Handling of data.
We settled on storing the training data such as, user ID, score, time, etc. on an online database. Firebase was our choice almost right away. Easy enough and open enough for explorative design and prototyping. The user idea was kept without storing any personal data on the game database, the user ID is just a reference to the subject register stored on secure University platforms. This solution held up all the way to the launched version and is still being used.
This is how I handed over the project.
The system was up and running and ready to launch. We had our interns pack up the first kits and had them shipped for live usage. The data was collected. The software and hardware platforms was up and running.
I should say that this project is still in further development, as I write this there is development done in customization for a study of picky eaters among children. And outside of my own involvement this platform went to Japan in a project studying smell and memory with the elderly.
The very last thing I did before removing my hands from this was to compile a Do-It-Yourself guide for how to print the parts, buy the electric components, and set up the code to run it. The DIY-blueprint is available here.
The most interesting personal take-away from this project...
I hadn’t really worked with smell before going into this project. I read a lot of the available relevant research on the topic and to just spend time around colleagues in different capacities very involved in the latest smell research was very interesting and valuable to drive ‘my part’ of the project forward.
Working with smell in relation to UX and human computer interaction have led me to a few insights…
Our sense of smell does differ from our other senses in how it’s not as direct in relation to our cognition. In difference of our other senses, the signals from our olfactory receptors shoot passed the parts affecting linguistics for instance and straight to the reptilian end of the brain. As you can tell this is a layman’s description, a designer’s actually, but not a researcher of smell in particular. But this could inform us in the research of interaction design. It is from this that the term “in the tip of my nose” comes from. To be absolutely certain of what you’re smelling but not able to put a name to it. We know and recognise smells without language. Pairing that knowledge with words is a secondary action and something that can be trained and taught.
Another interesting more practical insight is that of how to deliver and communicate smell. In this project we ended up with the simplest way of delivering the smell to the user: by having a volatile liquid under the nose of the user. There are several other, more sophisticated ways. I’ve seen examples of awesome olfactory displays delivering scent in precisely controlled vortices shooting through the air. I’ve experimented myself with delivering scent by pushing it through ceramic piezo electric elements, much like air humidifiers work. And I’ve seen beautiful art installations delivering soap bubbles filled with scented and coloured smoke.
One must also understand that in the context of interaction and an interactive loop smell differs in how it lingers in the space, and how it “contaminates” surfaces. How it’s both slow for the brain to process, as well as slow and tricky to get rid of physically.
On another note, running this project with cross platform interactions and both physical and digital interfaces was a blast. Adding a complex material like smell and for me new protocols like RFID to the mix was a lovely challenge.
In terms of research smell is still an interesting and exiting field within interaction design.
//j –2022