Facial Skin Analysis on Mobile Devices


ELLIOT NAYLOR
KESS 2 Masters by Research

My background is in Computer Games Development during which I was granted the unique opportunity to develop a game for the Road Safety Trust called ‘Virtual Road World’ (VRW). Whilst developing this game and communicating with staff, I was encouraged to apply for a KESS 2 ‘Masters by Research’ (MbyRes) project to study skin quality analysis. This was immediately of interest to me as my mother suffers from a skin condition, so it was a chance to research something I have always been passionate about.

Facial Skin Analysis on Mobile Devices

The goal of this project was to automatically assess facial skin quality with an easy to use mobile application. Methods were developed for the analysis of skin roughness, wrinkles, freckles, and pimples. Skin-care products could then be automatically recommended based on the participants’ requirements, without the need to visit skin-care shops in-person. Algorithms were developed to extract the area of the face, remove facial features, and track the facial rotation. That data was then used to automatically assess facial skin quality. A joint technique was developed and released as a journal unifying roughness and wrinkle detection using Gabor filtering and Dynamic line tracking.

INDUSTRY COLLABORATION

This project was in collaboration with Young Now Ltd, a beauty consumer products company which operates exclusively via online routes. The company’s ties to Asia presented an opportunity to become the first Welsh private legal entity to enter the booming Chinese beauty market.

Training the machine learning algorithms (image processing and facial recognition algorithms) on Asian datasets presented a unique challenge and it was interesting having to cater the algorithms towards a different country. For example, the algorithms were designed for Android over Apple, as there were far more of the company’s customers’ mobile devices using the Android operating system.

IMPACT

The development of the mobile application, together with Young Now’s tailored skincare products, is likely to become the first technology-enabled fully automated skincare personalisation system. The aim of this project was to make cosmetic personalisation more accessible and to reduce the need for visiting physical shops. The processing of customers’ personalised, and often complex, skincare requirements could all be achieved on mobile devices without human contact, which is especially useful with social distancing currently being a global primary concern. Furthermore, skin needs are usually subjective making it difficult for human personnel to recommend the ideal skin care product. The skin-care algorithms add a much-needed objectivity and provide fertile grounds for future research into the direction of facial features analysis.

“Supervision of Elliot was a great experience. Elliot had great motivation and passion for the field of computer vision. Project meetings were always productive, as Elliot always came back with good new ideas of his own and innovative current research in the field that he has found. During his Masters by Research, Elliot came across a novel solution to work out skin roughness and was able to publish a research paper on it.”
Dr Marius Miknis, Elliot’s Director of Studies

Progression from MbyRes to KESS 2 PhD

By following on through the KESS 2 programme, I have been able to upgrade from my MbyRes to a fully funded PhD in ‘Clinical Decision Support System for Skin Monitoring and Melanoma Detection using Machine Learning’. It continues to be a challenging but rewarding experience progressing from IT (which I have always enjoyed) to a Masters in skin analysis and finally onto a PhD researching the automation of skin cancer detection in collaboration with the NHS.

The biggest challenge was the learning curve when branching into new subjects and more recently, the current implications of COVID-19 with the NHS. However, with the constant support of my USW supervisors (Prof Janusz Kulon, Prof Marius Miknis and Professor Andrew Ware), and my industrial Partner (Dr Richard Motely), company supervisors (Dr Adam Partlow and Professor Colin Gibson) and KESS 2, I was able to complete my Masters, pass my viva and continue into a PhD in medical imaging analysis.
So far, I have written a journal during my Masters project on “Joint Roughness and Wrinkle Detection Using Gabor Filtering and Dynamic Line Tracking” and a couple more are planned for release during my PhD. I hope that my work will be substantial and that others will find it useful.

How KESS 2 and USW supported me

KESS 2 has been the backbone of my career, and without their continued support my research would not have been possible. The financial support from my KESS 2 budget and the training sessions received from KESS 2 and the USW Grad School provided me with a chance to concentrate solely on the research projects.

I had many fond experiences throughout my time with KESS 2; one of which was the teaching of classes within USW. It has always been a passion of mine to teach people, but it has been difficult because of my social anxiety. I took every opportunity to improve my social skills by showing my work in presentations to staff and students within USW. It was always a rewarding experience to improve my communication skills and have my work critiqued by others. I believe the opportunity to teach has helped me overcome these issues and gain confidence in myself.

PUBLICATIONS

“Joint Roughness and Wrinkle Detection Using Gabor Filtering and Dynamic Line Tracking”
https://www.cscjournals.org/manuscript/Journals/IJCSS/Volume13/Issue5/IJCSS-1507.pdf


Knowledge Economy Skills Scholarships (KESS 2) is a pan-Wales higher-level skills initiative led by Bangor University on behalf of the HE sector in Wales. It is part-funded by the Welsh Government’s European Social Fund (ESF) Convergence programme for West Wales and the Valleys.

For further information about how your organisation could benefit from participating in KESS 2, please contact the USW KESS team via email: kess@southwales.ac.uk

The USW KESS team is based within Research and Innovation Services (RISe), part of Research and Business Engagement. RISe is a dedicated team providing specialist support for Research Infrastructure; Postgraduate Research Support; Research Impact; and Income Generation. http://research.southwales.ac.uk/