DAMDID 2023. S1T2. Elnur Abbasov. UI Labeling Data Quality Matter for Predicting Website Aesthetics
SESSION 1. Image Analysis
Elnur Abbasov and Maxim Bakaev Does
UI Labeling Data Quality Matter for Predicting Website Aesthetics
The adoption of today’s data-intensive digital services relies on the overall user experience (UX), which is shaped not just by “hard” functionality, but also by “soft” subjective satisfaction. In the latter, aesthetic impression plays an important role (particularly since visually pleasing products are known to be perceived as more usable) and became a popular prediction objective for Machine Learning (ML) based user behavior models. Since datasets in the field of Human-Computer Interaction are generally too scarce for application of deep learning methods that could operate on raw website screenshots, they often undergo preliminary labeling. Although the common notion is that the quality of the labeling is important for the end quality of the predictive models, there were few attempts to quantify the effect. In a previous study, we unexpectedly found significant negative correlations between the input data quality and the models’ quality for Aesthetics and Orderliness subjective impressions. Our current paper is dedicated to validating the findings with another 557 website screenshots, 31 human participants labeling them, and 22 participants verifying the quality of their work. The non-parametrical models (Nadaraya-Watson kernel regression) with feature selection demonstrated somehow better performance, and the combined dataset better aligned with the expected effect of the labeling quality. Although our overall results are inconclusive, they might be of interest to ML practitioners and web designers who seek to automate the prediction of UX dimensions.