Usability Problem Reports for Comparative Studies: Consistency and Inspectability

Download

Date

2008-01-01

Author

Vermeeren, Arnold P. O. S.
Attema, Jelle
Akar, Evren
de Ridder, Huib
van Doorn, Andrea J.
Erbuğ, Çiğdem
Berkman, Ali E.
Maguire, Martin C.

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

140
views

0
downloads

This study explores issues of consistency and inspectability in usability test data analysis processes and reports. Problem reports resulting from usability tests performed by three professional usability labs in three different countries are compared. Each of the labs conducted a usability test on the same product, applying an agreed test protocol that was collaboratively developed by the labs. Each lab first analyzed their own findings as they always do in their regular professional practice. A few weeks later, they again analyzed their findings, but then everyone applied the same method (SlimDEVAN: a simplified version of DEVAN, a method developed for facilitating comparison of findings from usability tests in an academic setting). It was found that levels of agreement between labs did not improve when they all used SlimDEVAN, suggesting that there was inherent subjectivity in their analyses. It was found that consistency of single analyst teams varied considerably and that a method like SlimDEVAN can help in making the analysis process and findings more inspectable. Inspectability is helpful in comparative studies based on identified usability problems because it allows for tracing back findings to original observations as well as for laying bare the subjective parts of the data analysis.

URI

https://hdl.handle.net/11511/51605

Journal

HUMAN-COMPUTER INTERACTION

DOI

https://doi.org/10.1080/07370020802536396

Collections

Graduate School of Natural and Applied Sciences, Article

Suggestions

OpenMETU
Core

End User Evaluation of the FAIR4Health Data Curation Tool Gencturk, Mert; Teoman, Alper; Alvarez-Romero, Celia; Martinez-Garcia, Alicia; Parra-Calderon, Carlos Luis; Poblador-Plou, Beatriz; Löbe, Matthias; Sinaci, A Anil (2021-05-27) The aim of this study is to build an evaluation framework for the user-centric testing of the Data Curation Tool. The tool was developed in the scope of the FAIR4Health project to make health data FAIR by transforming them from legacy formats into a Common Data Model based on HL7 FHIR. The end user evaluation framework was built by following a methodology inspired from the Delphi method. We applied a series of questionnaires to a group of experts not only in different roles and skills, but also from various...
Ecodat: A web-based application and database for limnological monitoring data Değermenci, Ali; Özen, Can; Department of Biotechnology (2022-10-10) This study is focused on the elaboration of database application solutions for saving measured data in limnological monitoring. The purpose of the study is to give background information about monitoring data and the creation of a smart-accessible database system using query language and database management system Microsoft SQL Server 2012 while also providing a web application where researchers can reach, interact and save their fieldwork using C# .NET Core MAUI/Blazor framework. The thesis includes a data...
Optimum design of steel structures via differential evolution algorithm and application programming interface of SAP2000 Dedekargınoğlu, Özgür; Hasançebi, Oğuzhan; Department of Civil Engineering (2012) The objective of this study is to investigate the use and efficiency of Differential Evolution (DE) method on structural optimization. The solution algorithm developed with DE is computerized into software called SOP2011 using VB.NET. SOP2011 is automated to achieve size optimum design of steel structures consisting of 1-D elements such as trusses and frames subjected to design provisions according to ASD-AISC (2010) and LRFD-AISC (2010). SOP2011 works simultaneously with the structural analysis and design ...
Data analysis experiences: self-produced vs. Alien Data Sevinç, Şerife (2017-01-14) Data analysis is one of the core phases of qualitative research. This presentation aimed to share data analysis experiences of a team of researchers in a mathematics education study using three- tiered modeling research as a methodologic orientation. This study had three tiers: Tier 1-Pre-service mathematics teachers, Tier 2- A team of researchers, and Tier 3-Principal researcher. The team of researchers in Tier 2 composed of four researchers having different interests and teaching experiences as well as di...
Improvements on hyperspectral classification algorithms Özdemir, Okan Bilge; Çetin, Yasemin (2013-06-28) This study investigates the effect of training set selection strategy on classification accuracy of hyperspectral images. This effect is analyzed in conjunction with three other factors, namely the use principal component analysis on the input data, and the use of spatial information and choice of classifier. Support Vector Machines (SVM) and Maximum Likelihood (ML) classifiers are used for demonstration. Meanshift segmentation and majority voting are used for inclusion of spatial information. The effect of...

Citation Formats

A. P. O. S. Vermeeren et al., “Usability Problem Reports for Comparative Studies: Consistency and Inspectability,” HUMAN-COMPUTER INTERACTION, pp. 329–380, 2008, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/51605.