Towards a Lightweight Approach for On-site Interaction Evaluation of Safety-critical Mobile Systems

Quantitative analysis of the survey

survey

Effectivity. Pragmatic and hedonic quality. Improvement potential.

For the evaluation of the app, we faced the problem that people participating in a large event, such as a soccer match, are not willing to spend time on completing a long survey or interview. Hence, the goal of this contribution was to select and perform an on-site mobile evaluation approach that fits this context and allows us to evaluate the user interaction. The evaluation took place during the FIFA World Cup 2014 and tested the usability of the app with 112 users in Brazil and in Germany.The moderator showed a picture of a fire in a stadium (see Figure 1, right) and asked the participants to imagine that the fire observed in the picture was burning on the other side of the stadium or venue and that they wanted to inform the firefighters by means of the RESCUER app. After this instruction, the moderator assigned the tasks to be performed by the participants. For each task, the observer completed the following items: Did the user accomplish the task successfully? Did the user accomplish the task without further questAfter performing all assigned tasks in the application, the users were asked to complete a Mini AttrakDiff questionnaire and a demographic data surveyions, after 1-2 questions, after 3-4 questions, or after 5 or more questions?

The evaluation took place during the FIFA World Cup 2014 and tested the usability of the app with 112 users in Brazil and in Germany.The participants of the study consisted of 64 male and 48 female visitors of the public viewing areas of the FIFA World Cup 2014 (N = 112). The participants’ age ranged from 13 to 68 years

How to evaluate the usability and interaction of a mobile application for emergency situations in the site of a large-scale event?

Rohrer (2014) presents a 3-dimensional framework for the systematic selection of UX research methods. Within this framework, several UX research methods are classified into three dimensions:Behavioral (what people do) vs. Attitudinal (what people say)Qualitative (why or how to fix) vs. Quantitative (how many and how much)Usage context (natural or near-natural use of the product, scripted use of the product, not using the product during the study, a hybrid of the others)

As a result of the evaluation, we found severe usability issues and concrete insights into how to solve them. For our evaluation approach, it means that on-site mobile evaluation is an appropriate method for improving the usability and interaction of safety-critical software systems.

we developed the RESCUER app, which runs on smartphones and allows the crowd to report about an emergency, thereby improving the process for rescuing humans in an emergency.The main purpose of the on-site evaluation was to assess the usability of the RESCUER app in the context of use.