PI: A silver bullet for the trustability of Android applications?

Tasks with Standard PI

To achieve our goals, we wanted to simulate an Android 12 behavior as accurately as possible, and we created prototypes of real Android applications trought Figma prototyping tool. Prototypes are used by developers when usability testing and user feedback sessions are needed. They are high-fidelity representations that demonstrate how users interact with the new product or feature. We choose to employ four popular Google Play Store apps to provide results near the real-world use case scenario as feasible. -- WhatsApp, Messenger, Spotify, and Twitter.

Task 1

User need to open the Twitter app, and from the opening screen, he need to tap the blue button to create a new tweet. Once the user taps the button, he can see the screen where usually write a tweet; in this case, the tweet is already written, and the user has to publish it by tapping the "publish tweet" button. At this point, the user sees the published tweet. After a few seconds, the user gets a Gmail notification that he has to tap to open. Once done, the email appears with a code inside.
Task 2

Once WhatsApp is opened, the user will find a chat with messages to read on the screen. Opening the chat, he is asked to take a picture by tapping the camera button. Once the button is tapped, the camera preview opens, and the user has to tap the button to take the picture. At this point, the chat opens, and the photo is sent as an attachment. After a few seconds, a notification comes to the user from Gmail on the same screen. Opening the notification, the user finds the code in the text of the email.

Task 3

The user needs to open the Messanger app and find the chat with Simone. Opening the chat, he is asked to make a voice call by tapping the handset icon in the top right corner. After tapping the icon, the user sees the screen of the current call on the screen, and after a few seconds, he/she will have to close the call by tapping the red button. In a few moments, the user receives a notification from Spotify in which he is asked, once tapped, to find the song titled "Love me like you do" and play it. The playback screen shows the code.
Task 4

The user must open the Spotify app and find the "Dance Gavin Dance" playlist among those proposed. Opening the playlist, he finds a list of songs, and among them, he has to find and play the song "Love me like you do". Once on the playback screen, the user receives a notification from Messanger that the user must open. At this point, he has to find the chat with messages to read, and by opening it, he will find the code on the screen.

User Survey

Each user interacted with four tasks using four prototypes, two possible combinations of behavior (explicit/not-explicit) and resource (camera/microphone). The participant's group was chosen randomly, and each participant could answer only once and in only one of the two groups. We created a task for every combinations of factor and co-factors. The differences between groups are related to the different value of PI (visible/hidden) while the values of the co-factors (CB and R) are identical: each group had two not-explicit behavior tasks and two explicit ones; in two, the used resource was the camera and in the other two was the microphone. As for the POPUP experiment, the value of CB and R is unchanged, while in the tasks where the PI was not present, POPUP is shown.

Replication Package

Figma prototypes with PI

The prototypes can be used by framing the QRcodes on the right with an Android smartphone. Once you frame the QRcode you will be redirected to a Figma page. To use the prototypes as in the experiment, it is essential to click on "View full screen" when the page loads.