STOP Data and Knowledge Management, Pre-Processing, and User Interaction layer second release

Introduction

The STOP project will address the health societal challenge of obesity with the specific objectives of mitigating the enormous and growing Health Care costs of obesity and related health issues. The project proposed solution is the foundation of an innovative platform to support Persons with Obesity (PwO) with better nutrition under the supervision of Healthcare Professionals. Therefore, the STOP Platform will capture various PwO data from different kind of smart sensor streams and Chatbot technology, manage and enrich available data with existing knowledge bases and fuse these by machine learned driven Data Fusion approaches for sophisticated AI data analysis.

The WP2 has a strong relation with other WPs, especially in refining and implementing of the STOP platform architecture which is for convenience as well depicted in Figure 1-1. To provide a better overview of the integration of this report with the architecture, we will shortly summarize some key points. The architecture contains a Persistant Layer (located in the cloud), which consists of a Structured Content area (Digital Library and Multimedia Archive) for the Content and a Knowledge Network (KN) for KOs (details see D2.1). The KM-EP is based on a flexible integration architecture with service harmonization capabilities (an architecture about KM-EP could be found in D2.1).

Figure ‑: STOP Platform Architecture Data & Knowledge Ecosystem

As marked with different colours in the figure, the platform is combined the works of WP2 and WP3. Overall, the project Work Package 2 (WP2), which is described in this deliverable, is focused on topics related to:

  • Sensory Data Fusion: Merging raw data with captured data into KOs
  • Data & Knowledge Management – KM-EP: Manages the SDF input and output
  • Information Visualisation: Visualise the Content and Knowledge

Other parts of the STOP platform, such as, e.g., AI Chatbot and Gamification App, are reported in other deliverables. This report is the third deliverable of the work package 2 with scope on the second STOP Platform release that was conceptualised in the deliverable 2.1 and realized in the deliverable 2.2 (see Figure 1-2).

Figure ‑: Relation of STOP deliverables

The aim of this report is to document the project achievements additional to the first release on the following objectives:

  • O2.2: Conduct software implementation and refinement activities of policies and workflows to enable access and use of data and meta data, managed through the STOP Platform, besides establishing security functionalities, Semantic Data Integration and Fusion.
  • O2.3: Conduct software implementation and refinement activities, of innovative Visualisation algorithms and techniques to support case pilot specific user interaction with managed data.

To address the scope of this report and above objectives, project work in WP2 is split in in five tasks. The structure of this report will follow the order of tasks (from DoA) that are already worked in during various secondments. The first four tasks are already relevant for this report and copy and pasted for convenience:

  • T2.1 Task lead: CIT. Contributors: FTK, Sensory interfaces Implement an initial selection of sensory interfaces for the STOP Platform. These interfaces will provide a generic approach to engage different types of wearable sensors into the STOP Platform. To be further managed and enriched through the Data Management component. (Secondment No. 2, 12, 23)
  • T2.2 Task lead: CIT. Contributors: FTK, Data Fusion Research, selection and integration of machine learning algorithms to be used sensory Data Fusion. (Secondment No. 2, 12, 23)
  • T2.3 Task lead: FTK. Contributors: CIT, OKK. Data & Knowledge Management: implementation, refinement and documentation Implementation, refinement and documentation of Semantic Data Integration and Fusion functionalities. This will enable broad subsequent processing and data exchange throughout system barriers and comprehensive Knowledge Management with available data. Actually, the semantic Data Fusion could as well be regarded as a semantic information integration process that enables a unified and unique data access interface. Therefore, amongst others a formal extensible ontology will be developed and validated to represent the relevant concepts and relations of the food domain. Address ethical implications that goes along with the platform implementation and refinement of workflow and access functionalities. (Secondment No. 3, 7, 25)
  • T2.4 Task lead: OKK. Contributors: CIT, FTK. Visualisation: Implementation, refinement and documentation. An intuitive visualisation dashboard will be designed that is personalised to each user. The visualisation will complete the loop in the quantified-self paradigm by enabling effective feedback for reflection, planning, actionable insights and for self-understanding. The visualisation will be co-designed with users to ensure that it is useful and user-centred. The visualisation will be interactive in nature and will allow the user to interrogate the data on multiple levels of detail where the sophistication and granularity is controlled by the user. Interpretation of the visualisation will be validated using the concurrent think-aloud protocol otherwise known as a cognitive walkthrough with users. Finally, the visualisation will be audited using design principles and heuristics from Gestalt psychology. (Secondment No. 8 & 13).

The report documents two cognitive walk through of the platform on its Web and Mobile application. For the web application, the report only describes the components that were developed after the first release. Each walkthrough demonstrates stepwise the functionality of the system components with screenshots and their descriptions.

Web Application: Cognitive Walkthrough

In this section, new components that were added after the first release are presented. The walkthrough is split into a description of the Google Fit importing feature and the new Fitness Virtual Event component.

Google Fit Import

Beside Fitbit, Google Fit is a popular fitness data storage. Almost all fitness devices and apps provide an option to connect to Google Fit in order to export/store their data there. Users have the possibility to import their fitness data from Google Fit by going to menu Fitness > Import > Google Fit (Figure 2-1).

Figure ‑: Import from Google Fit

Users will need to authorize the STOP portal to access their fitness data by logging in using their Google account. Figure 2-7 presents Google’s login interface.

Figure ‑: Login Google account

After authorized the STOP portal, users will be redirected back to the portal. Here, they can choose to access to new data created since the last import. If users haven’t import data from this service before, the system will suggest loading all data created from the last 3 months.

Users have the option to change the import date then click on the “Load New Records” button to download their fitness data (Figure 2-8).

Figure ‑: Load data from Google Fit

After loading fitness data from Google Fit, users can preview them. There are 3 types of fitness data loaded: Activities, Exercises, and Weights. As shown in Figure 2-4, users can click on the “Activities” tabs to browse the activity records. They can also search for specific record by using the search box.

Figure ‑: List loaded activities

Similar to activity, users can browse exercise records in the “Exercises” tab. The related data for each exercise is shown in Figure 2-5.

Figure ‑: List loaded exercises

Users can look at their weight-related data each day in the “Weights” tab as shown in Figure 2-6.

Figure ‑: List loaded weights

After viewing all their data, users click on “Import” button at the end to import them to the STOP portal. The system will show how many records were imported successfully as in Figure 2-9.

Figure ‑: Import data successfully

Fitness Virtual Event

Fitness Virtual Event was created to encourage users to exercise more. Portal admins can create a new event by going to menu Fitness > Events (Figure 2-8).

Figure 2‑8: List created events

In the “Add/Edit Event” page shown in Figure 2-9, admins need to fill out the information related to the event, such as name, description, start date and end date. There are several goals for an event. Admins can add new goal with the “New Option” button and choose the type of goal and its value. After completed this step, admins click on the “Create/Save” button to save the input to the database. Admins can stop this process and go back to the listing page anytime by clicking on the “Cancel” button.

Figure 2‑9: Edit an event

On the listing page, admins can delete an event by clicking on the “Delete” button at the end of the row. Figure 2-10 shows the system asking once more time before removing the record from the database.

Figure 2‑10: Delete an event

.

Users can find an event in the search page or the event can be advertised somewhere with a banner. When users click on this banner, they will be redirected to the event page (Figure 2-11). Here they can find out all information related to the event and choose to participate. In order to do so, they need to select their goal and click on the Participate button.

Figure 2‑11: Event information page

Users can also find out which participations have the best performance and filter out this list by event goals (Figure 2-12). They can also search for information in this list. For example, which position they or their friends are right now. This will help users get motivated and achieve better results.

Figure 2‑12: Event result page

Users can review all events they participated in the Fitness Overview page (Figure 2-13), which can be navigated in menu Fitness > Overview. Here information regarding to their participation and performances, such as, chosen target, joined data, and current progress, is displayed.

Figure 2‑13: My events page

Mobile Application: Cognitive Walkthrough

In this section, the STOP Fitness Manager mobile app is introduced. The application is presented in a cognitive walkthrough where its features and functionalities are described in a series of screenshots and descriptions.

User Login

In order to use the app, users are required to login using their username and password. Unregistered users press on “Create a new account” to go to the STOP portal (see Figure 3‑1). They can also request a new password using the “Forgot Password?” link.

Figure 3‑: User login UI

Fitness Overview

After logged in, users are redirected to their fitness overview. It provides a quick look at users’ fitness current progress compared to their target each day. In Figure 3-2, users’ status in water consumption, food consumption, exercises, and steps, are visualized. Users can press on a section to add a new record.

Figure 3‑: Create a fitness profile

Activity Manager

To manage activity data, users can press on the Activities tab. In Figure 3-3, all activity records are listed. Users can either edit an existing record by press on it or create a new activity by press on the “+” button.

Figure 3‑: List created activities

In the “Add/Edit Activity” screen as shown in Figure 3-4, users need to fill out the required fields, such as “Date”. Other fields, such as “Time”, “Distance”, “Steps”, are optional. After filling out all the information, users press on “Save” to create a new activity or save the editing record. Furthermore, users can also press on the “🡨” button to come back to the listing screen.

Figure ‑: Edit an activity

On the listing screen, users swipe left on a record to remove it from the database. The system will ask for users’ confirmation to make sure that they did not do this by mistake (Figure 3-5).

Figure ‑: Delete an activity

Exercise Manager

Next to the “Activities” tab is the “Exercises”, where users can manage their exercise records (Figure 3-6). Similar to “Activities”, users can press on a record to edit it or the “+” button to create a new exercise.

Figure ‑: List created exercises

In the “Add/Edit Exercise” screen shown in Figure 3-7, users need to fill out the information related to their exercise, such as name, date, description, distance, etc. After completed this step, users press on the “Save” button to save the input in the database. Users can stop this process and go back to the listing page anytime by pressing on the “🡨” button.

Figure ‑: Edit an exercise

On the listing screen, users can delete a record by swiping left on the record row. Figure 3-8 shows the system asking users once more time before removing the record from the database.

Figure ‑: Delete an exercise

Meal Manager

Users can manage their food consumption by selecting the Nutrition tab (Figure 3-9). Here they can add a meal that they just ate or edit the created record in the list.

Figure ‑: List created meals

In the “Add/Edit Meal” screen in Figure 3-10, users need to enter the calories they consumed then press on the “Save” button to save the record. Otherwise, users can stop this process and go back to the listing screen anytime by pressing on the “🡨” button.

Figure ‑: Add a meal

Similar to other type of data, users can delete a meal by swiping left on record row. The system will need users’ confirmation once more time before removing the record from the database as shown in Figure 3-11.

Figure ‑: Delete a meal

Water Consumption Manager

Users can manage their water consumption by switching to the Water tab. Here they can add a drink to the system or edit the added drink in the list as shown in Figure 3-12.

Figure ‑: List created water consumptions

In the “Add/Edit Water” screen as shown in Figure 3-13, users fill out the timestamp and volume of the drink then press on the “Save” button to save it in the database. Otherwise, users can stop this process and go back to the listing screen by pressing on the “🡨” button.

Figure ‑: Drink water

In the listing screen, users can delete a drink by swiping left on the record row. The system will ask users if they are sure before removing the drink from the database (Figure 3-14).

Figure ‑: Delete a water consumption

Weight Manager

In the Weights tab, users can manage the changes of their weight over time. Here they can add, edit, or delete their weight records. Figure 3-15 shows a list of user’s weight records.

Figure ‑: List created weight records

In the “Add/Edit Weight” screen in Figure 3-16, users type in their weight along with BMI and Fat value then press on the “Save” button to save the record in the database. Users can also press on the “🡨” button to stop this process and go back to the listing screen.

Figure ‑: Add a weight record

In the listing screen presented in Figure 3-17, users can delete a record by swiping left on the record row. The system will ask users if they are sure before removing the record from the database.

Figure ‑: Delete a weight record

Mobile Application Download

The application was released on both Android and iOS platform. Users can download it using the following URL:

Summary and Future Works

In this deliverable, we have reported the work of developing and deploying the second version of the STOP Portal. This includes the updated web application where new import module was added to support data exchange from Google Fit and a new component called “Fitness Virtual Event” was implemented to encourage users to exercise more. Furthermore, a mobile application was introduced to enable users to interact with the STOP Portal on the go. The app has 6 modules that covers users’ fitness data. With this app, users can manage their progress, activity, exercise, meal and water consumption, and weight in a quick and easy manner. The works reported in this deliverable has fulfilled the goal of the WP2, which is to implement and refine Data and Knowledge Management, Pre-Processing and User Interaction Component and Service.

The developed prototype will be the base and paves the way to the incoming pilot randomized controlled trial, which will be conducted in M44. To support this trial, a tutorial describes new features and updates can be composed and uploaded to the Portal to help users getting familiar with the new version.