|
|
|
:arrow\_backward: [Home](home)
|
|
|
|
|
|
|
|
[[_TOC_]]
|
|
|
|
|
|
|
|
---
|
|
|
|
# Part 4: Harmonising Across Different Studies
|
|
|
|
In this part of the workshop, you will harmonise the variables you imported to the repository earlier.
|
|
|
|
|
|
|
|
The target variables are described in an harmonisation dictionary available [here](https://gitlab.inesctec.pt/wp4-recap/wp4_workshop/raw/master/wp4_workshop_harmonization_dictionary.xlsx).
|
|
|
|
The goal is to transform the original variables into the ones in the harmonisation dictionary using the tools provided by the Data Repository.
|
|
|
|
|
|
|
|
## Using _Views_ in the Data Repository
|
|
|
|
The harmonisation process in the Data Repository is fully based on the concept of `views`.
|
|
|
|
A view is a virtual table in which variables have been derived from some other table.
|
|
|
|
|
|
|
|
In the context of harmonisation, we can pull and transform data from a table and then save the resulting harmonised variables in a view:
|
|
|
|
|
|
|
|
<img src="img/harmonisation/overview.png" alt="overview" width="700"/>
|
|
|
|
|
|
|
|
Harmonisation via Opal is self-documenting. After the harmonisation is complete, all the scripts that were used are automatically saved as variable attributes and are thus always available for inspection.
|
|
|
|
|
|
|
|
## Upload the Harmonisation Dictionary
|
|
|
|
The first step in the harmonisation process is to upload the harmonisation dictionary into the repository.
|
|
|
|
|
|
|
|
* Log into the Data Repository
|
|
|
|
* Click on `Projects` on the top bar and then select your project
|
|
|
|
* Click on the `Files` tab on the left panel and then click on `Upload`
|
|
|
|
* Browse to the harmonisation dictionary file and click `Upload`
|
|
|
|
|
|
|
|
## Create a View over the Original Table
|
|
|
|
Just like regular tables, views can be created by using a dictionary. In this case, you need to create a view using the harmonisation dictionary you just uploaded:
|
|
|
|
|
|
|
|
* Click on the `Tables` tab on the left panel and then click on `Add Table` > `Add View`
|
|
|
|
|
|
|
|
<img src="img/harmonisation/create_view_1.png" alt="create_view_1" width="700"/>
|
|
|
|
|
|
|
|
* Type in a name for the view (e.g. epice-pt_data_harmonised)
|
|
|
|
* In the table references, select the table from which the variables will be derived (the table you created on part 1 of the tutorial) and click `Add`
|
|
|
|
* Click `Browse` and select the harmonisation dictionary
|
|
|
|
* Click `Save`
|
|
|
|
|
|
|
|
<img src="img/harmonisation/create_view_2.png" alt="create_view_2" width="500"/>
|
|
|
|
|
|
|
|
The view you have created contains the target variables in the harmonisation dictionary and also a reference to the original table. But no data has yet been pulled from the original table, and so the view does not yet contain any data.
|
|
|
|
|
|
|
|
## Harmonise Each Variable
|
|
|
|
To pull and transform data from the original table, you need to individually harmonise each variable in the view.
|
|
|
|
The harmonisation approach for each variable depends on whether it is a categorical variable or not:
|
|
|
|
|
|
|
|
* **The Target variable is categorical**
|
|
|
|
For categorical target variables you can use the graphical web interface:
|
|
|
|
|
|
|
|
1. Select a categorical variable on the view
|
|
|
|
1. Click on `Derive` > `Categorise another variable to this`
|
|
|
|
|
|
|
|
<img src="img/harmonisation/harmonise_categorical_1.png" alt="harmonise_categorical_1" width="700"/>
|
|
|
|
|
|
|
|
1. Select the variable from which values will be derived (a variable from the original table) and click `Next`
|
|
|
|
1. Map the original values to the new values defined in the harmonisation dictionary
|
|
|
|
|
|
|
|
<img src="img/harmonisation/harmonise_categorical_2.png" alt="harmonise_categorical_2" width="450"/>
|
|
|
|
|
|
|
|
1. Click `Next` and then `Finish`
|
|
|
|
|
|
|
|
* **The Target variable is continuous**
|
|
|
|
For continuous target variables you have to use [MagmaJS](http://opaldoc.obiba.org/en/latest/magma-user-guide/methods.html) scripts.
|
|
|
|
All the scripts needed to harmonise the EPICE-PT variables are available [here](magmajs).
|
|
|
|
1. Select a continuous variable on the view
|
|
|
|
1. Click on the `Script` tab and then `Edit`
|
|
|
|
1. Write the script and click `Save`
|
|
|
|
The script is applied to each row of the original table. For example, if your script is:
|
|
|
|
```javascript
|
|
|
|
$('weight').div(1000).round(2)
|
|
|
|
```
|
|
|
|
It means that you are pulling the values of a variable named `weight` from the original EPICE-PT table, dividing them by 1000 and then rounding the resulting number to two decimal places (this could be a script to convert grams to kilograms, for example).
|
|
|
|
The final values are then stored in the view. |
