Gitlab workflows

Showcasing different scenarios

Here we present different scenarios in which researchers can use Git

An overview of Git

This diagram summarizes the basic steps for using Git. We can perform these steps using the command line or using a program with a Graphic User Interface.

%%{init: { 'theme': 'base', 'sequence': {'showSequenceNumbers': true}} }%%

sequenceDiagram
    participant Workspace
    participant Staging Area (index)
    participant Local Repository
    participant Remote Repository

    note over Remote Repository: 🌐 Hosted Online
    
    rect rgb(235, 242, 240)
    opt Work on scripts locally and send to remote
    Workspace->>Staging Area (index): git add
    note right of Workspace: Prepares contents for commit
    
    note right of Staging Area (index): Records changes to the repository
    Staging Area (index)->>Local Repository: git commit
  
    note right of Local Repository: Upload local content to remote
    Local Repository->>Remote Repository: git push
    end
    end
    
    rect rgb(252, 243, 248)
    alt Fetch changes from remote then merge with local
    note right of Local Repository: ️Downloads refs and objects
    Remote Repository->>Local Repository: git fetch
    note right of Workspace: Incorporate changes 
    Local Repository->>Workspace: git merge
    
    else  or do fetch & merge  at once
    note right of Workspace: Downloads and incorporate changes  
    Remote Repository->>Workspace: git pull
    end
    end
    

Workflows in a single project

Here we describe several scenarios in which there are several researchers working on separate analyses or features within the same project. They use the same source data but their scripts are independent. We assume that each researcher is the main contributor to a set of scripts for a given analysis, study or experiment within the main project.

Multiple repositories

• All project-related
• The scripts subfolder is a clone of a remote repository
• Each researcher manages a subfolder within the repository and pushes/commit changes within that folder

Pros, Cons & Warnings

✅ One can manage permissions and share only the scripts related to one analysis or user. The local clones can still be within the same project folder (see example 1 below)
⚠️ The different repositories should be preferably under a common git group/user, namely , the lab’s Gitlab or Github.
⚠️ Researchers may keep their local clones in their personal folders (outside project: see example 2). In this case their internal collaborators or principal investigator will only be able to see their scripts through the repository , so changes should be committed regularly.
🔻 If there are many users and analyses this can result in many repositories associated to a project

Folder tree

Example 1

Project/              NAS folder accessible to all researchers       
├─ data
├─ outputs
├─ scripts        
│   ├─ researcher1*   [Cloned Git repository 1] 🌐
│   └─ researcher2    [Cloned Git repository 2] 🌐  
└─ ..

*subfolders may be named by analysis, experiment or study within the project

Example 2

Project/              NAS folder accessible to all researchers       
├─ data
├─ outputs
└─ ..

researcher1/         Location may differ from that of the project     
└─ scripts            [Cloned Git repository 1] 🌐

researcher2/           
└─ scripts            [Cloned Git repository 2] 🌐

Workflow

Each researcher has completely independent repositories. Researchers may edit and create code directly in the main branch or creating development branches and then merging with the main.

Repository 1

%%{init: { 'theme': 'base', 'gitGraph': {'showBranches': true,'showCommitLabel': false}} }%%
gitGraph
  commit
  commit
  branch branch1
  commit
  commit
  checkout main
  merge branch1
  commit

Repository 2

%%{init: { 'theme': 'base', 'gitGraph': {'showBranches': true,'showCommitLabel': false}} }%%
gitGraph
  commit
  commit
  branch branch1
  commit
  commit
  checkout main
  merge branch1
  commit

A single repository

• All project-related files in one location accessible by all contributing researchers
• The scripts project subfolder is a clone of a remote repository
• Each researcher manages a subfolder within the repository and pushes/commit changes within that folder

Pros, Cons & Warnings

✅ All scripts in one place
🔻Risk of accidental interference with colleague’s changes
🔻You can’t share or manage access to only one researcher’s scripts
⚠️ Researchers need to make sure they push and commit only their changes in their subfolders
⚠️Note that naming script subfolders by author is not informative and can work for a project in development, but not for publishing (sharing with external collaborators)

Folder tree

#| code-overflow: wrap 

Project/              ~NAS folder accessible to all researchers~       
├─ data
├─ outputs
├─ scripts*           [Cloned Git repository] 🌐
│   ├─ researcher1 
│   └─ researcher2     
└─ ..


*subfolders may be named by analysis, experiment or study within the project

Workflow

The different researchers access the project folder and work within their folder. When they are done they and push their changes and commit them to the main branch of the project’s repository

%%{init: { 'theme': 'base', 'gitGraph': {'showBranches': true,'showCommitLabel': false}} }%%
gitGraph
  commit
  commit
  branch researcher1
  commit
  commit
  checkout main
  branch researcher2
  commit
  commit
  checkout main
  merge researcher1
  merge researcher2
  commit