Using Runners
GitLab Runner is an essential component of GitLab CI/CD that acts as an agent for running jobs defined in CI/CD pipelines. It serves as an executor, allowing the distributed execution of tasks specified in the .gitlab-ci.yml configuration file.
Group Runners
We have several runners in our group to meet performance or hardware requirements for our jobs. These runners are shared for all projects in ONES-AI group.
cpu-runner Runner
This runner runs jobs that are untagged or tagged as cpu-runner. The runner has two 20C/40T Intel Xeon CPU and 256 GB memory to execute concurrent compilation jobs fast. The workloads are executed in docker containers. We allow 8 concurrent jobs in this runner.
gpu-runner Runner
This runner runs jobs that are tagged as gpu-runner. The runner has two 12C/24T Intel Xeon CPU and 256 GB memory to execute concurrent compilation jobs fast. It is also equipped with an Nvidia A100 card for GPU-intensive workloads. The workloads are executed in docker containers. We allow 1 concurrent job in this runner.
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.216.01 Driver Version: 535.216.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA A100 80GB PCIe Off | 00000000:63:00.0 Off | 0 |
| N/A 37C P0 44W / 300W | 13MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 1595 G /usr/lib/xorg/Xorg 4MiB |
+---------------------------------------------------------------------------------------+
demo-board Runner
This runner runs jobs that are tagged as demo-board. The workloads are executed in a special board.
rvx-board Runner
This runner runs jobs that are tagged as rvx-board. The workloads are executed in a special board.
Adding New Runners
When you need more runners, you can add them at ONES-AI Group > Build > Runners > New Group Runner menu.
Docker Runner
A Docker Runner in GitLab CI/CD is an execution environment provided by GitLab Runner that enables jobs to be executed inside Docker containers, offering isolation, reproducibility, and compatibility across diverse environments.
When adding a docker runner, select docker executor when registering a gitlab runner. You can install gitlab runner using the guidelines provided by GitLab.
SSH Runner (for demo boards)
The SSH executor in GitLab CI/CD refers to a configuration option in GitLab Runner that allows jobs to be executed remotely over SSH, providing a secure and flexible way to run CI/CD tasks on external machines or environments using the SSH protocol.
As our boards support SSH to run workloads remotely, we use SSH runner in this case. Follow the steps below to add a new SSH runner.
Prepare a Board
Install hardware.
Secure access to the board using network firewalls.
Locate a server which has SSH access to the board.
Install a SSH Runner
In the server where you have SSH access to the board, execute the followings:
Add a system user:
sudo adduser --system --home /var/lib/board-runner --shell /bin/bash --group board-runnerSwitch to the user:
sudo su board-runnerRegister a runner:
gitlab-runner registerSelect
sshexecutor when registering a gitlab runner.Provide credentials required to login to the target board.
Type exit:
exitConfigure a systemd service:
sudo nano /lib/systemd/system/board-runner.service[Unit] Description=GitLab Runner for ETRI board After=network.target syslog.target [Service] Type=simple User=board-runner Group=board-runner WorkingDirectory=/var/lib/board-runner ExecStart=/usr/bin/gitlab-runner run StandardOutput=syslog StandardError=syslog [Install] WantedBy=multi-user.target
Reload systemd:
sudo systemctl daemon-reloadStart service:
sudo systemctl start board-runner.serviceEnable service:
sudo systemctl enable board-runner.serviceCheck status:
systemctl status board-runner.service
Status should be active.
In this case, we used the username board-runner, but feel free to customize it according to your preferences.
Shell Runner (for rvx boards)
The shell executor in GitLab CI/CD refers to a configuration option in GitLab Runner that allows jobs to be executed locally.
We use the shell executor because our rvx board is connected via USB and requires commands to be executed from the system shell. Follow the steps below to add a new shell runner.
Prepare a Board
Install hardware.
Prepare all subsystems that this hardware requires.
Install a Shell Runner
In the system where you have access to the board, execute the followings:
Add a system user:
sudo adduser --system --home /var/lib/board-runner --shell /bin/bash --group board-runnerSwitch to the user:
sudo su board-runnerRegister a runner:
gitlab-runner registerSelect
shellexecutor when registering a gitlab runner.Type exit:
exitConfigure a systemd service:
sudo nano /lib/systemd/system/board-runner.service[Unit] Description=GitLab Runner for Board After=network.target syslog.target [Service] Type=simple User=board-runner Group=board-runner WorkingDirectory=/var/lib/board-runner ExecStart=/usr/bin/gitlab-runner run StandardOutput=syslog StandardError=syslog [Install] WantedBy=multi-user.target
Reload systemd:
sudo systemctl daemon-reloadStart service:
sudo systemctl start board-runner.serviceEnable service:
sudo systemctl enable board-runner.serviceCheck status:
systemctl status board-runner.service
Status should be active.
In this case, we used the username board-runner, but feel free to customize it according to your preferences. Please note that the user board-runner should have sufficient privilege to run commands to the target board. Additionally, be aware that jobs run by this runner are executed in the system shell, so it should be restricted to use only by trusted entities.
Runner Permissions
To run the pipelines using group runners, you must be a member of ONES-AI group. For others, you can use shared runners (or instance runners) provided by gitlab.com or you can bring your own project runners if possible.
Tip
For security reasons, pipelines for merge requests submitted from outside the ONES-AI group do not run on group runners. To run jobs on the group runner, a group member must make at least one commit to the branch of the merge request.
Project Runners
If you have specific requirements for a project, you can also add project-level runners. These runners are not shared to the group, and can be enabled for your projects one by one.
To add project runners, visit your project page, and go to Settings > CI/CD > Runners.
Please note that when sharing your project runner between multiple project, you need to have at lease maintainer privilege on both projects, and the runner should not be locked. For more information, you can refer to this.