# Using Runners ![Group Runners](/resources/images/runners.png) GitLab Runner is an essential component of GitLab CI/CD that acts as an agent for running jobs defined in CI/CD pipelines. It serves as an executor, allowing the distributed execution of tasks specified in the `.gitlab-ci.yml` configuration file. ## Group Runners We have several runners in our group to meet performance or hardware requirements for our jobs. These runners are shared for all projects in ONES-AI group. ### `cpu-runner` Runner This runner runs jobs that are untagged or tagged as `cpu-runner`. The runner has two 20C/40T Intel Xeon CPU and 256 GB memory to execute concurrent compilation jobs fast. The workloads are executed in docker containers. We allow 8 concurrent jobs in this runner. ### `gpu-runner` Runner This runner runs jobs that are tagged as `gpu-runner`. The runner has two 12C/24T Intel Xeon CPU and 256 GB memory to execute concurrent compilation jobs fast. It is also equipped with an Nvidia A100 card for GPU-intensive workloads. The workloads are executed in docker containers. We allow 1 concurrent job in this runner. ``` +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.216.01 Driver Version: 535.216.01 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA A100 80GB PCIe Off | 00000000:63:00.0 Off | 0 | | N/A 37C P0 44W / 300W | 13MiB / 81920MiB | 0% Default | | | | Disabled | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 1595 G /usr/lib/xorg/Xorg 4MiB | +---------------------------------------------------------------------------------------+ ``` ### `demo-board` Runner This runner runs jobs that are tagged as `demo-board`. The workloads are executed in a special board. ### `rvx-board` Runner This runner runs jobs that are tagged as `rvx-board`. The workloads are executed in a special board. ### Adding New Runners When you need more runners, you can add them at `ONES-AI Group > Build > Runners > New Group Runner` menu. #### Docker Runner A [Docker Runner](https://docs.gitlab.com/runner/executors/docker.html) in GitLab CI/CD is an execution environment provided by GitLab Runner that enables jobs to be executed inside Docker containers, offering isolation, reproducibility, and compatibility across diverse environments. When adding a docker runner, select `docker` executor when registering a gitlab runner. You can install gitlab runner using the [guidelines](https://docs.gitlab.com/runner/install/linux-repository.html) provided by GitLab. #### SSH Runner (for demo boards) The [SSH executor](https://docs.gitlab.com/runner/executors/ssh.html) in GitLab CI/CD refers to a configuration option in GitLab Runner that allows jobs to be executed remotely over SSH, providing a secure and flexible way to run CI/CD tasks on external machines or environments using the SSH protocol. As our boards support SSH to run workloads remotely, we use SSH runner in this case. Follow the steps below to add a new SSH runner. ##### Prepare a Board 1. Install hardware. 2. Secure access to the board using network firewalls. 3. Locate a server which has SSH access to the board. ##### Install a SSH Runner In the server where you have SSH access to the board, execute the followings: 1. Add a system user: `sudo adduser --system --home /var/lib/board-runner --shell /bin/bash --group board-runner` 2. Switch to the user: `sudo su board-runner` 3. Register a runner: `gitlab-runner register` 4. Select `ssh` executor when registering a gitlab runner. 5. Provide credentials required to login to the target board. 6. Type exit: `exit` 7. Configure a systemd service: `sudo nano /lib/systemd/system/board-runner.service` ```ini [Unit] Description=GitLab Runner for ETRI board After=network.target syslog.target [Service] Type=simple User=board-runner Group=board-runner WorkingDirectory=/var/lib/board-runner ExecStart=/usr/bin/gitlab-runner run StandardOutput=syslog StandardError=syslog [Install] WantedBy=multi-user.target ``` 8. Reload systemd: `sudo systemctl daemon-reload` 9. Start service: `sudo systemctl start board-runner.service` 10. Enable service: `sudo systemctl enable board-runner.service` 11. Check status: `systemctl status board-runner.service` Status should be `active`. In this case, we used the username `board-runner`, but feel free to customize it according to your preferences. #### Shell Runner (for rvx boards) The [shell executor](https://docs.gitlab.com/runner/executors/shell.html) in GitLab CI/CD refers to a configuration option in GitLab Runner that allows jobs to be executed locally. We use the shell executor because our rvx board is connected via USB and requires commands to be executed from the system shell. Follow the steps below to add a new shell runner. ##### Prepare a Board 1. Install hardware. 2. Prepare all subsystems that this hardware requires. ##### Install a Shell Runner In the system where you have access to the board, execute the followings: 1. Add a system user: `sudo adduser --system --home /var/lib/board-runner --shell /bin/bash --group board-runner` 2. Switch to the user: `sudo su board-runner` 3. Register a runner: `gitlab-runner register` 4. Select `shell` executor when registering a gitlab runner. 5. Type exit: `exit` 6. Configure a systemd service: `sudo nano /lib/systemd/system/board-runner.service` ```ini [Unit] Description=GitLab Runner for Board After=network.target syslog.target [Service] Type=simple User=board-runner Group=board-runner WorkingDirectory=/var/lib/board-runner ExecStart=/usr/bin/gitlab-runner run StandardOutput=syslog StandardError=syslog [Install] WantedBy=multi-user.target ``` 8. Reload systemd: `sudo systemctl daemon-reload` 9. Start service: `sudo systemctl start board-runner.service` 10. Enable service: `sudo systemctl enable board-runner.service` 11. Check status: `systemctl status board-runner.service` Status should be `active`. In this case, we used the username `board-runner`, but feel free to customize it according to your preferences. Please note that the user `board-runner` should have sufficient privilege to run commands to the target board. Additionally, be aware that jobs run by this runner are executed in the system shell, so it should be restricted to use only by trusted entities. ### Runner Permissions To run the pipelines using group runners, you must be a member of `ONES-AI` group. For others, you can use shared runners (or instance runners) provided by gitlab.com or you can bring your own project runners if possible. :::{tip} For security reasons, pipelines for merge requests submitted from outside the ONES-AI group do not run on group runners. To run jobs on the group runner, a group member must make at least one commit to the branch of the merge request. ::: ## Project Runners If you have specific requirements for a project, you can also add project-level runners. These runners are not shared to the group, and can be enabled for your projects one by one. To add project runners, visit your project page, and go to `Settings > CI/CD > Runners`. Please note that when sharing your project runner between multiple project, you need to have at lease maintainer privilege on both projects, and the runner should not be locked. For more information, you can refer to [this](https://docs.gitlab.com/ee/ci/runners/runners_scope.html#enable-a-project-runner-for-a-different-project).