Software Environments

To help GitLab CI/CD build jobs faster and keep our infrastructure secure, we incorporate several software components in CI/CD environments.

Local Docker Mirror

A local Docker registry mirror offers faster image retrieval and reduced bandwidth usage by caching images locally. It improves reliability, ensuring access to images even if external connections are disrupted, and enhances control and security within private networks.

Benefits of a Local Docker Registry Mirror

  • Speed: Faster access to Docker images through local caching.

  • Bandwidth Efficiency: Less network data transfer, conserving bandwidth.

  • Reliability: Constant image availability, even with network issues.

  • Security and Control: Improved security with internal network image transfers.

Usage

After setting up a registry server, client systems can connect to and use this Docker registry server. When configured, any image pull requests from the client system will go through this registry server. This setup streamlines the distribution and sharing of images, reducing network traffic and improving download speeds.

Client System Setup (cpu-runner)

The Docker registry server on etri-storage is configured for efficient image management. To connect your client system to this setup, simply follow these steps, keeping in mind that adjustments can be made to meet specific needs.

To integrate with the etri-storage Docker registry, the Docker daemon on the client system requires configuration. This is done by editing the /etc/docker/daemon.json file.

Add the following configuration:

{
    "registry-mirrors": ["http://storage.ones-ai.lab:6000"],
    "exec-opts": ["native.cgroupdriver=systemd"],
    "log-driver": "json-file",
    "log-opts": {
        "max-size": "100m"
    },
    "storage-driver": "overlay2"
}

Save the file and restart the Docker daemon to apply these changes: sudo systemctl restart docker

After this, your Docker commands will be directed to the local registry mirror automatically. Simply execute any docker command and it should work: docker run hello-world

Setting up Docker Registry Mirror

You can configure docker-compose to setup a Docker registry mirror. In our case, we employ etri-storage server for this.

File Path(etri-storage) : /docker-mirror/docker-compose.yml

version: "3"
services:
  dockerregistrymirror:
    container_name: etri-docker-registry
    image: registry:latest
    ports:
      - "6000:5000"
    environment:
      - REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io
      - REGISTRY_PROXY_USERNAME=###
      - REGISTRY_PROXY_PASSWORD=###
    restart: always

Note: Add REGISTRY_PROXY_USERNAME and REGISTRY_PROXY_PASSWORD if pulling private images is required.

Testing Registry

Test Registry Connectivity:

To confirm that the mirror is operational:

curl http://storage.ones-ai.lab:6000/v2/_catalog

Pull an Image (e.g., nginx):

For instance, to pull the nginx image:

docker pull nginx

Recheck the registry:

Verify that the image is now in your local mirror:

curl http://storage.ones-ai.lab:6000/v2/_catalog

Logs and monitoring

Regularly check logs for any operational insights or issues:

docker logs [DOCKER REGISTRY NAME] # Replace with your Docker registry container name, e.g., etri-docker-registry

MINIO: Local Object Storage

MinIO is an open-source, high-performance local object storage system. It’s compatible with `Amazon S3 , emphasizing self-hosted, private cloud environments for efficient management of large-scale unstructured data like photos, videos, and logs. This solution stands out for its scalability and security in personalized storage setups.

Usage

Connect

mc alias set etri-storage [IP Address] [ACCESS_KEY] [SECRET_KEY]

Check Server Info:

mc admin info etri-storage

Create User

mc admin user add etri-storage [ACCESS_KEY] [SECRET_KEY]

Attach Policy:

mc admin policy attach etri-storage readwrite --user=[ACCESS_KEY]

Set Alias for New User:

mc alias set ones-ai [IP Address] [ACCESS_KEY] [SECRET_KEY]

Create Bucket

mc mb ones-ai/nestc-pub

Upload Files

mc cp test.txt ones-ai/nestc-pub

Download Files

mc cp ones-ai/nestc-pub/test.txt ./local-directory

List Files in Bucket

mc ls ones-ai/nestc-pub

Remove Files in Bucket

mc rm ones-ai/nestc-pub/test.txt

Using in Pipelines

In your pipeline, you can use MinIO storage using mc.

publish:
  stage: publish
  image:
    name: minio/mc
    entrypoint: [""]
  before_script:
    - mc alias set $MINIO_ALIAS $MINIO_SERVER_URL $ACCESS_KEY $SECRET_KEY
  script:
    - mc rm $MINIO_ALIAS/$S3_BUCKET/ --recursive
    - mc cp build_ci $MINIO_ALIAS/$S3_BUCKET/ --recursive
  when: manual
  allow_failure: true

The usage of MinIO’s CLI is very similar to AWS S3. MinIO CLI Documentation

  • $MINIO_ALIAS: ones-ai

  • $MINIO_SERVER_URL: cpu-runner (ip address)

MinIO Docker Configuration

To configure MinIO with Docker, use the provided docker-compose.yml file.

File Path(etri-storage): /minio/docker-compose.yml

version: '3'
services:
  minio:
    image: minio/minio:latest
    command: server /data --console-address ":8080"
    container_name: etri-storage
    environment:
      MINIO_ROOT_USER: xxxx
      MINIO_ROOT_PASSWORD: xxxx
    restart: always

    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 5s
      retries: 5

    ports:
      - "80:9000"
      - "8080:9001"
    volumes:
      - minio-data:/data

volumes:
  minio-data:

Alpacon

Alpacon provides secure access to our servers, allowing you to effortlessly log in to the system and access the servers for which you have authorization.

Using Web

Open a Web browser, and visit https://ones-ai.alpacon.io. Upon logging in, you’ll encounter a list of our servers. Choose a server from the list and establish a connection by clicking the Websh button.

Using CLI

Install Alpacon CLI using the instructions for your platform.

Login

alpacon login

This command will prompt you to input username and password. For the server URL, use https://ones-ai.alpacon.io.

Connect

List servers that you have access:

alpacon servers ls

Connect to a server you want to access:

alpacon websh compute

Please wait until [compute] becomes connected...
Websh for [compute] became ready.

eunyoung@compute:~$

For root access:

alpacon websh -r compute