MinIO Object Storage Server

type access

  • Operating System:

  • Terminal:

  • Shell:

  • Editor:

  • Package Manager:

  • Programming Language:

type access

  • Operating System:

  • Terminal:

  • Shell:

  • Editor:

  • Package Manager:

  • Programming Language:

type access

  • Operating System:

  • Terminal:

  • Shell:

  • Editor:

  • Package Manager:

  • Programming Language:

MinIO is a high-performance object storage suite, capable of storing unstructured data such as photos, videos, log files, backups, and container images.

The app deploys a standalone (single-node single drive) MinIO server instance, where the user can select the storage location directly on the UCloud file system.

Object storage volume

MinIO creates an object-based storage volume, which is a self-contained repository organized in buckets. The latter have a flat structure in which the data is divided into discrete units called objects and distributed among the hardware. The storage structure provides a unique identifier, that allows the objects to be found over a distributed filesystem, and metadata describing the data.

As an example, consider a repository folder called minio-repo with the following structure:

minio-repo
├── data1
│   ├── file1
│   ├── file2
│   ├── file3
│   └── file4
├── data2
│   ├── image1.png
│   └── image2.png
└── data3
    └── doc.txt

The folder is mounted using the Object storage volume mandatory parameter.

In this case the storage volume contains three buckets with different file types. MinIO always adds the existing data as buckets and objects when the server starts.

Note

Files at the root of the repository folder do not display in MinIO. Existing data must be in folders in the starting path. Top level folders become MinIO buckets.

The user can work with buckets and objects, using the MinIO Console, the MinIO Client, or one of the MinIO Software Development Kits (SDK).

MinIO Console

The MinIO Console is launched together with the server and it can be accessed by clicking

Administrator login credentials are generated when the job starts and are printed in the app's standard output.

The MinIO Console provides an intuitive graphical user interface for managing buckets and objects. It looks like in the image below.

drawing

The user can perform the following operations from the MinIO Console:

  • Create/remove buckets

  • Enable bucket versioning

  • Enable bucket quota

  • Upload/download files and folders

  • Delete file and folders

  • Image/text file preview

In addition, the MinIO Console can be used for administration tasks like identity and access management, metrics and log monitoring, or server configuration.

MinIO Client

The MinIO Client mc command-line tool provides an alternative way to manage object-based storage volumes via the app's built-in terminal interface,

By default, the MinIO Client connects to the server with the alias name ucloud.

The following list provides a quick overview of using the mc command. For a more in-depth tutorial, check the official guide.

Test connection to the server

$ mc admin info ucloud

Tip

● localhost:9000
  Uptime: 27 minutes
  Version: 2022-08-02T23:59:16Z
  Network: 1/1 OK
  Drives: 1/1 OK
  Pool: 1

Pools:
  1st, Erasure sets: 1, Disks per erasure set: 1

1.2 GiB Used, 3 Buckets, 12 Objects, 20 Versions
1 drive online, 0 drives offline

Get administrator login credentials

$ mc alias list

Tip

ucloud
  URL : http://localhost:9000
  AccessKey : YWY4OGJmNDM1MDhhNzU4YTFjZDI0MmRh
  SecretKey : MmZkOTMzMjgyYTY3NTVmNDhlNGEzMThk
  API : s3v4
  Path : auto

List buckets and objects

$ mc ls --recursive ucloud

Tip

[2022-08-11 05:02:16 UTC] 5.4MiB STANDARD data1/file1
[2022-08-09 08:45:11 UTC] 11MiB STANDARD data1/file2
[2022-08-09 08:45:11 UTC] 11MiB STANDARD data1/file3
[2022-08-09 08:45:11 UTC] 16MiB STANDARD data1/file4
[2022-08-11 08:03:19 UTC] 1.2MiB STANDARD data2/image1.png
[2022-08-11 07:40:29 UTC] 1.2GiB STANDARD data2/image2.png
[2022-08-11 07:51:45 UTC] 29B STANDARD data3/doc.txt

Make a bucket

$ mc mb ucloud/data4

Tip

Bucket created successfully `ucloud/data4`.

Remove a bucket

$ mc rb ucloud/data4

Tip

Removed `ucloud/data4` successfully.

Copy objects

$ mc cp -r ucloud/data1 /work/

Move objects

$ mc mv ucloud/data1/file1 /work/
$ mc ls ucloud/data1/
$ mc mv /work/file1 ucloud/data1/
$ mc ls ucloud/data1/

Manage bucket versioning

$ mc version enable ucloud/data1

Tip

ucloud/data1 versioning is enabled

$ mc version suspend ucloud/data1

Tip

ucloud/data1 versioning is suspended

$ mc version info ucloud/data1

Tip

[2022-08-11 08:37:35 UTC] 5.4MiB STANDARD 93d9be13-c716-46bd-8c39-930fe9a5b3e5 v5 PUT file1
[2022-08-11 08:37:25 UTC] 0B STANDARD 33c5b6a5-9d04-4498-87ae-874803090a08 v4 DEL file1
[2022-08-11 05:02:16 UTC] 5.4MiB STANDARD 94386d9a-f68d-4cdd-9232-523f76390d47 v3 PUT file1
[2022-08-11 05:01:28 UTC] 0B STANDARD d2f19203-390f-47ea-ba90-552c527ffe15 v2 DEL file1
[2022-08-09 08:45:10 UTC] 5.4MiB STANDARD 329c44eb-6c96-4bd5-a241-57a2d8eca608 v1 PUT file1
[2022-08-09 08:45:11 UTC] 11MiB STANDARD e3ea200b-de26-48b7-9c2d-088b6b61d321 v1 PUT file2
[2022-08-09 08:45:11 UTC] 11MiB STANDARD 675dc696-2586-4741-819d-7fce1206d433 v1 PUT file3
[2022-08-09 08:45:11 UTC] 16MiB STANDARD c16c3cce-96f3-495f-bf21-7b2bfc2921a7 v1 PUT file4

Manage anonymous bucket policies

$ mc policy set public ucloud/data1

Tip

Access permission for `ucloud/data1` is set to `public`

$ mc policy list ucloud/data1

Tip

data1/* => readwrite

$ mc policy set upload ucloud/data2

Tip

Access permission for `ucloud/data2` is set to `upload`

$ mc policy list ucloud/data2

Tip

data2/* => writeonly

$ mc policy set download ucloud/data3

Tip

Access permission for `ucloud/data3` is set to `download`

$ mc policy list ucloud/data3

Tip

data3/* => readonly

Manage users

$ mc admin user add ucloud user1 user1password

Tip

Added user `user1` successfully.

$ mc admin user list ucloud

Tip

enabled    user1

$ mc admin user info ucloud user1

Tip

AccessKey: user1
Status: enabled
PolicyName:
MemberOf:

$ mc admin user disable ucloud user1

Tip

Disabled user `user1` successfully.

$ mc admin user remove ucloud user1

Tip

Removed user `user1` successfully.

Manage groups

$ mc admin group add ucloud group12 user1 user2

Tip

Added members {user1,user2} to group group12 successfully.

$ mc admin group list ucloud

Tip

group12

$ mc admin group info ucloud group12

Tip

Group: group12
Status: enabled
Policy:
Members: user1,user2

$ mc admin group disable ucloud group12

Tip

Disabled group `group12` successfully.

$ mc admin group remove ucloud group12 user1 user2

Tip

Removed members {user1,user2} from group group12 successfully.

$ mc admin group remove ucloud group12

Tip

Removed group group12 successfully.

Manage canned policies

$ mc admin policy add ucloud getonly ./getonly.json

Tip

Added policy `getonly` successfully.

with the policy settings specified in the JSON file getonly.json:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}
$ mc admin policy list ucloud

Tip

readwrite
writeonly
consoleAdmin
diagnostics
getonly
readonly

$ mc admin policy set ucloud getonly user=user1

Tip

Policy `getonly` is set on user `user1`

$ mc admin user info ucloud user1

Tip

AccessKey: user1
Status: enabled
PolicyName: getonly
MemberOf:

$ mc admin policy unset ucloud getonly user=user1

Tip

Policy `getonly` is unset on user `user1`

$ mc admin policy remove ucloud getonly

Tip

Removed policy `getonly` successfully.

Restart the server

$ mc admin service restart ucloud

Tip

Restart command successfully sent to `ucloud`. Type Ctrl-C to quit or wait to follow the status of the restart process.
...
Restarted `ucloud` successfully in 1 seconds

Shell aliases

Shell aliases can be used to override common Unix command-line tools. For example consider the following Bash script:

#!/bin/bash

tee -a ~/.bashrc << END
alias ls='mc ls'
alias cp='mc cp'
alias cat='mc cat'
alias mkdir='mc mb'
alias pipe='mc pipe'
alias find='mc find'
END

The script can be executed before starting the MinIO server using the optional Initialization parameter.

MinIO object sharing

The app leverages the MinIO multi-user feature to share data with external collaborators.

New user login accounts can be created by the administrator through the MinIO Console or the MinIO Client. The server can be configured to deny or allow access to buckets to each of these users.

To ensure remote connection, the app must be deployed with a public IP address via the corresponding optional parameter. The IP address must be configured with Transmission Control Protocol (TCP) and ports 9000 and 9001 for API and Console access, respectively.

Note

MinIO users can access the MinIO Console interface with their credentials at the URL: http://<PUBLIC-IP>:9001.

Pre-signed URLs

A MinIO client API should be installed on the remote machine (see, e.g., instructions reported here).

The mc alias set command is used to add a specific alias name to access the MinIO server via a client API:

$ mc alias set <ALIAS> http://<PUBLIC-IP>:9000 <USERNAME> <PASSWORD>

where <USERNAME> and <PASSWORD> are the login credentials of the MinIO user. The permission level for buckets and objects depends on the specific user access policies.

Once an alias name is assigned, the mc share command can be used to generate temporary pre-signed URLs with integrated access credentials for downloading/uploading objects from/to a MinIO bucket.

For example:

$ mc share download <ALIAS>/data1/file1

Tip

URL: http://<PUBLIC-IP>:9000/data1/file1
Expire: 7 days 0 hours 0 minutes 0 seconds
Share: http://<PUBLIC-IP>:9000/data1/file1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=user1%2F20220811%2Feu-north-0%2Fs3%2Faws4_request&X-Amz-Date=20220811T150626Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&versionId=93d9be13-c716-46bd-8c39-930fe9a5b3e5&X-Amz-Signature=fe29ae79b8faccdebacbe0e1790c22a99c5022246206713f3375e42ebb24c943

Note

Temporary sharable URLs do not work when generated from the MinIO Console.

MinIO SDK APIs

MinIO supports several SDK client APIs to access object storage volumes.

Some examples are reported below.

package main

import (
    "log"

    "github.com/minio/minio-go/v7"
    "github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
    endpoint := "<PUBLIC_IP>:9000"
    accessKeyID := "USERNAME"
    secretAccessKey := "PASSWORD"
    useSSL := true

    // Initialize minio client object.
    minioClient, err := minio.New(endpoint, &minio.Options{
        Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
        Secure: useSSL,
    })
    if err != nil {
        log.Fatalln(err)
    }

    log.Printf("%#v\n", minioClient) // minioClient is now setup
}
MinioClient minioClient =
    MinioClient.builder()
        .endpoint("<PUBLIC_IP>:9000")
        .credentials("USERNAME", "PASSWORD")
        .build();
var Minio = require('minio')

var minioClient = new Minio.Client({
    endPoint: '<PUBLIC_IP>:9000',
    port: 9000,
    useSSL: true,
    accessKey: 'USERNAME',
    secretKey: 'PASSWORD'
});
import os
from minio import Minio

client = Minio('<PUBLIC-IP>:9000',
               access_key='<USERNAME>',
               secret_key='<PASSWORD>',
               secure=False)

client.list_buckets()

For a more in-depth guide of the different APIs, check here.