Minio

Allikas: Imre kasutab arvutit
Redaktsioon seisuga 9. september 2026, kell 23:40 kasutajalt Imre (arutelu | kaastöö) (→‎Komponent - frontend)
Mine navigeerimisribaleMine otsikasti

Sissejuhatus

TODO

Tööpõhimõte

Väited

  • s3 protokoll töötab põhimõtteliselt ainult üle http:// (https://) - ei tööta nt üle gRPC
  • s3 autentimine (või on see autoriseerimine) toimub token sarnase väärtuse abil, mille klient lisab päringule
  • s3 ligipääsu tagava materjali moodustab klient 'access key id + secret access key' abil
  • põhimõtteliselt toimub nö tavalise s3 kliendi pöördumine ja presigned url kasutamine sama autentimise skeemi abi: ühel juhul post päringuga lisatakse headerite komplekt, teisel juhul query string abil key=value komplekt

tavalise kliendi puhul 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# docker compose -f docker-compose-presign-demo.yml exec -T svc_client curl -sv --user "admin:YWq7KKRaaspKCFNGNGJ4XzLg" --aws-sigv4 "aws:amz:us-east-1:s3" "http://presign-minio:9000/demo/testfile.txt"
* Host presign-minio:9000 was resolved.
* IPv6: (none)
* IPv4: 172.19.0.3
*   Trying 172.19.0.3:9000...
* Established connection to presign-minio (172.19.0.3 port 9000) from 172.19.0.2 port 55214
* using HTTP/1.x
* aws_sigv4: String to sign (enclosed in []) - [AWS4-HMAC-SHA256
20260909T202921Z
20260909/us-east-1/s3/aws4_request
46f4550f91c9822bf87c992e48e2e335efdd9848fd5279a1a4b190737fdfa27d]
* aws_sigv4: Signature - fb77e337cf1980811c1d9051ec630f804f7ce5cdbacb25954af26e86645f01b3
* Server auth using AWS_SIGV4 with user 'admin'
> GET /demo/testfile.txt HTTP/1.1
> Host: presign-minio:9000
> Authorization: AWS4-HMAC-SHA256 Credential=admin/20260909/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fb77e337cf1980811c1d9051ec630f804f7ce5cdbacb25954af26e86645f01b3
> X-Amz-Date: 20260909T202921Z
> x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> User-Agent: curl/8.22.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 34
< Content-Type: text/plain
< ETag: "28ca219778e750943fd411228f318861"
< Last-Modified: Wed, 09 Sep 2026 10:45:49 GMT
< Server: MinIO
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< Vary: Accept-Encoding
< X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
< X-Amz-Request-Id: 18D3C11261539973
< X-Content-Type-Options: nosniff
< X-Ratelimit-Limit: 1963
< X-Ratelimit-Remaining: 1963
< X-Xss-Protection: 1; mode=block
< Date: Wed, 09 Sep 2026 20:29:21 GMT
<
{ [34 bytes data]
hello from the presigned url demo
* Connection #0 to host presign-minio:9000 left intact

presigned url puhul 

# curl -s http://presign-minio:9000/demo/testfile.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=admin%2F20260909%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260909T115544Z&X-Amz-Expires=360&X-Amz-SignedHeaders=host&X-Amz-Signature=7855764e79cd3af21355167bf3340ec9cbb1c0fe7dc3b76f0ab34388f901f2cf

Paigaldamine

Paigaldamine toimub juhendi https://min.io/docs/minio/linux/operations/installation.html variatsiooni alusel 

Deploy MinIO: Single-Node Single-Drive -> Binary

Tundub, et variatsioon 'DEB (Debian/Ubuntu)' 2023 aasta suve lõpus midagi väga juurde ei anna, st kuigi paigladatakse .deb pakett ei teki kastuajat ega systemd unitit.

Keskkonna ettevalmistamine

kasutaja tekitamine 

# groupadd minio
# useradd -g minio -m -d /opt/minio -s /bin/bash minio

Failisüsteemi ettevalmistamise tulemusena peab olema kasutada /srv/minio kataloogis piisavalt ruumi minio andmete jaoks ning kataloog peab olema minio:minio omanduses.

f

Tarkvara kopeerimiseks sobib öelda kasutamiseks ettevalmistamiseks 

$ mkdir bin
$ cd bin
$ wget https://dl.min.io/server/minio/release/linux-amd64/minio
$ chmod 0755 minio

seejärel tuleb moodustada systemd unit 

# cat /etc/systemd/system/minio.service
[Unit]
Description=Minio
Documentation=https://docs.minio.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/opt/minio/bin/minio
 
[Service]
WorkingDirectory=/opt/minio
 
User=minio
Group=minio
 
PermissionsStartOnly=true
 
EnvironmentFile=-/opt/minio/conf/minio.conf
 
ExecStartPre=/bin/bash -c "[ -n \"${MINIO_VOLUMES}\" ] || echo \"Variable MINIO_VOLUMES not set in /opt/minio/conf/minio.conf\""
 
Restart=always
RestartSec=1
 
ExecStart=/opt/minio/bin/minio --compat server $MINIO_OPTS $MINIO_VOLUMES --console-address 0.0.0.0:9001 --address 0.0.0.0:9002
 
StandardOutput=journal
StandardError=inherit
 
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536
 
# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0
 
# SIGTERM signal is used to stop Minio
KillSignal=SIGTERM
 
SendSIGKILL=no
 
SuccessExitStatus=0
 
[Install]
WantedBy=multi-user.target

ning seadistusfail 

$ mkdir /opt/minio/conf
 
$ cat /opt/minio/conf/minio.conf
MINIO_VOLUMES=/srv/minio
MINIO_ROOT_USER=minio
MINIO_ROOT_PASSWORD=parool

Käivitamiseks sobib öelda 

# systemctl daemon-reload
# systemctl start --enable minio
# systemctl status minio

Tulemusena on ootus, et brauseriga saab pöördada arvuti ip aadressile port 9001, nt http://192.168.1.2:9001/

Minio rakenduse sisuosas ettevalmistamine

Minio rakenduse enda sisemiselt kasutamiseks ettevalmistamine sisaldab

  • kasuta moodustamine
  • storage ressursi moodustamine (nnn bucket)
  • ligipääsupiirangute seadistamine

Minio kasutamine

TODO

Presigned URL

Konteinerid kokku 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/volume/frontend# docker ps
CONTAINER ID   IMAGE                                      COMMAND                  CREATED       STATUS       PORTS                                NAMES
d951ef9fe43c   p_presign-demo-svc_frontend                "nginx -g 'daemon of…"   2 hours ago   Up 2 hours   127.0.0.1:8080->80/tcp               cn_presign-frontend
77175e844ced   p_presign-demo-svc_middleend               "/entrypoint.sh"         2 hours ago   Up 2 hours   80/tcp                               cn_presign-middleend
14c0bf3ac19c   minio/minio:RELEASE.2025-09-07T16-13-09Z   "/usr/bin/docker-ent…"   2 hours ago   Up 2 hours   127.0.0.1:9000-9001->9000-9001/tcp   cn_presign-minio
5ee127b7bb74   curlimages/curl:8.22.0                     "sleep infinity"         2 hours ago   Up 2 hours                                        cn_presign-client

presigned url tekitamine - kehtib 6 minutit 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# docker compose -f docker-compose-presign-demo.yml exec -T svc_client curl -s "http://cn_presign-middleend/presign?bucket=demo&key=testfile.txt&expiry=6m"
http://presign-minio:9000/demo/testfile.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=admin%2F20260909%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260909T115544Z&X-Amz-Expires=360&X-Amz-SignedHeaders=host&X-Amz-Signature=7855764e79cd3af21355167bf3340ec9cbb1c0fe7dc3b76f0ab34388f901f2cfroot@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc#

Presigned url kasutamine otse vastu minio teenust 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# docker compose -f docker-compose-presign-demo.yml exec -T svc_client curl -s "http://presign-minio:9000/demo/testfile.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=admin%2F20260909%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260909T115544Z&X-Amz-Expires=360&X-Amz-SignedHeaders=host&X-Amz-Signature=7855764e79cd3af21355167bf3340ec9cbb1c0fe7dc3b76f0ab34388f901f2cf"
hello from the presigned url demo

base64 teisendatud presigned url tekitamine 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# PRESIGNED='http://presign-minio:9000/demo/testfile.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=admin%2F20260909%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260909T115544Z&X-Amz-Expires=360&X-Amz-SignedHeaders=host&X-Amz-Signature=7855764e79cd3af21355167bf3340ec9cbb1c0fe7dc3b76f0ab34388f901f2cf'
root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# TOKEN=$(printf "%s" "$PRESIGNED" | sed -E "s#^https?://[^/]+##" | base64 -w0 | tr "+/" "-_" | tr -d "=")
root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# echo "http://127.0.0.1:8080/$TOKEN"
http://127.0.0.1:8080/L2RlbW8vdGVzdGZpbGUudHh0P1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9YWRtaW4lMkYyMDI2MDkwOSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjA5MDlUMTE1NTQ0WiZYLUFtei1FeHBpcmVzPTM2MCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmWC1BbXotU2lnbmF0dXJlPTc4NTU3NjRlNzljZDNhZjIxMzU1MTY3YmYzMzQwZWM5Y2JiMWMwZmU3ZGMzYjc2ZjBhYjM0Mzg4ZjkwMWYyY2Y

base64 teisendatud presigned url kasutamine 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# curl -s "http://127.0.0.1:8080/$TOKEN"
hello from the presigned url demo

peale expire aja möödumist 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc# curl -s "http://127.0.0.1:8080/$TOKEN"
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>testfile.txt</Key><BucketName>demo</BucketName><Resource>/demo/testfile.txt</Resource><RequestId>18D3A5E17030C8AC</RequestId><HostId>dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8</HostId></Error>root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/dc#

Komponent - frontend

Teisendava nginx lahenduse tekitamine 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/volume/frontend# ls -ld *
-rw-r--r-- 1 root root  819 Sep  9 10:39 Dockerfile
-rw-r--r-- 1 root root  587 Sep  9 10:39 frontend.js
-rw-r--r-- 1 root root 1048 Sep  9 10:45 nginx.conf

Dockerfile 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/volume/frontend# cat Dockerfile
# Deliberately NOT nginx:alpine - that image's pre-installed nginx.org
# build conflicts with Alpine's own nginx-mod-http-js package (built
# against Alpine's own nginx, different provenance) - confirmed live
# 2026-09-09 (apk dependency solver refused: "nginx-1.31.3-r1: breaks:
# nginx-mod-http-js-1.30.4-r1"). Plain alpine + apk's own matched
# nginx+nginx-mod-http-js pair (from the same repo, same version) avoids
# the conflict entirely.
#
# Base pinned by digest (alpine:3.24, confirmed live via
# `docker images --digests` on the target host 2026-09-09).
FROM alpine:3.24@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b

RUN apk add --no-cache nginx nginx-mod-http-js

COPY nginx.conf /etc/nginx/nginx.conf
COPY frontend.js /etc/nginx/njs/frontend.js

CMD ["nginx", "-g", "daemon off;"]

ning 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/volume/frontend# cat nginx.conf
# Alpine's own nginx package puts the njs module here (not
# /usr/lib/nginx/modules/ as the nginx.org build would) - confirmed live
# 2026-09-09.
load_module modules/ngx_http_js_module.so;

pid /run/nginx.pid;
worker_processes 1;
events { worker_connections 1024; }

http {
    js_path "/etc/nginx/njs/";
    js_import frontend from frontend.js;

    server {
        listen 80;
        js_var $decoded_uri;

        location / {
            js_content frontend.decode;
        }

        location @backend {
            # proxy_pass with a variable forces runtime DNS resolution,
            # which needs an explicit resolver - nginx does NOT fall back
            # to the system resolver for this. 127.0.0.11 is Docker's own
            # embedded DNS server on the container's default network -
            # confirmed live 2026-09-09 (omitting this produced
            # "no resolver defined to resolve minio" / 502).
            resolver 127.0.0.11 valid=30s;
            proxy_pass http://presign-minio:9000$decoded_uri;
        }
    }
}

ja frontend.js 

root@dh-ubuntu-2604-litellm-01:/srv/presign-demo/volume/frontend# cat frontend.js
function decode(r) {
    var token = r.uri.slice(1);
    var padded = token.replace(/-/g, '+').replace(/_/g, '/');
    while (padded.length % 4 !== 0) {
        padded += '=';
    }

    var decoded;
    try {
        decoded = Buffer.from(padded, 'base64').toString();
    } catch (e) {
        r.return(400, 'invalid base64 token\n');
        return;
    }

    if (decoded.charAt(0) !== '/') {
        r.return(400, 'decoded token is not a valid path\n');
        return;
    }

    r.variables.decoded_uri = decoded;
    r.internalRedirect('@backend');
}

export default { decode };

Kasulikud lisamaterjalid

  • TODO
Pärit leheküljelt "https://www.auul.pri.ee/wiki/index.php?title=Minio&oldid=5242"