EN: MinIO Distributed Storage - Short Intro

EN: MinIO Distributed Storage - Short Intro

Here we go the documentation quotes:

MinIO is High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Using MinIO build high performance infrastructure for machine learning, analytics and application data workloads.

If you have ever heard about AWS S3 one of the cheapest and biggest storage services from Amazon company, then you know that startup companies usually goes with it, but by using MinIO, it is easier touse storage for free on your local PCs, especially if you startup.

What is beautiful in AWS S3, it's a great API and SDK for most of languages, and this API already became as standard. So how it relates to MinIO? It's because MinIO supports S3 API and you can rely on using AWS layer of how to upload, get, delete and store your files.

What else is useful, first of all is that you have AWS compatible implementation of distributed storage for free. Plus, you can use following link even without installation of MinIO servers https://play.min.io/ and check APIs.If you want to try using MinIO, it's easy, than for instance, installing Ceph as implementation of AWS S3 on your servers. You can start right away, by running docker container, or as I said earlier, by using remote storage https://play.min.io/

docker pull minio/minio
docker run -p 9000:9000 minio/minio server /data
Run MinIO on Docker to start using storage API locally

Also if you are working on Mac OS, then you can use following commands, to install your own MinIO server with Homebrew.

Install minio packages using Homebrew👇

Copy brew install minio/stable/minio minio server /data

To access your MinIO server you need to know ACCESS KEY and SECRET KEY, it is recommended to set those keys in environment variables.

More details you can find at official documentation: https://docs.min.io/

Also in the following documentation you can find out many clients.

At last let see how distributed architecture looks

Minio architecture

I have noticed that in a site documentation there was not fully correct examples of Java clients usage for uploading file via putObjects  in implementation of MinioClient , there were missing argument PutObjectOptions - which accepts full files size and part size of uploading files, also you can add meta data as key-value. Please look at documented sample ->  github - there actual information.

Thats all! Thanks for reading till these lines! Please, right your comments, which subjects and samples you interesting to see in my posts.

P.S. docker-compose file and commands to start 4 servers - 8 nodes

docker-compose pull
docker-compose up
version: '3.7'

# starts 4 docker containers running minio server instances. Each
# minio server's web interface will be accessible on the host at port
# 9001 through 9004.
services:
  minio1:
    image: minio/minio:RELEASE.2020-05-01T22-19-14Z
    volumes:
      - data1-1:/data1
      - data1-2:/data2
    ports:
      - "9001:9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio2:
    image: minio/minio:RELEASE.2020-05-01T22-19-14Z
    volumes:
      - data2-1:/data1
      - data2-2:/data2
    ports:
      - "9002:9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio3:
    image: minio/minio:RELEASE.2020-05-01T22-19-14Z
    volumes:
      - data3-1:/data1
      - data3-2:/data2
    ports:
      - "9003:9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio4:
    image: minio/minio:RELEASE.2020-05-01T22-19-14Z
    volumes:
      - data4-1:/data1
      - data4-2:/data2
    ports:
      - "9004:9000"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server http://minio{1...4}/data{1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2: