Month: June 2020

Integrating Keycloak and Harbor Registry with OpenID Connect

The documentation for setting up an OpenIDC identity provider / authentication method for Harbor Registry can be found in the harbor docs.

Harbor has supported OIDC since version 1.8.

Importantly:

You can change the authentication mode from database to OIDC only if no local users have been added to the database. If there is at least one user other than admin in the Harbor database, you cannot change the authentication mode.

So if you have existing local users, you will need to remove them - unfortunately doing this from the admin frontend does not actually delete them. you have to enter the postgres db and delete associated projects and then the users.

Information for setting up the client on keycloak side can be found on the red hat docs page

Getting Started

Having said all that...

  1. As Admin, go to Administration -> Configuration -> Authentication

  2. Select Auth mode as OIDC

  3. Fill in the required information as per the below screenshot:
    harbor-oidc-config-keycloak

  4. Click Test Configuration

OIDC Endpoint

For keycloak you can get your realm's OIDC details by going to:

https://<base_url>/auth/realms/<realm_name>/.well-known/openid-configuration

But for the OIDC configuration you remove everthing up to /.well-known... including the back slash.
So the OIDC endpoint should be:

https://<base_url>/auth/realms/<realm_name>

Deleting Existing Harbor Users

If you are using harbor on kubernetes - you can enter the postgres pod and execute in the shell:

docker exec -it harbor-db bash
psql -U postgres
\c registry
select * from harbor_user
delete from harbor_user where user_id > 2

Source

Prerequisite Packages and Compiling Python 3 on CentOS

What are the prerequisite packages for a complete python3 compilation install?

You will always get issues like Pip not being able to access pypi because the openssl module was not installed. Other things need the gcc compiler and such.

Recently I got this warning:

Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.

It is very annoying

Install Prerequisites

yum groupinstall development
yum install zlib-devel gcc openssl-devel bzip2-devel libffi-devel xz-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel expat-devel

Compile python

cd /opt
curl -O https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tgz
tar xzf Python-3.8.3.tgz
./configure
make
sudo make install

More Issues

There may be a warning after running make:

The necessary bits to build these optional modules were not found:
_curses               _curses_panel         _dbm               
_gdbm                 _sqlite3              _tkinter           
_uuid                 readline     

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           

This post mentions that some more are required (which I retrospectively added above):

    sudo yum install yum-utils
    sudo yum groupinstall development
    # Libraries needed during compilation to enable all features of Python:
    sudo yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel expat-devel

Even after that I still get:

    The following modules found by detect_modules() in setup.py, have been
    built by the Makefile instead, as configured by the Setup files:
    _abc                  atexit                pwd                
    time   

Self-hosted dropbox file storage syncing and backup with Minio and Clients

I've been looking at recreating folder syncing and backup options.

Minio is a great cloud storage options for block storage. It is not the storage itself it does one thing well - act as an interface to your files which is similar to the amazon S3 file storage API.
I wanted to make use of it for storing my backups and syncing my files.

Options I looked at for the client:

Chosen backup Solution

I chose Duplicati 2 (Beta) and configured the backup using the below settings

minio-duplicati-settings