Month: January 2020

Junos PyEZ how to fix the xmlSAX2Characters:huge text node

When trying to acquire a huge bit of config from juniper routers and parse it with PyEZ sometimes you get an error. Something like:

pyez xmlSAX2Characters: huge text node, line 256071, column 53 (<string>, line 256071)

PyEZ uses ncclient the python netconf client behind the scenes.It isn't well documented in their docs though.

Allowing a huge text nodes in pyEZ, you should connect to your device and pass the huge_tree=True parameter:

with Device(host='x.x.x.x', user=junos_username, passwd=junos_password, huge_tree=True) as dev:

Kubernetes Questions – Please answer them

What is the Difference between a Persistent Volume and a Storage Class?

What happens when pods are killed, is the data persisted - How do you test this?

What is the difference between a Service and an Ingress?

By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine.

If you check the pod ip and there is an open containerPort then you should be able to access it via the node - with curl.

What happens when a node dies? The pods die with it, and the Deployment will create new ones, with different IPs. This is the problem a Service solves.

A Kubernetes Service is an abstraction which defines a logical set of Pods running somewhere in your cluster, that all provide the same functionality

When created, each Service is assigned a unique IP address (also called clusterIP)

This address is tied to the lifespan of the Service, and will not change while the Service is alive

communication to the Service will be automatically load-balanced

  • targetPort: is the port the container accepts traffic on
  • port: is the abstracted Service port, which can be any port other pods use to access the Service

Note that the Service IP is completely virtual, it never hits the wire

Kubernetes supports 2 primary modes of finding a Service - environment variables and DNS - DNS requires a COreDNS addon

Ingress is...

An API object that manages external access to the services in a cluster, typically HTTP

How do you know the size of the PV's to create for the PVC's of a helm chart?

Are helm chart declarative or imperitive?

What is a kubernetes operator?

How do you start a new mysql docker container with an existing data directory?

Usage against an existing databaseIf you start your mysql container instance with a data directory that already contains a database (specifically, a mysql subdirectory), the $MYSQL_ROOT_PASSWORD variable should be omitted from the run command line; it will in any case be ignored, and the pre-existing database will not be changed in any way.

The above did not work for me.

trademate-db_1   | Initializing database
trademate-db_1   | 2020-01-16T05:59:38.689547Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
trademate-db_1   | 2020-01-16T05:59:38.690778Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
trademate-db_1   | 2020-01-16T05:59:38.690818Z 0 [ERROR] Aborting
trademate-db_1   | 
trademate_trademate-db_1 exited with code 1

How do you view the contents of a docker volume?

You can't do this without a container for named volumes (the one's docker manages). So kak...

How do you run python debugger and attach inside a docker container?

Its a mess up...just develop locally

If you were mounting a conf file into an nginx image from docker-compose - how do you do that in production? Do you bake it into the image?

Yes you should.

Do something like this:

FROM nginx:1.17

COPY ./config/nginx/conf.d /etc/nginx/conf.d

# Remove default config
RUN rm /etc/nginx/conf.d/default.conf 

How do you deploy all the k8s spec files in a folder at once? If not is there a specific order to deploy them in?

This Service should exists before the replicas - as it adds environment variables to containers on the pods in the replicaset based on services created.

Should gunicorn and nginx containers be in the same pod?

Docker Saving 361MB on an Image File with dockerignore

Initially I created a Dockerfile to run my django app. I chose python alpine to save on image size.

There were a few issues with that but that is fixed in the below Dockerfile

FROM python:3.8-alpine

RUN mkdir -p /code/requirements

RUN pip install --upgrade pip --no-cache-dir

# Installing requirements.txt from project
COPY ./requirements/*.txt /code/requirements/.
RUN apk add --no-cache --virtual .build-deps gcc libffi-dev openssl-dev musl-dev mariadb-dev \
    && pip install --no-cache-dir -r /code/requirements/production.txt \
    && apk del .build-deps gcc libffi-dev musl-dev openssl-dev mariadb-dev

COPY . /code/

# Give access to non root user
RUN rm -rf /code/.git* && \
    chown -R 1001 /code && \
    chgrp -R 0 /code && \
    chmod -R g+w /code

USER 1001


CMD ["sh", "-c", "python collectstatic --no-input; python migrate; python runserver"]

The image size is a whopping 536MB

    REPOSITORY                                                                   TAG                 IMAGE ID            CREATED             SIZE
    trademate-app                                                                latest              c68e2e23464f        40 seconds ago      536MB

After checking the history:

$ docker history trademate-app 
IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
6f2ed6c9d629        About a minute ago   /bin/sh -c #(nop)  CMD ["sh" "-c" "python ma…   0B                  
da5c27376179        About a minute ago   /bin/sh -c #(nop)  EXPOSE 8000                  0B                  
49e374f1bd68        About a minute ago   /bin/sh -c #(nop)  USER 1001                    0B                  
f705cd2c2415        About a minute ago   /bin/sh -c rm -rf /code/.git* &&     chown -…   189MB               
d48349f175fa        2 minutes ago        /bin/sh -c #(nop) COPY dir:47dc9677cabb09e63…   193MB               
848177bf54b2        2 minutes ago        /bin/sh -c apk add --no-cache --virtual .bui…   37.9MB              
b5ab3803fe68        8 minutes ago        /bin/sh -c #(nop) COPY file:7b8d1a2c6c47119d…   215B                
ffe7a4e57a38        8 minutes ago        /bin/sh -c pip install --upgrade pip --no-ca…   5.03MB              
d7e45c46def5        8 minutes ago        /bin/sh -c #(nop) WORKDIR /code                 0B                  
8278e34b58a3        8 minutes ago        /bin/sh -c mkdir -p /code/requirements          0B                  
204216b3821e        2 months ago         /bin/sh -c #(nop)  CMD ["python3"]              0B                  
<missing>           2 months ago         /bin/sh -c set -ex;   wget -O "$P…   6.24MB              
<missing>           2 months ago         /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_SHA256…   0B                  
<missing>           2 months ago         /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_URL=ht…   0B                  
<missing>           2 months ago         /bin/sh -c #(nop)  ENV PYTHON_PIP_VERSION=19…   0B                  
<missing>           2 months ago         /bin/sh -c cd /usr/local/bin  && ln -s idle3…   32B                 
<missing>           2 months ago         /bin/sh -c set -ex  && apk add --no-cache --…   98.6MB              
<missing>           2 months ago         /bin/sh -c #(nop)  ENV PYTHON_VERSION=3.8.0     0B                  
<missing>           2 months ago         /bin/sh -c #(nop)  ENV GPG_KEY=E3FF2839C048B…   0B                  
<missing>           2 months ago         /bin/sh -c apk add --no-cache ca-certificates   551kB               
<missing>           2 months ago         /bin/sh -c #(nop)  ENV LANG=C.UTF-8             0B                  
<missing>           2 months ago         /bin/sh -c #(nop)  ENV PATH=/usr/local/bin:/…   0B                  
<missing>           2 months ago         /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B                  
<missing>           2 months ago         /bin/sh -c #(nop) ADD file:fe1f09249227e2da2…   5.55MB

So you can see the removal of the git folder and env folder was screwing me along with the copy.

So I added a .dockerignore file containing:


and removing the rm -rf /code/.git* command.

The image is now 175 MB

REPOSITORY                                                                   TAG                 IMAGE ID            CREATED             SIZE
trademate-app                                                                latest              0040fef600ad        8 minutes ago       175MB

Saving a whopping 361MB

Any other improvements would be welcomed...send me an email.