Showing posts with label Containers. Show all posts
Showing posts with label Containers. Show all posts
Monday, May 4, 2020
Docker Guide For DEVNET Engineers
Almost 2 years ago, I have written post on docker containers which covers basic and advanced topics. Below is the summary of all posts and can be utilized if anyone is preparing for DEVNET or trying to become DEVNET engineer. 1. Beginners Guide To Docker 2. Docker Beginners Guide - Troubleshooting 3. Types Of Docker Installations 4. How Secure The Data Within The Container: Mounting Container Files With Host 5. Building Docker Image From Scratch 6. Understanding Layered Architecture of Docker Container Images
Click Here To Read Rest Of The Post...
Tuesday, May 15, 2018
Understanding Layered Architecture of Docker Container Images
“Building Docker Image From Scratch” is easy way to build any kind of containerization image. Let’s get bit more deep into it and understand what it is doing at the low level. Everyone knows in Linux everything is considered as file only. The whole operating system is nothing but a collection of files and folders. In the previous post, we have seen there are 2 steps required as per “Dockerfile” and same has been embedded in the container layer file system. As a result we have got a container image which is nothing but a collection of layered filesystem. Container images are templates from which containers are created. These images are not just one monolithic block, but are composed of many layers. The first layer in the image is also called the base layer:
Click Here To Read Rest Of The Post...
Sunday, May 13, 2018
Building Docker Image From Scratch
Beginners Guide to dockers part 1, has covered the architecture of dockers. Along with this, so far, I have covered different types of dockers installation, how to download, install and delete docker image and Docker Beginners Guide - Troubleshooting This post is more focused on creating a docker image and it can be used anywhere in your project basis on the requirements. Lets create a new folder in windows directory called create-image and change the current directory to create-image.
PS C:\Lab\create-image>
Now create a new file called Dockerfile.txt in the current directory with below mentioned commands.
PS C:\Lab\create-image> cat .\Dockerfile.txt
FROM centos:7
RUN yum install -y wget
PS C:\Lab\create-image>
Let's create a new image by using Dockerfile.txt created in the previous step.
PS C:\Lab\create-image> docker image build -t my-new-image -f ./Dockerfile.txt .
Below is the output after running the above command.
PS C:\Lab\create-image> docker image build -t my-new-image -f ./Dockerfile.txt .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM centos:7
7: Pulling from library/centos
Digest: sha256:989b936d56b1ace20ddf855a301741e52abca38286382cba7f44443210e96d16
Status: Downloaded newer image for centos:7
---> e934aafc2206
Step 2/2 : RUN yum install -y wget
---> Running in cfc91e766858
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
* base: ftp.cuhk.edu.hk
* extras: ftp.cuhk.edu.hk
* updates: ftp.cuhk.edu.hk
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7_4.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
wget x86_64 1.14-15.el7_4.1 base 547 k
Transaction Summary
================================================================================
Install 1 Package
Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
Public key for wget-1.14-15.el7_4.1.x86_64.rpm is not installed
warning: /var/cache/yum/x86_64/7/base/packages/wget-1.14-15.el7_4.1.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f
4a80eb5: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) "
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-4.1708.el7.centos.x86_64 (@CentOS)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : wget-1.14-15.el7_4.1.x86_64 1/1
install-info: No such file or directory for /usr/share/info/wget.info.gz
Verifying : wget-1.14-15.el7_4.1.x86_64 1/1
Installed:
wget.x86_64 0:1.14-15.el7_4.1
Complete!
Removing intermediate container cfc91e766858
---> 4a991aace711
Successfully built 4a991aace711
Successfully tagged my-new-image:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and director
ies added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions f
or sensitive files and directories.
If you remeber, in the Dockerfile created in the previous step has two steps, the same can be found in the above output also. Let's do he postmartem of the above output. The first thing the builder does is package the files in the current build context and sends the resulting .tar file to the Docker daemon.
Sending build context to Docker daemon 2.048kB
Now we have the next output mentioned in step1/2. It will pull the centos from the docker registry if not available locally
Step 1/2 : FROM centos:7
7: Pulling from library/centos
Status: Downloaded newer image for centos:7
Below is the shortend output of next output mentioned in step2/2. It will run the "yum" command as mentioned in Dockerfile and download the wget package. Finally it will remove the inter mediator container and finally you can see the container name at the end "4a991aace711"
Step 2/2 : RUN yum install -y wget
---> Running in cfc91e766858
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
* base: ftp.cuhk.edu.hk
* extras: ftp.cuhk.edu.hk
* updates: ftp.cuhk.edu.hk
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7_4.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
wget x86_64 1.14-15.el7_4.1 base 547 k
Transaction Summary
================================================================================
Install 1 Package
Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
Running transaction
Installing : wget-1.14-15.el7_4.1.x86_64 1/1
install-info: No such file or directory for /usr/share/info/wget.info.gz
Verifying : wget-1.14-15.el7_4.1.x86_64 1/1
Installed:
wget.x86_64 0:1.14-15.el7_4.1
Complete!
Removing intermediate container cfc91e766858
---> 4a991aace711
Successfully built 4a991aace711
Successfully tagged my-new-image:latest
Finally you can check your image by running "docker images" command.
PS C:\Lab\create-image> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my-new-image latest 4a991aace711 5 minutes ago 263MB
Click Here To Read Rest Of The Post...
Tuesday, May 8, 2018
Docker Beginners Guide - Troubleshooting
Beginners Guide to dockers part 1, has covered the architecture of dockers. Along with this, so far, I have covered different types of dockers installation and how to download, install and delete docker image. In this post, I will be covering more about docker commands which are really very useful while working with docker containers. Obtaining Docker Version Information: This command will help to provide information about the docker server and client for your current working configuration. The output of the command states that CE version is being used with API and GO version details. Client deployed on windows platform, it is showing OS is windows. But for server it is showing that OS is linux. This installation has client and server installed on same machine. I am using docker tools to acces the docker server installed on same machine.
PS C:\lab\test-html> docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:48 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:22:38 2018
OS/Arch: linux/amd64
Experimental: false
PS C:\lab\test-html>
Obtaining Docker System Infomration: The next important command is the docker system info command. This command provides information about what mode the Docker engine is operating in (swarm mode or not), what storage driver is used for the union filesystem, what version of the Linux kernel we have on our host, and much more. Please have a careful look at the output generated by your system when running the command. Analyze what kind of information is shown below.
PS C:\lab\test-html> docker system info
Containers: 15
Running: 1
Paused: 0
Stopped: 14
Images: 14
Server Version: 18.03.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: ii9vy20o5gwxxmkcve6livhyw
Is Manager: true
ClusterID: x4wi39gjagt28xhvplmv26t64
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.65.3
Manager Addresses:
192.168.65.3:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.87-linuxkit-aufs
Operating System: Docker for Windows
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.934GiB
Name: linuxkit-00155d010105
ID: DGPH:X3AJ:ZH2F:36YZ:72TG:42Q7:BZIF:P7K4:CXTB:R67Y:NIGZ:7HAG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 47
Goroutines: 180
System Time: 2018-05-07T13:00:01.7021005Z
EventsListeners: 2
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
PS C:\lab\test-html>
Listing The Resource Consumption: The below output is showing how much resources are currently used. Below mentioned output tells that on system I am currently having 14 images locally cached of which 4 are in active use. An image is considered to be in active use if currently at least one running or stopped container is based on it. These images occupy 1.39 GB disk space. I have 15 running containers on my system and 1 stopped ones for a total of 14 containers. I can reclaim the space occupied by the stopped containers which is 472B. I also have 8 active volumes on my host that together consume about 136.5 MB of disk space. Since none of the volumes are in use, I can reclaim 100% of the space.Finally, my Build Cache is currently empty and thus of course I cannot reclaim any space there too.
PS C:\lab\test-html> docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 14 4 1.594GB 1.395GB (87%)
Containers 15 1 474B 472B (99%)
Local Volumes 8 0 136.5MB 136.5MB (100%)
Build Cache 0B 0B
PS C:\lab\test-html>
More detailed output can be see by typing the below mentioned command in the terminal window.
PS C:\lab\test-html> docker system df -v
Images space usage:
REPOSITORY TAG IMAGE ID CREATED ago SIZE SHARED SIZE UNIQUE SiZE CONTAINERS
my-website 1.0 1bbf94f77299 33 hours ago ago 18MB 18MB 71B 1
alpine-ping latest ee997c1cb716 2 days ago ago 5.647MB 4.148MB 1.499MB 0
ubuntu latest 452a96d81c30 9 days ago ago 79.62MB 0B 79.62MB 0
grafana/grafana latest ed6c9eb28b01 10 days ago ago 238.1MB 0B 238.1MB 0
hello-world latest e38bc07ac18e 3 weeks ago ago 1.848kB 0B 1.848kB 0
nginx alpine ebe2c7c61055 3 weeks ago ago 18MB 18MB 0B 0
centos latest e934aafc2206 4 weeks ago ago 198.6MB 0B 198.6MB 3
busybox latest 8ac48589692a 4 weeks ago ago 1.146MB 0B 1.146MB 2
alpine latest 3fd9065eaf02 3 months ago ago 4.148MB 4.148MB 0B 9
google/cadvisor latest 75f88e3ec333 5 months ago ago 62.21MB 0B 62.21MB 0
docker4w/nsenter-dockerd latest cae870735e91 6 months ago ago 187.5kB 0B 187.5kB 0
tutum/influxdb latest c061e5808198 18 months ago ago 289.7MB 224.4MB 65.32MB 0
tutum/influxdb 0.8.8 1f72f5000d33 2 years ago ago 279.9MB 224.4MB 55.53MB 0
fr3nd/collectd latest 8713c4a4964f 2 years ago ago 649.7MB 0B 649.7MB 0
Containers space usage:
CONTAINER ID IMAGE COMMAND LOCAL VOLUMES SIZE CREATED ago STATUS NAMES
247ac71ee498 alpine "/bin/sh" 0 335B 32 hours ago ago Exited (0) 31 hours ago reader
6cb7116dd7b1 alpine "/bin/sh" 0 137B 32 hours ago ago Exited (0) 32 hours ago writer
40c8e9220f5b alpine "-it --name writer e…" 0 0B 32 hours ago ago Created dazzling_chebyshev
0be877206ab2 alpine "-it --name writer a…" 0 0B 32 hours ago ago Created reverent_hopper
641f1371b872 alpine "-it --name writer a…" 0 0B 32 hours ago ago Created trusting_kalam
48ada1420e8e alpine "-it --name writer -…" 0 0B 32 hours ago ago Created clever_kapitsa
95ffa6d5fef4 alpine "-it --name write-pe…" 0 0B 32 hours ago ago Created elastic_lamport
fa5c8db3b550 alpine "-it --name write-pe…" 0 0B 32 hours ago ago Created kind_wing
3f432b2883f8 my-website:1.0 "nginx -g 'daemon of…" 0 2B 33 hours ago ago Up 33 hours my-site
634013c5b179 alpine "ssh 127.0.0.1" 0 0B 2 days ago ago Created pedantic_edison
c99bb072398a busybox "-it exec /bin/sh" 0 0B 2 days ago ago Created zen_kare
b0825921b5c5 busybox "exec /bin/bash" 0 0B 2 days ago ago Created priceless_bardeen
23e23f63ce00 centos "exec ping -c 127.0.…" 0 0B 3 days ago ago Created dazzling_turing
dfebc236e05f centos "/etc/*release*" 0 0B 3 days ago ago Created priceless_allen
9efc9e9ef4ae centos "ifconfig -a" 0 0B 3 days ago ago Created hungry_hugle
Local Volumes space usage:
VOLUME NAME LINKS SIZE
f9af4287f9891dd7c3b42d77b1657f6c1405fe622dcbe6e3f909ae40aad470ba 0 33.63MB
my-data 0 0B
00b2ec861da114fad3d09e1c174c918a34dcd88d14c7b4069e2c159c1ab06885 0 33.58MB
103fe18d6d3f8c0b2cae8d798edd1aeaa3b1fd448f838634749d9bf9f83e50fd 0 2.028MB
22de30eca1e1859257568153d4c2b79567832b56076ea7adea3db725bbd5fea4 0 869B
3dbfc337ac718416e53c2ead7dfb234b151862984741c2d1dae518cfe1169609 0 33.64MB
6276c3f403c651df7d5fa86f4d23c3033a134412393beb3e545892c1f716cef4 0 0B
6b864603f18146911e5c165a995b480321df71f4c03c1a7a252bc458c157deca 0 33.6MB
Build cache usage: 0B
PS C:\lab\test-html>
Checking the Status of the Docker Events: I liked the below mentioned command because it actually tells me what is happening in the background whenever I run any of the docker command.
PS C:\Lab\test-html> docker system events
I would suggest to open a new window and run the docker standard start or stop command. I am going to run a hello-world docker container and correspondingly we can see the list of events which are happening in background in the preious open terminal where we have issued the "Docker System Events" command.
PS C:\lab\test-html> docker run hello-world
Events generated on another terminal:
PS C:\Lab\test-html> docker system events
2018-05-07T18:43:52.277871400+05:30 container create e2e7a06a242fabcf36fea77d4f245c5f6d629f2b60c6170b47a45f0c056932f7 (image=hello-world, name=wonderful_euler)
2018-05-07T18:43:52.287628100+05:30 container attach e2e7a06a242fabcf36fea77d4f245c5f6d629f2b60c6170b47a45f0c056932f7 (image=hello-world, name=wonderful_euler)
2018-05-07T18:43:52.508228600+05:30 network connect 0e7a9971d8030271ef932b0556ca31e88bfede6ed213b1b1c1923847bb96c506 (container=e2e7a06a242fabcf36fea77d4f245c5f6d629
f2b60c6170b47a45f0c056932f7, name=bridge, type=bridge)
2018-05-07T18:43:52.996369000+05:30 container start e2e7a06a242fabcf36fea77d4f245c5f6d629f2b60c6170b47a45f0c056932f7 (image=hello-world, name=wonderful_euler)
2018-05-07T18:43:53.081629600+05:30 container die e2e7a06a242fabcf36fea77d4f245c5f6d629f2b60c6170b47a45f0c056932f7 (exitCode=0, image=hello-world, name=wonderful_eul
er)
2018-05-07T18:43:53.432605800+05:30 network disconnect 0e7a9971d8030271ef932b0556ca31e88bfede6ed213b1b1c1923847bb96c506 (container=e2e7a06a242fabcf36fea77d4f245c5f6d
629f2b60c6170b47a45f0c056932f7, name=bridge, type=bridge)
The above mentioned list of commands and respective outputs are really helpful while troubleshooting.
Here is the beginning of my post. And here is the rest of it.
Click Here To Read Rest Of The Post...
Monday, May 7, 2018
How Secure The Data Within The Container: Mounting Container Files With Host
So far I have posted the below posts: 1. Beginners guide to docker which has covered the basic understanding about dockers 2. Types of Docker installation 3. Download, Install and Delete Dockers As we all know how great the linux file system and security features are. But when we dealt with containers, the same kind of security resides in it also. To get better understanding of how does the file mounting system works, I tried to launch a container named "writer" with read and write permissions to a folder inside it and same time it has been mounted with my local windows directory. After that, I have launched same container with another name "reader" which has only read only rights and same directory is also mounted to my local directory with read only permissions too. Then I tried to make changes in writer container by creating a file name "simple.txt" and found the same file is in my local windows mounted directory. The same simple.txt file can be read by the reader container also but it it tries to write anything on it, it says access denied. This clearly demonstrates how securely we can share the files in the container with other containers without loosing any feature functionality. As per below image, there are 2 x Alpine container with writer and reader name has been spin up in the docker container. Both the containers are sharing the /data folder to host directory with right and read access.
docker run -it --name writer -v C:/lab/test-html:/data alpine /bin/sh
Let's try to create simple.txt file and write some text in it
/ # echo "Creating a file which is shared in C:/lab/test-html folder in read and write format" > /data/sample.txt
/ #
Check the contents of the file by typing the below command in terminal
/ # cat /data/sample.txt
After this, let spin the another container and mount the /data folder with read only access.
PS C:\lab\test-html> docker run -it --name reader -v C:/lab/test-html:/data:ro alpine /bin/sh
Write some content in the simple.txt file in reader container.
/ # echo "I have spin new apline container as name reader and trying to write the lines in it" > /data/sample_read.txt
We can see after hitting the above command in the terminal, we can see this has been rejected because reader container has only read only access to simple.txt file.
/bin/sh: can't create /data/sample_read.txt: Read-only file system
We can verify the same by running the below command in the terminal. Below mentioned output is the extract from the output. Rest output has been omitted
PS C:\lab\test-html> docker inspect reader
"HostConfig": {
"Binds": [
"/host_mnt/c/lab/test-html:/data:ro"
],
Click Here To Read Rest Of The Post...
Saturday, May 5, 2018
Types Of Docker Installations
Before moving ahead, we need to understand the few definitions which are utmost required to get better understanding of containers. The definitions are below mentioned: Container OS: The container OS is also known as Base OS. It refers to an image that contains an operating system like windows, centos etc. Container Host: Container host is also known as Host OS. In the case of Linux and non-Hyper-V containers, the Host OS shares its kernel with running Docker containers. Operating System Kernel: The Kernel manages lower level functions such as memory management, file system, network and process scheduling. Container can be deployed in three different ways as below mentioned: If someone known apart that, please share your comment will edit the post accordingly. 1. Install Docker On Linux Machine 2. Install Docker On Windows Machine Without Hyper-V 3. Install Docker On Windows Machine With Hyper-V
Install Docker On Linux Machine
Install Docker On Windows Machine Without Hyper-V
Install Docker On Windows Machine With Hyper-V
Click Here To Read Rest Of The Post...
Wednesday, May 2, 2018
Beginners Guide To Docker
Docker comes into two different kind of flavors community edition(CE) and enterprise edition(EE). Community edition can be downloaded by anyone and can be used for development and testing purpose without any professional support. Enterprise Edition comes with yearly based license and backed by 24 x 7 support and are supported with bug fixes much longer than their community edition. Food For Thought: Learn More About Docker Security Docker Architecture Docker uses a client-server architecture. The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers. The Docker client and daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon. The Docker client and daemon communicate using a REST API, over UNIX sockets or a network interface. Docker client interacts with docker daemon by sending the commands, Docker daemon listems for Docker API's request and manages images, volumes and networks. Docker registry is the place where the docker images get stored. docker hub is the ppular cloud based registry to host the docker images. Docker is written in GO which open source programming language. Docker uses a technology called namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container. These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace.
PS C:\windows\system32> docker --version
Docker version 18.03.1-ce, build 9ee9f40
PS C:\windows\system32>
In the next post, I will be covering more hands on exercise on dockers.
References: Docker Website
Click Here To Read Rest Of The Post...
Subscribe to:
Posts (Atom)