Enabling Snaps in Linux Mint 20+

Linux MInt has taken the decision to disable the ability to install snapd and subsequently containerised apps (or snaps) from the Ubuntu Snap Store.

Linux Mint users are met with an error when attempting to install snapd or snap store applications

You can disable the block on installing snapd as follows.

sudo rm /etc/apt/preferences.d/nosnap.pref
sudo apt update
sudo apt install snapd

This will allow you to install those specific containerised versions of apps that you want, e.g. Whatsdesk (a containerised Whatsapp client for your Linux desktop).

sudo snap install whatsdesk
Facebooktwitterredditpinterestlinkedinmail

Expand the existing Swapfile on Linux

Note that this only applies if you’re using a swapfile as opposed to a swap partition. This will apply if you’re using full disk encryption since the swap file is then also encrypted. Most modern linux distro’s will behave in this way by default.

Ignore the first dd command since it contains a typo (shown)

If you’re using KDE Plasma as your desktop environment, for entertainment purposes, open Memory.

The Memory Information dialog opens to look like this.

Now expand the window by dragging a corner and you’ll see some neat graphs of memory usage…

Memory and Swap Usage graphs in KDE Plasma Desktop Environments “Memory” app

Now with your Konsole to one side of the graphs, observe the changes going on on the system as you execute each command. Really quite cool. KDE Plasma is great. By far my favourite desktop environment. I recommend it but live in it for a week and figure it all out since it’s quite comprehensive. You’ll likely not go back!

Facebooktwitterredditpinterestlinkedinmail

Windows 10 De-Bloat and Customise

Download latest Windows 10 ISO from here and perform a Windows 10 Pro installation to a freshly procured virtual machine. Say No to pretty much everything. Log into your Microsoft account during installation if you have other synchronised Windows devices already.

If using Oracle Virtualbox, insert VirtualBox Guest Additions CD and install, Reboot VM

If using an Sun Microsystems/Oracle Keyboard, change Host Key to Alt Graph on Keyboard since there’s no Right-Ctrl key which is the default.

Search for Windows Updates, Check for Updates, Install Windows Updates, Reboot VM

Open Powershell as Administrator and run the following command taken from here

iex ((New-Object System.Net.WebClient).DownloadString('https://git.io/JJ8R4'))

Remove components you don’t require by clicking on each button. The script to remove the feature will execute in the PowerShell console behind.

I like to install Rainmeters and I simply stick with the default meters/widgets so I can easily keep track of the VM resource consumption at a glance, if things should slow up..

This is a running list of optimizations I’ve made in the past on problematic Windows 10 desktop systems to speed them up/fix them.

remove avast AV
remove avast VPN
restore windows defender AV
remove windows c++
check startup programs, remove superflouous items
analyze and defrag hard disk
reset windows updates
remove transparency effects
install rainmeter and fix items to desktop
make rainmeters disappear on mouse over
install brave browser
remove immediate access to MS Edge browser
set dark themes in windows and brave
change power settings to optimize for performance
change power settings to not power down or sleep when plugged in
change windows updates to dynamically adjust times when it performs windows updates based on usage
change display effects to remove all special effects apart from drop shadows and show window contents when dragging.
Disabled Windows Search Service (Search Indexing)
Enable restart and notification for restart when Windows Updates needs it
troubleshoot windows updates.
system installed updates on restart
feature update to windows 10 version 2004 being prepared. leave laptop on to complete.
Synced time and set system to sync region automatically
Run cmd as admin, run SFC /scannow to check for missing / damaged system files. Some found and repaired.

Facebooktwitterredditpinterestlinkedinmail

Ethereum Private Blockchains.

This post contains the steps to setup a private Ethereum blockchain.

First, you’ll need a Linux vm. Please note that this post assumes a medium/advanced level of knowledge. As such, it will be much more succinct than most posts in other categories.

INSTALL ETHEREUM

sudo apt-get install software-properties-common
Add the ethereum repository: sudo add-apt-repository -y ppa:ethereum/ethereum
Update the package list and upgrade entire system
Install Ethereum.

CREATE DIRECTORY TO STORE THE BLOCKCHAIN

Create a directory to store the blockchain

CREATE TWO ACCOUNTS

Create first account
Create second account

CREATE GENESIS BLOCK

{
  "config": {
    "chainId": 15,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "ethash": {}
  },
  "difficulty": "1",
  "gasLimit": "8000000",
  "alloc": {
    "0x849b1448F8Cca9F3BCeE21F1ebcC1545f12b8e25": { "balance": "300000" },
    "0x140DBCb0eC85B62A66aCBf52DeEFd9F01abCc22A": { "balance": "400000" }
  }
}

INITIALIZE THE GENESIS BLOCK

Add the accounts created into the genesisblock.json file and initialize the blockchain with the following command…

geth -datadir="/home/matt/privatechain" init "/home/matt/privatechain/genesisblock.json"

If your genesisblock.json file is correctly written, the block will initialize with “Successfully wrote genesis state” message.

CHAINDATA AND LIGHTCHAINDATA

The chaindata and lightchaindata directories in geth indicate that your blockchain network has been created.

cd into ~/privatechain/geth and you’ll see chaindata and lightchaindata subdirectories
The genesis block.

MINING THE BLOCK

geth -datadir “/home/matt/privatechain” -mine

While the first block is being mined, ethers will be being added to your address. It may take a while depending on your parameters set in your genesisblock.json file

Typical output seen during mining

Now that mining has begun, you can see the new files that have appeared in the privatechain filesystem

geth.ipc transactions.rlp and the .ldb database files appear once the genesis block has been mined

SUMMARY

The steps performed above can be summarised as follows.

You can stop the mining process using the miner.stop() command

SMART CONTRACTS

“Smart contracts” is the name to describe a program that is compiled to execute on the blockchain network. Such programs running on Ethereum blockchains are written in the Solidity language.

SMART CONTRACT DEVELOPMENT ENVIRONMENT

Download ganache from here

It’s an .appimage weighing in at around 146MB. TruffleSuite Ganache is produced by ConsenSys Software Inc.

The TruffleSuite Ganache Downloads page.

chmod +x the ganache appimage file and execute it.

chmod +x ganache to allow execution of it

GANACHE

Enable/Disable analytics and click Continue
Click on Quickstart
Change Port Number to 8545 for use with Metamask or Brave Crypto Wallet Browser extension
Enable Crypto Wallets in Brave Browser
Click the option to Restore a Local Wallet

Enter the seed phrase from Ganache into the Seed phrase field in Brave

Copy and Paste this into the Restore Seed Phrase field in Brave Crypto Wallet Restore and set a new password for the wallet.
Once your wallet is displayed, change the Ethereum MainNet to Localhost:8545 to connect to the Ganache private blockchain.

Connect to the local Ethereum blockchain network on port 8545 (Ganache’s internal eth network) and you’ll see an initial balance of 100 ETH tokens.

Click on the coloured icon next to Localhost:8545 and Import a Wallet

In Ganache, click on the Key icon on one of the wallets (choose the 2nd one in the list of pre-configured wallets). Copy and Paste the private key into the Import screen shown above.

Copy and Paste the private key of the first ganache ethereum wallet into the Browser Crypto Wallet Importer
You’ll import the 2nd wallet, so now Brave displays the balance of the first and second Ganache wallets. Ganache comes with a few more wallets for you to play with in your private ethereum development network.

You can see in the screenshot above, two Accounts have been imported, each with a balance of 100 ETH tokens. You can import as many of the ETH wallets from Ganache as you like/need for your Solidity Smart Contract project.

Since Ganache is connected to our local private blockchain running on our localhost on port 8545, you can see the details of the genesis block in the BLOCKS section

REMIX SOLIDITY IDE

Next we need an IDE for creating code when developing, compiling and deloying Solidity smart contracts that will use our connected wallets to pay the gas fees required to store the smart contract on our private blockchain. Remix is a web-based IDE for this purpose.

The web based Remix IDE for Solidity smart contracts running on Ethereum blockchains.
Click on the Deploy and Run Transactions icon (shown) in the left-most panel, select Injected Web 3 and it’ll attempt to connect to your Brave Crypto Wallet. Tick the ETH wallet you want it to use and Authorise the connection of the wallet to Remix. Note that any actual transactions invoked when deploying compiled smart contract code as byte code to the private blockchain will still require manual authorisation by the wallet owner.
Note that none of the pre-defined smart contracts have yet been compiled into bytecode so cannot be executed using our connected wallet.
Click on the Compiler icon (shown) in the left-most panel and compile the smart contract.
The successfully compiled files can be deployed.

Compile all the scripts 1_Storage, 2_Owner, 3_Ballot and 4_Ballot_Test files. Deploy the 4_Ballot_Test file.

CONFIRM TRANSACTIONS

The connected wallet will prompt for a confirmation for the transaction.

The transactions in the smart contract will require confirmation by the owner of the connected wallet before being submitted to the private blockchain we created earlier where they’ll be mined.

AUDIT TRANSACTIONS STORED IN MINED BLOCKS

The genesis block and our two transactions above can be seen in the BLOCKS tab in Ganache.

In Ganache that we connected to our local private Ethereum blockchain, you can click on the BLOCKS section and see the mined blocks that contain our recent two transactions, the date stamp of the transaction and gas used.

Click on the transactions button for each mined block to see the details of the transaction in that block.
You can view the transactions in the TRANSACTIONS section if you don’t know what BLOCK contains it in the BLOCKS section.

SUMMARY

So far, we have…

Installed Ethereum

Created a private Ethereum blockchain on our localhost

Initialised the private blockchain

Mined the Genesis block in our private blockchain

Installed Ganache and connected it to our private blockchain

Imported two of the ETH wallets in Ganache to our Web Browser’s crypto wallet extension

Connected to Remix Solidity Smart Contract IDE

Connected Remix to one of our ETH Wallets

Compiled some Solidity Code Smart Contracts

Deployed the BallotTest smart contract to our private blockchain, confirming the transactions and accepting the gas fees.

Audited the mined blocks containing our transactions in Ganache

EXTRAS

The other smart contracts were compiled and deployed, and new blocks containing those transactions show up in Ganache.

The blocks in our private blockchain
The transactions in the blocks on our private blockchain
The transaction data contained in the contract creation transaction
The resulting balance in our connected ETH wallet used to pay the gas fees on the network
The transactions in our wallet.

FINAL WORD

Note that all this information is referenced from the same blockchain, instead of the necessity of trusted third parties storing the same data in multiple places that could be changed and a conflict introduced into the history of the transactions in potentially mismatched ledgers. In addition to having an accurate central ledger that all parties reference, we can introduce more nodes in our network that each maintain a copy of the same blockchain.

The public ETH network that maintains the main net has many thousands of ETH nodes. It is this decentralised consensus that gives the value to blockchain, along with public keys being used to verify that the parties transacting have the necessary funds without revealing the private keys necessary to unlock those funds to the public internet.

The input and outputs between the wallet addresses are performed by the closed network, keeping everybody safe. The issuance of new coins is also controlled by the network to prevent abuse of the inflation rate that could affect the purchasing value of the tokens by driving the value down by diluting the supply ad infinitum (hyper inflation caused by excessive quantitative easing, effectively) .

Facebooktwitterredditpinterestlinkedinmail

Terraform: Infrastructure as code.

Originally developed by Hashicorp, Terraform allows you to define your infrastructure, platform and services using declarative code, i.e. you tell it what scenario you want to get to, and it takes care on the in-between steps in getting you there.

CODE PHASE

E.g., if you wanted to get from a current state of “nothing” to a state of “virtual machine (vm)” and “kubernetes cluster (k8s)” networked together by a “virtual private cloud (vpc)”, then you’d start with an empty terraform file (ascii text file with an extension of .tf) and inside there’d be three main elements, the vm, the k8s cluster and the vpc above. This is called the “Code” phase.

PLAN PHASE

The plan phase compares the current state with the desired state, and forms a plan based on the differences, i.e. we need a VM, we need a K8s cluster and we need a VPC network.

APPLY PHASE

Next, is the Apply phase. Terraform works against the Cloud providers using real API’s, your API token, spin up these resources and output some autogenerated output variables along the way such as the kubernetes dashboard URL etc

PLUGGABLE BY DESIGN

Terraform has a strong open source community and is pluggable by design. There are modules and resources available to connect up to any IAAS cloud provider and automate the deployment of infrastructure there.

Although Terraform is widely considered to support the provisioning of Infrastructure as code from various Infrastructure as a Service (IaaS) cloud providers, it has expanded its use cases into Platform as Service (PaaS) and Software as a Service (SaaS) areas as well.

DEVOPS FIRST

Terraform is a DevOps tool. that is designed and works with the DevOps feedback loop in mind. i.e. If we take our desired scenario above of a VPC, VM and Kubernetes cluster and decide that we want to add a load balancer, then we would add a Load Balancer requirement to the .tf file in the code phase and the plan phase would compare the current state and see that the load balancer is the only change / only new requirement. By controlling the infrastructure using code in the terraform pipeline instead of directly configuring and changing the infrastructure away from it’s original terraform “recipe”, you can avoid “configuration drift”. This is called a “DevOps first” approach and is what gives us the consistency and scalability we want from a cloud based infrastructure, managed using DevOps practices.

INFRASTRUCTURE AS CODE

These days it’s increasingly important to automate your infrastructure as applications can be deployed into production hundreds of times a day. In addition, infrastructure is fleeting and can be provisioned or de-provisioned according to the demand to provide the service that meets customer requirements but also keep control of costs from cloud providers.

IMPERATIVE vs DECLARATIVE APPROACH

An imperative approach allows you to storyboard and define how your infrastructure is provisioned from nothing through to the final state, using a CLI in say, a bash script. This is great for recording how the infrastructure was initially provisioned, and can also make creating similar environments for testing etc but it doesn’t scale well and you are still at risk of others making undocumented changes that send your Dev, Test etc environments out of sync (they should always match) and risks that afore mentioned configuration drift.

So, we use a declarative approach instead, defining the FINAL STATE of the infrastructure using a tool like terraform, and letting the public cloud provider handle the rest. So instead of defining every step, you just define the final state.

IMMUTABLE vs MUTABLE INFRASTRUCTURE

Imagine a scenario where you have your scripts and you run them to get to v1.0 of your infrastructure. You then have a new requirement for a database to be added to the current infrastructure mix of VPC, VM and K8s elements. So, you modify your declarative code, execute it against your existing Dev environment, then assuming you’re happy, make the same change to 100 or 1000 other similar environments, only for it to not work properly, leaving you in a configuration drift state.

To eliminate the risk of this occurring, we can copy and modify our original code that got us to v1.0 and then execute it to create an entirely new and separate v2.0 state. This also ensures that your infrastructure can move to scale. It is expensive while there are v1.0 and v2.0 infrastructures running simultaneously but is considered best practice, and you can always revert to the v1.0 which remains running while v2.0 is deployed.

So, this immutable infrastructure approach, i.e. can not be changed/mutated once deployed, is preferable in order to reduce/eliminate risks with changing mutable infrastructure.

INSTALLING TERRAFORM

I found that terraform was not available via my package repositories, nor from my Software Manager on Linux Mint. So, I downloaded the Linux 64 bit package of the hour from terraform’s website

Unzipping the downloadable reveals a single executable file, terraform.
After unzipping terraform, move the executable to /usr/local/bin

I’m using an AWS account, so I’ll need to use the AWS Provider in Terraform.

Next create a .tf file for our project that will initialize the terraform aws provider in the aws region we want and run the terraform init command.

#OpenShot Terraform Project
provider "aws" {
  region  = "eu-west-2"
}
Our openshot.tf file simply declares the provider (aws) and the region and is read by the terraform init command executed in the same directory.
Initialization takes a few seconds.

If you haven’t already got one then you’ll need to set up an account on aws.amazon.com. It uses card payment data but is free for personal use for one year. The setup process is slick as you’d expect from amazon, and you’ll be up and running in just a few minutes.

Log into your AWS Management Console.

In your AWS Cloud Management Console, Click Services, EC2.
Click Instances, Launch Instance.
Click AWS Marketplace and search for openshot

We don’t need to select it since we’re using terraform to build our infrastructure as code, so we can have terraform perform this search. The OpenShot AMI is a terraform DataSource AMI since its an image that already exists – not a Resource AMI i.e. we’re using an existing AMI not creating a new one. Terraform can perform this search for us in our .tf file.

Note that the documentation on terraform’s website uses /d/ or /r/ in the URL for datasources and resources of similarly named elements.

The Google search aws ami datasource terraform will take you here

We can add a few more lines to our openshot.tf file to have terraform search for the AMI in the AWS Marketplace as follows.

#OpenShot Terraform Project
provider "aws" {
  region  = "eu-west-2"
}

data "aws_ami" "openshot_ami" {
  most_recent = true
  owners = ["aws-marketplace"]

  filter {
    name = "name"
    values = ["*OpenShot*"]
  }
}

The next thing we need to do is create a security group, so that we can access our EC2 instance.

Google search for terraform aws security groups will take us here

Copy and paste the example code into our openshot.tf file and make a few adjustments to allow access from http and ssh. Note that you should restrict SSH access to your own IP address to prevent exposing your SSH server to the world.

#OpenShot Terraform Project
provider "aws" {
  region  = "eu-west-2"
}

data "aws_ami" "openshot_ami" {
  most_recent = true
  owners = ["aws-marketplace"]

  filter {
    name = "name"
    values = ["*OpenShot*"]
  }
}
resource "aws_security_group" "openshot_allow_http_ssh" {
  name        = "openshot_allow_http_ssh"
  description = "Allow HTTP and SSH inbound traffic"

  ingress {
    description = "Inbound HTTP Rule"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    description = "Inbound SSH Rule"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    #NOTE: YOU SHOULD RESTRICT THIS TO YOUR IP ADDRESS!
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "openshot_allow_http_ssh"
  }
}

We need to configure a Key Pair for our Amazon EC2 Instance for the SSH to ultimately work. We’ll return to this later when we configure the EC2 instance details in the openshot.tf file we’re constructing.

Configure a SSH Key Pair here.
Creating an SSH keypair for our openshot project

Save the openshot-ssh.pem file to your project folder when prompted.

TERRAFORM PLAN

Next, we can test this with the terraform plan command.

terraform plan fails at this stage as we’ve not specified the credentials for our AWS account.

AWS CREDENTIALS

The AWS credentials can be specified as environment variables or in a shared credentials file in your home directory in ~/.aws/credentials more on the options here

The ~/.aws/credentials file can be created by installing the awscli package from your repositories, and running aws configure

Note that you should create an IAM user in your Amazon Management Console, and not create credentials for your aws root user account. I have created an account called matt for example. You’ll receive an email with a 12 digit ID for each IAM user which you’ll need to log in as well as the username and password for that user. The root user just uses an email address and password to log in.

Once you’ve logged in to your AWS Management Console, the credentials are obtained here…

Click on your account user@<12-digit-number>, My Security Credentials

After pasting the Access Key ID in, you’ll be asked for the Secret key next. In the event you don’t have it, you can simply CTRL-C out of aws configure, go back to the aws management console and generate a new one. You’ll only be shown the private key one time, so be sure to copy it, then re-run aws configure and enter the new access key id and secret key. I specified eu-west-2 (London) as my default location and json as my output format.

Note that the credentials are stored in plain text in ~/.aws.credentials

Depending on your situation, you may be able to Deactivate the previous Access Key ID and delete it from AWS Management Console if it’s never going to be used.

TERRAFORM PLAN USING CREDENTIALS FILE

Now if we re-run our terraform plan we see it succeeds.

re-run terraform plan and this time it used the creds found in ~/.aws/credentials

You can see that the Plan outcome at the end is 1 to add, 0 to change, 0 to destroy.

EC2 INSTANCE

Now we’re ready to specify our EC2 Instance and we just need the final section and edit it as shown below based on the data source and resource names specified elsewhere in the terraform file, adding key_name=”openshot-ssh” to refer to our .pem file we created earlier when we generated a SSH key pair for the EC2 Instance on the AWS Management console.

#OpenShot Terraform Project
provider "aws" {
  region  = "eu-west-2"
}

data "aws_ami" "openshot_ami" {
  most_recent = true
  owners = ["aws-marketplace"]

  filter {
    name = "name"
    values = ["*OpenShot*"]
  }
}
resource "aws_security_group" "openshot_allow_http_ssh" {
  name        = "openshot_allow_http_ssh"
  description = "Allow HTTP and SSH inbound traffic"

  ingress {
    description = "Inbound HTTP Rule"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    description = "Inbound SSH Rule"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    #NOTE: YOU SHOULD RESTRICT THIS TO YOUR IP ADDRESS!
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "openshot_allow_http_ssh"
  }
}

resource "aws_instance" "web" {
  ami           = "data.aws_ami.openshot_ami.id"
  instance_type = "c5.xlarge"
  security_groups = ["{aws_security_group.openshot_allow_http_ssh.name}"]
  key_name = "openshot-ssh"

  tags = {
    Name = "OpenShot"
  }
}

Re-run terraform plan and we can see our

Re-running terraform plan shows our openshot ami resource on our aws ec2 instance
and ends with the message Plan: 2 to add, 0 to change, 0 to destroy.

OUTPUT

Lastly, we can add an output section that outputs the public IP for our OpenShot AMI

output "IP" {
  value = "${aws_instance.web.public_ip}"
}

TERRAFORM APPLY

When our terraform plan is ready, we can execute the command terraform apply and terraform will re-execute terraform plan before prompting for input prior to executing the script against the aws cloud provider and building our infrastructure.

terraform apply

Facebooktwitterredditpinterestlinkedinmail

Basic Linux Commands

Here are some Linux commands that everyone should be familiar with. In fact, you could argue that these are the first commands to memorise and build out your repertoire from there.

#BASIC LINUX COMMANDS

#Clear the terminal window
  clear
#Show kernel version
  uname -a
#Show all tunable kernel parameters in the /proc/sys directory
  sudo /sbin/sysctl -a
#Set a kernel parameter on the fly without persistence
  sudo /sbin/sysctl -w kernel.sysrq="1"
#Set a kernel parameter with persistence
  /etc/sysctl.conf
#Kernel parameters startup script
  /etc/rc.d/rc.sysinit
#Show network interfaces
  ifconfig
  ip addr show
#Configure network interface with persistence
  /etc/sysconfig/network
  /etc/sysconfig/network-scripts/ifcfg-eth0
#Show all filesystems and space
  df -ah
#Show service status
  service udev status
  systemctl status udev
#How much disk space is used by a given directory
  du ~/Downloads
#What TCP and UDP ports is the listem listening on?
  netstat -tulpn
  sudo netstat -tulpn #gives more info on process name
#Show information about a given process
  ps aux | grep containerd
#Show free memory stats
  free
#List block storage devices known to the system
  lsblk
#Show mounted storage devices
  mount
#Show filesystems that should be mounted at boot
  cat /etc/fstab
#Mount everything in /etc/fstab
  mount -a
#Mount a block storage device
  mount /dev/sdb1 /mnt
#LVM Commands
  pvdisplay pvcreate  pvremove  pvchange
  vgdisplay vgcreate  vgextend  vgremove  vgchange
  lvdisplay lvcreate  lvextend  lvremove  lvchange
  mkfs.ext4
#Copy files
  cp
  rsync
  dd
#Show command history
  history
#Look up a command
  man -k <search-string>
  man grep
Facebooktwitterredditpinterestlinkedinmail

XML, JSON and YAML

In DevOps, you’ll find yourself interacting with or creating files that are either XML, JSON or YAML format, depending on what you’re doing. All these file formats are a way of representing data, and a comparison of the three different file formats representing the same data is shown below.


In DevOps, you’ll find yourself interacting with or creating files that are either XML, JSON or YAML format, depending on what you’re doing. All these file formats are a way of representing data, and a comparison of the three different file formats representing the same data is shown below.

XML, JSON and YAML files representing the same data.

Facebooktwitterredditpinterestlinkedinmail

Kubernetes

WELCOME TO THE FUTURE OF INFRASTRUCTURE

In a single sentence: Kubernetes intends to radically simplify the task of building, deploying and maintaining distributed systems.

Kubernetes or K8s is open-source software, originally created by Google as a way to take on the burden of management of container sprawl for applications/microservices running on multiple containers (vm’s sharing a single kernel on a Linux/UNIX OS) that scale out onto potentially tens or hundreds of individual containers across multiple hosts. Google then handed it to the Cloud Native Computing Foundation.

In kubernetes, there is a master node and multiple worker nodes.

Each worker node manages multiple pods. Pods are just a bunch of containers, clustered together as a working unit.

Application developers design their application based on pods. Once those pods are ready, you tell the master node the pod definitions and how many need to be deployed.

Kubernetes takes the pods, deploys them to the worker nodes. In the event a worker node goes down, kubernetes deploys new pods onto a functioning worker node. The complexity of managing many containers is removed.

It is a large and complex system for automating, deploying, scaling and operating applications running on containers.

MINIKUBE

Rather than create a seperate post on MiniKube, I’ll incorporate it here on my Kubernetes post. MiniKube is a way of learning Kubernetes by running a single node cluster locally on a laptop/desktop machine. The commands required to get it up and running on Linux Mint/Debian are shown below.

INSTALLATION OF MINIKUBE ON DEBIAN/MINT

#INSTALL VIRTUALBOX
sudo apt-get install virtualbox virtualbox-qt virtualbox-ext-pack virtualbox-guest-additions-iso virtualbox-guest-utils

#INSTALL KUBECTL
sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

#INSTALL MINIKUBE VIA DIRECT DOWNLOAD
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube

#ADD MINIKUBE EXECUTABLE TO PATH
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/

#VERIFY INSTALLATION
minikube start --driver=virtualbox

#CHECK STATUS
minikube status

#STOP MINIKUBE
minikube stop

#START MINIKUBE
minikube start

#ENABLE BASH COMPLETION
kubectl completion bash

#ENABLE KUBECTL AUTOCOMPLETION
echo 'source <(kubectl completion bash)' >>~/.bashrc
kubectl completion bash >/etc/bash_completion.d/kubectl

#IF YOU USE A COMMAND ALIAS FOR KUBECTL
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -F __start_kubectl k' >>~/.bashrc

#START AND OPEN MINIKUBE DASHBOARD
minikube dashboard
I had to reboot and enable VTX (virtualisation support) in the BIOS on my HP EliteDesk.

You can check the status of MiniKube and Stop and Start it, using these commands.

#CHECK STATUS
minikube status

#STOP MINIKUBE
minikube stop

#START MINIKUBE
minikube start
There is an issue with VirtualBox that needs to be resolved.
Running minikube config set driver virtualbox, minikube delete and minikube start resolved the issue
The minikube virtual machine can be seen running in the VirtualBox console (virtualbox-qt)

Now that minikube is running, we’re ready to open the console and see our Kubernetes single-node cluster that is running on our local machine.

The minikube dashboard command enables the dashboard and opens the URL in your web browser.
The kubernetes dashboard, running on our minikube virtual machine on VirtualBox is displayed in the Brave browser.

BACK TO KUBERNETES…

CONTAINERS AND ORCHESTRATION

To understand kubernetes, we must first understand containers and orchestration. So make sure you’ve read and understood Docker first.

A Kubernetes (K8’s) Cluster was originally developed by Google to manage containers on a very large scale.

In Kubernetes, a Management Cluster (Kubectl) wraps a container called a “Pod” around each Docker container, and clusters Pods into a Replica set which is wrapped into a “Deployment“. A Service wraps around a Deployment and within the Service, a Daemonset controls what runs on each Pod for consistency and a Load Balancer, Cluster IP and Node Port controls the internal networking to simplify communications between Services. An Ingress Controller serves to protect the Service’s External IP from being visible, by controlling inbound URL and API requests and routing them to the Services, Deployments, Pods and Container via the Load Balancer and NodePorts. A Cluster IP facilitates communications between multiple pods via a single, virtual IP.

Using Pod Autoscalers and Cluster Autoscalers, the number of containers that can be deployed and automatically scaled to meet service level requirements is managed using the kubectl command on the Kubectl management cluster.

Rolling updates or a roll back of an image deployed to thousands of pods is done using the kubectl rolling-update command.

kubectl run --replicas=1000 my-web-mywebserver      #Runs a 1000 instances of an image in a kubernetes cluster
kubectl scale --replicas=2000 my-web-mywebserver    #Scales cluster up to 2000 containers
kubectl rolling-update my-web-server --image=web-server:2 #Performs rolling update to a Deployment of Pods
kubectl rolling-update my-web-server --rollback #Rolls a Deployment Pod image back to previous version
kubectl run hello-minikube #deploy an app to the cluster
kubectl cluster-info #view info about the cluster
kubectl get nodes #view nodes that are part of the cluster
Components in a Kubernetes System: API Server, etcd Key Store, Scheduler, Controller, Container Runtime and Kubelet service

A Kubernetes system consists of the following six components.

API SERVER

This is the front-end component that interacts with management commands, users, other management components (3rd party vendors storage etc).

ETCD

This is the distributed, reliable key value store, where information required to access all the containers is kept and maintained.

SCHEDULER

This is responsible for distributing work to the containers across multiple Nodes. It looks for newly created containers and assigns them to nodes.

CONTROLLER

This is the brain behind orchestration. It is responsible for noticing when endpoints, containers or nodes close down.

CONTAINER RUNTIME

The software used to run containers – in our case Docker but Kubernetes can be used to manage other container technologies too.

KUBELET

The agent that runs on each node in the cluster. It is responsible for making sure the containers are running on the nodes as expected. It listens for instructions from the Controller.

KUBE PROXY SERVICE

The Kube Proxy Service is another service that runs on each Node that makes sure all networking required to facilitate the necessary communication between containers is in place.

KUBERNETES ARCHITECTURE

So to summarise so far, a Kubernetes Architecture consists of the following key components spread across the Master Node and Worker Nodes in the Cluster.

KUBEADM

At the beginning of this post, we cover the steps in setting up a KUBECTL single-node kubernetes cluster, with a single worker node. Setting up a multiple node kubernetes cluster manually is a tedious task. KubeAdm is a command line tool that allows us to set up multiple node clusters more easily.

STEPS

  1. You will need multiple virtual machines set up, ready to act as Docker hosts and as the Management and Worker Nodes.
Our existing minikube virtual machine is the only machine currently running in virtualbox – we need more if we are going to run multiple worker nodes.

Note that the minikube VM must be started from the command line using the command

minikube start –driver=virtualbox

not from Oracle VM VirtualBox Manager (virtualbox-qt).

The minikube VM must be OFF before starting it with the command minikube start –driver=virtualbox

Open the minikube web console in your default browser with the command

minikube dashboard

Open Kubernetes Dashboard in your default browser after starting MiniKube.
minikube dashboard opens the above URL in your Browser.
The Kubernetes Web Console (click to enlarge)

XML, JSON and YAML

In DevOps, you’ll find yourself interacting with or creating files that are either XML, JSON or YAML format, depending on what you’re doing. All these file formats are a way of representing data, and a comparison of the three different file formats representing the same data is shown below.

XML, JSON and YAML files representing the same data.

PODs with YAML

Facebooktwitterredditpinterestlinkedinmail

Docker

READ THIS FIRST! On HP workstation equipment, you’ll likely need to enable VTX Extensions in the BIOS to allow for the execution of virtualisation technology. I use a hp EliteDesk 800 G2 Mini and the factory BIOS setting has VTX disabled by default. If in any doubt, check your BIOS now, or you WILL encounter problems later and could waste hours troubleshooting.

When it comes to containers, Docker is the most popular technology out there. So, why do we need containers? How do they differ from VM’s? Very briefly, a VM (as in a VMWare ESXi Virtual Machine) is an entirely self contained installation of an entire operating system that sits on top of a “bare metal hypervisor” layer that sits on top of the physical hardware. Unlike installation of an OS on the physical hardware, the bare metal hypervisor layer allows multiple installations of multiple OSes to co-exist on the same hardware – entirely separate apart from the physical resources they share. There may be cluster of ESXi hosts, managed by a Central vCenter server that allows better distribution of multiple VM’s across multiple physical hosts for the purposes of Distributed Resource Scheduling and High Availability. This is a more efficient use of physical servers, but still quite wasteful on resources and storage since many Windows VM’s for example, would be running the same binaries and storing the same information many times over.

Containers on the other hand are logically separated applications and their dependencies that all reside in an isolated “container” or “zone” or “virtual machine” or “jail” depending on the single instance of an underlying UNIX/Linux based OS. So, they share common components of the underlying OS which is a more efficient use of space and physical resources since only one instance of the operating system is running on the physical machine. This also reduces the overhead on patching and to some extent, monitoring, since in the case of an application hosted on multiple, entirely separate full stack VM’s in a virtual environment, only the parts of the stack with unique/incompatible dependencies are separated into their own container. This means that a container compared to a VM may be very small indeed, and containers are typically restarted in the time it takes to start a daemon or service, compared to the time it takes to boot an entire OS.

So in summary, a container is a more intelligent, more efficient way of implementing the various layers in a full stack application that won’t otherwise co-exist on the same OS due to their individual dependencies for slightly different versions of surrounding binaries and/or libraries.

On VMWare ESXi, there is no Operating System layer (shown in Orange), but VMWare Workstation or Oracle Virtualbox provides similar full OS VM separation within a software hypervisor running as an application in its own right atop an underlying desktop OS such as Windows or Linux. Hence the term bare-metal hypervisor (since the Hypervisor layer shown in Blue runs atop the Server hardware shown in Grey). Docker is similar to a software hypervisor, but rather than store multiple similar full OS/App stacks, it provides separation further up the stack, above the OS layer, such that just the unique requirements of the app/daemon/microservice are hosted in any given container, and nothing more in an effort to become as efficient as it’s possible to be.

REASONS FOR ADOPTING CONTAINERISATION.

In most information technology departments, there’ll be a team of developers who code and build apps using combinations of their own code, database, messaging, web server, programming languages that may each have different dependencies in terms of binaries/libraries and versions of the same binaries/libraries. This is referred to the Matrix from Hell as each developer will be building, knocking down, rebuilding their own development environment that likely runs on their own laptop. There’ll be a development environment too, the intention of which is to mirror that of the production environment although there’s no guarantees of that. There may be other Staging or Pre-Production environments too, again with no guaranteed consistency despite everybody’s best efforts. The problems arise when deploying an app from a Development environment to the Production environment only to find it doesn’t work as intended.

The solution to this problem is to put each component in the application you ultimately want to ship into production/cloud into its own container, i.e.

All the components in the application running on a single Linux OS that has Docker installed can be placed in their own container, i.e…
Once an application component is contained within its own container, all the dependencies that component has (other linux packages and libraries) will also be contained within the same container. So each component has exclusive access to just the packages and libraries it needs, without the potential to interfere with and break adjacent components on the same underlying host operating system.

In order to isolate the dependencies and libraries in this way, a typical Docker container will have its own Processes, Network interfaces and Mount points, just sharing the same underlying kernel.

Two containers with their own processes, network interfaces and mounts share the kernel of the OS that Docker is running on.

Linux containers typically run on LXC, a low level hypervisor that is tricky to set up and maintain, hence Docker was born to provide higher level tools to make the process of setting up containers easier.

Since Docker containers only share the kernel of the underlying OS, have their own processes, network interfaces and mounts, it is possible to run entirely different linux OS’es inside each container, since the only part of the underlying OS that Docker is running on, is the underlying kernel of that OS!

If you want, entirely different linux distributions that are able to run on the same kernel can be run in docker containers! However, since the kernel is very small, this is arguable as wasteful as simply using VMWare ESXi to host individual VM’s each running a different linux distro?!

Since the underlying kernel is the shared component, only OS’s that are capable of running on the same kernel can exist on the same docker host. Windows could not run in the scenario above, and would need to be run on a Windows Server based Docker host instead. This is not a restriction for the VMWare ESXi bare-metal hypervisor of course, where Windows and Linux can co-exist on the same physical host since their kernels are contained within the VM, along with everything else.

HOW IS DOCKER CONTAINERISATION DONE?

The good news is that containers are nothing new. They’ve been about for over a decade and most software vendors make their operating systems, databases, services and tools available in container format via Docker Hub.

Some of the official container images available on Docker Hub.

Once you identify the images you need and you install docker on your host, bringing up an application stack for the component you want, is as easy as running a docker command.

docker run ansible  #downloads and runs a container running ansible
docker run mongodb  #downloads and runs a container running mongodb
docker run redis    #downloads and runs a container running redis
docker run nodejs   #downloads and runs a container running node.js

A docker image can be installed mulitple times, for example in a scenario where you want multiple instances of node.js (you’d need a load balancer in front of the docker host(s)) so that in the event of a node.js container going down, docker would re-launch a new instance of that container.

So, the traditional scenario, where a developer puts together a bunch of components and builds an application, then hands it to operations, only for it to not install or run as expected because the platform is different in some way, is eliminated by ensuring all dependencies of each component is contained in its own container, and is thus guaranteed to run. the docker image can subsequently be deployed on any docker host, on prem or in the cloud.

It worked first time!

INSTALLING DOCKER

My everyday desktop machines all run Linux Mint for it’s ease of use and it’s propensity to just work when it comes to my various desktop computing requirements. You’d likely not run it on your servers though, instead choosing Debian or Ubuntu (which Mint is actually based on but not guaranteed to be exactly the same). Your server linux distro choice should be based on support, and by that I mean support for any problems that arise and support in terms of software vendors and in our case, docker image availability.

So, since I’m blogging on this same Mint machine, I’m going to install Docker via the Software Manager for immediacy. I will however cover installation on Ubuntu below.

Quickest and most reliable way of installing a working Docker on Mint, is to use the Software Manager.

The first step is to get docker. Start here.

Take the Docker for Linux option, unless you’re running it on a Windows or Mac Desktop machine

Follow the instructions here to install docker. The steps are also shown below.

#INSTALLING DOCKER ON UBUNTU
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo docker run hello-world
sudo docker run hello-world will download the hello-world container, install it and produce the output shown.
always precede your ‘docker’ command with sudo. It needs those root level privileges to communicate with the docker daemon.
sudo docker run -it ubuntu bash opens a root shell on container ‘ade951cb999’

From the ubuntu bash container, issuing a few linux commands quickly shows that the container has no reach outside of itself but shares stats returned by the kernel, such as disk space for the /dev/sda2 partition (the docker hosts root partition), cpu, memory and swap utilisation .

The hostname is different, there is only one process running ‘bash(excluding the ps -ef command itself), it can see how much physical disk space is used on the docker host (67% /dev/sda2), it has it’s own set of directories (/home shows as being empty) and the output from top shows only the 1 running process.

Standard linux commands being run inside the ubuntu bash container
The CPU, Memory and Swap statistics are the same as the Docker host that the container is running on since they share the same kernel.
/etc/issue says Ubuntu despite the Docker host being Linux Mint
The Docker host running Linux Mint.
cat /etc/*release* reveals more information about the operating system running in the container.

To display the version of docker thats been installed, use sudo docker version

sudo docker version

DOCKER IMAGES AND COMMANDS

Remember, you probably need to precede docker commands with sudo.

Here are some initial commands before I stop to make an important point…

#DOCKER COMMANDS
docker run nginx  #checks for a local copy of the nginx container image, if there isn't one, it'll go out to docker hub
                  # and download it from there.  For each subsequent command, it'll use the local copy.
docker ps         #lists all running containers
docker ps -a      #lists all containers present, running or otherwise
docker start <container-id> or <container-name>   #(re)start a non-running container
docker stop <container-id> or <container-name>    #stop a running container
docker rm <container-id> or <container-name>      #remove container
docker images     #shows all docker images downloaded from docker hub on the local system
docker rmi  nginx   #removes the docker image from the system (make sure non are running)
docker pull ubuntu  #pulls ubuntu image to local system but dont run it until docker run command is issued
Containers are only running while the command executed inside them is running. Once the process stops, the container stops running. This is an important distinction from VM’s that stay running and consuming system resources, irrespective. Note also the final column, a randomly assigned “name” for the container.

An important distinction between containers and VM’s is that whereas a VM stays running all the time, a container is only running while the command inside it is running. Once the process for the command completes, the container is shutdown, thus handing back any and all resources to the docker host.

Taking the 1st, 2nd and 3rd columns from the sudo docker ps -a command above for closer inspection, you can see that there is a container ID, the docker image, and the command run within that docker image, e.g.

Earlier we executed the command sudo docker run ubuntu bash and the docker host checked for a local copy of the ubuntu image, didn’t find one, so downloaded one from docker hub. It then started the container, and ran the bash command within that container, and thus we were left as a running bash command prompt on our container running ubuntu. As soon as we typed exit, the bash terminal closed, and since there were no running processes remaining on that container, the container was subsequently shut down.

Another container, docker/whalesay was also downloaded and ran the command cowsay Hello-World! before exiting and unlike ubuntu bash dropped us back at our own prompt on the docker host. This is because once the cowsay Hello-World! command had executed, there was no further need for the container, so it was shut down by the docker host.

docker exec mystifying_hofstadter cat /etc/hosts    #execute a command on an existing container
docker start <container-id> or <container-name> #starts an existing non-running container
docker stop <container-id> or <container-name> #stops a running container that's been STARTed

So, docker run <image-name> <command> will get a docker image, start it and execute a command within it, downloading the image from docker hub if required. But once the image is stored locally and that container exists on our docker host system, albeit in an exited state, what if we want to run the command again. Well, docker run <image-name> <command> will create another duplicate container and execute the command in that. We want to execute the same command in the existing container. For that we use the docker start command followed by the docker exec command and optionally finish up with the docker stop command e.g.

Before using docker exec to execute a command on an existing container, you’ll need to docker start it first.

DETACH and ATTACH (background and foreground)

If you’re running a container that produces some kind of output to the foreground but you want to run it and return to the prompt instead, like backgrounding a linux command, you can docker run -d <container> to run it in detached mode. To bring that running container to the foreground, you can docker attach <container>. Where <container> is the id or name of the container.

docker run -d kodekloud/simple-webapp               #runs a container in the background if it would otherwise steal foreground
docker run -a <container-id> #bring detached container (running in the background) to the foreground

If you have a docker image like redis, that when run with docker run, will stay running interactively in the foreground, there is no way to background it (detach it) without hitting CTRL-C to kill the process, then re-run it with docker run -d so that it runs in detached mode. However, if you run your containers with docker run -it then you can use the key sequence CTRL-P CTRL-Q to detach it without killing it first. Reattach using the docker attach <container> command. According to the docker run –help page, -i runs in in interactive mode and -t allocates a pseudo tty (terminal) to the running container.

DOCKER COMMANDS AND HELP SYSTEM

Docker has a very nicely thought out help system. Simply type docker and all the management commands and docker commands are listed along with descriptions. Type docker <command> –help and you’ll see more information on that particular command.

docker commands. Use docker <command> –help to dig deeper.

RUN TAG

If we run a container e.g. redis with docker run redis, we can see in the output the the version of redis is Redis version=5.0.8

The version TAG is Redis version=5.0.8 in our redis container.

If we wanted to run a different version of redis, say version 4.0, then we can do so by specifying the TAG, separated by a colon e.g. docker run redis:4.0

Run a different version of the redis container by specifying the TAG in the the docker run redis:4.0 command

In fact, if you specify no TAG, then what you’re actually doing is specifying the :latest tag, which is the default if no tag is specified. To see all the Tags supported by the container, go to docker hub, search for the container and you’ll see the View Available Tags link underneath the command.

RUN -STDIN

Already mentioned above, if you have a simple shell script that prompts for user input, then produces an output, e.g.

#!/bin/bash
echo "What is your name?"
read varname
echo "Hello $varname. It's nice to meet you."
exit
hello.sh needs to prompt the user for input before producing an output.

If this simple program were containerised with docker, when run, it needs to prompt the user for input before it can produce an output. So, the command needed to run this container, would be docker run -i -t <image>. The i runs the container in interactive mode so you can enter stdin, and the t allocates a pseudo terminal so you get to see the stdout.

PORT MAPPING

Before talking about port mapping, I’ll first cover how to see the internal ip address assigned to the container and the port the container is listening on. The output of docker ps will display a PORTS column, showing what port the container is listening on, then use docker inspect <container-name> to see the IP Address.

display the port using docker ps and use docker inspect to display the internal ip address.

The internal IP address is not visible outside of the docker host. In order for users to connect to the port on the container, they must first connect to a port on the docker host itself, that is then mapped to the port on the container i.e.

Here we see port 80 on the docker host is mapped to port 5000 on the container running a web app.

To map a local port on the docker host to a listening port on the container, we use the docker run -p 80:5000 <image-name> command. The -p stands for publish and creates a firewall rule allowing the flow of traffic through the specified ports. By default a container doesn’t publish any of its ports.

Users can connect to the IP and Port on the Docker host, and be tunnelled through to the container.

VOLUME MAPPING AND PERSISTENT DATA

If you’re running a container that stores data, any changes that occur are written inside that container. e.g. a mysql container will store it’s tablespace files in /var/lib/mysql inside the container.

A MySQL database will write data to it’s internal file system. But how does that work?

docker run -v /opt/datadir:/var/lib/mysql mysql mounts the directory /opt/datadir on the docker host into /var/lib/mysql on the mysql container. Any data that is subsequently written to /va/rlib/mysql by the mysql container, will land on /opt/datadir on the docker host, and as such will be retained in the event that the mysql container is destroyed by docker rm mysql.

CONTAINER INFORMATION

Already mentioned before, the docker inspect command returns many details about the container in JSON format. ID, Name, Path, Status, IP Address and many other details.

LOGS AND STDOUT

So, you can run the docker run -it redis command and see the standard output, but if you have an existing container that you start with docker start <container-name> and then attach to it using docker attach <container-name> you won’t see any stdout being sent to the screen. This is because unlike running it interactively with an assigned tty, simply starting the container and attaching to it, will not assign a tty. In order to view the stdout on the container, use the docker logs <container-name> and you’ll see the same output that you would if you used the docker run -it redis command. Obviously, using docker run redis would create a new container using the redis image, not start an existing redis container.

Starting and attaching to a container that produces stdout will not display the stdout
Using the docker logs <container-name> command to view the stdout on that container.

ENVIRONMENT VARIABLES

Consider the following python code web-page.py to create a web server that serves a web page with a defined background colour and a message. If the python program has been packed up into a docker image called my-web-page, then you’d run it using the docker run my-web-page command, connect to it from the web browser on the docker host on port 8080 to view the page.

import os
from flask import Flask

app = Flask (__name__)

color = "red"

@app.route("/")
def main():
    print(color)
    return render_template('hello.html', color=color)
                           
if __name__ == "__main__":
    app.run(host="0.0.0.0", port="8080")

The python program has a variable color=red within it but you want to be able to pass in values for that variable from the docker host when you run the container. To do this, you can move the variable outside of the python code by replacing the line of color=red with color = os.environ.get(‘APP_COLOR’)

import os
from flask import Flask

app = Flask (__name__)

color = os.environ.get('APP_COLOR')

@app.route("/")
def main():
    print(color)
    return render_template('hello.html', color=color)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port="8080")

On the docker host, you can create and export a variable export APP_COLOR=blue; python web-page.py and refresh the page and the colour will change since it’s value is being read from an external variable on the docker host.

To run a docker image and pass in a variable, you can use the command

docker run -e APP_COLOR=orange my-web-page

to pass the variable APP_COLOR-orange into the container image my-web-page before the container is started.

To find the environment variable set on a container, you can use the docker inspect <container-name> command, and in the JSON file, under the “config”: { section, “env”: { subsection, you’ll see “APP_COLOR=blue” variable, along with some other variables too.

docker inspect will show the variables passed in from the docker host

CREATING A DOCKER IMAGE

So, you now have a good idea on how to interact with docker images and docker containers running on your linux system. We’ve even seen some code that can be containerised but we’ve not elaborated on how you get from a python program or shell script to a docker container image. Lets cover that important topic next.

Firstly, lets ask “Why would you want to dockerize a program?”. There are two answers to that question. The first is that you cannot find what you want on docker hub already, so you want to make it yourself. The second is that you have a program on your development laptop/environment and want to containerise it for ease of shipping to operations teams or docker hub and deployment on production infrastructure.

So, taking the above example of a web server and web page python script called web-page.py that uses the python flask module. If i were to build a system to serve that page, I’d follow the following steps.

  1. Install the Ubuntu OS
  2. Perform a full update of all packages using apt-get update && apt-get dist-upgrade
  3. Install python3.x and any dependencies using apt-get install python3
  4. Install python3.x script module dependencies using the python pip package manager
  5. Create/Copy the python source code into /opt directory
  6. Run the web server using the flask command.

DOCKER FILE

A docker file is basically the sequence of commands required to perform the sequence of steps listed above. It is written in an INSTRUCTION, Argument format. Everything on the left in CAPS is an Instruction, and everything that follows it is an argument.

It looks like this…

#Dockerfile for cyberfella/my-web-page
FROM Ubuntu 

RUN apt-get update
RUN apt-get install python

RUN pip install flask
RUN pip install flask-mysql

COPY . /opt/source-code

ENTRYPOINT FLASK_APP=/opt/source-code/web-server.py flask run

Docker has a “layered architecture”. This means that when you run docker build (shown below), it builds a docker image that contains only the parts of the application added by each line in the Dockerfile. Think of the Dockerfile as the recipe, the docker build command as the chef and the docker image as the dish (and I suppose docker hub as the restaurant if we take this food analogy all the way to its logical conclusion!).

The first line contains a FROM instruction. This specifies the base OS or another docker image.

To build the docker image, use docker build Dockerfile -t cyberfella/my-web-page command. This will create a local docker image called my-web-page.

Note that if your Dockerfile lives in a directory called MyApp, then you need to specify the folder that contains the Dockerfile in it in the docker build command, e.g. sudo docker build MyApp -t cyberfella/my-web-page

docker build adds each layer based on the actions of each instruction in the Dockerfile, to create the Docker image. Each layer only stores the changes from the previous layer, which is reflected in the size of the image file.
docker build output clearly shows the activity based off each line in the Dockerfile

If the docker build process fails during the processing of one of the layers, then once you have fixed the issue, the build process will start again at the failed layer since all previous layers are stored in the docker hosts cache. This means docker is very quick to rebuild an image, compared to the first time the image is built.

Re-running docker build will use the contents of the cache for layers that were previously successfully built. This makes the docker build process faster over time as small changes are all that are implemented with each subsequent run of docker build.

To push the image to Docker Hub, use the command docker push cyberfella/my-web-page

If docker push fails, you’ve probably not used docker login to log in to your repository on docker hub!

Note that cyberfella is my Docker Hub login name. You will need to register on Docker Hub first so that there’s a repository to push to and you’ll need to log into your repository from the docker host using the docker login command. You can also link your Docker Hub repository with your GitHub repository and automate new Docker image builds when updated code is pushed to GitHub!

You can see the size of each layer in the image by using the command docker history <image-name>

docker history <image-name> displays the amount of data being added to the image in each layer of the docker build process

COMMANDS ARGUMENTS AND ENTRYPOINTS

If you recall the last Instruction line in the Dockerfile was CMD and the Argument that follows it is the name of the command or daemon that you want your container to execute. These arguments can take the form of the command you’d ordinarily type in a Shell, or can be specified in JSON format, e.g.

The sleep 5 command can be specified as sleep 5, or as [“sleep”, “5”] in JSON format, whereby the first element is the executable and the second element the argument passed into the executable.
Building a Dockerfile that consists of two Instruction and Argument lines, FROM ubuntu \ CMD [“sleep”, “5”] took around a second. This is because the cache already contained the build artifacts from a prior successful FROM ubuntu instruction.

When we run the cyberfella/ubuntusleeper container, it will sleep for 5 seconds and then exit, just as if we ran the command sleep 5 from our Ubuntu terminal. Remember, if we wanted to run our ubuntusleeper container for 10 seconds, we don’t need to rebuild it, we can optionally pass in the amount of time we want as a parameter, e.g. sudo docker run cyberfella/ubuntusleeper sleep 10 passing in the executable and argument as parameters that will override the default parameters set in the Dockerfile when the image was built.

This still looks a little untidy. the imagename ubuntusleeper implies that the container will sleep, so we don’t want to have to pass in sleep 10 as parameters going forward. We’d prefer to just have to enter sudo docker run cyberfella/ubuntusleeper 10. This is where the ENTRYPOINT Instruction comes in. If we add an Instruction to our Dockerfile ENTRYPOINT [“sleep”] then any argument passed in, will be passed into that executable by default.

After appending ENTRYPOINT [“sleep”] the re-build of the docker image and pushing it to docker hub takes mere seconds.

This works great until the image is run without any parameters, at which point it’ll error with a missing operand error. To overcome this, edit the Dockerfile to contain the ENTRYPOINT instruction first and the CMD instruction after, but remove the sleep command from the CMD instruction since it will default to using it from the ENTRYPOINT instruction.

FROM ubuntu ENTRYPOINT [“sleep”] CMD [“5”] is all you need in your Dockerfile in order to pass in no parameters or just the parameter for the sleep duration in seconds. This only works for Dockerfiles written in JSON format.

Lastly, if you wanted to override the ENTRYPOINT parameter when running the docker image and replace sleep with, say, sleep2.0 then this can be done by specifying the new entrypoint on the docker run command, e.g. sudo docker run –entrypoint sleep2.0 cyberfella/ubuntusleep 5

DOCKER NETWORKING

When you install Docker, it creates three networks, bridge, None and host.

Bridge is the default network a container gets attached to. To specify a different network for the container to get attached to, use the –network= parameter in the docker run command, e.g sudo docker run –network=host cyberfella/ubuntusleep

The bridge network is the default network that containers get attached to on the docker host. They can all talk to one another, but there is no connectivity to the outside world (see PORT MAPPING above in this post for how to map ports on the docker host to ports on the containers)
The host network removes any network isolation from the docker host. This means that the IP address is the same as the docker host. This also means that you can’t host similar containers serving over the same port and that you cannot ship containers to other docker hosts so easily, without causing some disruption.

The none network, simply put, means the container is not connected to any network.

USER DEFINED NETWORKS

It is possible to create more than the default bridge network, so that groups of containers can talk to one another, but you can introduce some isolation between different groups of containers on the same docker host.

User defined networks allow only certain containers to be able to communicate with one another on a given docker host.

The command to set up this extra bridge network on the docker host, would be…

sudo docker network create –driver bridge –subnet 182.18.0.0/16 custom-isolated-network

To list all networks on the docker host, use the docker network ls command.

Remember the docker inspect command can be used to view IP addresses assigned to docker containers? To view the network settings for a specific container, use the command sudo docker inspect blissful_wozniak

Note that a docker container is only connected to the network and has an IP address when it is running. There is no IP address information for non-running containers in an existed state.

On the left, the network information for a non-running container, compared to a running container on the right.

EMBEDDED DNS

Containers can contact one another using their names. For example, If you have a MySQL Server and a Web Server that need to communicate with one another, it is not ideal to configure IP Addresses in a mysql.connect(), since there is no guarantee that a container will get the same IP address from the docker host each time it is started.

The docker host creates a separate namespace for each container and uses virtual ethernet pairs to connect the namespaces together.

containers should be configured to communicate using their names.

DOCKER STORAGE

So, where does the docker host store all it’s files? In /var/lib/docker

The docker host’s filesystem and containers directories.
Recall how the build process is a layered architecture, where each build stores only the changes made within each layer.

If we turn the build process upside down so we’re starting at the bottom layer with a base image from docker hub, then working our way up to a newly built container, we will call these layers the Image Layers. When we run that container with the docker run command, docker host creates a new Container Layer on top where it stores the changes occurring on that running container. This container layer exists while the container is running and is destroyed once the container exits. There is no persistent storage by default.

COPY ON WRITE MECHANISM

The difference between our Image Layers and Container Layers on a running container, is that the Image Layers are Read Only or “Baked in”. Only the Container Layer is Read/Write. If we attempt to modify the code of the script specified in our Entrypoint, then we are not modifying the code in the Image Layer. We are taking a copy of that code from the Image Layer/Entrypoint and modifying that copy.

Remember, when the container exits, it is destroyed and there is no persistent storage by default. Running a container and modifying its entrypoint code will not change the underlying container image contents.

This is known as the Copy on Write mechanism i.e at the point that you write a change, a copy is made and changes written to that copy.

At the point at which the container exits, the image layers and the container layer and is data are destroyed until a container is run again from the built image consisting only of immutable image layers.

If we have a container that we want to write some data that we want to be persistent, i.e. survive after the container is destroyed, then we need to create a volume outside of the container and mount it inside the container.

VOLUME MOUNTING

Create a volume called data_volume using the docker volume create command data_volume command on the docker host that will run your container.

A folder called data_volume will then be created in the /var/lib/docker/volumes/ directory on the docker host.

To mount the volume in your container, run the container with the command docker run -v data_volume :/var/lib/mysql mysql (where mysql is the image name in this example and /var/lib/mysql is the directory on the container that you wish to be the mount point for the data_volume volume on the docker host).

If you run a container and specify a non-existent volume, e.g. docker run -v data_volume2:/var/lib/data ubuntu then rather than error, it will actually create a volume called data_volume2 and mount it into the specified directory, in our case /va/rlib/data on a container running using the ubuntu image.

MOUNTING EXTERNAL STORAGE (BIND MOUNTING)

If the docker host has mounted some external storage, into lets say /data/mysql on the docker host, and we wish to mount that on our running container, then we use the docker run -v /data/mysql:/var/lib/mysql mysql command to accomplish this, specifying the mount point on the docker host rather than the volume, followed by the mount point on the container and the image name just as before.

The -v switch is the old method. The newer standard is to use –mount instead, an example of bind mounting using the newer –mount way, is shown below

docker run --mount type=bind,source=/var/san-mysqldata-vol01,target=/var/lib/mysql mysql

The docker storage drivers control the presentation of one type of filesystem to the docker container that mounts it. Commonly supported file system types are aufs, zfs, btrfs, device mapper, overlay, overlay2

DOCKER COMPOSE

We’ve covered how to run docker containers using the docker run command, and how we can run multiple containers built from different images, but if we want to run an application built from multiple components/containers, a better way to do that is to use “docker compose”, specifying the requirements in a YAML file.

A docker-compose.yml file for your app stack might look like this,

#docker-compose.yml
services:
web:
image: "cyberfella/webapp"
database:
image: "mongodb"
messaging:
image: "redis:alpine"
orchestration:
image: "ansible"

The stack is then brought up with the command docker compose up

Lets look at a more comprehensive example, often used to demonstrate docker. Consider the following voting application architecture.

Imagine if we were going to run this manually, using docker run commands and the images were already built/available on docker hub, we’d start each container like this…

docker run -d --name=vote -p 5000:80 voting-app
docker run -d --name=redis redis
docker run -d --name=worker worker
docker run -d --name=db postgres:9.4 db
docker run -d --name=result -p 5001:80 result-app

But our application does not work because despite running all these required detached containers and port-forwards for our app, they are not linked together. In fact, on our docker host, we may have multiple instances of redis containers running. The console will display errors akin to “waiting for host redis” or “waiting for db” etc.

So how do we associate the containers for our app with one another, amongst other containers potentially running on our docker host?

Going back to our architecture, the python voting app is dependent on the redis service, but the voting app container cannot resolve a container with the name redis, so in our command to start the voting app container, we’d use a –link option to make the voting app container aware of the container named redis with the hostname of redis. This is why we used the –name=redis redis option in our docker run commands above.

# Use --link <container-name>:<host-name> option
  docker run -d --name=vote -p 5000:80 --link redis:redis voting-app 

This actually creates an entry in the hosts file on the voting app container for the redis host. So what other links would we need here?

Well our result-app container needs to know of the db container in order to display the results…

docker run -d --name=vote -p 5000:80 --link redis:redis voting-app # --link <container-name>:<host-name>
docker run -d --name=redis redis
docker run -d --name=worker --link db:db ---link redis:redis worker
docker run -d --name=db postgres:9.4 db
docker run -d --name=result -p 5001:80 --link db:db result-app

Our .NET worker container needs to know of the db and redis containers…

docker run -d --name=vote -p 5000:80 --link redis:redis voting-app # --link <container-name>:<host-name>
docker run -d --name=redis redis
docker run -d --name=worker --link db:db ---link redis:redis worker
docker run -d --name=db postgres:9.4 db
docker run -d --name=result -p 5001:80 --link db:db result-app

Our redis and db containers don’t need to make contact with anything since all communications to these are inbound in our app.

docker run -d --name=vote -p 5000:80 --link redis:redis voting-app # --link <container-name>:<host-name>
docker run -d --name=redis redis
docker run -d --name=worker --link db:db ---link redis:redis worker
docker run -d --name=db postgres:9.4 db
docker run -d --name=result -p 5001:80 --link db:db result-app

With our docker run commands ready, we can now think about putting together our docker-compose.yml file for our app.

#DOCKER COMPOSE EQUIVALENT TO DOCKER RUN COMMANDS
redis:
  image:  redis
db:
  image: postgres:9.4
vote:
  image: voting-app
  ports:
    - 5000:80
  links:
    - redis
result:
  image: result-app
  ports:
    - 5001:80
  links:
    - db
worker:
  image: worker
  links:
    - redis
    - db    

Underneath each section for each of our containers, we’ve specified the image we wish to use. Two of our containers are just images available from docker hub but the other three are using our own in-house application code. Instead of referencing the built image of these in-house containers, the code can be referenced in our docker-compose.yml file instead. The docker host will build our container using the Dockerfile and surrounding code in the directory that contains all the elements required to build our container image using the lines build: ./vote and build: ./result and build: ./worker instead of image: voting-app etc. i.e.

#DOCKER COMPOSE EQUIVALENT
redis:
  image:  redis
db:
  image: postgres:9.4
vote:
  build: ./vote
  ports:
    - 5000:80
  links:
    - redis
result:
  build: ./result
  ports:
    - 5001:80
  links:
    - db
worker:
  build: ./worker
  links:
    - redis
    - db

There are currently 3 different formats of docker-compose files since docker compose has evolved over time. Lets look at the differences between v1.0 and v2.0

docker-compose version 1 connects all containers to the default bridged network, whereas docker-compose version 2 creates a new, separate dedicated bridge network on the docker host, and connects each container to that – presumably for better isolation and better security.

version: 2 must be specified at the top of the docker-compose.yml file and all version 1 elements move under a section called services:) Because of this isolated bridge network, links are created automatically between the containers, so the link: lines are also no longer required.

Version 2 also supports a depends on: feature, to control the start up order of containers.

#DOCKER COMPOSE V2 EQUIVALENT TO DOCKER RUN COMMANDS
version: 2
services:
  redis:  
    image: redis
  db:
    image: postgres:9.4
  vote:
    image: voting-app
    ports:
      -5000:80
    depends_on:
      - redis
  result:
    build: ./result
    ports:
      - 5001:80
  worker:
    build: ./worker

Version 3 looks like Version 2 but has version: 3 at the top (obviously). It also is used by Docker Swarm and Docker Stacks, which we will come on to later.

Going back to Docker Compose Networks, what if we wanted to separate our traffic on the front end App and Results servers from our back end Redis, Worker and DB?

We’d need to create a network for the voting and results containers to connect to, i.e.

and all components to a separate back end network, i.e.

Our version 2 docker-compose.yml file would need to have a map of networks added, i.e.

#DOCKER COMPOSE V2 FRONT-END AND BACK-END NETWORKS
version: 2
services:
  redis:
    image: redis
    
  db:
    image: postgres:9.4
    
  vote:
    image: voting-app
    
  result:
    image: result
    
networks:
  front-end:
  back-end:

then each of our containers needs a networks: section that defines which networks to connect to, e.g.

#DOCKER COMPOSE V2 FRONT-END AND BACK-END NETWORKS
version: 2
services:
  redis:
    image: redis
    networks:
      -back-end

  db:
    image: postgres:9.4
    networks:
      - back-end

  vote:
    image: voting-app
    networks:
      - front-end
      - back-end

  result:
    image: result
    networks:
      - front-end
      - back-end

networks:
  front-end:
  back-end:

DOCKER REGISTRY

PUBLIC REGISTRY

The docker registry is the place where the docker images come from. We run a docker container with a command like docker run nginx and then magically, if the image doesn’t already exist, docker host goes and downloads the latest nginx image from the docker hub repository nginx/nginx:latest i.e. if only nginx is specified, then docker assumes that the docker hub user name is nginx and the repository name is nginx and if no tag is specified, it’ll assume the :latest tag and download the latest version of the image that exists in the nginx/nginx respository. In summary, the first nginx is the docker hub user account and the second nginx is the repository name where the image lives. If there are multiple images for that container in the repository, then they’ll have a tag to denote the version or that they are the latest version of the container image. Since we did not specify the docker registry, it was assumed that we meant the docker.io registry, i.e. docker.io/nginx/nginx:latest

There are other registries too, where docker images can be found, such as Googles Kubernetes registry, gcr.io/kubernetes-e2e-test-images/dnsutils for performing end-to-end tests on the cluster for example. These are publicly accessible images that anybody can download.

PRIVATE REGISTRY

If you have images that should not be made available publicly, you can create a private registry in house. Alternatively AWS, Azure or GCP provide a private registry when you create an account on their cloud platform.

To obtain a container image from a private registry, you must first log into the private registry with your credentials using the docker login command

docker login private-registry-name.io 

You need to log into a registry before you can push or pull an image.

When you use a Cloud account provider like AWS or Azure, a private registry is created when you open an account. If you wanted to deploy an on-premise private docker registry however, you could do so using the command docker run -d -p 5000:5000 –name registry registry:2

To create a private repo, tag an image prior to pushing it to it, or to pull it from a local or network private repo, use the commands shown below.

docker login private-registry.io                    #logs in to private image registry (prompts fro creds)
docker run -d -p 5000:5000 --name registry registry:2   #creates on-premise private docker registry
docker image tag my-image localhost:5000/my-image   #Tag the container image prior to pushing to local private repo
docker push localhost:5000/my-image                 #push image to local private repo
docker pull localhost:5000/my-image                 #pull image from local private repo
docker pull 192.168.56.100:5000/my-image            #pull from private repo on network

DOCKER ENGINE

When you install docker engine on a host, you’re actually installing three components, the Docker Command Line Interface, the REST Application Programming Interface and the Docker Daemon.

An important point is that the Docker CLI need not reside on the docker host in order to communicate with the Docker Daemon. If using a docker CLI on a laptop to talk to a remote Docker Host for example, use the following command…

docker -H=remote-docker-engine:2375 forms the first part of a command executed on a remote docker host
docker -H=remote-docker-engine:2375                 #first part of a command to execute on a remote docker host
docker -H=10.123.2.1:2375 run nginx                 #runs nginx on a remote docker host

CONTAINERIZATION

How does Docker work under the hood.

Docker utilises namespaces to isolate workspace. Process ID, Network, Timeslicing, InterProcess communication and Mount are created in their own namespace thereby providing isolation within containers

NAMESPACE PID

When a linux system boots up, the root process PID 1 is the first process to start, followed by other services with incremental, unique process ID’s. Because of this, each container must think it is a unique system, originating from a root process with a PID of 1.

Each container thinks that it is it’s own system originating from a root PID of 1, just like any other Linux system.

Since the processes are all running on the same host system and Process ID’s must be unique, we cannot have more than one process ID of 1, 2,3 etc. This is where process namespaces come into play. If we were to look, we’d see the same process running on both the container and the docker host, but with different PID’s.

Notice how the there is one container running a /bin/bash process. On the docker host, only one of the /bin/bash processes is running as the root user. When I stop the container, that is the process that dies.

By default, there is no restriction on how many system resources a container can use.

There is a way to restrict how much CPU and Memory a container can use. Docker uses cgroups to do this. To limit the amount of CPU and/or Memory a container uses, use the following commands in docker run.

docker run --cpus=.5 ubuntu                         #ensures ubuntu image doesnt use more than 50% CPU on the docker host
docker run --memory=100m ubuntu                     #limits the amount of ram to 100M

Remember, Docker containers use the underlying kernel so for linux containers, you need to run docker on a linux host. Thats not to say the docker host can’t be a VM running on a hypervisor on a Windows physical host. If you want to play around with Docker, but only have access to a WIndows machine, then you can install Oracle Virtualbox on Windows and create a Linux Ubuntu, Debian or CentOS VM. Then, install docker on the Linux VM and try out all the different commands for yourself. Docker provides the Docker Toolbox which contains all the pieces of software you might want in order to try it out, but instead of having to download the software from many different places, Docker put it all together in one place for your convenience. This is or older versions of Windows that don’t meet the new Docker Desktop Standard.

Download Docker Toolbox from https://docs.docker.com/toolbox/toolbox_install_windows/

The latest option for Windows 10 Enterprise or Professional or Windows Server 2016, is Docker Desktop for Windows, which removes the need for Oracle Virtualbox and uses Windows Hyper-V virtualisation instead.

Docker Desktop uses Hyper-V and the default option is Linux containers, however, Docker has announced that you can now package Windows containers for use on a Windows host, using Docker Desktop.

To use Windows containers, you must switch to Windows Containers mode in Docker Desktop in the menu.

Unlike Linux, in Windows there are two types of Windows Container. The first is Windows Server. The second is Hyper-V isolation, whereby a highly optimised VM guarantees total kernel isolation from the WIndows host.

Windows Server containers and Hyper-V Isolation Containers. The latter guarantees Kernel isolation from the host.

Windows containers can be deployed to Windows Server Core or Nano Server.

CONTAINER ORCHESTRATION

Orchestration solutions such as Docker Swarm and Kubernetes (Google K8’s) allow for the deployment and scaling of many docker hosts and many docker containers across those cluster with advanced networking and sharing of cluster resources and storage between them. Docker Swarm is easy to set up but lacks some of the more advanced features now available on other solutions like Mesos and Kubernetes. Kubernetes is one of the highest ranked projects on GitHub.

DOCKER SWARM

To create a Docker Swarm Cluster, run the following commands on your designated Manager and Workers respectively. The docker swarm init command will generate a token code to use on the workers in that cluster.

docker swarm init --advertise-addr 192.168.1.12     #Initialize Swarm Manager
docker swarm join --token SWMTKN-1-<token-code> <token-mgr-ip>:2377   
docker swarm join-token manager #Add a amanger to a Swarm
docker service create --replicas=3 <image-name>      #Deploy a docker image to a Swarm

Just as with the docker run command, the docker service command supports many of the same commands e.g.

docker service create --replicas=3 --network frontend <image>   #Deploy a docker image to a Swarm, connected to front end network
docker service create --replicas=3 -p 8080:80 mywebserver   #Deploy mywebserver image with port forwarder

KUBERNETES

There is a more in-depth review of Kubernetes here, where the journey into the topic of containerisation continues.

Facebooktwitterredditpinterestlinkedinmail

Savage leaky programs

It’s come to my attention recently that despite a fresh install of Linux Mint, certain programs seem to leak like a basket and hang around after they’re closed too.

I’d noticed my machine freezing intermittently and adding the memory monitor panel item revealed that the system memory was filling up.

The blue mem bar fills up over time when Brave is left open. Disappointing for such an otherwise excellent Web Browser.

xreader and brave seemed to be the main culprits but since rebuilding my desktop machine, I’ve not been using many other programs apart from ledger live to track the value of my crypto currency portfolio while the fed prints money ad infinitum during the coronavirus pandemic. I digress.

Killing processes gets old really quick, so I wrote a quick’n’dirty little shell script to do it for me. Rather than killing individual processes, it savages all processes by the same name.

I shall call it savage.sh and share it with the world, right here. Not on github.

Killing all running processes for ledger and brave using savage.sh
#!/bin/bash
# savage.sh finds all process ID's for the specified program running under your own user account and kills them
# in order to free up system resources.  Some programs have severe memory leaks and consume vast amount of RAM and 
# swap if left running over time.
#
# Usage: savage.sh 
#
# Written by M. D. Bradley during Coronavirus pandemic, March 2020

#Variables
user=`whoami`
memfree=`free | grep Mem | awk {'print $4'}`
#Code
echo "Program to kill e.g. xreader?: "
read program
pidcount=`ps -fu $user | grep $program | awk {'print$program'} | wc -l`
ps -fu $user | grep $program | awk {'print$2'} | while read eachpid; do 
	kill $eachpid >/dev/null 2>&1
done
memfree2=`free | grep Mem | awk {'print $4'}`
freedmem=$(( memfree2 - memfree ))
if [ $pidcount -eq 1 ] 
then
	echo "Found $pidcount process running for $program"
	echo "Killed it.  Freed up $freedmem bytes."
fi
if [ $pidcount -gt 1 ] 
then
	echo "Found $pidcount processes running for $program"
	echo "Savaged them. Freed up $freedmem bytes."
fi
Facebooktwitterredditpinterestlinkedinmail