Understanding basics Unix/Linux..
Well, it is important to know unix/linux operating system and its commands to work effectively in cloud devops. Understanding unix networking concept and unix basic concept is key to grasp cloud compute resource much better.
I am not sysadmin or unix admin but being devops engineer, you need to know at least this basics to work effectively unix based cloud compute product. I really feel proud when I am sharing what I have been learning in IT devops profile. This is my learning experience on very basics unix concepts.
Happy Learning :)
1- To identity list of users who are currently logged into unix machine..
“w” command to display list of users with more information who are currently logged into unix machine and it show when those user logged in and how much time since they are logged in into unix machine.
[ec2-user@ip-172-31-4-5 ~]$ w08:51:36 up 44 min, 3 users, load average: 0.00, 0.00, 0.00USER TTY FROM LOGIN@ IDLE JCPU PCPU WHATec2-user pts/0 152.57.34.111 08:10 32:35 0.02s 0.02s -bashec2-user pts/1 152.57.34.111 08:26 14:56 0.02s 0.02s -bashec2-user pts/2 152.57.34.111 08:51 0.00s 0.01s 0.00s w[ec2-user@ip-172-31-4-5 ~]$
“who” command to display list of users logged into linux machine with some information, I would go with “w” command to get more information
[ec2-user@ip-172-31-4-5 ~]$ whoec2-user pts/0 2021-11-12 08:10 (152.57.34.111)ec2-user pts/1 2021-11-12 08:26 (152.57.34.111)ec2-user pts/2 2021-11-12 08:51 (152.57.34.111)[ec2-user@ip-172-31-4-5 ~]$
“whoami”- To check with which user id your current terminal running
[ec2-user@ip-172-31-4-5 ~]$ whoamiec2-user[ec2-user@ip-172-31-4-5 ~]$
“id” — To display user and group information for that logged in user
[ec2-user@ip-172-31-4-5 ~]$ iduid=1000(ec2-user) gid=1000(ec2-user) groups=1000(ec2-user),4(adm),10(wheel),190(systemd-journal)[ec2-user@ip-172-31-4-5 ~]$
2- To check memory and cpu usage in unix..
Well, there are few commands available to check memory and cpu usage much better in unix.
“free -m” — The free command is the most simple and easy to use command to check memory usage on linux. Here is a quick example
[ec2-user@ip-172-31-4-5 ~]$ free -mtotal used free shared buff/cache availableMem: 982 88 448 0 445 761Swap: 0 0 0[ec2-user@ip-172-31-4-5 ~]$
The m option displays all data in MBs. The total os is 761 MB is the total amount of RAM installed on the system, that is 1GB. The used column shows the amount of RAM that has been used by linux, in this case around 982MB. The output is pretty self explanatory. The catch over here is the cached and buffers column. The second line tells that 88 MB is free. This is the free memory in first line added with the buffers and cached amount of memory.
Linux has the habit of caching lots of things for faster performance, so that memory can be freed and used if needed.
The last line is the swap memory, which in this case is lying entirely free.
“/proc/meminfo” — The next way to check memory usage is to read the /proc/meminfo file. Know that the /proc file system does not contain real files. They are rather virtual files that contain dynamic information about the kernel and the system
[ec2-user@ip-172-31-4-5 ~]$ cat /proc/meminfoMemTotal: 1006128 kBMemFree: 459200 kBMemAvailable: 780056 kBBuffers: 2088 kBCached: 427712 kBSwapCached: 0 kBActive: 208036 kB
---
“vmstat” — The vmstat command with the s option, lays out the memory usage statistics much like the proc command. Here is an example
[ec2-user@ip-172-31-4-5 ~]$ vmstat -s1006128 K total memory90472 K used memory208268 K active memory274056 K inactive memory459200 K free memory2088 K buffer memory454368 K swap cache
“top command” The top command is generally used to check memory and cpu usage per process. However it also reports total memory usage and can be used to monitor the total RAM usage. The header on output has the required information. Here is a sample output
top - 09:02:44 up 55 min, 3 users, load average: 0.00, 0.00, 0.00Tasks: 92 total, 1 running, 55 sleeping, 0 stopped, 0 zombie%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 stKiB Mem : 1006128 total, 458392 free, 91160 used, 456576 buff/cacheKiB Swap: 0 total, 0 free, 0 used. 779316 avail MemPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND1 root 20 0 125544 5400 3920 S 0.0 0.5 0:02.07 systemd2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H
To check cpu usage in unix apart from top command, use below two commands.
“mpstat” — Report per-processor or per-processor-set statistics
[ec2-user@ip-172-31-4-5 ~]$ mpstatLinux 4.14.252-195.483.amzn2.x86_64 (ip-172-31-4-5.us-east-2.compute.internal) 11/12/2021 _x86_64_ (1 CPU)09:05:47 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle09:05:47 AM all 0.34 0.00 0.10 0.13 0.00 0.01 0.29 0.00 0.00 99.13
“sar” — System activity reporter
[ec2-user@ip-172-31-4-5 ~]$ sarLinux 4.14.252-195.483.amzn2.x86_64 (localhost) 11/12/2021 _x86_64_ (1 CPU)08:07:11 AM LINUX RESTART08:10:01 AM CPU %user %nice %system %iowait %steal %idle08:20:01 AM all 0.02 0.00 0.02 0.00 0.02 99.93
3- More on bash scripting side.
This is simple question but important to know how to run bash script in debug mode and options available to run bash script in debug mode.
I have written simple bash script to calculate sum of two command line arguments(integer number) and print on terminal.
#!/bin/bash# adding -xv in first command to enable debugging mode while running bash script and -> #!/bin/bash -xv# Display commands and their arguments as they are executed.set -xecho "Hello Basav, lets understand how to debug shell script in debug mode"echo "To display sum of two number by taking two argument via command line"sum=$(($1 + $2))echo "sum of two number is $sum"# To unset to stop displaying commands on terminalset +x# To print bash script name itselfecho $0
“bash -x test.sh” — This command help to run bash script in debug mode , well you can use bash -v test.sh as well.
bash -x test.sh 30 50+ set -x+ echo 'Hello Basav, lets understand how to debug shell script in debug mode'Hello Basav, lets understand how to debug shell script in debug mode+ echo 'To display sum of two number by taking two argument via command line'To display sum of two number by taking two argument via command line+ sum=80+ echo 'sum of two number is 80'sum of two number is 80+ set +xtest.sh
“#!/bin/bash -xv” — This approach can be used if you want to make your script debugging permanent in your script.
To read command line argument in your script, use below
“echo $0” -To print script name in your script output.
“echo $1”- To print first argument passed in command line.
“echo $2” — To print second argument passed in command line.
“echo $?” — To check exit code of last executed command in linux.
This below example help you to understand much better.
[ec2-user@ip-172-31-4-5 ~]$ ./test.sh 30 70#!/bin/bash -xv# adding -xv in first command to enable debugging mode while running bash script# Display commands and their arguments as they are executed.set -x+ set -xecho "Hello Basav, lets understand how to debug shell script in debug mode"+ echo 'Hello Basav, lets understand how to debug shell script in debug mode'Hello Basav, lets understand how to debug shell script in debug modeecho "To display sum of two number by taking two argument via command line"+ echo 'To display sum of two number by taking two argument via command line'To display sum of two number by taking two argument via command linesum=$(($1 + $2))+ sum=100echo "sum of two number is $sum"+ echo 'sum of two number is 100'sum of two number is 100# To unset to stop displaying commands on terminalset +x+ set +x# To print bash script name itselfecho $0./test.sh[ec2-user@ip-172-31-4-5 ~]$
4- Understanding Unix Process Lifecycle and Process Table.
In the unix, everything is running as process and understanding process lifecycle is crucial being devops engineer.
Well, knowing state of each process running in the unix and understanding each and every state make me confident on basics of unix and before understanding lifecycle of unix process, its important to understand what is process in the unix.
Process :
Processes in Linux follow patterns similar to that of humans. Just like people, processes are born and do carry out regular tasks; taking rest, sleeping in between and finally, being killed or dying. Processes, being the most fundamental aspects in Linux, are necessary to carry out tasks in the system. You create a process by running binary program executable. The binary program executable comes from a piece of code, transitions to a process, while acquiring states during its lifetime and demise.
Process Lifecycle :
When you execute program in unix operating system, it will create new process with state — New State. It then goes to Ready State, at this moment the process is waiting to be assigned a processor by the OS. Once the Processor is assigned, the process is being executed and turns in Running State. If process find required resource and it directly execute goes into Terminate state, however, if it has some external dependency like IO operations or some other high priority jobs to be executed first, then it will goes to Wait State
Now, lets understand zombie process in the unix.
Zombie processes in Linux are sometimes also referred to as defunct or dead processes. They’re processes that have completed their execution, but their entries are not removed from the process table.
Process Table:
Linux maintains a process table of all the processes running, along with their states. You can check process table information in some of the specific Unix/Linux operating system using ls /proc
Let’s briefly overview the various process states:
- Running (R): These processes are currently running or runnable.
- Waiting (S/D): These are the processes that are waiting for an event or a resource. The wait can either be an interruptible sleep (S) or an uninterruptible sleep (D).
- Stopped (T): We can stop a Linux process by sending an appropriate signal.
- Zombie (Z): When a process finishes its task, it releases the system resources it was using and cleans up its memory. However, its entry from the process table is not removed, and its status is set as EXIT_ZOMBIE.
To check unix process status using ps aux or ps ux command and you can look at column name STAT
[ec2-user@ip-172-31-4-5 ~]$ ps auxUSER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMANDroot 1 0.0 0.5 125544 5460 ? Ss 08:07 0:02 /usr/lib/systemd/systemd --switched-roroot 2 0.0 0.0 0 0 ? S 08:07 0:00 [kthreadd]root 4 0.0 0.0 0 0 ? I< 08:07 0:00 [kworker/0:0H]
Well, when do you except process state will goes into zombie state, suppose your child process complete it execution and it send SIGCHLD signal to parent process and parent process involve system wait() call to check exit code of child process from process table and it clean up accordingly. However, if parent process fail to create wait() system call successfully while creating process in the unix and so it ignore SIGCHLD signal from child process, it never clean up child process id from process table therefore, that process id exit code remain exist in process table.
How to handle zombie process in the unix
We can manually send the SIGCHLD signal to the parent of a zombie process. Consequently, it will intimate the parent to trigger the wait() system call, which will clean up the defunct child process from the process table.
$ ps -A -ostat,pid,ppid | grep -e '[zZ]'
Z 108 103$ kill -s SIGCHLD 103
However, it isn’t really guaranteed that sending the SIGCHLD signal to the parent will kill a zombie process. It works only in cases where parent processes can handle the SIGCHLD signals.
If the method discussed in the previous section is unable to clear the defunct process, we should consider killing its parent process:
$ kill -9 103However, killing a parent can affect all its child processes. Hence, we should exercise extra caution and must identify the impact before killing a parent process.
Thank you so much for reading this, I am new in writing blog on internet and appreciate for your feedback to create such meaning full blog on Technology.
