Linux Infrequently Asked and Obscure Questions
Composed by Ryan Tennant

Table of Contents ( System | Network )
  1. System ( General | Shell | Kernel | Devices | Filesystem )

    1. General
      1. How do I untar a file with absolute paths to a absolute location?
      2. How do I do a recursive grep?
      3. How do I find out the number of files used on local filesystems?
      4. How do I use a FIFO?
      5. How do I list available signals?
      6. How do I show how a process will respond to a given signal?
      7. How do I remove a file that begins with a - ?
      8. ls(1) no longer works, how can I view directory contents?
      9. How can I tell what the various ERROR codes mean?
      10. How can I create a file of arbitrary size?
      11. How can I get seconds from epoch?
      12. How do I get yesterdays date?
      13. How do I get access, modify, creation time of a file?
      14. What is load average?
      15. What is the run queue?
      16. How can I copy directory contents to a remote machine?
      17. How do I archive directories with 155+ character directory names or 100+ character file names?
      18. How do I convert hexadecimal to decimal and vice versa?
      19. How do I convert binary to decimal and vice versa?
      20. What can I do about zombie processes?
      21. How do I list just directories, but not their contents?
      22. How do I grep for multiple patterns?
      23. How do I configure the system wide dynamic linker?

    2. Shell
      1. My setuid shell script keeps running as the real user, why?
      2. Why is cd() a shell built-in rather than an executable?
      3. How do I redirect stderr into stdout?
      4. How do I rename files by extension like MS-DOS?
      5. AWK Tricks
        1. Round up number to next whole number
      6. SED Tricks

    3. Kernel
      1. Where do I put kernel configuration?
      2. How do I add more PTYs?
      3. What is shared memory?
      4. How do I know the limits for shared memory kernel tunables?
      5. What is a semaphore?
      6. How do I know the limits for semaphore kernel tunables?
      7. How do I increase the number of file descriptors available to an application?
      8. What is a register window?
      9. What is the current memory page size?
      10. How do I adjust interprocess communications (IPC) variables?

    4. Devices
      1. How do I make the system aware of new devices?
      2. Where are device drivers located?
      3. How do I configure a device driver?
      4. How do I list currently loaded device drivers?
      5. How do I change the default terminal setting on the system console?

    5. Filesystem
      1. How do I get a list of superblocks on a filesystem?
      2. How do I grow/shrink an ext2/3/4fs filesystem?
      3. What are inodes 0, 1, and 2 used for?
      4. What do I do if I have a corrupt boot block?
      5. How do I disable "access time" updates for file?
      6. What is the difference between file mode 1 and 5?
      7. How can i force an unmount of a filesystem?
      8. How do I mount an ISO image file?


  2. Network ( Physical Layer | Transport Layer )

    1. Physical Layer
      1. How do I find the speed my network card is at?
      2. How do I configure what my network card is capable of?
      3. How do I display what my link partner is capable of?
      4. How can I tell if my card is active on the network?

    2. Transport Layer
      2. I have a large amount of connections in state CLOSE_WAIT, what can be done to reduce this number in the future?
      3. How can I increase my TCP Window size?
      4. What do all the TCP states actually mean?

Questions and Answers
  1. ^^ System

    1. ^^ General

      1. ^^ How do I untar a file with absolute paths to an absolute location?
        1. Caution: GNU tar strips absolute paths as a safety precaution. Consider the potential for danger with absolute path names in a tar archives
        2. Method 1
          1. /bin/tar xPf file.tar

      2. ^^ How do I do a recursive grep?
        1. Method 1
          1. /bin/grep -r PATTERN .
          2. displays filename:match
        2. Method 2
          1. /usr/bin/find . | /usr/bin/xargs /usr/bin/grep PATTERN
          2. displays filename:match
        3. Method 3
          1. /usr/bin/find . -exec /usr/bin/grep PATTERN {} /dev/null \;
          2. displays filename:match

      3. ^^ How do I find out the number of files used on local filesystems?
        1. /bin/df -i

      4. ^^ How do I use a FIFO?
        1. /usr/bin/mkfifo fifo
        2. /usr/bin/gzip < fifo > file.gz &
        3. /bin/cat file > fifo
          1. compresses file into file.gz

      5. ^^ How do I list available signals?
        1. /usr/bin/kill -l
        2. Read /usr/include/bits/signum.h (may also include signum-generic.h or signum-arch.h)
          1. SIGHUP /* Hangup */
          2. SIGINT /* Interrupt */
          3. SIGQUIT /* Quit */
          4. SIGILL /* Illegal Instruction */
          5. SIGTRAP /* Trace Trap */
          6. SIGABRT /* Abort */
          7. SIGBUS /* Bus Error */
          8. SIGFPE /* Floating Point Exception */
          9. SIGKILL /* Kill | Unblockable */
          10. SIGUSR1 /* User-defined 1 */
          11. SIGSEGV /* Segmentation Fault */
          12. SIGUSR2 /* User-defined 2 */
          13. SIGPIPE /* Broken Pipe */
          14. SIGALRM /* Alarm */
          15. SIGTERM /* Termination */
          16. SIGSTKFLT /* Stack Fault */
          17. SIGCHLD /* Child status has changed */
          18. SIGCONT /* Continue */
          19. SIGSTOP /* Stop | Unblockable */
          20. SIGTSTP /* Keyboard stop */
          21. SIGTTIN /* TTY IN */
          22. SIGTTOU /* TTY OUT */
          23. SIGURG /* Urgent socket condition */
          24. SIGXCPU /* CPU limit exceeded */
          25. SIGXFSZ /* File size limit exceeded */
          26. SIGVTALRM /* Virtual alarm */
          27. SIGPROF /* Profiling alarm */
          28. SIGIO /* IO now possible */
          29. SIGPWR /* Power failure restart */
          30. SIGSYS /* Bad system call */

      6. ^^ How do I show how a process will respond to a given signal?
        1. Similar to the psig command in Solaris
        2. grep ^Sig /proc/$$/status
          1. SigPnd shows signals that are pending
          2. SigBlk shows signals that are blocked
          3. SigIgn shows signals that are ignored
          4. SigCgt shows signals that are caught
        3. Convert to binary and then compare to signals list
          1. Signals list can be found at I.A5
        4. Example: What signals are being caught by the current running process?
          1. grep ^Sig /proc/$$/status 
            SigQ:   1/63725
SigPnd: 0000000000000000
SigBlk: 0000000000000002
SigIgn: 0000000000384000
SigCgt: 0000000008013003
          2. printf "ibase=16; obase=2; %X\n" "0x0000000008013003" | /usr/bin/bc 
            1000000000010011000000000011
          3. Read right to left. 1 is true, 0 is false.
          4. 1=(1)SigHup, 1=(2)Sigint, 0=(3)SigQUIT, 0=(4)SigILL, 0=(5)SigTrap... 1=(13)SigPIPE, etc.

      7. ^^ How do I remove a file that begins with a - ?
        1. This problem, contrary to popular belief, has nothing to do with the shell. It has to do with how rm(1) parses options.
        2. Method 1
          1. /usr/bin/rm ./-file
        3. Method 2
          1. /usr/bin/rm -- -file
          2. many programs use getopt(), thus they'll interpret - as an argument, to tell getopt() there are no more arguments to parse, use --

      8. ^^ ls(1) no longer works, how can i view directory contents?
        1. echo *
          1. This method uses the shell built-in echo() in conjunction with the * matching properties to generate listing of current directory.

      9. ^^ How can I tell what the various ERROR codes mean?
        1. /usr/bin/man -s 3 errno

      10. ^^ How can I create a file of arbitrary size?
        1. /usr/bin/dd < /dev/zero > file bs=1M count=10
          1. Creates a 10 Megabyte file
        2. /usr/bin/dd < /dev/zero > file bs=1M seek=10 count=1
          1. Creates a 10 Megabyte file
          2. This method does not require many reads and writes since the file is sparse.

      11. ^^ How can I get seconds from epoch?
        1. date +%s
        2. /usr/bin/perl -e 'printf "%d\n", time;'
        3. /usr/bin/python3 -c 'import time; print(int(time.time()))'

      12. ^^ How do I get yesterdays date?
        1. date -d yesterday

      13. ^^ How do I get access, modify, creation time of a file?
        1. Access time (atime)
          1. /usr/bin/ls -ul filename
            1. From the stat system call manual page, the field st_atime is changed by file accesses, for example, by execve(2), mknod(2), pipe(2), utime(2) and read(2) (of more than zero bytes). Other routines, like mmap(2), may or may not update st_atime.
        2. Modify time (mtime)
          1. /usr/bin/ls -l filename
            1. From the stat system call manual page, the field st_mtime is changed by file modifications, for example, by mknod(2), truncate(2), utime(2) and write(2) (of more than zero bytes). Moreover, st_mtime of a directory is changed by the creation or deletion of files in that directory. The st_mtime field is not changed for changes in owner, group, hard link count, or mode.
        3. Creation time
          1. There is no way to determine creation time in the ext2/3/4 filesystems
        4. Change time (ctime)
          1. /usr/bin/ls -cl filename
            1. From the stat system call manual page, the field st_ctime is changed by writing or by setting inode information (i.e., owner, group, link count, mode, etc.).
        5. All in one
          1. stat filename

      14. ^^ What is load average?
        1. Load average is the average number of processes currently running and waiting to run.
        2. Method 1
          1. /usr/bin/w -u
          2. displays load average over last 1, 5 and 15 minutes
        3. Method 2
          1. /usr/bin/uptime
          2. displays load average over last 1, 5 and 15 minutes

      15. ^^ What is the run queue?
        1. The run queue consists of processes ready to run, i.e not otherwise blocked or waiting for i/o, that are contending for cpu resources to become available.
        2. /usr/bin/vmstat 1 2
          1. Current run queue is indicated by the "r" heading
          2. First line of output is average since system boot

      16. ^^ How can I copy directory contents to a remote machine (without nfs)?
        1. Method 1
          1. /usr/bin/tar -cf - sourcepath | /usr/bin/ssh remote " cd /targetpath ; /usr/bin/tar -xBf - "
        2. Method 2
          1. /usr/bin/find sourcepath | /usr/bin/cpio -o | /usr/bin/ssh remote "cd /targetpath ; /usr/bin/cpio -id"

      17. ^^ How do I archive directories with 155+ character directory names or 100+ character file names?
        1. This is no longer an issue with modern versions of GNU tar (Version > 1.13.25)
        2. Historically, POSIX.1-1988 or 'ustar' versions only supported 256 character files. POSIX.1-2011 or 'posix' is now the default format for GNU tar (Version > 1.13.25) and can handle unlimited UID, file sizes, and file names.

      18. ^^ How do I convert hexadecimal to decimal and vice versa?
        1. Method 1 - bc
          1. echo "ibase=16; FF" | /usr/bin/bc
            1. converts FF into 255
            2. case sensitive
          2. echo "obase=16; 255" | /usr/bin/bc
            1. converts 255 to FF
        2. Method 2 - dc
          1. echo "16 10 o i FF p" | dc
            1. converts FF to 255
            2. case sensitive
          2. echo "10 16 o i 255 p" | dc
            1. converts 255 to FF
        3. Method 3 - printf
          1. printf '%d\n' 0xff
            1. converts ff to 255
          2. printf '%x\n' 255
            1. converts 255 to ff

      19. ^^ How do I convert binary to decimal and vice versa?
        1. Method 1 - bc
          1. echo "ibase=2; 11111111" | /usr/bin/bc
            1. converts 11111111 into 255
          2. echo "obase=2; 255" | /usr/bin/bc
            1. converts 255 to 11111111
        2. Method 2 - dc
          1. echo "2 10 o i 11111111 p" | dc
            1. converts 11111111 to 255
          2. echo "10 2 o i 255 p" | dc
            1. converts 255 to 11111111

      20. ^^ What can I do about zombie processes?
        1. Get process id (pid) and parent process id (ppid) of Zombie process
          1. ps -e -o stat,ppid,pid,comm | awk '$1 == "Z" {print}'
        2. Example : PPID 1000, PID 1001
          1. gdb -p 1000
            1. (gdb) call waitpid(1001,0,0)
            2. (gdb) quit

      21. ^^ How do I list just directories, but not their contents?
        1. Method 1
          1. /bin/ls -d */
          2. commonly aliased to 'lsd'
        2. Method 2 (requires read access to listed directories)
          1. /usr/bin/find /path/to/dir/* -type d -prune
          2. displays full path results

      22. ^^ How do I grep for multiple patterns?
        1. pattern1 OR pattern2
          1. /bin/egrep -e '(pattern1|pattern2)'
        2. pattern1 AND pattern2
          1. Method 1 (Recommended)
            1. /bin/egrep -e '(pattern1.*pattern2|pattern2.*pattern1)'
          2. Method 2 (Quick and Dirty)
            1. /bin/grep pattern1 | grep pattern 2

      23. ^^ How do I configure the system wide dynamic linker?
        1. Warning: This changes the library search path, so make sure to include the default paths when making changes.
        2. Default configuration file: /etc/ld.so.conf (and typically /etc/ld.so.conf.d directory)
        3. /sbin/ldconfig -v


    2. ^^ Shell

      1. ^^ My setuid shell script keeps running as the real user, why?
        1. Setuid shell scripts are dangerous
        2. Linux ignores the set-user-ID and set-group-ID bits on scripts. (see man -s2 execve)

      2. ^^ Why is cd() a shell built-in rather than an executable?
        1. Quick Answer
          1. a child process cannot modify the environment of the parent
        2. Long Answer
          1. a shell fork()s and then exec()s the requested executable. in doing so, the newly created process begins life with the environment of the parent process. the new child process then manipulates the environment in the manner requested, in this case a modification of the directory stack, and returns to the parent. however, since this change occurred in the child address space, the parent's environment was never changed, and therefore the requested operation did not take place.

      3. ^^ How do I redirect stderr?
        1. Bourne Shell
          1. to stdout
            1. command 2>&1
          2. to file
            1. command 2> file
          3. to null
            1. command 2> /dev/null
        2. C Shell
          1. to stdout
            1. command >& /dev/tty
          2. to file (without affecting stdout)
            1. ( command > /dev/tty ) >& file
          3. to null (without affecting stdout)
            1. a. ( command > /dev/tty ) >& /dev/null

      4. ^^ How do I rename files by extension like MS-DOS?
        1. DOS Example: move *.doc *.txt
        2. Korn Shell / Bourne Again Shell
          1. for x in *.doc; do mv "$x" "${x%.doc}.txt"; done

      5. ^^ AWK Tricks

        1. ^^ Round up number to next whole number
          1. Example (4.1 to 5, -4.1 to -4)
          2. awk '{printf "%d\n", ($0 % 1) ? (int($0)+(($1>0)?1:0)) : $0}'

      6. ^^ SED Tricks

    3. ^^ Kernel

      1. ^^ Where do I put kernel configuration?
        1. /etc/sysctl.conf

      2. ^^ 2. How do I add more PTYs?
        1. Kernel 2.6.4 and later
          1. PTY currently in use
            1. cat /proc/sys/kernel/pty/nr
          2. PTY maximum
            1. cat /proc/sys/kernel/pty/max
            2. Example : 8192
              1. echo 8192 > /proc/sys/kernel/pty/max
        2. Kernel 2.6.3 and earlier
          1. CONFIG_UNIX98_PTYS at compile time
          2. 256 default, 2048 maximum

      3. ^^ What is shared memory?
        1. Just as it sounds. Shared memory is an Interprocess Communication (IPC) mechanism used by multiple processes to access common memory segments.

      4. ^^ How do I know the limits for shared memory kernel tunables?
        1. Read /usr/include/sys/shm.h and /usr/include/bits/shm.h

      5. ^^ What is a semaphore?
        1. A non-negative integer that is incremented or decremented relative to available resources.

      6. ^^ How do I know the limits for semaphore kernel tunables?
        1. Read /usr/include/sys/sem.h and /usr/include/bits/sem.h

      7. ^^ How do I increase the number of file descriptors available to an application?
        1. File descriptors are used for more than just open files, they also provide the framework for socket i/o.
        2. The kernel dynamically allocates resources for open files. There is no maximum number of file descriptors per system.
        3. User limits
          1. /etc/security/limits.conf
          2. * soft nofile 1024
          3. * hard nofile 1024
        4. System limits
          1. Active System
            1. Method 1
              1. cat /proc/sys/fs/file-max
              2. echo "65536" > /proc/sys/fs/file-max
            2. Method 2
              1. sysctl fs.file-max
              2. sysctl -w fs.file-max=65536
          2. Persistent
            1. Modify /etc/sysctl.conf
            2. fs.file-max = "65536"

      8. ^^ What is a register window?
        1. A register window is used by the operating system to store the current local and in registers upon a system interupt, exception, or trap instruction.
        2. register windows are important to preserve the state of the stack between function calls.

      9. ^^ What is the current memory page size?
        1. getconf PAGESIZE

      10. ^^ How do I adjust interprocess communications (IPC) variables?
        1. Message Queue
          1. Modify /etc/sysctl.conf
            1. msgmax
              1. Maximum message size
              2. sysctl kernel.msgmax
            2. msgmnb
              1. Maximum bytes in queue
              2. sysctl kernel.msgmnb
            3. msgmni
              1. Number of message queue identifiers
              2. sysctl kernel.msgmni
            4. msgssz
              1. [Not Configurable] Message segment size
            5. msgtql
              1. [Not Implemented] Number of system message headers
            6. msgseg
              1. [Not Configurable] Number of message segments
        2. Semaphores
          1. Modify /etc/sysctl.conf
            1. semmsl semmns semopm semmni all are configured through kernel.sem separate by whitespace
            2. semmni
              1. Number of identifiers
            3. semmns
              1. Number of semaphores in the system
            4. semmnu
              1. [Not Implemented] Number of processes in the undo facility
            5. semmsl
              1. Maximum semaphores per id
            6. semopm
              1. [Not Implemented] Maximum operations per call
            7. semume
              1. [Not Implemented] Number of undo structures per process
        3. Shared Memory
          1. Modify /etc/sysctl.conf
            1. shmmax
              1. Maximum size of a shared memory segment.
              2. sysctl kernel.shmmax
            2. shmmin
              1. Minimum size of a shared memory segment.
              2. sysctl kernel.shmmni
            3. shmall
              1. Maximum number of total shared memory segments that can be created
              2. sysctl kernel.shmall
            4. shmseg
              1. [Not Implemented] Maximum number of shared memory segments per process

    4. ^^ Devices

      1. ^^ How do I make the system aware of new devices?
        1. Disks
          1. SCSI
            1. echo "- - -" > /sys/class/scsi_host/host#/scan
        2. Memory
          1. grep line /sys/devices/system/memory/*/state
          2. If memory appears as offline, set it to online
            1. echo online > /sys/devices/system/memory/memory#/state

      2. ^^ Where are device drivers located?
        1. /lib/modules/<kernel>

      3. ^^ How do I configure a device driver?
        1. /etc/modprobe.d/<driver>

      4. ^^ How do I list currently loaded device drivers?
        1. /sbin/lsmod

      5. ^^ How do I change the default terminal setting on the system console?
        1. RedHat 5
          1. /etc/grub.conf
            1. Add "console=ttyS0,115200" to kernel line, multiple console lines will attempt to output to multiple devices simultaneously
          2. /etc/inittab
            1. S1:2345:respawn:/sbin/agetty -L 115200 ttyS0 vt100
          3. /etc/securetty
            1. Add entry for ttyS0

    5. ^^ Filesystem

      1. ^^ How do I get a list of superblocks on a filesystem?
        1. dumpe2fs /dev/xvda1 |grep -i superblock

      2. ^^ How do I grow/shrink an ext2/3/4fs filesystem?
        1. First, grow the underlying partition, LVM or MD
        2. resize2fs <device>

      3. ^^ What are inodes 0, 1, and 2 used for?
        1. Inode 0 is unusable. It is used to mark unused inodes.
        2. Inode 1 is for bad block information.
        3. Inode 2 is "/" or "root" of the filesystem.
        4. Inode 5 is the boot loader inode.
        5. Inode 6 is the undelete directory inode.
        6. On ext3, Inode 7 is the reserved groups descriptor inode.
        7. On ext3, Inode 8 is the journal inode.

      4. ^^ What do I do if I have a corrupt boot block?
        1. Boot from CD / Net
        2. linux rescue
        3. chroot /mnt/sysimage
        4. grub-install <bootdisk>

      5. ^^ How do I disable "access time" updates for a filesystem?
        1. This is useful for web servers and news servers to prevent unnecessary file I/O.
        2. Add "noatime" to mount options in /etc/fstab

      6. ^^ What is the difference between file mode 1 and 5?
        1. mode 1 allows exec() of the binary
        2. mode 5 allows exec() of the binary and processes to mmap() pages from within userspace
        3. this is why shared libraries generally need to be PROT_READ and PROT_EXEC at page level and -r-x at file level

      7. ^^ How can I force an unmount of a filesystem?
        1. Force
          1. umount -f <mountpoint>
        2. Lazy
          1. umount -l <mountpoint>

      8. ^^ How do I mount an ISO image file?
        1. mount -o loop -t iso9660 <image.iso> <mountpoint>

  2. ^^ Network

    1. ^^ Physical Layer

      1. ^^ How do I find the speed my network card is at?
        1. ethtool <interface>

      2. ^^ How do I configure what my network card is capable of?
        1. Speed
          1. ethtool -s <interface> speed <speed>
        2. Advertise auto negotiate capability
          1. ethtool -s <interface> autoneg on

      3. ^^ How do I display what my link partner is capable of?
        1. If supported, mii-tool <interface>

      4. ^^ How can I tell if my card is active on the network?
        1. tcpdump -i <interface>

    2. ^^ Transport Layer

      1. ^^ I have a large amount of connections in state CLOSE_WAIT, what can be done to reduce this number in the future?
        1. Decrease Close Wait / Time Wait Interval
          1. Current
            1. cat /proc/sys/net/ipv4/tcp_fin_timeout
          2. Reduce to 5 seconds
            1. echo "5" > /proc/sys/net/ipv4/tcp_fin_timeout

      2. ^^ How can I increase my TCP Window size?
        1. Increasing the transmit window value in excess of the 16bit window defined in RFC793, as SEG.WND, causes the Window Scaling option as defined in RFC1323.
        2. Window Scaling is enabled by default on Linux (post-k2.2)
        3. cat /proc/sys/net/ipv4/tcp_window_scaling

      3. ^^ What do all the TCP states actually mean?
        1. CLOSED (0)
          1. Socket is closed
        2. LISTEN (1)
          1. Socket is passive, awaiting a connection request
        3. SYN_SENT (2)
          1. Socket is active, has sent a SYN
          2. Session not yet active
        4. SYN_RECEIVED (3)
          1. Socket is active, has sent and received SYN
          2. Session not yet active
        5. ESTABLISHED (4)
          1. Socket is active
          2. Session is active, has completed handshake
        6. CLOSE_WAIT (5)
          1. Socket is closed, received FIN, waiting for close
          2. Session is terminating
        7. FIN_WAIT (6)
          1. Socket is closed, sent FIN, waiting for FIN ACK
          2. Session is terminating
        8. CLOSING (7)
          1. Socket is closed, exchanged FIN, waiting for FIN ACK
          2. Session is terminating
        9. LAST_ACK (8)
          1. Socket is closed, received FIN, waiting for FIN ACK
          2. Session is terminating
        10. FIN_WAIT_2 (9)
          1. Socket is closed, received FIN ACK
          2. Session is complete
        11. TIME_WAIT (10)
          1. Socket is closed, waits for ( 2 * max segment life )
          2. Session is complete
Disclaimer: This document is not supported in any way and no warranty is provided. Errors in certain commands executed as root can cause system failure and data corruption. Please exercise caution and remember to backup your data.