Linux Resources

Tools

• ag - A code-searching tool similar to ack, but faster.

  • ggreer/the_silver_searcher

• awk - pattern scanning and processing language

  • Usage:
    • awk -F "," '{print $2}' t.txt 以逗号做分割从文件中提取数据
    • awk '{sum += $2} END {print sum}' t.txt 每行第二列数字之和
  • AWK 简明教程 - 酷壳

• cheat - linux 命令小抄，比 help 和 man 命令更容易理解，通过实例告诉你怎么使用。

  • chrisallenlane/cheat

• cloc - Count Lines of Code

  • AlDanial/cloc

• curl - transfer a URL

  • Usage:
    • curl example.com
    • curl -A 'Baiduspider' example.com 指定 User Agent
    • curl -X POST --data "data=xxx" example.com 使用 POST 方法
  • curl 网站开发指南

• dmesg - display the system message buffer

  • The dmesg Command

• free - Display amount of free and used memory in the system

  • Usage: free -h or free -m 查看内存使用情况
  • Linux 的 free 命令详解

• grep - print lines matching a pattern

  • Usage:
    • grep -B 3 -A 2 foo README.txt
    • grep -C 3 foo README.txt, grep a file, but show several surrounding lines
    • tail -f web.log | grep -E "\" 50[0-9] ", grep 5xx from the tail of file
    • tac web.log | grep -E "\" 50[0-9]", grep from file in reverse
  • What's the difference between “grep -e” and “grep -E”
  • ‘grep’ regular expression syntax

• iostat - input/output statistics

  • Usage: iostat -k 1
  • Linux iostat 监测 IO 状态
  • 实例讲解 iostat

• lsof - list open files

  • Usage: lsof -i TCP:80, to find which program is using the port 80.

• make - GNU make utility to maintain groups of programs

  • What does "./configure; make; make install" do?

• netcat - TCP/IP swiss army knife

  • Usage: Echo Server: nc -l 1567, Client: nc localhost 1567
  • Linux Netcat 命令——网络工具中的瑞士军刀

• netstat - prints information about the Linux networking subsystem

  • Usage:
    • netstat -a | grep 2901 查看某端口的网络连接情况
    • netstat -a | grep TIME_WAIT | wc -l 计算处于 TIME_WAIT 状态的连接数
  • Linux netstat 命令详解

• ps - process status

  • ps -ef: UID PID PPID C STIME TTY TIME CMD

• sar - system activity information.（最全面的系统分析工具）

  • Usage: sar -n DEV 1 2, sar -n TCP,ETCP 1 2 检查网络接口的吞吐量
  • Useful Sar (Sysstat) Examples for UNIX / Linux Performance Monitoring

• stat - display file or file system status

  • stat(1) - Linux man page

• strace - trace system calls and signals

  • Usage:
    • strace -p pid 绑定 pid 进程追踪系统调用
    • strace -f -r -p pid -f 表示同时追踪子进程，-r 表示同时打印系统调用时间
    • strace -o filename -p pid 把详细日志写入文件
  • 使用truss、strace 诊断软件的"疑难杂症"

• tcpdump - dump traffic on a network

  • Usage:
    • tcpdump -i lo0 port 8000, listen 8000 port on the lo0 interface.
    • sudo tcpdump -A src port 8080, print each pocket on 8080 port from src.
    • sudo tcpdump -A dst port 8080, print each pocket on 8080 port to dst.        * tcpdump -vvvs 1024 -l -A host fir3net.com 打印 HTTP Header    - A tcpdump Primer with Examples
  • Linux tcpdump 命令详解

• tmux - terminal multiplexer

  • Tmux：Linux 从业者必备利器

• vmstat - Report virtual memory statistics

  • Usage: vmstat 1
  • Linux 监控工具 vmstat 命令详解
  • Linux 内存 buffer 和 cache 的区别

Best Practices

• How to find the largest files in linux?

  • Find All Large Files On A Linux System
    • Usage: find / -type f -size +20M -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2
  • http://www.cyberciti.biz/faq/how-do-i-sort-du-h-output-by-size-under-linux/

• file encoding

  • file - determine file type
    • Usage: file a.csv
  • iconv - Convert encoding of given files from one encoding to another
    • Usage: iconv -f UTF8 -t GB18030 a.csv > b.csv

• listen host and port

  • lsof -i TCP:80, to find which program is using the port 80.
  • tcpdump -i lo0 port 8000, listen 8000 port on the lo0 interface.

• split a file using a numeric suffix

  • awk '{filename = "prefix." int((NR-1)/10000) ".txt"; print >> filename}' inputfile

• Linux 的性能诊断

  • Linux 性能分析的第一分钟
  • Linux Performance Analysis in 60,000 Milliseconds

Shell Script

• How to start multiple processes in Bash
• How can I make variables “exported” in a bash script stick around?

References:

• Master the command line, in one page
• Linux 工具快速教程