Download as PDF, PPTX
![Pipes
• Introduced to UNIX by Douglas
McIlroy in 1973
• Pipes are the glue in UNIX
component based programming (aka
shell scripting)
• Powerful idiom for stream processing
• The character | is used by all known
shells
Examples
lsof | grep ^Keynote | wc -l
ifconfig | grep -e ^[a-z] | cut -f1 -d:
"Pipeline" by TyIzaeL - Licensed under Public domain via Wikimedia Commons
http://commons.wikimedia.org/wiki/File:Pipeline.svg#mediaviewer/File:Pipeline.svg](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-8-2048.jpg)
![Basic syntax
• White spaces: space and
tab
• Comments: # and to end-
of-line
• Statements: either end-
of-line of ; (semicolon)
• Variables and functions:
Letters, digits and
underscore
#!/bin/bash
# Monte Carlo calculation of pi
NSTEPS=500
NHITS=0
i=0
while [ $i -lt $NSTEPS ]; do
x=$(echo $RANDOM/32767 | bc -l)
y=$(echo $RANDOM/32767 | bc -l)
d=$(echo "sqrt($x*$x+$y*$y) < 1.0" | bc -l)
if [ $d -eq 1 ]; then
NHITS=$(($NHITS + 1))
fi
i=$(($i + 1))
done
PI=$(echo "4.0*$NHITS/$NSTEPS" | bc -l)
echo "PI = $PI"
Example: pi.sh](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-20-2048.jpg)
![Branches
• Simple branching with if then
else fi
• Enclose condition with []
• elif is possible, too
• Use case in esac when you
can many cases and single
condition
String operators:
-z is empty?
-d is directory?
-f is file?
== equal to
!= not equal to
Integer operators:
-eq equal to
-lt less than
-ne not equal to
-gt greater than
Example: branches.sh](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-22-2048.jpg)
![Loops
• Simple loops: for … in … ; do …
done
• The seq utility can generate list of
numbers
• Conditional loops: while … ; do …
done
• Line-by-line: while read line ;
do … done
Example: loops.sh
One-liner (similar to watch)
while [ true ]; do
clear;
echo $RANDOM;
sleep 1;
done](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-23-2048.jpg)
![.bash_profile
# Ignore a few commands in history
export HISTIGNORE="pwd:ls:ls -l:cd"
# don't put duplicate lines in the history. See bash(1) for more options
# don't overwrite GNU Midnight Commander's setting of `ignorespace'.
HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups
# ... or force ignoredups and ignorespace
HISTCONTROL=ignoreboth
# Bash completion
if [ -f $(brew --prefix)/etc/bash_completion ]; then
. $(brew --prefix)/etc/bash_completion
fi
if [ -f $(brew --prefix)/share/bash-completion/bash_completion ]; then
. $(brew --prefix)/share/bash-completion/bash_completion
fi
# Prompt - including git
PS1='u@h:w$(__git_ps1 " (%s)") $ '](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-38-2048.jpg)
![branches.sh
#!/bin/bash
if [ -z "$1" ]; then
name="Arthur"
else
name="$1"
fi
if [ "$name" != "Arthur" ]; then
echo "Not Arthur"
else
echo "Hello Arthur"
fi
answer=$((6*7))
if [ $answer -eq 42 ]; then
echo "Life, the universe,
and everything"
fi](https://image.slidesharecdn.com/paorlgrst5ks3gqynkaj-signature-65f67f50ad078e1a677f61f34346981cb5dbbf9c657f7903f26cc69c11ac9933-poli-150421095210-conversion-gate02/75/Unleash-your-inner-console-cowboy-41-2048.jpg)
Uploaded byKenneth Geisshirt
Unleash your inner console cowboy
This document provides an overview of using the bash shell as a productivity tool. It discusses using the terminal, basic bash usage including redirection, pipes, configuration files, keyboard shortcuts, history, and tab completion. Useful utilities like sed, awk, find, grep and scripting basics are covered. Tools for developers like Homebrew, Git, Xcode, and Cocoapods are also summarized. The document aims to help users "unleash their inner console cowboy" and do more work directly from the command line.
More Related Content
Similar to Unleash your inner console cowboy
Unleash your inner console cowboy
- 1. Unleash your inner consolecowboy Kenneth Geisshirt [email protected] Realm Inc. @realm http://github.com/Realm/ http://realm.io/ Ivan Constantin, https://www.flickr.com/photos/ivan70s/
- 2. Today’s goal • Presentbash as a productivity tool • stop using the mouse 🐁 • Write scripts to automate your work • Begin to use advanced tools in your daily work Become a console cowboy
- 3. Agenda • The terminaland the shell • Basic usage of bash • Living on the command-line • Useful utilities • Scripting • Home brew • Tools for developers • git • Xcode • Cocoapods
- 4.
- 5. Which terminal? • iTerm2is much better • Easier to change tab (⌘ left + right, CTRL+TAB) • Change Desktop navigation to ⌥ left + right • Add CTRL left + right to iTerm2 preferences • Keeps SSH connection alive • http://iterm2.com/
- 6. Which shell? OS Xcomes with many shells ➤ bash, csh, ksh, sh, tcsh, and zsh Parée, https://www.flickr.com/photos/pareeerica/ Since 10.3, bash has been the default shell OS X 10.9.4 carries bash 3.2.51 (2010-03-17) Stephen R. Bourne (Bell lab) introduced the shell to UNIX in 1977 Home brew has many great bash related packages
- 7. Redirection UNIX idioms • atool should do one thing but do it well • text is the universal data format }Output of one utility is input for the next Bash implements redirection: • stdout to file: > • stdin from file < • append stdout to file: >> • stderr to stdout: 2>&1 Examples: echo “Hello” > hello cat < hello echo “World” >> hello clang notfound.m > error 2>&1
- 8. Pipes • Introduced toUNIX by Douglas McIlroy in 1973 • Pipes are the glue in UNIX component based programming (aka shell scripting) • Powerful idiom for stream processing • The character | is used by all known shells Examples lsof | grep ^Keynote | wc -l ifconfig | grep -e ^[a-z] | cut -f1 -d: "Pipeline" by TyIzaeL - Licensed under Public domain via Wikimedia Commons http://commons.wikimedia.org/wiki/File:Pipeline.svg#mediaviewer/File:Pipeline.svg
- 9. Configuration • $HOME/.bash_profile and$HOME/.bashrc are your personal configuration • alias - useful for often used options • Setting prompt (PS1) and search path (PATH) • Reload configuration: source ~/.bash_profile See my .bash_profile at the end
- 10. Keyboard short-cuts • Bashuses Emacs bindings by default 😄 (help bind for details) Movement • CTRL-a beginning of line • CTRL-e end of line • CTRL-← One word left • CTRL-→ One word right Cut-n-paste • CTRL-space mark • ␛🔙 Delete word • CTRL-d Delete character • CTRL-_ undo • CTRL-k delete until end of line • CTRL-y yank from kill- ring Request a dem o Remap CTRL-←/→
- 11. History • Bash storesyour command history • history - show latest commands • !! - run last command • !number - run command number again • CTRL-r - search in history • Exclude commands from history: • export HISTIGNORE=“pwd:ls:ls -l:cd”
- 12. Completion • Use TABto let Bash complete as much as possible • Use TAB+TAB to show possible completions • Bash has programmable completion → you can specify what Bash does • Large collections of completion recipes exist (home brew is your friend) Request a dem o
- 13. Living on thecommand-line • cd - - go back to previous folder • file file - guess file content (magic numbers) • lsof - list open files • ps (aux or -ef) and top - show processes • Simple watch: while true ; do clear ; command ; sleep n ; done
- 14. OS X specificcommands • open file - Starts registered program and open file • say “Hello world” - speech synthesis (download extra voices/languages in System preferences) • ls | pbcopy - copy stdin to paste board • pbpaste - paste to stdout • dns-sd -B _ssh._tcp - show Bonjour enabled SSH hosts
- 15. Useful utilities Find files:find . -name ‘*.o’ -delete Patterns: grep -r list * Cut field: cut -f1,3 -d: /etc/passwd Word count: wc -l *.cpp Transform: tr “ “ “_” < README.org Sort lines: sort -t: -n -r -k 4 /etc/passwd Last lines: tail /etc/passwd First lines: head /etc/passwd Request a dem o
- 16. sed - thestream editor • sed is used to edit files non- interactively • Option -E gives an editing (regular) expression • s/FISH/HORSE/g - substitute • /FISH/d - delete lines joinash, https://www.flickr.com/photos/joinash/ Option -i is tricky: • GNU sed has optional extension • BSD sed requires extension (‘’ is useful) Request a dem o
- 17. awk - aprocessing tool • awk is a programming language by itself • Matching lines are processed • line is split in fields (spaces are default) Example: adding.sh Patterns: BEGIN - before opening file END - after closing file A. Aho, P. Weinberger, B. Kernighan
- 18. Scripting Examples can be foundas last slides
- 19. Bash for programmers •Bash is a complete programming language • Shell scripts grow and become ugly 😞 • Execution: • sh script.sh • chmod +x script.sh; ./script.sh • Interpreted language → slow
- 20. Basic syntax • Whitespaces: space and tab • Comments: # and to end- of-line • Statements: either end- of-line of ; (semicolon) • Variables and functions: Letters, digits and underscore #!/bin/bash # Monte Carlo calculation of pi NSTEPS=500 NHITS=0 i=0 while [ $i -lt $NSTEPS ]; do x=$(echo $RANDOM/32767 | bc -l) y=$(echo $RANDOM/32767 | bc -l) d=$(echo "sqrt($x*$x+$y*$y) < 1.0" | bc -l) if [ $d -eq 1 ]; then NHITS=$(($NHITS + 1)) fi i=$(($i + 1)) done PI=$(echo "4.0*$NHITS/$NSTEPS" | bc -l) echo "PI = $PI" Example: pi.sh
- 21. Variables• Case-sensitive names •No declarations, no types • Strings: “…” are substituted; ‘…’ are not • Assignment (=): no spaces! • $(…) assignment from stdout including spaces • I often use awk ‘{print $1}’ to remove spaces • $((…)) arithmetic • $varname - value of variable varname Example: variables.sh Built-in variables: • $# is the number of argument • $1, $2, … are the arguments • $$ is the process ID • $? is exit code of last command
- 22. Branches • Simple branchingwith if then else fi • Enclose condition with [] • elif is possible, too • Use case in esac when you can many cases and single condition String operators: -z is empty? -d is directory? -f is file? == equal to != not equal to Integer operators: -eq equal to -lt less than -ne not equal to -gt greater than Example: branches.sh
- 23. Loops • Simple loops:for … in … ; do … done • The seq utility can generate list of numbers • Conditional loops: while … ; do … done • Line-by-line: while read line ; do … done Example: loops.sh One-liner (similar to watch) while [ true ]; do clear; echo $RANDOM; sleep 1; done
- 24. Functions • Functions canincrease readability of your scripts • arguments are $1, $2, … • local variables can be used • return an integer and get it as $? • Use global variable to return a string 😒 Example: functions.sh
- 25. Tips and tricks •Use set -e to exit early • or use || exit 1 • set -O pipefail and you can get the exit code of the first failing program in a pipe • xcpretty never fails but xcodebuild might • Use tee to write to stdout and file • To trace (debugging): set -x or sh -x
- 26. Tips and tricks •Always use “$var” when dealing with file names (and strings) • str=“fish horse”; for i in $str; do echo $i; done • str=“fish horse”; for i in “$str”; do echo $i; done • Call mkdir -p when creating folders • Create temp. files with mktemp /tmp/$$.XXXXXX • Using variable to modify behaviour of script: • FLAGS=“-O3 -libc++=stdlibc++” build.sh • Subshells: (cd foo && rm -f bar)
- 27.
- 28. Home brew • Homebrew provides calories for console cowboys • You don’t have to be root to install • Software is installed in /usr/ local/Cellar, and symlinked to / usr/local/bin • Brew cask is for binary distribution • http://brew.sh and http:// caskroom.io Greg Peverill-Conti, https://www.flickr.com/photos/gregpc/ Examples: brew search bash brew info bash brew install bash brew update
- 29. Tools for developers •Apple provides some basic tools • nm - display symbol table • c++filt - Prettify C++ and Java names • otool -L - display which shared libraries are required • libtool - create libraries • lipo - manipulate fat/universal binaries zzpza, https://www.flickr.com/photos/zzpza/ Examples: nm book.o | c++filt otool -L RealmInspector
- 30. git • Home brewpackages: • git, git-extras • Symlink /usr/local/bin/git to / usr/bin • Bash completion works • commands, branches, etc. • Fancy prompt: PS1='u@h:w$(__git_ps1 " (%s)") $ ‘ git log --graph --simplify-by-decoration --pretty=format:'%d' --all Examples: git count -all git contrib "Kenneth Geisshirt"
- 31. Xcode • You canbuild Xcode projects at the command-line xcodebuild -scheme SpainPlain - configuration Release -sdk iphonesimulator • Targets: clean, build, test • You can add shell scripts to build phases
- 32. xcpretty • The outputof xcodebuild can be hard to read • xcpretty makes it prettier • Installation: • sudo gem install xcpretty • Usage: • xbuildcode … | xcpretty
- 33. xctool • Yet anotherbuild helper • Installation: • brew install xctool • Usage: • xctool -scheme SpainPlain - configuration Release -sdk iphonesimulator build
- 34. Cocoapods • Dependency andbuild system for iOS and OS X developers • Installation: • sudo gem install cocoapods • Usage • Create a new Xcode project and quit Xcode • Edit Podfile and run pod install • Open workspace in Xcode • http://cocoapods.org
- 35. Further information • ClassicalShell Scripting. R. Arnolds and N.H.F. Beebe. O’Reilly Media, 2005. • The sed FAQ: http://sed.sourceforge.net/ sedfaq.html • Advanced Bash-Scripting Guide: http:// www.tldp.org/LDP/abs/html/ http://realm .io W e are hiring
- 36. varriables.sh #!/bin/bash message_1="Hello" message_2="World" message="$message_1 $message_2" echo $message nusers=$(grep-v ^# /etc/passwd | wc -l | awk '{print $1}') echo "Number of users: $nusers" answer=$((6*7)) echo "The life, the universe, and everything: $answer"
- 37.
- 38. .bash_profile # Ignore afew commands in history export HISTIGNORE="pwd:ls:ls -l:cd" # don't put duplicate lines in the history. See bash(1) for more options # don't overwrite GNU Midnight Commander's setting of `ignorespace'. HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups # ... or force ignoredups and ignorespace HISTCONTROL=ignoreboth # Bash completion if [ -f $(brew --prefix)/etc/bash_completion ]; then . $(brew --prefix)/etc/bash_completion fi if [ -f $(brew --prefix)/share/bash-completion/bash_completion ]; then . $(brew --prefix)/share/bash-completion/bash_completion fi # Prompt - including git PS1='u@h:w$(__git_ps1 " (%s)") $ '
- 39. .bash_profile - con’t #Color ls etc. alias ls="ls -G" alias ll="ls -l" alias fuck='sudo $(history -p !!)' # rerun as root # Building export MAKEFLAGS=-j4 # LejOS export NXJ_HOME=$HOME/local/leJOS_NXJ_0.9.1beta-3 export PATH=$PATH:$NXJ_HOME/bin
- 40. adding.sh #!/bin/bash tmpfile="$(mktemp /tmp/$$.XXXXXX)" for iin $(seq 1 20); do echo $RANDOM >> "$tmpfile" done awk 'BEGIN {total=0 } END { print total } { total+=$1 }' $tmpfile rm -f "$tmpfile"
- 41. branches.sh #!/bin/bash if [ -z"$1" ]; then name="Arthur" else name="$1" fi if [ "$name" != "Arthur" ]; then echo "Not Arthur" else echo "Hello Arthur" fi answer=$((6*7)) if [ $answer -eq 42 ]; then echo "Life, the universe, and everything" fi
- 42. branshes.sh - con’t case"$name" in "Arthur") echo "Welcome onboard" ;; "Trillian") echo "You know Arthur" ;; *) echo "Who are you?" ;; esac
- 43. loops.sh#!/bin/bash # Multiplication table fori in $(seq 1 10); do echo "$i $((3*$i))" done # All .sh files for f in $(ls *.sh); do echo "$f $(head -1 $f | cut -c3-) $(wc -l $f | awk '{print $1}')" done # read self lile-by-line i=1 cat $0 | while read line ; do nchars=$(echo "$line" | wc -c | awk '{print $1}') echo "$i $nchars" i=$(($i+1)) done | sort -n -k 2
- 44. report.text This is atest test. I can change it. A test of sed is about to happen.