Welcome to Software Carpentry Etherpad!

This pad is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents.

Use of this service is restricted to members of the Software Carpentry and Data Carpentry community; this is not for general purpose use (for that, try etherpad.wikimedia.org).

Users are expected to follow our code of conduct: http://software-carpentry.org/conduct.html

All content is publicly available under the Creative Commons Attribution License: https://creativecommons.org/licenses/by/4.0/

Website of the workshop: http://rits.github-pages.ucl.ac.uk/2016-12-13-UCL_software_carpentry/



Introduction to programming with Python

Before starting

- Download the data: http://swcarpentry.github.io/python-novice-inflammation/data/python-novice-inflammation-data.zip and save it in a new directory in your desktop: LearningPython
- Make that directory as a git repository
- Create it a repository in github with the same name and set the remote as that one
- 

The lesson on turning Python code into scripts that you can run from the command line is at http://swcarpentry.github.io/python-novice-inflammation/10-cmdline/

Excel - Python material (Day 2, Afternoon):


Survey data spreadsheet file for the Excel lesson is available to download from Figshare here: https://ndownloader.figshare.com/files/2252083


Version Control with Git


Getting started

You should already have:
  1. Installed git
  2. Created an account for yourself at https://github.com/

Exercises

Visit https://b.socrative.com/login/student/ 
Room number: RITS

Session notes


Links & Resources

The software carpentry lesson this was based on: http://swcarpentry.github.io/git-novice/

More learning resources:

Useful tools:

Using Word files with git:

Excel seems to be a bit more complicated, but various people have solutions:


Automating tasks with the Unix shell


Lesson materials: http://swcarpentry.github.io/shell-novice/

Getting started

1. Download these files: http://swcarpentry.github.io/shell-novice/data/shell-novice-data.zip
2. Extract the zip archive
3. Copy the data-shell folder to your desktop


Exercises

https://b.socrative.com/login/student/

Room number: UQLWDDJ6


Glossary

(keep track of what commands do here)

Commands: https://swcarpentry.github.io/shell-novice/reference/

    pwd - path working directoryvcd - show you where you are
    whoami - tells you who you are 
    ls - tells you what's in the directory 
        files in whites
        
        directories in blues
    ls --help or man ls to see the help  (ls --help doesn't work in Mac)
    ls Desktop  - shows what's in Desktop
    ls Desktop/data-shell - shows what's in Desktop/data-shell
    ls -a  - shows hidden files, like `.` and `..`
    mkdir - to create a new directory
    when naming files - do not use space in between
    rm - use with caution..it will delete file permanently
    rm - can't be used to delete  a directory
    rm - r  deletes a directory and all it's contents
    rm -r -i  asks for permission for each step
    mv first argument second argument (to move files from directory (first argument) to another directory (second argument)
    mv can rename a file or directory (if these don't exist)
    
    
    cd - is to change directory,
    cd Desktop/data-shell - will move us to the Desktop/data-shell directory
    relative paths vs absolute paths
    absolute paths start from the root (/)
    cd ../ - goes one level up
    tab key completes file name (cd no + tab) will give north-pacific-gyre (if within data-shell)
    cd  - by itself goes to the home directory
    
    vim and emacs are more advanced text editors
    rm - permanent remove/delete file (won't be in recycle)

  wc  - it's used to count words, lines and characters
   wc -l  - counts number of lines.
sort - is used to sort
   -n  ; to sort using numerically method, so 2 goes before 10
head - shows the first lines of a file
  head -n 1 filename ; shows only the first line of the file
  
  Pipe => "|" is used to transfer the output of a command as an input to the next one:
      wc -l *.pdb | sort -n |  head -n 1
        \                       \            \-> shows the first line
          \                       \-> sort the lines by numbers
            \-> print the number of lines for each file that ends by : .pdb

Loops: 
$ for file in NENE*[AB].txt
> do
> echo $file
> bash goostats $file stats-$file
> done
That will run the program "goostats" for each $file (called NENE*[AB].txt ) and generate a new file called stats-$file

Scripts
Store a list of commands that you want to execute in a file, and run them whenever you like.
You can pass arguments to scripts to make them do different things each time they're run.


Geeky aside

The script that I used for the split screen display is:
    
#!/bin/bash
#
# Create terminal for Software Carpentry lesson
# with the log of the commands at the top.

# Fix for Mac OS Sierra; also need 'brew install coreutils'
export EVENT_NOKQUEUE=1

# Where we'll store the executed history.  Defaults to /tmp/log-file,
# but you can override from the calling process.  For example:
#
#   LOG_FILE=/tmp/my-log ./swc-shell-split-window.sh
LOG_FILE="${LOG_FILE:-/tmp/log-file}"

# The number of lines of history to show.  Defaults to 5, but you can
# override from the calling process.
HISTORY_LINES="${HISTORY_LINES:-5}"

# Session name.  Defaults to 'swc', but you can override from the
# calling process.
SESSION="${SESSION:-swc}"

export SHELL_SESSION_HISTORY=1

# If $LOG_FILE exists, truncate it, otherwise create it.
# Either way, this leaves us with an empty $LOG_FILE for tailing.
> "${LOG_FILE}"

# Create the session to be used
# * don't attach yet (-d)
# * name it $SESSION (-s "${SESSION}")
# * start reading the log
tmux new-session -d -s "${SESSION}" "gtail -f '${LOG_FILE}'"

# Get the unique (and permanent) ID for the new window
WINDOW=$(tmux list-windows -F '#{window_id}' -t "${SESSION}")

# Get the unique (and permanent) ID for the log pane
LOG_PANE=$(tmux list-panes -F '#{pane_id}' -t "${WINDOW}")
LOG_PID=$(tmux list-panes -F '#{pane_pid}' -t "${WINDOW}")

# Split the log-pane (-t "${LOG_PANE}") vertically (-v)
# * make the new pane the current pane (no -d)
# * load history from the empty $LOG_FILE (HISTFILE='${LOG_FILE}')
# * lines which begin with a space character are not saved in the
#   history list (HISTCONTROL=ignorespace)
# * append new history to $HISTFILE after each command
#   (PROMPT_COMMAND='history -a')
# * launch Bash since POSIX doesn't specify shell history or HISTFILE
#   (bash)
# * when the Bash process exits, kill the log process
tmux split-window -v -t "${LOG_PANE}" \
        "HISTFILE='${LOG_FILE}' HISTCONTROL=ignorespace PROMPT_COMMAND='history -w' bash --norc; kill '${LOG_PID}'"

# Get the unique (and permanent) ID for the shell pane
SHELL_PANE=$(tmux list-panes -F '#{pane_id}' -t "${WINDOW}" |
        grep -v "^${LOG_PANE}\$")

tmux send-keys -t "${SHELL_PANE}" " cd" enter

# Unset all aliases to keep your environment from diverging from the
# learner's environment.
tmux send-keys -t "${SHELL_PANE}" " unalias -a" enter

# Set nice prompt displaying
# with cyan
# the command number and
# the '$'.
tmux send-keys -t "${SHELL_PANE}" " export PS1=\"\[\033[1;36m\]\! \w $\[\033[0m\] \"" enter

# Clear the history so it starts over at number 1.
# The script shouldn't run any more non-shell commands in the shell
# pane after this.
tmux send-keys -t "${SHELL_PANE}" "history -c" enter

# Send Bash the clear-screen command (see clear-screen in bash(1))
tmux send-keys -t "${SHELL_PANE}" "C-l"

# Wait for Bash to act on the clear-screen.  We need to push the
# earlier commands into tmux's scrollback before we can ask tmux to
# clear them out.
sleep 0.1

# Clear tmux's scrollback buffer so it matches Bash's just-cleared
# history.
tmux clear-history -t "${SHELL_PANE}"

# Need add an additional line because Bash writes a trailing newline
# to the log file after each command, tail reads through that trailing
# newline and flushes everything it read to its pane.
LOG_PANE_HEIGHT=$((${HISTORY_LINES} + 1))

# Resize the log window to show the desired number of lines
tmux resize-pane -t "${LOG_PANE}" -y "${LOG_PANE_HEIGHT}"

# Turn off tmux's status bar, because learners won't have one in their
# terminal.
# * don't print output to the terminal (-q)
# * set this option at the window level (-w).  I'd like new windows in
#   this session to get status bars, but it doesn't seem like there
#   are per-window settings for 'status'.  In any case, the -w doesn't
#   seem to cause any harm.
tmux set-option -t "${WINDOW}" -q -w status off

tmux attach-session -t "${SESSION}"