SCP (Secure Copy) Examples

Copy the file "foo.txt" from a remote host to the local host

$ scp you@remotehost.edu:foo.txt /local/dir

Copy the file "foo.txt" from the local host to a remote host

$ scp foo.txt you@remotehost.edu:/remote/dir

Copy the directory "foo" from the local host to a remote host's directory "bar"

$ scp -r foo you@remotehost.edu:/remote/dir/bar

Copy the file "foo.txt" from remote host "rh1.edu" to remote host "rh2.edu"

$ scp you@rh1.edu:/remote/dir/foo.txt you@rh2.edu:/remote/dir/

Copying the files "foo.txt" and "bar.txt" from the local host to your home directory on the remote host

$ scp foo.txt bar.txt you@remotehost.edu:~

Copy the file "foo.txt" from the local host to a remote host using port 2264

$ scp -P 2264 foo.txt you@remotehost.edu:/remote/dir

Copy multiple files from the remote host to your current directory on the local host

$ scp you@remotehost.edu:/remote/dir/\{a,b,c\} .
$ scp you@remotehost.edu:~/\{foo.txt,bar.txt\} .

SSH Cheatsheet

SSH Notes

Configure aliases for hosts

For example, to set up named aliases for dev and qa, create a file at ~/.ssh/config:

Host dev
    HostName foo.somewhere.com
    User dev_username

Host qa
    HostName foo.somewhere.com
    User qa_usermname

Usage:

$ ssh dev

Passwordless auth with ssh-copy-id

Use ssh-copy-id to copy your public key to the remotehost. After you've logged in once, you get passwordless authentication thereafter.

$ ssh-copy-id -i ~/.ssh/id_rsa.pub remotehost

This will not work if you are connecting to a Windows remotehost.

Executing Multiple Commands with ssh

the here document syntax

The here document syntax lets you include everything in between << EOF and EOF as a list of commands. Like so:

$ ssh remotehost << EOF
  cd some_folder
  git pull
EOF

a script with commands only

Use a script of just the commands.

File: runner.sh

#!/usr/bin/env bash
cd some_project
./do_it.sh
ls

Use:

$ ssh user@remote 'bash -s' < /path/to/runner.sh

Transfer files with SFTP

You can use sftp in the same way ssh is used, to get an old-school ftp cli interface.

Compare files with diff

$ ssh user@host cat /path/to/remotefile | diff /path/to/localfile –

Port forwarding

Connects to a remote port, forward it to local.

$ ssh -L 4444:localhost:4000 remotehost

After that, from localhost, requests to http://localhost:4444 will get a response from http://remotehost:4000

Hello World with Node, Express, and React

Node and npm handle the server-side components of the app. We will build a basic Express web framework serving Jade templated files, which is the default for the express-generator.

On the client-side, React will be used for the View layer. React lets you write your client-side code in clean javascript components, rather than mucking around with multiple blocks of html and scripts.

Set up the Express server

• Install node on your system.

• Globally install express-generator

  npm install express-generator -g

• Use express-generator to create a basic app structure.

  $ express demo-react
  
     create : demo-react
     create : demo-react/package.json
     create : demo-react/app.js
     create : demo-react/public
     create : demo-react/public/images
     create : demo-react/routes
     create : demo-react/routes/index.js
     create : demo-react/routes/users.js
     create : demo-react/public/stylesheets
     create : demo-react/public/stylesheets/style.css
     create : demo-react/views
     create : demo-react/views/index.jade
     create : demo-react/views/layout.jade
     create : demo-react/views/error.jade
     create : demo-react/bin
     create : demo-react/bin/www

install dependencies:

    $ cd demo-react && npm install

run the app:

    $ DEBUG=demo-react:* npm start

Then load http://localhost:3000/ in your browser to access the app.

Set up the client-side React app

For React to work, it has to transpile it's .jsx componets. The easiest way to do this is with In-browser transformation. This method is not designed for production use, since it slows down the pages somewhat. But it's perfectly fine for dev and prototyping work.

We will use Bower to gather the React and Babel scripts needed for transformation.

Install Bower

    $ npm install bower -g

Initialize bower, accepting the defaults.

    $ bower init

We will make one customization here. Let's put the bower_components directory into the public dir, for convenience.

• Create a .bowerrc file in the root
• In the .bowerrc, write this one line: {"directory" : "public/bower_components"}

• Then install your bower components

  bower install babel --save
  bower install react --save

Configure the Jade templates for React

Here's how you set up your Jade templates to include your React components.

The main views/layout.jade needs to be updated to:

• source the react script,
• source the babel browser transformation script,
• and provide a new block called called morejs

See the new bottom three lines in layout.jade

    doctype html
    html
      head
        title= title
        link(rel='stylesheet', href='/stylesheets/style.css')
      body
        block content
        script(src='/bower_components/react/react.min.js')
        script(src='/bower_components/babel/browser.min.js')
        block morejs

In our views/index.jade,

• Add a #greeter element, which is where React is going to inject our component.

• Add a reference to that new component in the morejs block. Note the type="text/babel"

  extends layout
  
  block content
    h1= title
    p Welcome to #{title}
    #greeter
  
  block morejs
    script(src='/javascripts/helloworld.jsx', type="text/babel")

And the greeter component React JSX code itself, at /public/javascripts/helloworld.jsx is:

    React.render(
        <h1>Hello, world from React.js!</h1>,
        document.getElementById('greeter')
    );

That's it. Refresh your browser at http://localhost:3000/ . You should see

---

Express

Welcome to Express

Hello, world from React.js!

---

Longest Common Substring Algorithms

Given strings "ABAB" and "BABA"...

A dynamic programming style algorithm constructs a matrix of the substring lengths.

    |   |   | A | B | A | B |
    |   | O | O | O | O | O |
    | B | O | O | 1 | O | 1 |
    | A | O | 1 | 0 | 2 | 0 |
    | B | O | 0 | 2 | 0 | 3 |
    | A | O | 1 | 0 | 3 | 0 |

In this matrix, the 5th column tests the substring ABA of the ABAB sequence. So cell [5, 5] compares ABA to BAB ... the lcs is BA, length 2.

The other approach is to construct a generalized suffix tree for the strings, then find the deepest internal nodes which have leaf nodes from all the strings in the subtree below it.

A Simple Suffix Tree Implementation:

Dijkstras Algorithm for shortest path:

Or, here's a more complete implementation.

Function Caching via Memoization

To explain memoization, see these different versions of Fibonacci:

Purely recursive. Highly inefficient. Recalculates every value for every calculation.

Iterative. Much faster.

Here is the memoization technique. A 'cache' dictionary of { (args) : (return_val), } is created. This allows us to quickly locate and reuse any value that has already been calculated.

Here is the same memoization design, implemented as a decorator.

Update Evernote in Wine on Ubuntu

Wine on Ubuntu has problems running the Evernote updates. You get something like: "Evernote was already installed by another user...." The 'find/remove regedit keys' method, described here, and here, did not work for me. I didn't find the reversed the digits in regedit. Well, I only use wine for Evernote. So I used this subtle technique to remove all wine installation. (Don't do this if you have other Wine apps you are keeping.)

sudo apt-get purge wine
rm -rf ~/.wine
sudo apt-get install wine

Then ran the Evernote installer as usual (from files, rt-click on installer).

Fabric is cool.

[Fabric](http://www.fabfile.org/) is cool. More python, less Bash. For example, I can automate my django db rebuild process, like so:

Programming Makey Makey clicks to send strings

Sending a string

One way to send a string instead of a keystroke is to modify the Mouse Button Events in the makey.ino file. To make a left mouse click send 'Hello World':

1. In the Arduino IDE, open the makey-makey.ino file.
2. In the main code file, *not* the settings.h, find the lines

if (inputs[i].keyCode == MOUSE_LEFT) {
  Mouse.press(MOUSE_LEFT);
}

3. Comment out the Mouse.Press line, and replace it with a Keyboard.print

if (inputs[i].keyCode == MOUSE_LEFT) {
  //Mouse.press(MOUSE_LEFT);
  Keyboard.print("Hello World.");
} 

4. Upload the sketch. Pin A1 is the Makey left-mouse click. Touch the Ground and Pin A1 to try it.

Sending a carriage return with button click

Modify the settings.h file this time, since only one key is being modified. For example, pin A0 defaults to a right-mouse-click...change it to send KEY_RETURN.

int keyCodes[NUM_INPUTS] = {
  // top side of the makey makey board
  KEY_UP_ARROW,      // up arrow pad
  KEY_DOWN_ARROW,    // down arrow pad
  KEY_LEFT_ARROW,    // left arrow pad
  KEY_RIGHT_ARROW,   // right arrow pad
  ' ',               // space button pad
  MOUSE_LEFT,        // click button pad
  // female header on the back left side
  'w',                // pin D5
  'a',                // pin D4
  's',                // pin D3
  'd',                // pin D2
  'f',                // pin D1
  'g',                // pin D0
  // female header on the back right side
  MOUSE_MOVE_UP,      // pin A5
  MOUSE_MOVE_DOWN,    // pin A4
  MOUSE_MOVE_LEFT,    // pin A3
  MOUSE_MOVE_RIGHT,   // pin A2
  MOUSE_LEFT,         // pin A1

  // COMMENT MOUSE_RIGHT TO REPLACE IT WITH KEY_RETURN
  //MOUSE_RIGHT      // pin A0
  KEY_RETURN         // pin A0
};

Sending a carriage in sketch code

Instead or modifying the inits, you can write it in one of the Mouse.press routines. Keyboard.write(KEY_RETURN);

Using a Makey-Makey with Ubuntu

I've spent a lot of time working with software, but had never been exposed to the basic principles of electronics. The Makey-Makey is a great introduction to the world of circuitry. Its 'cool trick' is hacking just about anything into a keyboard. There's a lot of video of folks wiring all kinds of household items into input devices.

But underlying all the fun stuff is real first look at how electronic hardware works. The Sparkfun electronic hobbyist's site has an excellent collection of tutorials presenting the fundamental science and application of electronic circuitry.

Arduino is a very popular open-source electronics prototyping platform...your local RadioShack should offer a variety of Arduino-ready components. The Makey Makey is programmed using the Arduino IDE (Integrated Development Environment). Here's how to install the Arduino IDE in Ubuntu. (I used Arduino IDE version 1.0.5, and Ubuntu version 13.10.)

First, install the Arduino IDE. The one in the Ubuntu repository worked fine for me.

sudo apt-get install arduino

This will install the IDE software to: /usr/share/arduino/

To use the Arduino IDE with Makey-Makey, you'll need to install a driver. Download the zipped driver with:

 wget https://dlnmh9ip6v2uc.cloudfront.net/tutorialimages/MaKey_MaKey_QuickStart/MaKeyMaKey-13-8-12.zip

Then Unzip the driver to the IDE's hardware directory:

unzip MaKeyMaKey-13-8-12.zip -d /usr/share/arduino/hardware

If that worked properly, you should see the Makey-Makey listed in the Board menu.

To modify basic functionality of the Makey, you'll need to grab the source code.
Get and extract the source:

wget https://dlnmh9ip6v2uc.cloudfront.net/tutorialimages/MaKey_MaKey_QuickStart/MaKeyMaKey-13-8-12.zip  

Move it your home/sketchbook's

mv hardware/Makey ~/sketchbook/hardware/

Permissions

To ensure your IDE can communicate with the Makey, add your user to the following three groups, like so:

sudo usermod -a -G dialout $USER
sudo usermod -a -G tty $USER
sudo usermod -a -G uucp $USER

Resolve Ubuntu ModemManager conflicts

For the IDE to be able to update the Makey settings, you'll need edit ModemManager blacklist settings. (This fix is taken from discussions here and here.) Using your favorite text editor, you'll need to create/edit a file at:

sudo vi /etc/udev/rules.d/77-mm-arduino-blacklist.rules

In your editor, append the following line to that file:

ATTRS{idVendor}=="1b4f", ATTRS{idProduct}=="2b74", ENV{ID_MM_DEVICE_IGNORE}="1"

Save the file.

Unplug your Makey. Close the IDE, Logout/Login. Plug Makey back in. From there, you'll be ready to reprogram your Makey, following along with the MaKey MaKey Quickstart Guide (Part 2).

Using python pymssql to query SQLServer from linux

Getting pyssmql to work was pretty easy. (I had less luck using pyodbc.)

sudo apt-get install freetds

sudo pip install pymssql

Create entries for your sqlservers in /etc/freetds/freetds.conf like so:

[dev01]
host = YOUR_SQL_SERVER_HOST
port = 1433
tds version = 8.0


Here's a simple script using it:


import pymssql
import getpass

pwd = getpass.getpass()

conn = pymssql.connect(host='YOUR_SQL_SERVER_HOST',
                       user='YOUR_DOMAIN\\YOUR_USERNAME', password=pwd)
cur = conn.cursor()
cur.execute('use YOUR_DB')

query = "select top 10 id, name, lastmodifieddate from campaign"

cur.execute(query)

for row in cur:
    print "{0}\n{1}\n{2}\n".format(row[0], row[1], row[2])

sqsh -- install, configure, usage

install:

sudo apt-get install freetds
sudo apt-get install sqsh

Create entries for your sqlservers in /etc/freetds/freetds.conf like so:

[dev01]
host = dev01server
port = 1433
tds version = 8.0


Create settings file in ~/.sqshrc
\set username=MYDOMAIN\myusername
\set password=
\set style=vert


Usage:
user@box:~$ sqsh -Sdev01

sqsh-2.1 Copyright (C) 1995-2001 Scott C. Gray
This is free software with ABSOLUTELY NO WARRANTY
For more information type '\warranty'
1> use testdb
2> go
1> select field1, field2 from table where field1 = 'test';
2> go
field1: test
field2: other data
(1 row affected)
1> exit
user@box:~$

access remote django dev server from local browser

To access a remote Django development server from your local browser....

On the local host, SSH to the remote host:
$ ssh -v -L 9000:localhost:8000 <the remote address>
On the remote host, run the Django dev server:
/path/to/my/django/project$ python manage.py runserver 0.0.0.0:8000
On the local host, go to http://localhost:9000 in the browser

Or...

Using LocalForward in your ~/.ssh/config, add entry

Host myremote
  User eliot
  HostName my.remotehost.com
  LocalForward 9000 localhost:8000

SQuirrel SQL client for Ubuntu

Provides a nice gui for accessing Microsoft SQLServer databases. (Handles MySQL also.)

Get it from here: http://www.squirrelsql.org/

Configuring it is a little tricky...use the guide here:
http://informatics.malariagen.net/2011/08/15/squirrel-sql-a-nice-database-gui-frontend-for-ubuntu/

To configure it for MySQL as well, follow similar steps after installing the mysql-jdbc driver.
sudo apt-get install libmysql-java

sshfs

I've made some effort to duplicate my work desktop Ubuntu environment on my laptop and home desktop systems. When that becomes tiresome, I use SSH from my remote systems to work on my desktop files. But that restricts me to using only the command line editors available on the Ubuntu system. But then EUREKA, SSHFS! Now I mount the work Ubuntu file system to a local drive. Let's me use Sublime and other local dev apps with the files on the mapped drive. Nice.

mysql + django on the mac is a pain

Issues with mixing 32 bit python build with 64 bit mysql build.

Switching to PostgreSQL.

installed ack in cygwin

perl -MCPAN -e "CPAN::Shell->force(qw(install App::Ack));"

built vim from source to enable python support for cygwin

uninstalled vim ...apt-cyg remove vim
got source vim-7.3.tar.bz2 from vim.org
2 steps to unzip....install bunzip
bunzip vim-7.3.tar.bz2
tar -xvf vim-7.3.tar

installed ncurses
installed libncurses-devel .. had to be done via cygwin-setup
installed make
installed view

cd vim73
./configure --enable-pythoninterp --enable-perlinterp --enable-rubyinterp --enable-gui=no --without-x --enable-multibyte --prefix=/usr


unlink /usr/bin/view
unlink /usr/bin/rvim
unlink /usr/bin/rview
unlink /usr/bin/vimdiff


make && make install

I finally got MongoDB running in cygwin! Trying to make it work with Django....

Automated web config transformation for Uhuru AppCloud

I use the Uhuru AppCloud (http://uhurusoftware.com/) for hosting some of my applications. It's an excellent resource for serving your apps from the cloud. 

For .NET applications, they provide the Uhuru Cloud Manager plugin for Visual Studio that makes deployment simple. The one piece of configuration that is required by the developer is to change the Web.config's database references to point to their new locations in the cloud. I wanted to automate this bit of configuration using Visual Studio tools. 

Visual Studio uses it's Configuration Manager to implement multiple deployment environments when deploying, but its intended to be used with the VS One-Click publishing  tool, rather than the Cloud Manager plugin. 

Here’s a way to use the standalone Web Configuration Transformation Tool to perform the same transform that Visual Studio’s Configuration Manager uses to implement multiple deployment environments. This lets you keep store and your Uhuru configuration in the project for use when needed, while keeping your Default configuration file with its local settings for Debug mode testing. 


I. Use Visual Studio to create your transform template.
  1. In VS, choose Build > Configuration Manager > <New...> and name it UhuruCloud, for example.
  2. In Solution Explorer, right-click on your Web.Config file and choose 'Add Config Transforms'
  3. Edit the resulting Web.UhuruCloud.config for the transform. (See http://msdn.microsoft.com/en-us/library/dd465326(VS.100).aspx )
  4. For example, my Web.config file contained this:

<connectionStrings>
    <add name="ApplicationServices"
      connectionString="data source=localhost;Integrated Security=SSPI;"
      providerName="System.Data.SqlClient" />
</connectionStrings>

    and the transform xsl I needed looked like this:

<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
    <connectionStrings>
        <add name="ApplicationServices"
          connectionString="{mssql-2008#SQL Server}"
          xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
    </connectionStrings>
</configuration>


II. Using the standalone Config Transform standalone Tool.
  1. Get the tool from http://ctt.codeplex.com/Wikipage?ProjectName=ctt
  2. The syntax of the tool is like so:

ctt.exe s:source.config t:transform.config d:destination.config

That’s just the most basic usage. Other good Examples are provided in the distribution. For me, it was handy to have a quick batch script to make that call easier. Especially while trying to get the paths just right.

File: ctt_run.bat
@ECHO off
rem ...Set (S)ource, (T)emplate, and (D)estination files here
SET S=C:\Projects\MySolution\MyApp\Web.config
SET T=C:\ Projects \ MySolution \ MyApp \Web.UhuruCloud.config
SET D=Web.New.config
@ECHO on
ctt.exe s:%S% t:%T% d:%D%

That’s it. (My example wrote the transformed file 'Web.New.config'.) 



I also posted this at StackOverflow
http://stackoverflow.com/questions/12031254/web-configuration-transformation

And in UhuruCloud's forum
http://support.uhurusoftware.com/entries/21870063

Put a git repository into DropBox

Not locked to GitHub anymore. Local repos on laptop and Mac pull from master on filesystem DropBox dir.