rsync over SSH

sshMany of us use SSH multiple times on a daily basis times to do simple, complicated, and often redundant tasks. Often times the tasks are those which could be scripted and automated. For instance, if you have to synchronize files with a server often throughout the day, a cron job would be the ideal way because then it will be done automatically and you don’t have to worry about it. If you use SSH keys without a password to access a server, you can expand on it by using rsync to synchronize those local and remote directories.

Here is the command you would use to make this happen:

rsync -e 'ssh -i ~/.ssh/id_rsa' -rulvhtpz /Users/user/file_to_sync user@host.com:~/

rsync options used

-r, recursive throughout directories
-u, skip files that are newer at the destination (meaning only update old files)
-l, copy symlinks as symlinks
-v, verbose; show all output as it happens
-h, display output in human readable format
-t, preserve times of files
-p, preserve permissions
-z, compress files during transfer to preserve bandwith

Making rsync convenient:

rsync is really nice when it comes to automation. Adding rsync to a crontab entry comes really handy. There are all kinds of options for cron – to view them, check out my knowledge base article on it.

If we want rsync to run automatically at 12pm and 4pm, this is what we would do:

Open up your terminal app and type the following:

crontab -e

Add the following lines to the file:

00 12 * * * rsync -e 'ssh -i ~/.ssh/id_rsa' -rulvhtpz /Users/user/file_to_sync user@host.com:~/
00 16 * * * rsync -e 'ssh -i ~/.ssh/id_rsa' -rulvhtpz /Users/user/file_to_sync user@host.com:~/

If you’re really clever

If you are a programmer and want your code to automatically synchronize to a remote server, add a macro to your IDE that somehow that adds the rsync code to a button in. For instance, if you add the rsync command to the save button command, maybe it will kill two birds with one stone.

For more information

Go into your terminal and type man rsync or rsync -h

SSH Tunneling; encrypted surfing with Virtual Hosts

Encrypted SurfingWhether you’re the Secretary of Defense, or just an average Joe trying to survive with some peace of mind and security, encryption is a good thing. Have many virtual hosts on your unencrypted Apache server, but want encryption for whichever virtual host you specify? Here is the solution! Note, this is written for Linux clients – not Windows. You can tweak the instructions to work with Windows by using Putty and creating a tunnel that way.

First of all, here is the command to tunnel for Linux:

ssh -f -L 10000:your_virtual_host.com:80 user@myserver.com -N

Explanation of the above command:

  • ssh starts the ssh client
  • -f forks the ssh client into the background
  • -L forwards the command to the binded source_port:server:destination_port
  • -N tells ssh not to execute a command on the remote server once you are logged in to it

After you have started the tunnel using the command above, you will stay logged into it as long as the terminal is open.

Next, you would open your web browser and go to the following address with the address bar:

http://your_virtual_host.com:10000

By going to that address, it fails? What? Ohhhhh that’s right, you need to add the entry for that site to your hosts file:

Open the file:

vi /etc/hosts

(If you were doing this in Windows, the file is at C:WindowsSystem32driversetchosts)

Add this line to it:

127.0.0.1 your_virtual_host.com

Now go to the address again and it should work:

http://your_virtual_host.com:10000

If you were to run a packet sniffing program such as WireShark, you could monitor your network adapter (wlan0 or eth0 – whichever one you are using) and see that everything going to myserver.com (which is where you are tunneled into via ssh – using it to access you_virtual_host.com) is encrypted! Whoo hoo now you can log into your unencrypted website without worrying that people can see your plain text password going over the network.

If you were to monitor your loopback interface (lo) then you would see all the clear text data – except it never leaves your computer unencrypted.

NOTE: Once you have added your_virtual_host.com to your /etc/hosts file, it will always look for that domain on the local machine which means you need to open to tunnel to access it. As a result, if you try to ssh into your_virtual_host.com you will see that the connection is refused. The way around this is to ssh into a different domain on the same server (notice that I ssh’d into myhost.com instead of my_virtual_host.com).

Every host you add to the 127.0.0.1 line in the /etc/hosts file that is on your server will work on the same port (10000 or whatever port you specify – you can use any port you want that isn’t taken by another program). So, if you have my_other_domain.com on the server and, in your web browser, you go to http://my_other_domain.com then it will work as well with the existing tunnel.

View the next article for setting your computer to automatically start the tunnel when you log in.

If this works for you or not, please comment below and let me know.

Remote Tunneling into your Home/Business/School Network

If you are in need of a secure connection, or your establishment blocks ports that you need to use for remote access, then SSH tunneling is the answer. This article explains how to set up Putty for tunneling. Putty is available for Windows at http://www.chiark.greenend.org.uk/~sgtatham/putty/

Open up Putty and follow these steps:

  • Under “Category,” click on Session
  • Under “Basic options,” fill in the Hostname or IP address of the server you wish to connect to.
  • Under “Saved Sessions,” Fill in a name to describe the connection you are creating.

At this point, do NOT click save or open or anything else. Continue to the next steps.

  • Click on the + (plus) sign towards the bottom of the list at SSH.
  • Click on Tunnels in the SSH list.
  • For the source port, type whatever port you want (I typically use the same port that I will use for destination). For demonstration, I use port 10000.
  • For Destination, type localhost:10000 (Use localhost – do not substitute anything else).
  • Click Add
  • Back in the “Category” list, go to the top and click on “Session” again.
  • Click the Save button.

Now Putty is set up to SSH tunnel you to whatever port you need to access. Follow the rest of the steps and you are done!

  • Open the Putty connection that you just created back at the Sessions page.
  • It will open up a command line interface. Login as prompted.
  • Leave the Putty command line interface window open.
  • Open your web browser
  • In the address bar, type http://localhost:10000 (or whatever port you used).
  • If you are accessing a SSL website, use https://localhost:10000


Introduction to Php

PHP, The Pre Hypertext Preprocessor is a programming language that allows web developers to create dynamic content that interacts with databases. PHP is basically used for developing web based software applications (http://www.webasyst.net/glossary.htm). Recently some projects have been started that are porting PHP to be used to write client side programs – just like what you use on your personal computer that is not on the internet.

Rather than rambling on and on about php, this article is going to be written to give you a simple reference of how to perform some simple PHP coding. I will start off by simply showing you how to write content to a page, and how to do it most efficiently. I will show you how to properly comment your code, and how to use variables. How to execute queries to a MySql database and how to display the data from the MySql database.

First off, PHP is an embedded server-side scripting language. This means that it can be placed inside of HTML code and work with it. All PHP code is parsed (executed) on the server so that the only thing sent to a web browser is pure HTML. This is unline JavaScript which is parsed on the client’s computer. You can see this if you view the source of a page with a .php extension. All you will see is HTML.

hello.php

<?php
$title = 'hello world!';
?>

Here is the typical HTML content to your page

Here is the title:
<?php echo $title; ?>

To start, we create a page called hello.php. When you write a page that has PHP language, it needs to have the .php extension so that the server will know to parse the PHP code.

As you can see, the above code is primarily written in HTML. The parts in color are written in php.

Line 1 is the <?php code that you use to tell the server that you are writing inPHP. It can go anywhere in a HTML page – even above the <html> tags.

Line 2 has $title which is a variable. Variables are words that start with adollar sign ( $ ) that store data that is assigned to them. In this case, since it says <?php $title = ‘Hello World’; ?> then $title is equal to the string: Hello World!

Line 3 has the closing php tags – this tells the server not to parse PHP again until it sees the opening . remember, you always have to add a closing ?> tag as well.

Line 6 has the HTML title tags which contain the title of the page – typicalHTML. The only difference is that instead of making the title to say Hello World!, I made it to say $title. Actually, both of them say the exact same thing when printed or echo’d to the page in your web browser. In this case, since it is in the will actually be printed to the browser title bar (usually at the top of the browser for Firefox or Internet Explorer, etc.).

Line 10, although in the body of the page that is showing pure HTML, has a little bit of PHP code in it. <?php echo $title; ?>, although it looks like it does in the code, it will actually print what was assigned to the $title variable back at the top of the page. It will print Hello World!

Upload/Download using your private key on Windows with WinSCP

This tutorial shows you how to use your private key for authentication to our server using WinSCP

If you are a web hosting client of kasit.com, then you should have received a private key in your email. It is called something like something_kaslnetwork_key. Download the private key from your email and put it on your desktop.

  • Create a directory in your home folder named ssh
  • move something_kaslnetwork_key to your ssh folder

First you need to convert your private key to a different format. To do this, download PuTTYgen from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html (Make sure it is PuTTYgen and nothing else).

Start PuTTYgen and follow these steps:

  • Click on the “Load” button
  • Select the private key I gave you that’s in your ssh folder
  • Press “Open”
  • Press “OK” on the “Successfully imported foreigh key…” box
  • Click “Save Private Key” (NOT PUBLIC KEY)
  • Click “Yes” to save it without a password
  • Name the file the same as the existing file you have, but keep the .ppk extension.
  • Close PuTTYgen – you’re done with converting the key.

Now proceed to the following section on WinSCP

Using WinSCP

You can download WinSCP from http://winscp.net. It is recommended that you downoad the stable version. Install it, and start it.

Fill in the following fields as such:

Host name: kasnetwork.com
Port number: 22
User name: Your username that you have been using on the server
Password: Leave Blank
Private key file: something_kaslnetwork_key.ppk (in your ssh folder that you created)
File Protocol: SFTP (Leave the “Allow SCP fallback” box checked

You should now be able to drag and drop files between your computer and the server

Please note that you will *not* have permissions to modify anything in the initial directory – only in the sub-directories. If this is a problem, contact me and I can move whatever you want to whereever you want.

Upload/Download using your Private Key in Linux or OSX

If you use Linux or Mac OSX, this is a tutorial for transferring files between our server and your computer

First of all, if you are a web hosting client of kaslnetwork.com and have not received your private key by email, then please contact me and I will email it to you.

Now that you have your private key (called something like coyote_kaslit_key), put it on your Desktop, open up your terminal and run these commands (replace coyote with your username (if your key is named something completelydifferent, then use the name of your key):

cd ~
mkdir .ssh
cp ~/Desktop/coyote_kaslit_key .ssh/
chmod 700 .ssh
chmod 600 .ssh/*
ssh-add .ssh/coyote_kaslit_key
sftp coyote@kaslnetwork.com

If you get a prompt after that last command that looks like sftp> then it worked and you are now on our server in an encrypted file transfer program.

You can now use your favorite sftp client (some ftp clients include sftp connection methods) to connect. Be sure to use your private key (coyote_kaslit_key) to connect to our server on port 22