Backing up a server to S3

AWS LogoIf you run a server of any kind then you should know how important it is to regularly backup the server. Now even though backing up the server is a great thing to do, it doesn’t do any good if the backup is in the same location as the server itself; if the building burns down, everything is gone. It is best to have an off-site backup location in another city or even state or country.

This is where Amazon’s S3 service comes in handy. S3 is a service where you can store data of any type for a low monthly fee ( $0.14 per Gigabyte at the time of this writing ). S3 uses “buckets” to store your data – think of a bucket as a directory. To get started, do the following:

  • Sign up for am Amazon account at amazon.com.
  • Log in at aws.amazon.com. You will see a bunch of tabs at the top of the page. Click on the one for S3.
  • Create a new bucket and name it something relevant to what you are going to store inside of it.
  • Click on your name at the top-right of the page and select “security credentials”
  • Copy your access key and your security key (you can do this later – but just be aware of where they are)

Now you are set up to use Amazon’s S3. Next, you need to install a S3 client. I am running Ubuntu Server and the client I use is called s3cmd. To install s3cmd, do this (it should already be in your repository for Ubuntu Server):

sudo apt-get install s3cmd

After it is installed, you must configure the program (I did it as root since I will be running this program as root). To configure it, do this:

sudo s3cmd --configure

Go through the configuration wizard. Use the access and security keys you discovered above.

s3cmd has all kinds of options. The option I am using is the sync option since I will set the servers to sync every x amount of hours. If you want to learn about more s3cmd options, run this command:

s3cmd -h

The command I use to back up my servers is as follows (I run it as a cron job, but you can also run it manually):

s3cmd -rpH sync /home s3://my_bucket
  • the “r” option makes the sync go recursively (into sub-directories, etc)
  • the “p” option preserves permissions and ownership of files and dirs
  • the “H” option shows the file sizes in human readable format
  • “sync” tells it to sync between the server and s3
  • “/home” is the directory I am recursively backing up
  • “s3://my_bucket ” is the bucket location in S3 where everything will be backed up to.

SSH Tunneling; encrypted surfing with Virtual Hosts

Encrypted SurfingWhether you’re the Secretary of Defense, or just an average Joe trying to survive with some peace of mind and security, encryption is a good thing. Have many virtual hosts on your unencrypted Apache server, but want encryption for whichever virtual host you specify? Here is the solution! Note, this is written for Linux clients – not Windows. You can tweak the instructions to work with Windows by using Putty and creating a tunnel that way.

First of all, here is the command to tunnel for Linux:

ssh -f -L 10000:your_virtual_host.com:80 user@myserver.com -N

Explanation of the above command:

  • ssh starts the ssh client
  • -f forks the ssh client into the background
  • -L forwards the command to the binded source_port:server:destination_port
  • -N tells ssh not to execute a command on the remote server once you are logged in to it

After you have started the tunnel using the command above, you will stay logged into it as long as the terminal is open.

Next, you would open your web browser and go to the following address with the address bar:

http://your_virtual_host.com:10000

By going to that address, it fails? What? Ohhhhh that’s right, you need to add the entry for that site to your hosts file:

Open the file:

vi /etc/hosts

(If you were doing this in Windows, the file is at C:WindowsSystem32driversetchosts)

Add this line to it:

127.0.0.1 your_virtual_host.com

Now go to the address again and it should work:

http://your_virtual_host.com:10000

If you were to run a packet sniffing program such as WireShark, you could monitor your network adapter (wlan0 or eth0 – whichever one you are using) and see that everything going to myserver.com (which is where you are tunneled into via ssh – using it to access you_virtual_host.com) is encrypted! Whoo hoo now you can log into your unencrypted website without worrying that people can see your plain text password going over the network.

If you were to monitor your loopback interface (lo) then you would see all the clear text data – except it never leaves your computer unencrypted.

NOTE: Once you have added your_virtual_host.com to your /etc/hosts file, it will always look for that domain on the local machine which means you need to open to tunnel to access it. As a result, if you try to ssh into your_virtual_host.com you will see that the connection is refused. The way around this is to ssh into a different domain on the same server (notice that I ssh’d into myhost.com instead of my_virtual_host.com).

Every host you add to the 127.0.0.1 line in the /etc/hosts file that is on your server will work on the same port (10000 or whatever port you specify – you can use any port you want that isn’t taken by another program). So, if you have my_other_domain.com on the server and, in your web browser, you go to http://my_other_domain.com then it will work as well with the existing tunnel.

View the next article for setting your computer to automatically start the tunnel when you log in.

If this works for you or not, please comment below and let me know.

Creating a website (for beginners)

If you are a beginner to creating a website, this is for you!

First Thing’s first

If you are new to web design and HTML, I want you to read this whole article. It is very handy. But first, follow these steps to see what HTML looks like:

– Right-Click on this web page right here —-> <— (right there).
– Select “View Page Source” (or “View Source” in Internet Explorer).

The “source” is all the HTML code that I have written either by hand, that was generated with Php, or that was created in a WYSIWYG (What You See is What You Get) editor.

What do I call my web pages?

When you are creating a website, you can name your web pages whatever you want. However, you need to use an appropriate extension on your web page so that the web server will recognize it.

Here is a list of some important page names, and some not so important page names:

index.html – This is the name of a page that most webservers recognize as the default page to load when a website is loaded. For example, when you go to www.google.com, you are actually going to www.google.com/index.html – you just don’t actually index.html unless you manually type it in.

index.htm – This is the same as index.html. The only difference is there is no “l” at the end. Why don’t you tell me why if you find out?

index.php – This is the same as index.html or index.htm – EXCEPT you use .php when you are creating pages with Php code. It does not hurt anything to use .php – even if you aren’t programming in Php.

contact.php – If you create a page where users can see your contact information, you can call it contact.php, or you can call it fried_chicken.php. As long as it has a .html, .htm, or .php extension (there are many other extensions out there in the world too), that’s all that matters.

How do I make Windows show me what extension a file is?

If you are using a Windows computer (like XP, Vista, or Windows 7), then chances are that you can create a file on your computer named “file1.txt” and, all you will see for its name is “file1.” You NEED to change this ASAP so that you do not have naming issues for your web pages. I highly recommend that you follow these steps regardless if you are creating a website or not:

  • On Windows XP, open your “My Documents” folder and click on “Tools” > “Folder Options”
  • On Windows Vista or 7, open your “Documents” folder and click on “Organize” > “Folder and Search Options”
  • Next, click on the “View” tab.
  • Uncheck the box next to “Hide Extensions for Known File Types.”

Now you should be able to view file extensions. If you use Windows XP and those steps didn’t quite work for you, please let me know through email so that I can update this tutorial.

How do I put my web pages on the internet?

To put web pages on the server, you use FTP (File Transfer Protocol). The easiest way to use FTP on a Windows machine is to follow these steps:

  • Open up your “Documents” folder (your “My Documents” folder for Windows XP users).
  • In the location bar at the top of the page, type in the following (exactly – don’t add or change anything):
  • ftp.kaslnetwork.com
  • When you are prompted, enter your username and password and sign in.
  • Now you can drag and drop your files from your computer to the FTP server.

Once you are signed in, you will see a directory called public_html. That is the directory that contains your websites home directory (like yourdomain.com). When you are in FTP, everything you see in that window is on a remote server – NOT on your computer.

Where on the server should my web pages go?
Once you are connected to the FTP server (using the step above), follow these steps:

  • Click on public_html (entering the public_html directory).
  • You will see a directory like yourdomain.com
  • Click on yourdomain.com (entering the yourdomain.com directory).

Any files that you put in the yourdomain.com directory will be viewable by anybody who is surfing the internet. Here is a good way to layout your files and directories:

*NOTE1: Never use spaces in your file or directory names. Always use _ or – when you have multiple words, or just run them together.
*NOTE2: On a web server, file names are CASE SENSITIVE. Therefore, File1.html is different than file1.html

yourdomain.com

  • images
  • frog_picture.png
  • dog_smiling.jpg
  • horseJumping.gif
  • includes
  • database_connection_script.php (if you are using Php web pages (like page1.php (instead of page1.html)), you can include pages into other pages)
  • functions.php
  • random_stuff

your_2nd_domain.com (if you have more than one website)

  • images
  • strangeLizard.png
  • FastRabbit.jpg
  • Crazy_Wierd_Lady.gif
  • includes
  • header.php
  • footer.php
  • connect.inc
  • random_stuff
  • What program should I use to create my web pages?

    You can use any program you want. Here are some examples (for more information, visit http://en.wikipedia.org/wiki/List_of_HTML_editors):

    Non-Free:

    • Adobe Dreamweaver
    • Microsoft Expression Web

    Free:

    See Web Tools at the kaslnetwork.com home page for a list

    How do I create a hyperlink to another page?

    A hyperlink is a bit of text that, when you click on it in a web browser, takes you to a different web page. This is a hyperlink. You can use hyperlinks anywhere you want in web pages. You can also turn images into hyperlinks. Say you want to have users taken to your friends website. Here is how you would do it:

     <a href="http://yourfriendswebsite.com">Your Friends Website</a>
     <a href="http://yourfriendswebsite.com/contact_friend.html">Your Friends Website</a>

    You can also send users to a page on your own website using a relative link (without the http://yourfriendswebsite part):

    <a href="contact.html">Contact Us</a>

    How to I put a picture in my web pages?

    In order to show images on the internet, you need to first upload the image to the web server using FTP.

    • Connect to the FTP server using the method above (or any method you know of).
    • Navigate to your images directory (or whatever you called it. It is recommended that you put all images in an images directory for organization).
    • Drag the image file from your computer into the FTP window into the images directory.

    Now the image is on the internet. You can view it at http://yourdomain.com/images/your_image.png. To put the image in a web page, write the following code:

    If you want to modify your image size or add a border, here are some options you can add:

    <img src="/images/image.png" style="border: 1px; height: 200px; width: 300px;">

    If you want your images to link to another web page, you can turn them into hyperlinks like this:

    <a href="url.com"><img src="/images/image.png" style="border: 1px; height: 200px; width: 300px;"></a>

    To force the link to open in a new window or tab, add target=”blank” – see below:

    <a href="url.com" target="blank"><img src="/images/image.png" style="border: 1px; height: 200px; width: 300px;"></a>

    Linux DNS Server Setup

    What is the Domain Name System(DNS? The DNS is the glue that keeps the Internet together. DNS translates domain name(example.com) to an IP address and vice versa. It’s much easier for us to remember names than numbers.

    DNS is defined in Request for Comments (RFCs) 1034 and 1035.

    Basically, any major Linux distro can be used as a DNS, Web, E-mail, and FTP server, and Suse is not an exception. All in one box and one static IP address. That’s all you need! Of course, I am talking about an experimental machine. An educative aproach. If you are seriuos about the Internet, you are going to need minimun two DNS servers, an email server, a web server, an FTP server, a firewall machine may be using ipcop.org, a proxy server, etc.

    The DNS server is very easy to install. All you need to install is the BIND package, modified the /etc/named.conf file, and create your own zones files.

    If you are using any type of firewall, keep port 53 open for UDP and TCP. This is the port dedicated to DNS.

    Edit /etc/named.conf and add your reference zone files. Right after the first reference zone file, in this case zone “0.0.127.in-addr.arpa”, add yours, in my case I added two zone references: enicaragua.net and sixbone.com. Pay close attention to the open and close braces, quotes, and semicolons.

    zone "0.0.127.in-addr.arpa" in {
      type master;
      file "127.0.0.zone";
    };

    zone "enicaragua.net" in {
      type master;
      file "enicaragua.zone";
    };

    zone "sixbone.com" in {
      type master;
      file "sixbone.zone";
    };

    Zones files
    Suse puts the named dir in /var/lib/named. Here is where your zones file will be located. This is the sixbone.com domain’s zone file:

    unix:/home/ipv6 # more /var/lib/named/sixbone.zone
    ; DNS record for the sixbone.com domain

    $TTL 86400 ; max TTL
    $ORIGIN sixbone.com.

    @ IN SOA unix.sixbone.com. root.sixbone.com.(
      2003021401
      28800
      7200
      604800
      3600
    )

    @ IN A 68.183.62.111
    @ IN NS unix.sixbone.com.
    @ IN MX 10 unix.sixbone.com.
    www IN CNAME sixbone.com.
    IN CNAME sixbone.com.

    Now, we need to test the zone files, so we type: rndc reload, next we run rcnamed start. You should get not erros. Next type rcnamed status, you should see the number of zones running, and the last line will say the server is up and running.

    If you get an error, check braces, quotes, and semicolos in /etc/named.conf.

    If you want to know more about CNAME, A, NS, etc. Read The DNS & Bind Book

    Have you seen those sites that if you don’t put the www, you cannot get to it? Well, it’s all in the zone files. If you check the sixbone.com file, you see one reference with the www and one reference without it, these are called CNAME or alias. Bingo! By the same token, you create the same way an FTP server, so on and so forth. In the case of a Web server, you need to check your Apache conf files, and make sure that you create your virtual hosts reference as well. This is another topic. Finally, I run the last check using dig, your DNS friend, and this is what I get:

    unix:/home/ipv6 # dig sixbone.com
    ; <<>> DiG 9.3.1 <<>> sixbone.com
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55238
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
    ;; QUESTION SECTION:
    ;sixbone.com. IN A
    ;; ANSWER SECTION:
    sixbone.com. 86194 IN A 68.183.62.111
    ;; AUTHORITY SECTION:
    sixbone.com. 86194 IN NS unix.sixbone.com.
    ;; Query time: 34 msec
    ;; SERVER: 66.51.205.100#53(66.51.205.100)
    ;; WHEN: Thu Jan 13 03:27:51 2005
    ;; MSG SIZE rcvd: 64

    DNS is an exciting topic, and these few lines don’t even put a dent to it. For example, you need to register your own domain name with a domain name service provider. i.e ns.com, buydomains.com, etc. By creating these zone files in your server, your DNS server is not going to work, I mean, it won’t resolve any domain name at all, may be locally.

    How the DNS work? Here is a link that explain very well how it works: How DNS worksEach domain name must have two DNS servers. Some people named their ns1.example.com and ns2,example,com, in my case I named mine unix.sixbone.com unix is the name of the machine itself, or host name in the UNIX jargon. The first reference in the link above, is the root level. These are a series of machines, 13 total to be exact, distributed around the world. I think Japan has two, Europe has a couple more, the US has the most, etc.

    What are the names of these machines? In your /var/lib/named there is a file called root.hint, this file contain the list of the these machines. Here is the file in my Linux box:

    unix:/home/ipv6 # more /var/lib/named/root.hint
    ; This file holds the information on root name servers needed to
    ; initialize cache of Internet domain name servers
    ; (e.g. reference this file in the "cache . "
    ; configuration file of BIND domain name servers).
    ;
    ; This file is made available by InterNIC
    ; under anonymous FTP as
    ; file /domain/named.root
    ; on server FTP.INTERNIC.NET
    ; -OR- RS.INTERNIC.NET
    ;
    ; last update: Jan 29, 2004
    ; related version of root zone: 2004012900
    ;
    ;
    ; formerly NS.INTERNIC.NET
    ;
    . 3600000 IN NS A.ROOT-SERVERS.NET.
    A.ROOT-SERVERS.NET. 3600000 A 198.41.0.4
    ;
    ; formerly NS1.ISI.EDU
    ;
    . 3600000 NS B.ROOT-SERVERS.NET.
    B.ROOT-SERVERS.NET. 3600000 A 192.228.79.201
    ;
    ; formerly C.PSI.NET
    ;
    . 3600000 NS C.ROOT-SERVERS.NET.
    C.ROOT-SERVERS.NET. 3600000 A 192.33.4.12
    ;
    ; formerly TERP.UMD.EDU
    ;
    . 3600000 NS D.ROOT-SERVERS.NET.
    D.ROOT-SERVERS.NET. 3600000 A 128.8.10.90
    ;
    ; formerly NS.NASA.GOV
    ;
    . 3600000 NS E.ROOT-SERVERS.NET.
    E.ROOT-SERVERS.NET. 3600000 A 192.203.230.10
    ;
    ; formerly NS.ISC.ORG
    ;
    . 3600000 NS F.ROOT-SERVERS.NET.
    F.ROOT-SERVERS.NET. 3600000 A 192.5.5.241
    ;
    ; formerly NS.NIC.DDN.MIL
    ;
    . 3600000 NS G.ROOT-SERVERS.NET.
    G.ROOT-SERVERS.NET. 3600000 A 192.112.36.4
    ;
    ; formerly AOS.ARL.ARMY.MIL
    ;
    . 3600000 NS H.ROOT-SERVERS.NET.
    H.ROOT-SERVERS.NET. 3600000 A 128.63.2.53
    ;
    ; formerly NIC.NORDU.NET
    ;
    . 3600000 NS I.ROOT-SERVERS.NET.
    I.ROOT-SERVERS.NET. 3600000 A 192.36.148.17
    ;
    ; operated by VeriSign, Inc.
    ;
    . 3600000 NS J.ROOT-SERVERS.NET.
    J.ROOT-SERVERS.NET. 3600000 A 192.58.128.30
    ;
    ; operated by RIPE NCC
    ;
    . 3600000 NS K.ROOT-SERVERS.NET.
    K.ROOT-SERVERS.NET. 3600000 A 193.0.14.129
    ;
    ; operated by ICANN
    ;
    . 3600000 NS L.ROOT-SERVERS.NET.
    L.ROOT-SERVERS.NET. 3600000 A 198.32.64.12
    ;
    ; operated by WIDE
    ;
    . 3600000 NS M.ROOT-SERVERS.NET.
    M.ROOT-SERVERS.NET. 3600000 A 202.12.27.33
    ; End of File

    The last machine listed in this root.hint file is locates in Paris, France. Just use visualroute.com to find the rest.

    Keep in mind that we are “inserting” a machine to the biggest network of the world: The Internet. And this machine must be capable to let everybody know regardless where they are how to find example.com, how to find this domain name and its contents. For a small local network, you don’t need a DNS server, you can use the first approach since the ARPANET’s days of resolving names to IP addresses: the hosts file located in /etc/hosts