Tuesday, 18 November 2014

How to configure Load Balancing for 2 ISP

LAN: eth0: 192.168.0.1/24
ISP1: eth1: 192.168.1.1/24, gateway: 192.168.1.2/24
ISP2: eth2: 192.168.2.1/24, gateway: 192.168.2.2/24
So here is how I would do by using iptables method:

Route tables

First edit the /etc/iproute2/rt_tables to add a map between route table numbers and ISP names
...
10 ISP1
20 ISP2
...
So table 10 and 20 is for ISP1 and ISP2, respectively. I need to populate these tables with routes from main table with this code snippet (which I have taken from hxxp://linux-ip.net/html/adv-multi-internet.html)
ip route show table main | grep -Ev '^default' \
   | while read ROUTE ; do
     ip route add table ISP1 $ROUTE
done
And add default gateway to ISP1 through that ISP1's gateway:
ip route add default via 192.168.1.2 table ISP1
Do the same for ISP2
So now I have 2 route tables, 1 for each ISP.

Iptables

OK now I use iptables to evenly distribute packets to each route tables. More info on how this work can be found here (http://www.diegolima.org/wordpress/?p=36) and here (http://home.regit.org/?page%5Fid=7)
# iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
# iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT
# iptables -t mangle -A PREROUTING -j MARK --set-mark 10
# iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 20
# iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

NAT

Well NAT is easy:
# iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
# iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

Tuesday, 11 November 2014

How To Set Up Apache Virtual Hosts

Virtual Hosts are used to run more than one domain off of a single IP address. This is especially useful to people who need to run several sites off of one virtual private server. The sites display different information to the visitors, depending on with which the users accessed the site.There is no limit to the number of virtual hosts that can be added to a VPS.

Set Up

The steps in this tutorial require the user to have root privileges. You can see how to set that up in thehere Initial Server Setup. You can implement whatever username suits you.
Additionally, you need to have apache already installed and running on your virtual server If this is not the case, you can download it with this command:
sudo apt-get install apache2

Step One— Create a New Directory

The first step in creating a virtual host is to a create a directory where we will keep the new website’s information.
This location will be your Document Root in the Apache virtual configuration file later on. By adding a -p to the line of code, the command automatically generates all the parents for the new directory.
sudo mkdir -p /var/www/example.com/public_html
You will need to designate an actual DNS approved domain, or an IP address, to test that a virtual host is working. In this tutorial we will use example.com as a placeholder for a correct domain name.
However, should you want to use an unapproved domain name to test the process you will find information on how to make it work on your local computer in Step Seven.

Step Two—Grant Permissions

We need to grant ownership of the directory to the user, instead of just keeping it on the root system.
 sudo chown -R $USER:$USER /var/www/example.com/public_html 
Additionally, it is important to make sure that everyone will be able to read our new files.
 sudo chmod -R 755 /var/www
Now you are all done with permissions.

Step Three— Create the Page

Within our configurations directory, we need to create a new file called index.html
sudo nano /var/www/example.com/public_html/index.html
We can add some text to the file so we will have something to look at when the IP redirects to the virtual host.
<html>
  <head>
    <title>www.example.com</title>
  </head>
  <body>
    <h1>Success: You Have Set Up a Virtual Host</h1>
  </body>
</html>
Save and Exit

Step Four—Create the New Virtual Host File

The next step is to set up the apache configuration. We’re going to work off a duplicate—go ahead and make a copy of the file (naming it after your domain name) in the same directory:
 sudo cp /etc/apache2/sites-available/default /etc/apache2/sites-available/example.com

Step Five—Turn on Virtual Hosts

Open up the new config file:
 sudo nano /etc/apache2/sites-available/example.com
We are going to set up a virtual host in this file.
The first step is to insert a line for the ServerName under the ServerAdmin line.
  ServerName example.com 
The ServerName specifies the domain name that the virtual host uses.
If you want to make your site accessible from more than one name (for example, with www in the URL), you can include the alternate names in your virtual host file by adding a ServerAlias Line. The beginning of your virtual host file would then look like this:
<VirtualHost *:80>
        ServerAdmin webmaster@example.com
        ServerName example.com
        ServerAlias www.example.com
  [...]
The next step is to fill in the correct Document Root. For this section, write in the extension of the new directory created in Step One. If the document root is incorrect or absent you will not be able to set up the virtual host.
The section should look like this:
 DocumentRoot /var/www/example.com/public_html 
You do not need to make any other changes to this file. Save and Exit.
The last step is to activate the host, with the built in apache shortcut:
 sudo a2ensite example.com

Step Six—Restart Apache

We’ve made a lot of the changes to the configuration, and the virtual host is set up. However none of the changes that we made will take effect until Apache is restarted.
Use this command to restart apache:
 sudo service apache2 restart
You may see an error along the lines of
Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName 
The message is just a warning, and you will be able to access your virtual host without any further issues.

Optional Step Seven—Setting Up the Local Hosts

If you have pointed your domain name to your virtual private server’s IP address you can skip this step—you do not need to set up local hosts. Your virtual hosts should work. However, if want to try out your new virtual hosts without having to connect to an actual domain name, you can set up local hosts on your computer alone. For this step, make sure you are on the computer itself, not your droplet.
To proceed with this step you need to know your computer’s administrative password, otherwise you will be required to use an actual domain name to test the virtual hosts.
If you are on a Mac or Linux, access the root user (su) on the computer and open up your hosts file:
nano /etc/hosts 
If you are on a Windows Computer, you can find the directions to alter the host file on the Microsoft site
You can add the local hosts details to this file, as seen in the example below. As long as that line is there, directing your browser toward, say, example.com will give you all the virtual host details for the corresponding IP address.
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost

#Virtual Hosts 
12.34.56.789    example.com
However, it may be a good idea to delete these made up addresses out of the local hosts folder when you are done to avoid any future confusion.

Step Eight—RESULTS: See Your Virtual Host in Action

Once you have finished setting up your virtual host, you can see how it looks online. Type your ip address into the browser (ie. http://12.34.56.789)
It should look somewhat similar to my handy screenshot
Good Job!

Creating More Virtual Hosts

To add more virtual hosts, you can just repeat the process above, being careful to set up a new document root with the appropriate domain name, and then creating and activating the new virtual host file.

How To Set Up a MongoDB Database

1. Hard Drives

If you have a choice of selecting which hard drives you will have, go with enterprise grade dual SSD drives in RAID1. As we have covered before, they are great on performance and actually save you money.
Edit your /etc/fstab file in Linux and make sure to disable access time logging on your mount that will be used with MongoDB. Add noatime in 4th column:

Re-mount the partition:
[root@mongodb1 ~]# mount -o remount /
Verify that the new settings took effect:
[root@mongodb1 ~]# mount
/dev/sda on / type ext4 (rw,noatime)

2. CPU and Memory

Setting MongoDB as a VM on a hypervisor would let you scale up RAM and CPU cores later on. Amount of CPU cores and RAM that should be assigned depends on your infrastructure needs and budget.

3. Optimization

The most useful tip is to optimize your database queries:
  • Add indexes for commonly searched or sorted queries.
  • Use MongoDB’s explain() command.
  • Limit search results and limit fields that are being returned.
For testing purposes, we’ll spin up 3 droplets:

Installation

This procedure will be the same on mongodb1, mongodb2, and mongodb3. Installing MongoDB on CentOS is very simple. Add the following repository by editing
/etc/yum.repos.d/10gen.repo
[10gen]
name=10gen
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64
gpgcheck=0
enabled=1
Now install the packages:
[root@mongodb1 ~]# yum -y install mongo-10gen mongo-10gen-server
Enable MongoDB to start on reboot, and start the service:
[root@mongodb1 ~]# chkconfig mongod on && service mongod start
Starting mongod: forked process: 1387
all output going to: /var/log/mongo/mongod.log
child process started successfully, parent exiting
                                                           [  OK  ]
Now you should be able to see statistics on http://SERVER:28017/

Setting up Master-Slave replica set

We’ll assign mongodb1 as a master server. Add “master = true” to /etc/mongod.conf and do
service mongod restart

While mongodb2 and mongodb3 will be setup as slaves. Add “slave=true”, “source = mongodb1” to /etc/mongod.conf and do
service mongod restart

Now we should secure this database with a password or add iptables rules to ports 27017 (MongoDB) and 28017 (Web interface).
To create a user with password:
> use test
> db.addUser('admin', 'password');
{
        "user" : "admin",
        "readOnly" : false,
        "pwd" : "90f500568434c37b61c8c1ce05fdf3ae",
        "_id" : ObjectId("50eaae88790af41ffffdcc58")
}
We should also add firewall rules to restrict to other MongoDB servers, our IP, and save:
[root@mongodb1 ~]# iptables -N MongoDB
[root@mongodb1 ~]# iptables -I INPUT -s 0/0 -p tcp --dport 27017 -j MongoDB
[root@mongodb1 ~]# iptables -I INPUT -s 0/0 -p tcp --dport 28017 -j MongoDB
[root@mongodb1 ~]# iptables -I MongoDB -s 127.0.0.1 -j ACCEPT
[root@mongodb1 ~]# iptables -I MongoDB -s 192.34.57.64 -j ACCEPT
[root@mongodb1 ~]# iptables -I MongoDB -s 192.34.56.123 -j ACCEPT
[root@mongodb1 ~]# iptables -I MongoDB -s 192.34.57.162 -j ACCEPT
[root@mongodb1 ~]# iptables -A MongoDB -s 0/0 -j DROP
[root@mongodb1 ~]# /etc/init.d/iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
Repeat this procedure on your other MongoDB servers (mongodb2, mongodb3).
If you are using PHP for your frontend, you would need to install MongoDB module for PHP:
[root@webserver ~]# pecl install mongo
[root@webserver ~]# echo extension=mongo.so >> `php -i | grep /php.ini | awk '{print $5}'`
[root@webserver ~]# service httpd restart

Populate your database with data

Now we can begin testing our new setup. You can access the database from command shell by typing mongo :
[root@mongodb1 ~]# mongo
MongoDB shell version: 2.2.2
connecting to: test
Lets enter New York Times Bestsellers list into the database for testing:
> db.books.save( { title: 'Safe Haven', author: 'Nicholas Sparks' } )
> db.books.save( { title: 'Gone Girl', author: 'Gillian Flynn' } )
> db.books.save( { title: 'The Coincidence Of Callie And Kayden', author: 'Jessica Sorensen' } )
> db.books.save( { title: 'Fifty Shades of Grey', author: 'E.L. James' } )
> db.books.save( { title: 'Hopeless', author: 'Colleen Hoover' } )
To display all results:
> db.books.find()
{ "_id" : ObjectId("50eaaa4b633625147f205994"), "title" : "Safe Haven", "author" : "Nicholas Sparks" }
{ "_id" : ObjectId("50eaaa62633625147f205995"), "title" : "Gone Girl", "author" : "Gillian Flynn" }
{ "_id" : ObjectId("50eaaa8d633625147f205996"), "title" : "The Coincidence Of Callie And Kayden", "author" : "Jessica Sorensen" }
{ "_id" : ObjectId("50eaaaa0633625147f205997"), "title" : "Fifty Shades of Grey", "author" : "E.L. James" }
{ "_id" : ObjectId("50eaaab3633625147f205998"), "title" : "Hopeless", "author" : "Colleen Hoover" }
You should be able to see the same entries on mongodb2 and mongodb3 since they are a replica sets:

You could’ve entered all kinds of values for these books, such as publisher’s name, ISBN number, average customer rating, written language, and so on. In order to optimize your queries, however, it is best to limit number of results, and number of fields being returned.
For example, to return only 2 results we would use limit() at the end:
> db.books.find( {}, { title : 1 , author: 1 } ).sort( { timestamp : -1 } ).limit(2)
{ "_id" : ObjectId("50eaaa4b633625147f205994"), "title" : "Safe Haven", "author" : "Nicholas Sparks" }
{ "_id" : ObjectId("50eaaa62633625147f205995"), "title" : "Gone Girl", "author" : "Gillian Flynn" }
Once you have reached maximum capacity for your current setup, you can begin sharding your database. We will cover this in a future post.

How To Set Up Apache Virtual Hosts

The Apache web server is the most popular way of serving web content on the internet. It accounts for more than half of all active websites on the internet and is extremely powerful and flexible.
Apache breaks its functionality and components into individual units that can be customized and configured independently. The basic unit that describes an individual site or domain is called a virtual host.
These designations allow the administrator to use one server to host multiple domains or sites off of a single interface or IP by using a matching mechanism. This is relevant to anyone looking to host more than one site off of a single VPS.
Each domain that is configured will direct the visitor to a specific directory holding that site's information, never indicating that the same server is also responsible for other sites. This scheme is expandable without any software limit as long as your server can handle the load.
In this guide, we will walk you through how to set up Apache virtual hosts on an Ubuntu 14.04 VPS. During this process, you'll learn how to serve different content to different visitors depending on which domains they are requesting.

Prerequisites

Before you begin this tutorial, you should create a non-root user as described in steps 1-4 here.
You will also need to have Apache installed in order to work through these steps. If you haven't already done so, you can get Apache installed on your server through apt-get:
sudo apt-get update
sudo apt-get install apache2
After these steps are complete, we can get started.
For the purposes of this guide, my configuration will make a virtual host for example.com and another fortest.com. These will be referenced throughout the guide, but you should substitute your own domains or values while following along.
To learn how to set up your domain names with DigitalOcean, follow this link. If you do not have domains available to play with, you can use dummy values.
We will show how to edit your local hosts file later on to test the configuration if you are using dummy values. This will allow you to test your configuration from your home computer, even though your content won't be available through the domain name to other visitors.

Step One — Create the Directory Structure

The first step that we are going to take is to make a directory structure that will hold the site data that we will be serving to visitors.
Our document root (the top-level directory that Apache looks at to find content to serve) will be set to individual directories under the /var/www directory. We will create a directory here for both of the virtual hosts we plan on making.
Within each of these directories, we will create a public_html file that will hold our actual files. This gives us some flexibility in our hosting.
For instance, for our sites, we're going to make our directories like this:
<pre>
sudo mkdir -p /var/www/<span class="highlight">example.com</span>/publichtml
sudo mkdir -p /var/www/<span class="highlight">test.com</span>/public
html
</pre>
The portions in red represent the domain names that we are wanting to serve from our VPS.

Step Two — Grant Permissions

Now we have the directory structure for our files, but they are owned by our root user. If we want our regular user to be able to modify files in our web directories, we can change the ownership by doing this:
<pre>
sudo chown -R $USER:$USER /var/www/<span class="highlight">example.com</span>/publichtml
sudo chown -R $USER:$USER /var/www/<span class="highlight">test.com</span>/public
html
</pre>
The $USER variable will take the value of the user you are currently logged in as when you press "ENTER". By doing this, our regular user now owns the public_html subdirectories where we will be storing our content.
We should also modify our permissions a little bit to ensure that read access is permitted to the general web directory and all of the files and folders it contains so that pages can be served correctly:
sudo chmod -R 755 /var/www
Your web server should now have the permissions it needs to serve content, and your user should be able to create content within the necessary folders.

Step Three — Create Demo Pages for Each Virtual Host

We have our directory structure in place. Let's create some content to serve.
We're just going for a demonstration, so our pages will be very simple. We're just going to make anindex.html page for each site.
Let's start with example.com. We can open up an index.html file in our editor by typing:
<pre>
nano /var/www/<span class="highlight">example.com</span>/public_html/index.html
</pre>
In this file, create a simple HTML document that indicates the site it is connected to. My file looks like this:
<pre>
<html>
<head> <title>Welcome to <span class="highlight">Example.com</span>!</title> </head> <body><h1>Success! The <span class="highlight">example.com</span> virtual host is working!</h1> </body></html> </pre>
Save and close the file when you are finished.
We can copy this file to use as the basis for our second site by typing:
<pre>
cp /var/www/<span class="highlight">example.com</span>/publichtml/index.html /var/www/<span class="highlight">test.com</span>/publichtml/index.html
</pre>
We can then open the file and modify the relevant pieces of information:
<pre>
nano /var/www/<span class="highlight">test.com</span>/public_html/index.html
</pre> <pre> <html> <head> <title>Welcome to <span class="highlight">Test.com</span>!</title> </head><body> <h1>Success! The <span class="highlight">test.com</span> virtual host is working!</h1> </body></html> </pre>
Save and close this file as well. You now have the pages necessary to test the virtual host configuration.

Step Four — Create New Virtual Host Files

Virtual host files are the files that specify the actual configuration of our virtual hosts and dictate how the Apache web server will respond to various domain requests.
Apache comes with a default virtual host file called 000-default.conf that we can use as a jumping off point. We are going to copy it over to create a virtual host file for each of our domains.
We will start with one domain, configure it, copy it for our second domain, and then make the few further adjustments needed. The default Ubuntu configuration requires that each virtual host file end in .conf.

Create the First Virtual Host File

Start by copying the file for the first domain:
<pre>
sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/<span class="highlight">example.com</span>.conf
</pre>
Open the new file in your editor with root privileges:
<pre>
sudo nano /etc/apache2/sites-available/<span class="highlight">example.com</span>.conf
</pre>
The file will look something like this (I've removed the comments here to make the file more approachable):
<pre>
<VirtualHost *:80>
ServerAdmin webmaster@localhost DocumentRoot /var/www/html ErrorLog ${APACHELOGDIR}/error.log
CustomLog ${APACHELOGDIR}/access.log combined
</VirtualHost>
</pre>
As you can see, there's not much here. We will customize the items here for our first domain and add some additional directives. This virtual host section matches any requests that are made on port 80, the default HTTP port.
First, we need to change the ServerAdmin directive to an email that the site administrator can receive emails through.
<pre>
ServerAdmin <span class="highlight">admin@example.com</span>
</pre>
After this, we need to add two directives. The first, called ServerName, establishes the base domain that should match for this virtual host definition. This will most likely be your domain. The second, calledServerAlias, defines further names that should match as if they were the base name. This is useful for matching hosts you defined, like www:
<pre>
ServerName <span class="highlight">example.com</span>
ServerAlias <span class="highlight">www.example.com</span> </pre>
The only other thing we need to change for a basic virtual host file is the location of the document root for this domain. We already created the directory we need, so we just need to alter the DocumentRootdirective to reflect the directory we created:
<pre>
DocumentRoot /var/www/<span class="highlight">example.com</span>/public_html
</pre>
In total, our virtualhost file should look like this:
<pre>
<VirtualHost *:80>
ServerAdmin <span class="highlight">admin@example.com</span> ServerName <span class="highlight">example.com</span> ServerAlias <span class="highlight">www.example.com</span>DocumentRoot /var/www/<span class="highlight">example.com</span>/publichtml
ErrorLog ${APACHE
LOGDIR}/error.log
CustomLog ${APACHE
LOG_DIR}/access.log combined
</VirtualHost>
</pre>
Save and close the file.

Copy First Virtual Host and Customize for Second Domain

Now that we have our first virtual host file established, we can create our second one by copying that file and adjusting it as needed.
Start by copying it:
<pre>
sudo cp /etc/apache2/sites-available/<span class="highlight">example.com</span>.conf /etc/apache2/sites-available/<span class="highlight">test.com</span>.conf
</pre>
Open the new file with root privileges in your editor:
<pre>
sudo nano /etc/apache2/sites-available/<span class="highlight">test.com</span>.conf
</pre>
You now need to modify all of the pieces of information to reference your second domain. When you are finished, it may look something like this:
<pre>
<VirtualHost *:80>
ServerAdmin <span class="highlight">admin@test.com</span> ServerName <span class="highlight">test.com</span> ServerAlias <span class="highlight">www.test.com</span>DocumentRoot /var/www/<span class="highlight">test.com</span>/publichtml
ErrorLog ${APACHE
LOGDIR}/error.log
CustomLog ${APACHE
LOG_DIR}/access.log combined
</VirtualHost>
</pre>
Save and close the file when you are finished.

Step Five — Enable the New Virtual Host Files

Now that we have created our virtual host files, we must enable them. Apache includes some tools that allow us to do this.
We can use the a2ensite tool to enable each of our sites like this:
<pre>
sudo a2ensite <span class="highlight">example.com</span>.conf
sudo a2ensite <span class="highlight">test.com</span>.conf </pre>
When you are finished, you need to restart Apache to make these changes take effect:
sudo service apache2 restart
You will most likely receive a message saying something similar to:
 * Restarting web server apache2
 AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
This is a harmless message that does not affect our site.

Step Six — Set Up Local Hosts File (Optional)

If you haven't been using actual domain names that you own to test this procedure and have been using some example domains instead, you can at least test the functionality of this process by temporarily modifying the hosts file on your local computer.
This will intercept any requests for the domains that you configured and point them to your VPS server, just as the DNS system would do if you were using registered domains. This will only work from your computer though, and is simply useful for testing purposes.
Make sure you are operating on your local computer for these steps and not your VPS server. You will need to know the computer's administrative password or otherwise be a member of the administrative group.
If you are on a Mac or Linux computer, edit your local file with administrative privileges by typing:
sudo nano /etc/hosts
If you are on a Windows machine, you can find instructions on altering your hosts file here.
The details that you need to add are the public IP address of your VPS server followed by the domain you want to use to reach that VPS.
For the domains that I used in this guide, assuming that my VPS IP address is 111.111.111.111, I could add the following lines to the bottom of my hosts file:
<pre>
127.0.0.1 localhost
127.0.1.1 guest-desktop 111.111.111.111 <span class="highlight">example.com</span> 111.111.111.111 <span class="highlight">test.com</span> </pre>
This will direct any requests for example.com and test.com on our computer and send them to our server at 111.111.111.111. This is what we want if we are not actually the owners of these domains in order to test our virtual hosts.
Save and close the file.

Step Seven — Test your Results

Now that you have your virtual hosts configured, you can test your setup easily by going to the domains that you configured in your web browser:
<pre>
http://<span class="highlight">example.com</span>
</pre>
You should see a page that looks like this:
Apache virt host example
Likewise, if you can visit your second page:
<pre>
http://<span class="highlight">test.com</span>
</pre>
You will see the file you created for your second site:
Apache virt host test
If both of these sites work well, you've successfully configured two virtual hosts on the same server.
If you adjusted your home computer's hosts file, you may want to delete the lines you added now that you verified that your configuration works. This will prevent your hosts file from being filled with entries that are not actually necessary.
If you need to access this long term, consider purchasing a domain name for each site you need andsetting it up to point to your VPS server.

Conclusion

If you followed along, you should now have a single server handling two separate domain names. You can expand this process by following the steps we outlined above to make additional virtual hosts.
There is no software limit on the number of domain names Apache can handle, so feel free to make as many as your server is capable of handling.
<div class="author">By Justin Ellingwood</div>