Okay. This is new to me. I just learned few things on setting up VPS and host a live website on it.
Just a bookmark. Thanks to all original author.

1. This link to install LAMP.

2. Then install phpmyadmin. At one time, you might need to change php.ini to suit upload_max_filesize, in case of importing existing database.

3. Then install FTP server.

4. Install nano if you have to.

5. To uninstall phpmyadmin, just in case you need to do it.

6. To create new user and grant privileges in MySql.

7. Adding virtual host to the apache.

8. If required, add new user on Ubuntu with the home path.

9. Then in case web browser complaining about “you don’t have permission to access“.

10. To enable modrewrite if you are using .htaccess.
Notice the use of “sudo a2enmod rewrite”, then restart apache.

11. And some changes to the virtual host settings.

Don’t forget to check against /var/log/apache2/error.log if you stuck to any steps.