Optimizing your WordPress powered website is a continuous process. It depends on how heavily customized your WordPress site is, luckily the out of the box the theme I installed ranked average.

Check your site at Think With Google or GtMetrix to get baseline of where your current website stands.

This is my baseline stats.

I started with these and with 3 quick steps and it took me less than 40 mins.

Step 1:  Enable gzip compression on your website. Edit the .htaccess file and add the following

# Compress HTML, CSS, JavaScript, Text, XML and fonts
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
AddOutputFilterByType DEFLATE application/x-font
AddOutputFilterByType DEFLATE application/x-font-opentype
AddOutputFilterByType DEFLATE application/x-font-otf
AddOutputFilterByType DEFLATE application/x-font-truetype
AddOutputFilterByType DEFLATE application/x-font-ttf
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE font/opentype
AddOutputFilterByType DEFLATE font/otf
AddOutputFilterByType DEFLATE font/ttf
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml

# Remove browser bugs (only needed for really old browsers)
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
Header append Vary User-Agent

Step 2: Go into your theme folder and grab the css and js files and make a backup. Minify the existing css files at Minifier.org and upload them back, if nothing breaks you are gold.

Step 3: Optimize and compress images on your site, install the Smush It plugin. Free edition lets you process 50 images at a time.

With this 3 simple steps you can quickly do to increase your site performance. Then you can go further and implement more methods like shop around for a better host, remove unnecessary plugins, start using CDNs where you host your images and javascript libraries or install a caching plugin like W3 Total Cache.