Google have announced that page load time would be one of the ranking factors a lot time ago. Now this information is available in official webmaster manual. Page load time can be tasted on Google PageSpeed insights page.
In most cases, result of 80/100 can be scored simply by editing the web server settings by enabling deflate and caching mods. Under the cut there are .htaccess file and nginx configs, I’ve been using to optimize page load time for a long time. Make sure you also follow other recommendations provided by Google, because they can simnifically decrease page load time and thus improve your site visitors expirience.
If these configs do not satisfy you with performance you should consider using special modules developed by google, available both for nginx and apache at the google PageSpeed module page.