About half of web users expect a site to load in 2 seconds or less. If your website isn’t loaded within the minimum amount of time, then the users tend to leave the site and move on to other website.
Which means you’re not only losing your current visitors, but also decreasing conversion rates. In addition you run the risk of loosing traffic from those customers who may have referred your website to others.