If your web application isn't supported on a shared hosting platform and a VPS doesn't give you enough memory and hard disk storage then a dedicate physical server may be an option. This means you pay for an actual computer server, in a rack, in a data centre.

Advantages 

There are many advantages to having a dedicate server, most are the same as if you had the server in your own premises. You have 100% of the resources available for your own use and as a single server can handle 100's or 1000's of web sites and email boxes you are unlikely to overload it.

Disadvantages 

However, choosing a dedicated server brings with it all the responsibilities of running a server in house unless you want to ay for the ISP to perform the services for you. You'll have to source or pay for at least the following;

• A firewall to protect it, this is usually an extra cost
• Server monitoring, if a dedicated server goes down you're the one fixing it unless you have a support contract. 
• Operating system and application level patching
• General maintenance like log rotation, disk space monitoring etc.

Summary 

Generally we don't recommend dedicated servers unless you need lots of capacity and have a full time member of staff to keep an eye on it or have budget to pay the ISP or us to manage it for you.