Application Pages Site Pages
Application pages are stored in the server’s file system Site Pages is a concept where complete or partial page is stored within content database and then actual page is parsed at runtime and delivered to end-users. 
 SharePoint Designer tool cannot be used withapplication pages.  Site pages can be edited by using SharePointDesigner tool. 
Application pages cannot be used within sandboxed solutions. Site pages are used within Sandboxed solutions. 
An Application page cannot be customized and modified by end user, instead a developer is required. These are the normal .aspx pages deployed within SharePoint. A site page can be customized and modified by end user.
SharePoint specific features like Information Management Policies, Workflows, auditing, security roles can only be defined against site pages not against application pages. SharePoint specific features like Information Management Policies, Workflows, auditing, security roles can only be defined against site pages not against application pages.
These are typical ASP.Net aspx pages and can utilize all of the functionalities available within ASP.Net including code-behind, code-beside, inline coding etc.

These are compiled by .Net runtime like normal pages. If you deploy your custom ASPX pages within _layouts folder or within SharePointapplication using a virtual directory, you will not be able to use SharePoint master pages and have to deploy your master page within the virtual directory or _layouts folder. 

Since they are rendered not compiled hence it is not easy to add any inline code, code behind or code beside. Best way of adding code to these pages is through web-parts, server controls in master pages, user controls stored in “Control Templates” folder or through smart parts. If you want to add any inline code to master page, first you need to add  configuration within web.config: To add code behind to SharePoint master pages or page layouts.

Application page : Application pages are used to support application implementations inSharePoint Foundation. An application page is an ASP.NET Web page that is designed for use in a SharePoint Web site. Application pages are a specialized type of ASP.NET page.
Application pages are stored on the file system of the front-end Web server in the %ProgramFiles%Common FilesMicrosoft Sharedweb server extensions14TEMPLATELAYOUTS directory and exist for every site in a Web application. This folder is mapped to an Internet Information Services (IIS) virtual directory called _layouts. Every site and subsite will have access to the application pages by using the _layouts virtual directory. For example, http://myserver/_layouts/settings.aspx and http://myserver/subsite/_layouts/settings.aspx access the same application page on the front-end Web server unlike site pages, which are an instance for the specified site.
Application pages are not subject to the same restrictions as site pages. They allow in-line code without restriction. They cannot, however, use dynamic Web Parts or Web Part zones or be modified using SharePoint Designer. Modifying the default application pages is not supported inSharePoint Foundation. Custom application pages can be added to a subdirectory inside the _layouts folder.
Site Page : Site pages are pages that are created, edited, and customized by end users. They are primarily used for the content in a site. Site pages come in two types—a standard page and a Web Parts page. A standard page contains text, images, Web Parts, and other elements. A Web Parts page contains Web Parts in Web Part zones. They have a predefined layout that uses Web Part zones. Both types of site pages are edited using a Web browser or Microsoft SharePointDesigner.
Site pages are provisioned from a template page that is stored on the file system of the front-end Web server. When a site is provisioned, SharePoint Foundation creates a pointer to the instance of the page template on the file system. This allows SharePoint Foundation to avoid repeatedly creating copies of the pages, which are provisioned each time a site is created.
When a user customizes a site page, the template for the page is then stored in the contentdatabase. The page is retrieved from the content database every time it is requested by a user. A customized page can, however, be reset to the original template page through the Web browser or a tool such as SharePoint Designer.
Pages stored in Pages libraries or document libraries or at root level within SharePoint (Wiki pages) .
They are at user-level not at web-application or farm level and can be customized per site level.