Classic Mode: This is nothing but Windows Authentication (NTLM or Kerberos). In the past, If any web application was created with Classic Mode Authentication then you must extend the web application to a new site if you also want to enable forms based authentication (FBA), and each login (NTLM/FBA) was unique. To configure this application to use Forms Based Authentication you can convert from Classic Mode to Claims Based. But, there is no UI exist for doing this conversion. The only way around is through PowerShell.
$app = get-spwebapplication "URL"
$app.useclaimsauthentication = "True"
$app.Update()
if an application is created using Claims based and if you want to convert from Claims based to Classic Mode then it is not possible either through UI or through PowerShell.
Claims Based: In SharePoint 2010 for a web application we can enable both windows, forms authentication. In earlier implementation to do this, we have to create two web applications which has different zones and different authentication. But, with the new claims based authentication a single application can have the capability to configure both windows and forms under single URL. All this is possible because of the authentication framework is built on Microsoft Identify Foundation. And it uses “Geneva” framework to handle this authentication.
