To understand the need for a ArcSDE service, we need to understand the architecture better. And to do that, we need to go back in time to the version 8.3 onwards.
When you run the installation for ArcSDE (or PostInstall pre v10.1), it creates a repository in the database, which are basically supporting tables for your main data. (From 10.1 onwards there is no PostInstall, as I understand)
When you add a new featureclass in your SDE geodatabase, it is not stored as just one table. The Information is spread across multiple tables. When your version data, and edit it, even more tables are affected.
For the data to be saved in the correct tables, the client, or whichever software is saving the data, needs to know about these specific tables, and do things correctly. Even when reading from the geodatabase, the information has to be read from various tables and collected together to make a feature class as we know it.
Now your ArcGIS desktop and ArcGIS Server are smart enough to do that. But what about those clients that are not aware of all these different tables?
That is where the ArcSDE service comes in. It acts as an intermediary between the clients and the RDBMS back-end. You want to query the data from the command line? Sure you can do so by connecting to the service.
In the past ArcGIS desktop had limited knowledge of the RDBMS back-end. Over time ESRI has made ArcGIS desktop more and more aware of the repository tables, and given it the ability to manage the data directly. That is why nowadays there is very little need for the SDE service. ESRI has been pushing its clients to direct connect from the time of 9.3 onwards. And in the latest version making a three-tier connection is a pain.
(Ok so I sort of cheated here. This isn't the only reason for having A service for SDE, but it is the Main reason)
So to summarize, Unless you face some major problem in your workflow, you are not missing anything by not using a service. Using a Direct Connect is the way to go.
Now coming to your question about how to access the map services outside your network. (Actually this should be a seprate question, but I'll give you a short answer).
The Server is supposed to be installed on a seprate machine, which has access to the internet. Either it can be in the DMZ or your firewalls needs to be configured such that it allows some connections to your ArcGIS server. Your WebApp will the access your ArcGIS server using its public IP or hopefully a domain name.
SDE is a middle tier software that translates calls from ESRI, or other, clients into SQL that is appropriate for a certain database. If you are connecting to a geodatabase, then SDE also manages a certain amount of metadata that is stored in that database's SDE or dbo schema. What is ArcSDE
For Vector data on SQL Server 2008, there are actually spatial types provided by Microsoft which you can be used to store your data. But by default, SDE will create tables using a storage type called SDEBinary or just Binary. This is basically storing the geometry as binary data in a side table (F-Table), and maintaining a pseudo-index table to improve performance (S-Table). I call it a pseudo-index table because it isn't an index at the database level.
I don't think Microsoft has provided a type that can store rasters in the database (other then a BLOB). SDE can add an ST_Raster type, but that type will not be added by default.
When you run the post-install program, or the enable or create geodatabase GP tools at later releases, you are adding the tables in the SDE or dbo schema that track feature classes and tables, along with other functionality such as versioning and archiving. Additional tables are added by the Geodatabase to track feature database, subtypes, topologies, etc.
The difference between direct connect(2-tier) and application service connection(3-tier) is where the SDE process is running. That is, where the translation between the SDE calls from the client and the DBMS calls from SDE are actually being translated. With a direct connection, that process is running within the client application (ArcMap, ArcCatalog, ArcIMS, etc). With an application service connection, that same process is started by a separate service called the giomanager. This giomanager can run on any machine, but is commonly run on the same machine that is hosting the database. An Overview of ArcSDE Geodatabase Connections
So, to answer your questions:
Question1: It depends on how you are connecting. If in your connection dialog box you are entering a port number into the instance field, you are running application service connection. If you are entering something like 'sde:sqlserver:myHostName', then you are running a direct connection. Creating spatial database connections - Scroll down to 'Adding a direct connection to a geodatabase in SQL Server'.
Question2: It depends. There are many things that can cause performance issues for both SQLServer, DBMS and file based rasters. But in general, file based rasters outperform those stored in databases. This is because the architecture of a database, and ArcSDE, isn't really designed to store and retrieve giant chunks of binary data. Similarly, large and externally complex vector data does not perform well either. Rasters stored locally will outperform those stored remotely. But all else being equal, I would expect that a raster stored in a remote file geodatabase to perform more slowly then one stored in an SDE instance, which would perform slower then one stored as a tiff in a remote directory, which would be slower than one stored in Image Server. Geodatabase and ArcSDE Raster Basics
Best Answer
A very good discussion of whether to use ArcSDE or not can be found here.
At 10.1 there is no need to install the ArcSDE software unless you need to run an ArcSDE service. If all of your users are making Direct Connections to the geodatabase then the ArcSDE installation is not necessary. As well, most of the functionality offered by ArcSDE commands is now available in ArcGIS Desktop & through GP tools.To make a long story short, the vast majority of ArcGIS 10.1 users no longer need ArcSDE to connect to an enterprise geodatabase. Direct Connection is now the default connection method.