Configuring a distributed installation of ArcGIS Server

Note: Distributed installations of ArcGIS Server are only available at the Enterprise level. The Workgroup level of ArcGIS Server is licensed for deployment on one machine only.

Contents

Introduction

Choose a configuration

Install the appropriate features on each machine

Run the post installs

Log off or restart each machine

Create the ArcGIS Web Services account on each SOC machine

Add users to the agsadmin and agsusers groups

Prepare any server directories that your server will use

Grant share permissions to the server directories

Grant file (NTFS) permissions to the server directories

Associate virtual directories with your server directories

Grant permissions to data directories

Configure the log directory

Connect to the GIS server

Specify the log directory location

Add server directories

Add SOC machines

Add services

Troubleshooting

Summary

Appendix A: Accounts and permissions diagram

Appendix B: Directory diagram

Appendix C: Common problems and error messages

Introduction

ArcGIS Server has a scalable architecture which allows for deployment sizes ranging from one to many machines. When you first install ArcGIS Server, you will likely choose to install all components on one machine for development and testing purposes. Once you are ready to deploy your ArcGIS Server application, you will need to consider a distributed installation of ArcGIS Server so that you can achieve an acceptable level of performance for the number of users accessing the system.

A distributed installation of ArcGIS Server is when components of one ArcGIS Server system reside on multiple machines in the same local network. For example, the graphic below depicts a distributed installation because the Server Object Manager (SOM), Server Object Containers (SOCs), and Web server reside on separate machines.


A distributed installation of ArcGIS Server can include many machines, which must be configured to communicate correctly

A distributed installation of ArcGIS Server allows you the flexibility to scale out your deployment by adding more machines. Since the container processes do the GIS work and typically consume the most CPU resources, each SOC machine that you add to your system increases the number of users that your GIS server can accommodate.

Wisely distributing the ArcGIS Server components among multiple machines can help you make the most efficient use of your hardware resources. For example, if you have a limited number of machines available, you might consider installing the SOM on the same machine as the Web server, since the SOM uses relatively little memory. Your remaining hardware can then be put to use as SOC machines to increase the processing loads that your GIS server can handle.

Performing a distributed installation of ArcGIS Server differs from a regular ArcGIS Desktop or ArcGIS Engine installation because you must configure multiple machines to correctly communicate with each other. The SOM must be able to send a request for a service to any SOC machine in the system. Since each machine is working with the same set of data and directories, a common naming convention (such as UNC paths) must be used so that each machine can refer to the data and directories in the same way.

Security mechanisms can also pose a challenge in communication between machines. For example, a SOC account can require permissions to read or write data on a number of different machines. Because of the open communication required by the ArcGIS Server architecture, firewalls are not recommended between the components of ArcGIS Server (such as between the Web server and the SOM, or between the SOM and a SOC). This paper contains an alternative recommendation for securing your system with firewalls.

Configuring a distributed installation of ArcGIS Server requires you to perform a series of important administrative tasks in the correct order. The purpose of this document is to help you through the process of a distributed installation.

Choose a configuration

The first step in deploying a distributed ArcGIS Server system is designing the configuration. The SOM, SOC, Web Applications, and Web Application Developer Framework (ADF) are ArcGIS Server features that can be installed on the same machine, or distributed among multiple machines. The ADF Runtime must be installed on the same machine as the Web server.

Your data must be available on the same local area network as your GIS server. If you are not using Manager to administer your server, ArcCatalog must also be available on the network. However, your data and ArcCatalog do not have to reside on the same machine as any other components of your GIS server.

The ArcGIS Server Installation Guide contains diagrams of several deployment configurations you might consider when planning your system.

The System Design Strategies document at www.esri.com/systemdesign also contains diagrams of recommended distributed installations of ArcGIS Server. Most of this information is in Chapter 4.0: GIS Product Architecture. Additionally, this paper contains information about deploying ArcSDE with ArcGIS Server.

A note about firewalls

ESRI does not recommend or support firewalls between ArcGIS Server components. The recommended technique for protecting an ArcGIS Server system with firewalls is to configure a reverse proxy Web server within a perimeter network (also known as a DMZ, demilitarized zone, or screened subnet). In this scenario, the reverse proxy Web server receives incoming HTTP requests through a firewall that restricts traffic to a known port (usually port 80). It then sends the request through another firewall—using a port unknown to the end user—to the ADF Web server. This Web server hosts your ArcGIS Server Web applications and services, and resides in a secure internal network. The ADF Web server is then free to establish unrestricted Distributed Component Object Model (DCOM) communications with the other ArcGIS Server components. In this way, the entire GIS server operates within a secure internal network, and does not require firewalls between its components.


An ArcGIS Server system protected by firewalls. The reverse proxy Web server receives the client request through a known port on the first firewall, then directs the request through a different port on the second firewall to the ADF Web server. The ADF Web server then forwards DCOM requests to the GIS server. The second firewall restricts access through any other port.

In some cases, the firewalls included with Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1 may need to be configured to work with ArcGIS Server. For more information, see ESRI Knowledge Base article 27798.

Install the appropriate features on each machine

The ArcGIS Server Installation Guide contains detailed information about the software installation process. It also contains a list of system requirements and prerequisites for ArcGIS Server. You can open the guide by clicking the Install Guide button from the first panel of the ArcGIS Server Setup wizard.


The ArcGIS Server Installation Guide

After installing any necessary prerequisites as detailed in the System Requirements, you can begin installing the ArcGIS Server software on each machine in your system. As you navigate through the ArcGIS Server Setup wizard, you will see a panel that allows you to choose which components, or features, of ArcGIS Server to install. For each machine, choose only the features necessary for the machine to perform its function in the system.


Selecting which features of ArcGIS Server will be installed

Run the post installs

ArcGIS Server 9.2 has two post installation wizards: the GIS Server Post Install and the Web Applications Post Install. Follow the guidelines below to understand which post installs are required by each machine:

More about the GIS Server Post Install

The GIS Server Post Install has two parts: Configure ArcGIS Server, and Authorize ArcGIS Server. The sections of the GIS Server Post Install you need to complete on each machine may vary. For example, you will only need to authorize ArcGIS Server on machines that will function as SOCs. For a machine on which you installed the SOM only, the authorization portion of the GIS Server Post Install will be disabled.

During the GIS Server Post Install, you will be prompted to provide names and passwords for the accounts used by the GIS server, namely the SOM, SOC, and ArcGIS Web Services accounts. To understand what these accounts do, and best practices for configuring them, see Accounts used by the GIS server.

You should enter the same SOM, SOC, and ArcGIS Web Services account information on each machine that you run the post install. The accounts must have the same name and password on each machine. The GIS Server Post Install gives you the option to save a configuration file that contains the names and passwords that you entered for the accounts. When you run the GIS Server Post Install on other machines, you can use the configuration file to quickly load the same name and password information.


This panel of the GIS Server Post Install prompts you to specify the SOM and SOC accounts

For security purposes, ESRI recommends that you make the SOM and SOC accounts local instead of specifying domain accounts. This ensures that a malicious user could not use the accounts to acquire administrative privileges on other machines in your network.

Note that in Windows Computer Management, the full names of the SOM and SOC accounts are the ArcGIS Server Object Manager Account and ArcGIS Server Object Container Account, respectively. When granting permissions to these accounts in Windows, the full name will appear.

For additional information on the GIS Server Post Install, see Step 4a of the ArcGIS Server Install Guide.

More about the Web Applications Post Install

You will need to run the Web Applications Post Install on the Web server machine. This should be the machine on which you installed the Web Applications component of ArcGIS Server. If the post install does not appear automatically, you can launch it from the Start menu.

The primary purpose of the Web Applications Post Install is to link the Web server and the SOM in what is known as an ArcGIS Server instance. In large deployments of ArcGIS Server, configuring multiple instances can be a good way to organize the server system so that it can use a variety of licensing levels, security models, or application groups.

Consequently, the first thing you need to provide in the Web Applications Post Install is the name of the instance. The default name is ArcGIS. If you change the default, you should be aware that it will change many of the default examples of URL structure and paths to server files mentioned in the ArcGIS Server Help.

Where prompted to enter the GIS Server, enter the name of the machine running the Server Object Manager (SOM) component of ArcGIS Server.

When prompted to enter the ArcGIS Web Services account, keep in mind the guidelines from Accounts used by the GIS Server. You will rarely work with this account after running the post installs, and in most cases it’s sufficient to accept the default. You must enter the same account here that you entered when you ran the GIS Server Post Install on the SOM.

Log off or restart each machine

In order for the account settings created by the post installs to take effect, you need to log off and back on to each machine in your system before continuing to configure your ArcGIS Server system.

Create the ArcGIS Web Services account on each SOC machine

Note: The following step applies if you are using a local account for the ArcGIS Web Services account. If you are using a domain account, you can skip the next paragraph.

The ArcGIS Web Services account must be accessible to each SOM, SOC, and Web server machine in your ArcGIS Server configuration. The post installs accomplish the task of creating the ArcGIS Web Services account on the SOM and Web server. However, if you have machines in your configuration that have the SOC component only installed, you must use the operating tools to manually create the ArcGIS Web Services account on those machines. Use the same name and password that you used for the ArcGIS Web Services account on the SOM and Web server. On the SOC, you do not need to give this account any special permissions, nor do you need to add it to the agsadmin or agsusers groups.

Add users to the agsadmin and agsusers groups

After running the appropriate post installs on each machine, you need to specify which users will have administrative and regular-usage access to your server. The GIS Server Post Install creates two operating system groups on the SOM: agsadmin and agsusers. The agsadmin group is for administrators: for example, those who will add SOC machines and services to the server. You need to add yourself, and anyone else who will be administering the server, to the agsadmin group on the SOM machine.

The agsusers group is for those who will be making local connections to the GIS server, but do not need administrative access. You need to populate the agsusers group with a list of authorized users. Accounts already in the agsadmin group do not need to be added to agsusers.

You do not need to add the SOM and SOC accounts to the agsadmin and agsusers groups. These accounts are only used internally by the GIS server.

Prepare any server directories that your server will use

The GIS server makes use of three types of directories: Cache, Jobs, and Output. The server uses these directories to store map and globe caches, manage geoprocessing jobs, and write temporary files and output map images, respectively.

Each SOC machine in your system needs to be able to access the server directories. To make this possible, you can configure the folders representing your server directories to be shared, so that other machines on the network can access them. Suppose you have a folder on your hard drive at the path C:\ArcGIS\server_output which you want to make available to any computer on the network. You can share the folder and make it available to other users via a Universal Naming Convention (UNC) path. In the example above, for a machine named ‘myServer’, this path would take the form of \\myServer\server_output. Any machine on the network could access the folder using this naming convention.


You can share a folder in Windows by using the Sharing tab on the folder's Property page

When you choose to share a folder in Windows, you need to specify share permissions and file permissions (sometimes called NTFS permissions). Share permissions describe the levels of access different users will have to the folder. File permissions describe what the users can do with the folder’s contents. When a user attempts to access the folder, share permissions are considered first, followed by file permissions. In the case of conflict between the two permissions, the strictest permission is applied.

Grant share permissions to the server directories

For all server directories—cache, jobs, and output—you need to allow the SOC account at least Change level share permissions and the SOM account Full Control share permissions. You can set the share permissions in the folder’s Properties window. On the Sharing tab, click the Permissions button to view and edit the folder’s share permissions.


Granting share permissions in Windows XP

Grant file (NTFS) permissions to the server directories

Server cache, jobs, and output directories require that the SOC account have at least Read and Write file (NTFS) permissions. Additionally, the SOM account must have Full Control file permissions to these directories. You can set file permissions in the folder’s Properties window, on the Security panel.


Granting file permissions in Windows XP

Associate virtual directories with your server directories

A virtual directory allows Internet users to access the contents of a folder on your computer via a URL. When you associate a virtual directory with your server directory, you allow your Web applications to make use of the server directory’s contents.

You create a virtual directory on your Web server machine, using your Web server’s administration software; however, the server directory itself does not have to be on the same machine as the Web server. Remember the following points when creating a virtual directory:


Verifying that a virtual directory has Read access enabled in IIS 5

Grant permissions to data directories

The services created by ArcGIS Server rely on existing GIS resources. GIS resources are map documents, address locators, geodatabases, tools, and globe documents that you create using ArcGIS Desktop. All SOC machines must have access to this data. There are two options for ensuring that all SOC machines can access the data:

Option 1: Maintain one copy of the data in a shared folder. All SOC machines will access the data in this folder, using a UNC path.

Option 2: Maintain identical copies of the data on each SOC machine, using an identical folder structure. You can then use local paths to refer to the data. This configuration is potentially faster, since a SOC will never have to retrieve data from a different machine; however, it can be difficult to maintain if the data changes often. Also, this option is impractical for large datasets, map and globe caches, and data that will be edited.

For both options, you need to give the SOC account permissions for each data folder in the same way that you granted the SOC account access to your server directories.

If a folder contains data that will be used by a service, you need to do the following:

  1. If the folder is shared, grant Change share permissions to the SOC account for the folder.
  2. Grant Read and Write file permissions to the SOC account for the folder.

Steps on how to grant permissions are included earlier in this document.

These steps apply not only to folders that contain the source documents, but also to folders that contain data referenced within the document. Suppose you have a map document that displays two data layers. If the map document and the data reside in different directories, you should grant permissions as described in the steps above for both the folder containing the map document, and the folder containing the data.

A simpler solution would be to store the map document and its data in the same folder. You could then use relative paths within the map document to reference the data. This way you would only have to grant permissions for one folder.

Accessing ArcSDE data

If your data is accessed through ArcSDE, you need to make sure that your name and password are saved in the Database Connection. For detailed help with accessing ArcSDE data through ArcGIS Server, see the topic Preparing resources for publishing. This topic also explains what permissions to give to the SOC account if you're using a Database Server, and discusses what to do if your data is stored on a machine where no other components of ArcGIS Server are installed.

Configure the log directory

To aid server administration and troubleshooting, the server writes log files to a specified location. The default location is <ArcGIS install location>\Server\user\log, and the ArcGIS Server installation gives the SOM and SOC accounts permissions to this directory. These permissions are sufficient for an ArcGIS Server installation on one machine; however, for a distributed installation you need to make some extra configurations:

  1. Share the log directory.
  2. Ensure that the SOM account has Change level share permissions to the log directory.

If you want the logs to be written to a directory other than the default, you should follow the same steps listed above. Additionally, ensure that the SOM account has Read and Write file permissions to the log directory. (The GIS Server Post Install granted these permissions automatically for the default directory). If you do not configure the directory correctly, the server will write the logs to the default location.

Connect to the GIS server

At this point you are ready to connect to the GIS server. You can use either Manager or ArcCatalog to connect to and administer the server. Manager, which gets installed when you select the “Web Applications” component, should be installed on the Web server machine. ArcCatalog does not have to be installed on the same machine as any of your other ArcGIS Server components; it just has to be on the same local network and not behind any firewalls.

If you are using Manager to administer your server, see Logging into Manager.

If you are using ArcCatalog to administer your server, see Connecting to a GIS server for instructions on how to make an administrative connection.

Specify the log directory location

Since the server log directory defaults to a local path, you need to change it to a UNC path when implementing a distributed installation. Instructions about sharing the log folder and granting it the appropriate permissions are available earlier in this document. You can specify the log directory location in Manager or ArcCatalog. At the initial release of ArcGIS Server 9.2, you must use ArcCatalog the first time you change the default log file location.


Specifying the log directory location in ArcCatalog

Add server directories

Once you’ve connected to the server, you can specify one or more server directories that it can access. The Server Properties window contains a Directories tab where you can add server directories. Before you add a server directory, you should have created it in the file system and configured its sharing and permissions as explained earlier in this document. When you enter the name of the directory, be sure to use a UNC path.

In most deployments, you should create at least one cache directory, one jobs directory, and one output directory. Normally, these directories should have associated virtual directories as explained previously in this document.

See these ArcGIS Server Help topics for instructions on creating server directories:

When you create a map service or edit its properties, you can choose which server output directory it will use. If you select 'None', map images will be accessed using MIME data. You can also specify the server cache directory that a map service will use.

Similarly, when you configure geoprocessing services, you need to select a server jobs directory. This is where the server writes the results of geoprocessing jobs.

Add SOC machines

ArcGIS Server cannot function without machines that host SOC processes. Before you can create and use services, you need to inform the SOM which machines will function as SOCs.

Before adding the SOC machines to your server, make sure that you have run the GIS Server Post Install on each machine and have granted the SOC account permissions to the data and server directories that it will need to access.

To add a SOC machine, follow the instructions in the applicable ArcGIS Server Help topic:

Add services

With a connection to the server established and server directories, data directories, and SOC machines in place, you can begin publishing GIS resources on your server as services. To add a service, follow the directions in Adding a new service.

When you specify the GIS resource for the service (for example, a map document for a map service), you can use either:

Troubleshooting

With the many deployment possibilities of ArcGIS Server, the multiple steps involved in setting up the system, and the uniqueness of environments in each organization, troubleshooting is often a necessary part of the install process. An important source of help for troubleshooting will be the log files. These are found in <ArcGIS install location>\Server\user\log. For information on how to set the logging level and interpret the log files, see How log files work

Appendix C contains a list of problems or error messages that you may encounter when working with ArcGIS Server—especially during the distributed installation process—and suggested solutions. If you don’t see the problem there, also consult Common problems and solutions.

Summary

ArcGIS Server has a scalable architecture which allows for a distributed installation among any number of machines. The most direct way to add computing power to your GIS server is by adding server object container (SOC) machines. The machines in an ArcGIS Server system must be able to freely communicate with each other, unhindered by firewalls, file naming conflicts, or restricted permissions to data and folders.

To facilitate a successful distributed installation of ArcGIS Server, remember the following key points:

Appendix A: Accounts and permissions diagram

Setting up a distributed installation involves running post installs, creating users, and managing operating system groups on multiple machines. Below is a guide that shows what you’ll need to do on each machine. Each machine in the diagram contains some green text denoting which post install you must run on that machine. Items in blue are accomplished by the post install. Items in red are things that you must do. Note especially that you must manually add the ArcGIS Web services account on each dedicated SOC machine.

Appendix B: Directory diagram

ArcGIS Server uses various directories for reading and writing information. These directories:

This diagram displays the directories used by the GIS server, and the steps you need to take to configure them properly.


Appendix C: Common problems and error messages

Below is a list of common problems and error messages you might encounter when configuring ArcGIS Server. Error messages are shown in quotes.

Usually these appear when you first attempt to create a service. Error messages may appear in the log files, on your screen, or both.

When previewing a service in ArcCatalog, you see a white screen. The coordinates adjust correctly when you move the mouse, but nothing is visible.

This problem can occur if your map document contains invalid data layers. Open the document in ArcMap to verify that the data source paths are correct for all layers.

You may also see a blank screen when the server output directory is configured correctly, but its associated virtual directory is not. There are several ways the virtual directory could be configured incorrectly that would yield this result:

To troubleshoot these issues, open IIS Manager and verify that the virtual directory exists, points to the server output directory where your data resides, and has the correct permissions applied.

Server configuration and/or services are lost when ArcGIS Server Object Manager service is restarted

If you need to restart the SOM service, an error may occur with the MSXML parser version. ESRI Knowledge Base article 29524 gives more details.

"Access denied" or "The connection could not be made"

These messages sometimes appear as parts of the more detailed messages listed below. You can find general support for these errors at ESRI Knowledge Base article 29042.

"Server object instance creation failed on all SOC machines. Server Object instance creation failed on machine X."

This message occurs when the server fails to create a service. Usually this message is followed by additional information. Check the other errors in this section to see if one of them matches the additional information in your error message.

"Machine X is not a valid server container. Error: (-8001) You are not licensed for ArcGISServer."

This message appears when you attempt to use a SOC machine that has not been properly authorized. Verify that you have run the “Authorize GIS Server” portion of the GIS Server Post Install on all SOC machines. You can also open your license file with a text editor and verify that:

If any of the above two conditions are not met, you will need to request a new file from ESRI Customer Service.

"Access to output directory is denied."

This error can appear when you forget to give the SOC account appropriate access to your server output directory. You can either give the SOC account at least Read and Write permissions to the output directory (remember to give it both share and file permissions), or reconfigure the service so that it does not use an output directory.

See also ESRI Knowledge Base article 26554.

"The connection could not be made. Access denied: The SOM service on machine X is not started and does not allow startup by this user."

This error can appear if you fail to log out and back in to your machine after running the post installation. Certain DCOM permissions settings related to the SOM and SOC accounts are modified during the post installation. You need to log out, then log back in, for these to take effect.

"Access denied: The SOM service is not registered on machine X"

This error appears when the SOM component is not installed on the machine that you are attempting to connect to. If you’ve installed the SOM, verify that you have run the GIS Server Post Install, followed by logging off and back on to your machine.

"The request method (GET, POST, etc.) was not allowed for this particular resource"

One cause of this error is attempting to make an ArcGIS Server Internet connection to a machine that is not configured correctly for Web services. For example, this message can appear if the SOM component is not installed on the machine you are connecting to.

This message will also appear if you type an invalid instance name in the URL. For example, if you’re trying to connect to Server1 with the default instance name of ArcGIS and you type http://Server1/AArcGIS/services, you will likely get this error message.

"Couldn’t resolve host. The given remote host was not resolved"

This error can appear when you type in an invalid URL when attempting to make an ArcGIS Server Internet connection. It is most likely due to an error in the machine name part of the URL.

"Machine X is not a valid server container. Error: 0x80070005 (Access is denied.)"

This error appears when you are attempting to add a SOC machine to your server. Verify that you have installed the SOC component of ArcGIS Server on the machine you are attempting to add. This error message will also appear if you have not run or completed the ArcGIS Server Post Installation on the SOC machine, or if each SOC machine does not have identical names and passwords for the SOC account.

"Server context creation failed on machine X. File Y does not exist"

The most likely cause of this error is that the server cannot find the file you specified as the GIS resource for your service, such as a map document. Check the Service Properties to make sure that a valid path to the data was entered. Then, check the data itself to make sure that the SOC account has appropriate share and file permissions to access it, as described earlier in this document. You may also try checking your server output directory to make sure that the SOC account has appropriate permissions for it, and that any virtual directories pointing to the server output directory are configured correctly.

"Your selection cannot be displayed in the current view"

This error appears in the Preview tab of ArcCatalog. This can occur when you fail to specify the same user name and password for the SOM and SOC accounts across all machines in your system. As a result the server cannot establish a connection to one or more SOC machines.

This message will also appear when you attempt to preview a map service that is stopped or paused. Right-click the service, click Start, and the preview should appear in the Preview tab.

"Machine X is not a valid server container. Error: 0x80040154 (Class not registered)"

The probable cause of this error is that a machine you are trying to add as a SOC host does not have the SOC component of ArcGIS Server installed. To correctly configure the SOC machine, install the SOC software and run the Post Installation, ensuring that the SOM and SOC accounts have the same names and passwords as they do on the other machines in your configuration. Then log out and back in to the machine.

"The connection could not be made. GIS server is not running on machine X"

This error may actually signify that ArcCatalog cannot locate machine X. Make sure that you have entered the name of a valid machine on your network that would be accessible to the machine running ArcCatalog.

"The selection cannot be previewed"

This message appears in the Preview window of ArcCatalog if an irrelevant node is clicked, such as the GIS server name, or a GIS server folder. It will also appear if you attempt to preview a geodata or geoprocessing service. These services have no direct graphical output and cannot be previewed in ArcCatalog.