[GIS] Publishing a service on ArcGIS Server 10.4 when publisher and the server are working out of different folders

arcgis-10.4arcgis-publisherarcgis-server

The deployment scenario I am trying to use is: The publisher's machine and the server are working out of different folders. I have my development file share (hostname dev.share.me for this example) that my publishers all used to store their data. Its structure is:

\\dev.share.me\ArcGIS
    \Data
        \Open
            \Mosaic1
                \Image1.tif
                \Image2.tif
                \...
            \Mosaic1.gdb
                ...
        \Restricted
            \Topic1.gdb
    \Desktop
        \Topic1.mxd
    \Services
        \Topic1.sd

Then on my ArcGIS Server (NOT dev.share.me, perhaps dev.arcgis.me, but definitely separate disconnected servers), I have:

C:\ArcGIS
    \Data

Which is registered using:

Publisher Folder Path: \\dev.share.me\ArcGIS\Data
Server Folder Path: C:\ArcGIS\Data

And I copy all the data from \\dev.share.me\ArcGIS\Data to C:\ArcGIS\Data on my server. Once the data is on the server I repair the mosaic dataset using the repair function to change \\dev.share.me\ArcGIS\Data to C:\ArcGIS\Data.

Finally, I open my map file (Topic1.mxd) and

  1. Click File -> Share As -> Service
  2. Choose Save a service definition file
  3. Click Next
  4. Choose No available connection (leave Include data in service definition when publishing unchecked)
  5. Click Next
  6. Click Continue
  7. Under Capabilities add WMS
  8. Click Stage

However, when I attempt to publish the service through ArcGIS Server Manager webapp, I get this error:

<Msg time="2016-03-18T10:46:47,272" type="SEVERE" code="8252" source="System/PublishingTools.GPServer" process="3816" thread="73" methodName="" machine="MYSERVER" user="" elapsed="">Instance of the service 'System/PublishingTools.GPServer' crashed. Please see if an error report was generated in 'C:\arcgisserver\logs\MYSERVER\errorreports'. To send an error report to Esri, compose an e-mail to ArcGISErrorReport@esri.com and attach the error report file.</Msg>
<Msg time="2016-03-18T10:47:05,468" type="SEVERE" code="8254" source="Server" process="3816" thread="1" methodName="" machine="MYSERVER" user="" elapsed="">The containing process for 'System/PublishingTools' job 'j0a3a8bbeeded4ef8b926c021d52d3b80' has crashed.</Msg>

I'm not sure how I am departing from the approach outlined in: The publisher's machine and the server are working out of different folders. Is ArcGIS Server really this failure prone, or am I doing something fundamentally wrong here?

It may be worth mentioning that I am also using a Web Adaptor, and AM publishing through the Web Adaptor URL, not the Server URL.

—- UPDATE 2016/03/22 —-
Per the suggestions by @KirkKuykendall, I tuned the log level to debug, and tried again. Now I get a stack trace in the log:

<Msg time="2016-03-22T12:32:23,421" type="DEBUG" code="9999" source="System/PublishingTools.GPServer" process="3816" thread="73" methodName="" machine="ASIAS-DEV-GIS" user="" elapsed="">java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
        java.net.SocketException: Connection reset
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:229)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227)
        at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179)
        at com.sun.proxy.$Proxy43.handleRequest(Unknown Source)
        at com.esri.arcgis.discovery.ejb.util.EJBBase.handleRequestBase(EJBBase.java:548)
        at com.esri.arcgis.discovery.ejb.impl.GPServerSyncBean.handleRequest(GPServerSyncBean.java:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
        at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
        at org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:164)
        at org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:92)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
        at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
        at org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:122)
        at org.apache.openejb.core.stateless.StatelessContainer._invoke(StatelessContainer.java:221)
        at org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:174)
        at org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:136)
        at org.apache.openejb.server.ejbd.EjbRequestHandler.doEjbObject_BUSINESS_METHOD(EjbRequestHandler.java:238)
        at org.apache.openejb.server.ejbd.EjbRequestHandler.processRequest(EjbRequestHandler.java:129)
        at org.apache.openejb.server.ejbd.EjbDaemon.processEjbRequest(EjbDaemon.java:196)
        at org.apache.openejb.server.ejbd.EjbDaemon.service(EjbDaemon.java:149)
        at org.apache.openejb.server.ejbd.EjbServer.service(EjbServer.java:71)
        at org.apache.openejb.server.ejbd.KeepAliveServer$Session.service(KeepAliveServer.java:213)
        at org.apache.openejb.server.ejbd.KeepAliveServer.service(KeepAliveServer.java:233)
        at org.apache.openejb.server.ejbd.EjbServer.service(EjbServer.java:66)
        at org.apache.openejb.server.ServicePool$2.run(ServicePool.java:91)
        at org.apache.openejb.server.ServicePool$3.run(ServicePool.java:120)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:209)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
        at java.io.DataInputStream.readByte(DataInputStream.java:265)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:215)
        ... 37 more
</Msg>

But I'm still stumped… The first time I ran it, it was exactly 60 seconds after the publish request showed up in the log, so I started thinking a timeout of some sort. But subsequent attempts showed crashes in less time, so I gave up on that…

Best Answer

Its a bug. I worked with ESRI support to provide them with a procedure to reproduce in their environment. If/when they decide to provide a fix for this issue, I will update this answer. For now, I guess I'm outta luck...

The bug has been logged and is BUG-000065570. ... client can also discuss this Bug with their account manager and let them know that issue is impacting productivity and needs to be escalated. Additionally if you or your client have any other questions for ESRI Support on this matter, you can always reply to this email and I will try to assist. At this time our case will be marked as resolved as there is no active troubleshooting for us to take