GeoServer – Unable to Publish a GeoTiff File in GeoServer Using Curl Tool

apachegeoservergeoserver-rest-apigeotiff-tifftomcat

I have a workspace in GeoServer with many shapefiles and postgis tables. Now I'm writing a script that will publish GeoTiff files in that workspace using curl tool. I'm trying to succeed that in two steps.
At first I create a coveragestore with the following command:

curl -u admin:geoserver -v -XPOST -H 'Content-type: application/xml' -d '<coverageStore><name>input.tif</name><workspace>myworkspacee</workspace><enabled>true</enabled><type>GeoTIFF</type><url>/usr/share/geoserver/data/data/myworkspace/input.tif </url></coverageStore> http://my_mv_ip/geoserver/rest/workspaces/myworkspace/coveragestores'

which works fine. And then I execute:

curl -u admin:geoserver -v -XPUT -H "Content-type: image/tiff"  \
--data-binary @input.tif   \
http://my_vm_ip/geoserver/rest/workspaces/myworkspace/coveragestores/input/input.geotiff

The output from both commands in command line is the following:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying my_mv_ip...
* Connected to my_mv_ip (my_mv_ip) port 80 (#0)
* Server auth using Basic with user admin`
> POST /geoserver/rest/workspaces/myworkspace/coveragestores HTTP/1.1
> Host: my_mv_ip
> Authorization: Basic YWRtaW46Z2Vvc2VydmVy
> User-Agent: curl/7.47.0
> Accept: */*
> Content-type: application/xml
> Content-Length: 249 
> 
* upload completely sent off: 249 out of 249 bytes
< HTTP/1.1 201 Created
< Date: Mon, 14 Aug 2017 13:11:28 GMT
< Server: Noelios-Restlet-Engine/1.0..8
< Location: http://my_mv_ip/geoserver/rest/workspaces/myworkspace    /coveragestores/input
< Transfer-Encoding: chunked
< 
* Connection #0 to host my_vm_ip left intact
*   Trying my_mv_ip...
* Connected to my_mv_ip (my_mv_ip) port 80 (#0)
* Server auth using Basic with user admin
> PUT /geoserver/rest/workspaces/myworkspace/coveragestores/input    /input.geotiff 
HTTP/1.1
> Host: my_mv_ip
> Authorization: Basic YWRtaW46Z2Vvc2VydmVy
> User-Agent: curl/7.47.0
> Accept: */*
> Content-type: image/tiff
> Content-Length: 44295940
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue

After about 20 minutes it raises the following error:

We are completely uploaded and fine
< HTTP/1.1 502 Bad Gateway
< Date: Mon, 14 Aug 2017 12:10:44 GMT
< Server: Apache/2.4.7 (Ubuntu)
< Content-Length: 311
< Content-Type: text/html; charset=iso-8859-1 
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Bad Gateway</title>
</head><body>
<h1>Bad Gateway</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
</p>
<hr>
<address>Apache/2.4.7 (Ubuntu) Server at my_vm_ip Port 80</address>
</body></html>
* Connection #0 to host my_vm_ip left intact

I tried some solutions from:
Adding a new Coveragestore in a certain Workspace by REST
and:

Programming Geoserver 2.0.2 to add a raster data store and layer without the UI

But with no result. I'm using GeoServer 2.7.4 Tomcat7 and Ubuntu 14.04.

My Apache logs:

[Mon Aug 14 17:14:04.785741 2017] [proxy:error] [pid 14922:tid             139798341732096] (104)Connection reset by peer: [client 176.92.115.36:34098]     AH01084: pass request body failed to [::1]:8080 (localhost)
[Mon Aug 14 17:14:04.785886 2017] [proxy_http:error] [pid 14922:tid    139798341732096] [client 176.92.115.36:34098] AH01097: pass request body     failed to [::1]:8080 (localhost) from 176.92.115.36 ()

A piece from my tomcat logs:

14 Aug 17:14:04 ERROR [geoserver.rest] - Could not determine format. Try  setting the Content-type header. 
14 Aug 17:14:04 ERROR [geoserver.rest] -

Best Answer

Finally I found the answer in my question. It helped me the following topic: http://osgeo-org.1560.x6.nabble.com/Geoserver-REST-Question-td3796644.html and the answer of @iant because I figured out that the solution should be in front of my eyes.

Instead of:

curl -u admin:geoserver -v -XPUT -H "Content-type: image/tiff"  \
--data-binary @input.tif   \
http://my_vm_ip/geoserver/rest/workspaces/myworkspace/coveragestores/input/input.geotiff

I had to type:

curl -u admin:geoserver -v -XPUT -H "Content-type: image/tiff"  \
--data-binary @input.tif   \
http://my_vm_ip/geoserver/rest/workspaces/myworkspace/coveragestores/input/file.geotiff

In other words, I had just to replace input.geotiff with file.geotiff. And the problem solved. :-)

Related Solutions

[GIS] Add external WFS layer to geoserver using REST API

You have to create a FeatureType before you can create a Layer (i.e. publish it). If you have already created a test layer then fetch the feature type of that one (like http://geoserver.ianturton.com/rest/workspaces/topp/datastores/bgs/featuretypes/test_uk_625k_mapped_feature.xml) but you should be able to delete quite a lot of that and leave it for GeoServer to fill in.

<featureType>
  <name>test_uk_625k_mapped_feature</name>
  <nativeName>test_uk_625k_mapped_feature</nativeName>
  <namespace>
    <name>topp</name>
    <atom:link xmlns:atom="http://www.w3.org/2005/Atom" rel="alternate" href="http://geoserver.ianturton.com/rest/namespaces/topp.xml" type="application/xml"/>
  </namespace>
  <title>uk_625k_mapped_feature</title>
  <description>net.opengis.wfs20.impl.AbstractTypeImpl@1f89241 (value: Just for testing, lang: &lt;unset&gt;)</description>
  <keywords>
    <string>uk_625k_mapped_feature</string>
    <string>features</string>
  </keywords>
  <nativeCRS class="projected">PROJCS[&quot;OSGB 1936 / British National Grid&quot;, 
  GEOGCS[&quot;OSGB 1936&quot;, 
    DATUM[&quot;OSGB 1936&quot;, 
      SPHEROID[&quot;Airy 1830&quot;, 6377563.396, 299.3249646, AUTHORITY[&quot;EPSG&quot;,&quot;7001&quot;]], 
      TOWGS84[446.448, -125.157, 542.06, 0.15, 0.247, 0.842, -20.489], 
      AUTHORITY[&quot;EPSG&quot;,&quot;6277&quot;]], 
    PRIMEM[&quot;Greenwich&quot;, 0.0, AUTHORITY[&quot;EPSG&quot;,&quot;8901&quot;]], 
    UNIT[&quot;degree&quot;, 0.017453292519943295], 
    AXIS[&quot;Geodetic longitude&quot;, EAST], 
    AXIS[&quot;Geodetic latitude&quot;, NORTH], 
    AUTHORITY[&quot;EPSG&quot;,&quot;4277&quot;]], 
  PROJECTION[&quot;Transverse_Mercator&quot;, AUTHORITY[&quot;EPSG&quot;,&quot;9807&quot;]], 
  PARAMETER[&quot;central_meridian&quot;, -2.0], 
  PARAMETER[&quot;latitude_of_origin&quot;, 49.0], 
  PARAMETER[&quot;scale_factor&quot;, 0.9996012717], 
  PARAMETER[&quot;false_easting&quot;, 400000.0], 
  PARAMETER[&quot;false_northing&quot;, -100000.0], 
  UNIT[&quot;m&quot;, 1.0], 
  AXIS[&quot;Easting&quot;, EAST], 
  AXIS[&quot;Northing&quot;, NORTH], 
  AUTHORITY[&quot;EPSG&quot;,&quot;27700&quot;]]</nativeCRS>
  <srs>EPSG:27700</srs>
  <nativeBoundingBox>
    <minx>-132576.78915711527</minx>
    <maxx>743466.6249987332</maxx>
    <miny>-15669.960592884949</miny>
    <maxy>1248847.1762802668</maxy>
    <crs class="projected">EPSG:27700</crs>
  </nativeBoundingBox>
  <latLonBoundingBox>
    <minx>-11.801005644638966</minx>
    <maxx>4.349696858682689</maxx>
    <miny>49.5245056714837</miny>
    <maxy>61.11839723246307</maxy>
    <crs>GEOGCS[&quot;WGS84(DD)&quot;, 
  DATUM[&quot;WGS84&quot;, 
    SPHEROID[&quot;WGS84&quot;, 6378137.0, 298.257223563]], 
  PRIMEM[&quot;Greenwich&quot;, 0.0], 
  UNIT[&quot;degree&quot;, 0.017453292519943295], 
  AXIS[&quot;Geodetic longitude&quot;, EAST], 
  AXIS[&quot;Geodetic latitude&quot;, NORTH]]</crs>
  </latLonBoundingBox>
  <projectionPolicy>FORCE_DECLARED</projectionPolicy>
  <enabled>true</enabled>
  <store class="dataStore">
    <name>bgs</name>
    <atom:link xmlns:atom="http://www.w3.org/2005/Atom" rel="alternate" href="http://geoserver.ianturton.com/rest/workspaces/topp/datastores/bgs.xml" type="application/xml"/>
  </store>
  <maxFeatures>0</maxFeatures>
  <numDecimals>0</numDecimals>
  <overridingServiceSRS>false</overridingServiceSRS>
  <skipNumberMatched>false</skipNumberMatched>
  <circularArcPresent>false</circularArcPresent>
  <attributes>
    <attribute>
      <name>uuid</name>
      <minOccurs>1</minOccurs>
      <maxOccurs>1</maxOccurs>
      <nillable>true</nillable>
      <binding>java.lang.String</binding>
    </attribute>
    <attribute>
      <name>lex</name>
      <minOccurs>1</minOccurs>
      <maxOccurs>1</maxOccurs>
      <nillable>true</nillable>
      <binding>java.lang.String</binding>
    </attribute>
    <attribute>
      <name>rcs</name>
      <minOccurs>1</minOccurs>
      <maxOccurs>1</maxOccurs>
      <nillable>true</nillable>
      <binding>java.lang.String</binding>
    </attribute>
    <attribute>
      <name>lex_rcs</name>
      <minOccurs>1</minOccurs>
      <maxOccurs>1</maxOccurs>
      <nillable>true</nillable>
      <binding>java.lang.String</binding>
    </attribute>
    <attribute>
      <name>the_geom</name>
      <minOccurs>0</minOccurs>
      <maxOccurs>1</maxOccurs>
      <nillable>true</nillable>
      <binding>com.vividsolutions.jts.geom.Geometry</binding>
    </attribute>
  </attributes>
</featureType>

Experiment and when it works update the documentation for the next person along.

[GIS] 405 Method not Allowed on uploading a shapefile via REST API on GeoServer

Try running it without the apache proxy first, that will rule out it being in the web server. You should be able to point to localhost:8000 in order to get to GeoServer directly.

The 405 error is unusual for GeoServer.

EDIT: Looking at the documentation, here, it seems that you should be using POST rather than PUT. Can you try that?

EDIT2: The documentation seems to be misleading, as, from testing, I can use PUT and not POST with the kind of URL you're using. But regardless, I've tested and this code works:

#!/usr/bin/env python3

import requests

shapefile = "/Users/alex/Desktop/example_shapefile.zip"
url = 'http://localhost:8080/geoserver/rest/workspaces/test_workspace/datastores/test_datastore/file.shp'

headers = {
    "Content-type": "application/zip"
}

with open(shapefile, 'rb') as data:
    r = requests.put(url, data=data, auth=('admin', 'geoserver'), headers=headers)

print(r.status_code, r.text)

I'm not sure what you're doing in PHP, but I certainly didn't get a 405 error when using PUT, though I did with a POST.

Note that I did get an error about 'magic numbers' if I didn't set the headers with the Content-type.

Best Answer

Related Solutions

[GIS] Add external WFS layer to geoserver using REST API

[GIS] 405 Method not Allowed on uploading a shapefile via REST API on GeoServer

Related Question