getting blank tile while using mod_tile with mapnik to serve geotiff file

[pdpsinghr]

Hi, I stuck in one issue where i have geotiff files and i want to use mod_tile but problem is after trying everything i am still getting balnk tile in api response but i tride to generate same tile using python and mapnik i am able to do so.

i am adding all information here

1. mapnik.xml

<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
<Style name="raster-style">
<Rule>
        <RasterSymbolizer opacity="1.0" scaling="bilinear">
                <Colorizer type="linear" epsilon="0.01">
                        <stop color="black" value="0"/>
                        <stop color="white" value="255"/>
                 </Colorizer>
        </RasterSymbolizer>
</Rule>
</Style>
    <Layer name="GeoTIFF" status="true" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
        <StyleName>raster-style</StyleName>
        <Datasource>
            <Parameter name="type">gdal</Parameter>
            <Parameter name="file">/usr/share/renderd/mod_tile_poc/AQ.tif</Parameter>
            <Parameter name="estimate_extent">true</Parameter>
            <Parameter name="log">/usr/share/renderd/mod_tile_poc/mapnik.log</Parameter>
            <Parameter name="debug">debug</Parameter>
        </Datasource>
    </Layer>
</Map>

2. /etc/renderd.conf

[renderd]
pid_file=/run/renderd/renderd.pid
socketname=/run/renderd/renderd.sock
stats_file=/run/renderd/renderd.stats
tile_dir=/var/cache/renderd/tiles

[mapnik]
font_dir=/usr/share/fonts
font_dir_recurse=0
plugins_dir=/usr/lib/mapnik/3.0/input

[default]
URI=/osm_tiles/
TILEDIR=/var/lib/mod_tile
XML=/usr/share/renderd/example-map/mapnik.xml
HOST=0.0.0.0
TILESIZE=256

[example-map]
URI=/tiles/renderd-example
XML=/usr/share/renderd/example-map/mapnik.xml

[example-map-jpg]
TYPE=jpg image/jpeg jpeg
URI=/tiles/renderd-example-jpg
XML=/usr/share/renderd/example-map/mapnik.xml

[example-map-png256]
TYPE=png image/png png256
URI=/tiles/renderd-example-png256
XML=/usr/share/renderd/example-map/mapnik.xml

[example-map-png32]
TYPE=png image/png png32
URI=/tiles/renderd-example-png32
XML=/usr/share/renderd/example-map/mapnik.xml

[example-map-webp]
TYPE=webp image/webp webp
URI=/tiles/renderd-example-webp
XML=/usr/share/renderd/example-map/mapnik.xml

3. this is info about geotiff file

Driver: GTiff/GeoTIFF
Files: AQ.tiff
Size is 7220, 2810
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-180.500000000000000,84.099607528087063)
Pixel Size = (0.050000000000000,-0.050000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (-180.5000000,  84.0996075) (180d30' 0.00"W, 84d 5'58.59"N)
Lower Left  (-180.5000000, -56.4003925) (180d30' 0.00"W, 56d24' 1.41"S)
Upper Right ( 180.5000000,  84.0996075) (180d30' 0.00"E, 84d 5'58.59"N)
Lower Right ( 180.5000000, -56.4003925) (180d30' 0.00"E, 56d24' 1.41"S)
Center      (   0.0000000,  13.8496075) (  0d 0' 0.01"E, 13d50'58.59"N)
Band 1 Block=7220x1 Type=Int32, ColorInterp=Gray
  NoData Value=-255

[hummeltech]

Hello @pdpsinghr, It looks like your mapnik.xml file may be mal-formed, can you provide your renderd --foreground output?

Here's an example of our valid example mapnik.xml file:
https://github.com/openstreetmap/mod_tile/blob/c3453bc1e19247c145b0e87742710d5e076e338e/utils/example-map/mapnik.xml

[pdpsinghr]

Hi @hummeltech

I'm exploring ways to dynamically serve GeoTIFF files with mod_tile, where the hour parameter is provided via an API URL. I tried using a custom script in renderd.conf (ModTileXMLConfigEndpoint /usr/renderd/custom_script.py), but encountered issues. Do you know of any alternative methods or approaches?

i keep last 48 hours geotiff so the url will be dynamic along with the mapnik.xml.

[hummeltech]

@pdpsinghr, I would recommend you look into writing a "parameterization" function for this, here's a comment I wrote recently explaining it a little bit:
#396 (comment)

In your case, you might add a new function which would change the source GeoTIFF based on the URL path.

[pdpsinghr]

@hummeltech I am new to this one ,
so...
one another questions is do i have to handle all the changes in src/parameterize_style.cpp, like on basis of url change the geotiff url in mapnik in parameterize_map_language function only? or is there any plugin i can add?

[hummeltech]

@pdpsinghr, you would just need to create the function and add a configuration to your renderd.conf file. The value in the request will be passed via the parameter function parameter.
E.g.:
Create your new function in /src/parameterize_style.cpp:

 static void parameterize_geotiff_by_date(mapnik::Map &m, char *parameter) 
 {
 	char *geotiff_date = strdup(parameter);
 	# modify `mapnik::Map &m` to use `geotiff_date` in the `DataSource` file name
 }

Add a reference to it in init_parameterization_function in /src/parameterize_style.cpp:

parameterize_function_ptr init_parameterization_function(const char *function_name)
{
	if (strcmp(function_name, "") == 0) {
		g_logger(G_LOG_LEVEL_DEBUG, "Parameterize_style not specified (or empty string specified)");
		return NULL;
	} else if (strcmp(function_name, "language") == 0) {
		g_logger(G_LOG_LEVEL_DEBUG, "Loading parameterization function for '%s'", function_name);
		return parameterize_map_language;
	} else if (strcmp(function_name, "geotiff_by_date") == 0) {
		g_logger(G_LOG_LEVEL_DEBUG, "Loading parameterization function for '%s'", function_name);
		return parameterize_geotiff_by_date;
	} else {
		g_logger(G_LOG_LEVEL_WARNING, "unknown parameterization function for '%s'", function_name);
	}

	return NULL;
}

Add a configuration to renderd.conf:

[geotiff_by_date]
PARAMETERIZE_STYLE=geotiff_by_date
TILEDIR=/var/lib/mod_tile
XML=/usr/share/renderd/example-map/mapnik.xml

[pdpsinghr]

if i define renderd.conf file like this

[ats_tile]
URI=/tiles/ats_tile/2024_05_20_08
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_08_ats_tile.xml

URI=/tiles/ats_tile/2024_05_20_09
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_09_ats_tile.xml

URI=/tiles/ats_tile/2024_05_20_10
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_10_ats_tile.xml

URI=/tiles/ats_tile/2024_05_20_11
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_11_ats_tile.xml

then from cache i getting wrong tile mostly from different hour not for the hour specified
and in cache directory also it is on basis of x,y,z not on basis of hour

can we store cache according to hour and then go by x, y, z?

[hummeltech]

If you are going to create multiple map sections in your renderd.conf file, you will need to add the section headers:

[ats_tile_2024_05_20_08]

URI=/tiles/ats_tile/2024_05_20_08
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_08_ats_tile.xml

[ats_tile_2024_05_20_09]

URI=/tiles/ats_tile/2024_05_20_09
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_09_ats_tile.xml

[ats_tile_2024_05_20_10]

URI=/tiles/ats_tile/2024_05_20_10
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_10_ats_tile.xml

[ats_tile_2024_05_20_11]

URI=/tiles/ats_tile/2024_05_20_11
XML=/usr/share/renderd/example-map/ats_tile/2024_05_20_11_ats_tile.xml

then from cache i getting wrong tile mostly from different hour not for the hour specified
and in cache directory also it is on basis of x,y,z not on basis of hour

can we store cache according to hour and then go by x, y, z?

[cagelight]

Hello, I realize this is an old issue, but I've been encountering the same issue and have narrowed it down to 100% an interaction between renderd and the mapnik library, across many version. Even just a minimal mapnik.xml and renderd.conf that only should draw a georeferenced TIFF creates blank tiles.

[hummeltech]

@cagelight, Can you provide your minimal mapnik.xml and renderd.conf and georeferenced TIFF?

I can use it for testing and add it as a test.

[cagelight]

@hummeltech Sorry for the delay, I must have had notifications disabled.

We were using these for testing, confirmed valid referenced TIFFs:
https://www.naturalearthdata.com/downloads/50m-raster-data/50m-natural-earth-2/

And pretty much any minimal mapnik.xml will do, we tried many, many variants to no avail. So I'd be interested to see which (if any) work for you on the other hand.
But just an example of something we tried:

<?xml version="1.0" encoding="utf-8"?>
<Map srs="+init=epsg:4326">
  <Style name="raster">
    <Rule>
      <RasterSymbolizer/>
    </Rule>
  </Style>

  <Layer name="raster" status="on" srs="+init=epsg:4326">
    <StyleName>raster</StyleName>
    <Datasource>
      <Parameter name="type">gdal</Parameter>
      <Parameter name="file">/data/tiff/test.tif</Parameter>
    </Datasource>
  </Layer>
</Map>

[hummeltech]

@cagelight , Thanks, I'll add some tests and see if I can figure out what is going on.

[hummeltech]

@cagelight , you will need to reproject your GeoTIFF files to EPSG:3857 because EPSG:4326 is not a supported projection in mod_tile:

mod_tile/src/gen_tile.cpp

Lines 115 to 151 in e8ac8f1

	if (strstr(srs, "+proj=merc +a=6378137 +b=6378137") != NULL) {
	g_logger(G_LOG_LEVEL_DEBUG, "Using web mercator projection settings");
	prj = (struct projectionconfig *)malloc(sizeof(struct projectionconfig));
	prj->bound_x0 = -20037508.3428;
	prj->bound_x1 = 20037508.3428;
	prj->bound_y0 = -20037508.3428;
	prj->bound_y1 = 20037508.3428;
	prj->aspect_x = 1;
	prj->aspect_y = 1;
	} else if (strcmp(srs, "+proj=eqc +lat_ts=0 +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs") == 0) {
	g_logger(G_LOG_LEVEL_DEBUG, "Using plate carree projection settings");
	prj = (struct projectionconfig *)malloc(sizeof(struct projectionconfig));
	prj->bound_x0 = -20037508.3428;
	prj->bound_x1 = 20037508.3428;
	prj->bound_y0 = -10018754.1714;
	prj->bound_y1 = 10018754.1714;
	prj->aspect_x = 2;
	prj->aspect_y = 1;
	} else if (strcmp(srs, "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs") == 0) {
	g_logger(G_LOG_LEVEL_DEBUG, "Using bng projection settings");
	prj = (struct projectionconfig *)malloc(sizeof(struct projectionconfig));
	prj->bound_x0 = 0;
	prj->bound_y0 = 0;
	prj->bound_x1 = 700000;
	prj->bound_y1 = 1400000;
	prj->aspect_x = 1;
	prj->aspect_y = 2;
	} else {
	g_logger(G_LOG_LEVEL_WARNING, "Unknown projection string, using web mercator as never the less. %s", srs);
	prj = (struct projectionconfig *)malloc(sizeof(struct projectionconfig));
	prj->bound_x0 = -20037508.3428;
	prj->bound_x1 = 20037508.3428;
	prj->bound_y0 = -20037508.3428;
	prj->bound_y1 = 20037508.3428;
	prj->aspect_x = 1;
	prj->aspect_y = 1;
	}

Here's a command to do so (adapted from here):

$ gdalwarp -s_srs EPSG:4326 -t_srs EPSG:3857 -r bilinear -te -20037508.34 -20037508.34 20037508.34 20037508.3 NE2_50M_SR.tif NE2_50M_SR_web.tif

@pdpsinghr, this method might also resolve your issue.

[cagelight]

@hummeltech This did indeed fix my problem. Thank you for pointing us in the right direction!