Opened 9 years ago
Closed 9 years ago
#2892 closed defect (fixed)
t.rast.aggregate: error when input space time raster dataset store many rasters
| Reported by: | tmsz | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.0.4 |
| Component: | Temporal | Version: | 7.0.2 |
| Keywords: | wingrass, t.rast.aggregate, g.message | Cc: | |
| CPU: | x86-64 | Platform: | MSWindows 7 |
Description
Steps to reproduce (follow example from Soren Gebbert presentation: The temporal GRASS GIS framework, page 53):
- download the ECAD climate dataset from http://eca.knmi.nl/download/ensembles/data/Grid_0.25deg_reg/ as several netCDF files (tg_0.25deg_reg_1950-1964_v12.0.nc.gz, tg_0.25deg_reg_1965-1979_v12.0.nc.gz, tg_0.25deg_reg_1965-1979_v12.0.nc.gz, tg_0.25deg_reg_1995-2015_v12.0.nc.gz) and import them with r.in.gdal in a Lat/Lon location.
- import netCDF files
r.in.gdal -oe input=tg_0.25deg_reg_1950-1964_v5.0.nc output=temperature_mean offset=0
r.in.gdal -oe input=tg_0.25deg_reg_1965-1979_v12.0.nc output=temperature_mean offset=5480
r.in.gdal -oe input=tg_0.25deg_reg_1980-1994_v12.0.nc output=temperature_mean offset=10959
r.in.gdal -oe input=tg_0.25deg_reg_1995-2015_v12.0.nc output=temperature_mean offset=16439
total count of temperature_mean rasters = 23925
t.create type=strds output=temperature_mean_1950_2015_daily temporal=absolute title="European mean temperature 1950-2015" description="The European daily mean temperature"
- create text file with all temperature_mean rasters
python code:
import os map_list = os.path.join(os.path.expanduser('~'), 'map_list.txt') with open(map_list, 'w') as f: for i in range(23925): f.write('temperature_mean.{0}\n'.format(i + 1))
t.register -i type=rast input=temperature_mean_1950_2015_daily file="path to the map_list.txt file" start=1950-01-01 increment="1 days"
g.region -p rast=temperature_mean.1
t.rast.aggregate input=temperature_mean_1950_2015_daily method=average output=temperature_mean_1950_2015_monthly base=temperature_mean_monthly granularity="1 months"
Error message:
Traceback (most recent call last):
File "C:\Program Files (x86)\GRASS GIS
7.0.2/scripts/t.rast.aggregate.py", line 215, in <module>
main()
File "C:\Program Files (x86)\GRASS GIS
7.0.2/scripts/t.rast.aggregate.py", line 185, in main
nprocs))
File "C:\Program Files (x86)\GRASS GIS
7.0.2\etc\python\grass\script\core.py", line 580, in info
message(msg, flag='i')
File "C:\Program Files (x86)\GRASS GIS
7.0.2\etc\python\grass\script\core.py", line 551, in message
run_command("g.message", flags=flag, message=msg,
errors='ignore')
File "C:\Program Files (x86)\GRASS GIS
7.0.2\etc\python\grass\script\core.py", line 396, in
run_command
ps = start_command(*args, **kwargs)
File "C:\Program Files (x86)\GRASS GIS
7.0.2\etc\python\grass\script\core.py", line 364, in
start_command
return Popen(args, **popts)
File "C:\Program Files (x86)\GRASS GIS
7.0.2\etc\python\grass\script\core.py", line 67, in __init__
subprocess.Popen.__init__(self, args, **kwargs)
File "C:\Program Files (x86)\GRASS GIS
7.0.2\Python27\lib\subprocess.py", line 711, in __init__
errread, errwrite)
File "C:\Program Files (x86)\GRASS GIS
7.0.2\Python27\lib\subprocess.py", line 953, in
_execute_child
startupinfo)
WindowsError: [Error 206] The filename or extension is too
long
Error appear in the core.py file, class Popen (row 67). I try debug subprocess.py file. Error realated with creating process (subprocess.py file, method _subprocess.CreateProcess, row 946) when g.message.exe command is callled (created process) with long message argument as args parameter in method _subprocess.CreateProcess (attached file with args argument).
On Windows OS platform is command-line string limitation for total length of the following command line https://support.microsoft.com/en-us/kb/830473.
Attachments (1)
Change History (7)
by , 9 years ago
| Attachment: | args.tar.xz added |
|---|
follow-ups: 2 3 comment:1 by , 9 years ago
Hi,
I followed your workflow but I wasn't able to reproduce the error you get. I'm using grass71 (r67703) under Fedora 23. Have you tried with the new grass gis stable version 7.0.3?
Btw, your offsets in r.in.gdal didn't work for me, they left one map out and then it was not possible to register maps. So, here is how I changed them (and the total count is now 23922):
r.in.gdal -oe input=tg_0.25deg_reg_1950-1964_v12.0.nc output=temperature_mean offset=0 r.in.gdal -oe input=tg_0.25deg_reg_1965-1979_v12.0.nc output=temperature_mean offset=5479 --o r.in.gdal -oe input=tg_0.25deg_reg_1980-1994_v12.0.nc output=temperature_mean offset=10957 --o r.in.gdal -oe input=tg_0.25deg_reg_1995-2015_v12.0.nc output=temperature_mean offset=16436 --o
For the rest, the monthly aggregation works just fine here and no error regarding filename or extension. Have you tried shrinking names? Something like: basename=month_tmean?
comment:2 by , 9 years ago
Hi,
This error appear on OS Windows platform only. Offset for import rasters can be different on OS GNU/Linux. I think problem is not with raster name length.
When I run t.rast.aggregate, which inner create process (in the source code for create new process is called python method _subprocess.CreateProcess() on OS Windows platform) with argument which is command-line calling g.message command with argument message which is very long. On Windows OS platform is command-line string limitation for total length of the following command line (last previous link).
short preview, how is g.message called on OS Windows:
"C:\Program Files (x86)\GRASS GIS 7.0.2\bin\g.message.EXE" -i "message=granularity_list: [<grass.temporal.space_time_datasets.RasterDataset object at 0x040968F0>, ...
Check the text file (complete g.message command calling) in the attached previous archive.
comment:3 by , 9 years ago
Hi again,
I appologize for my bad offset value. Your offset value are correct.
follow-up: 5 comment:4 by , 9 years ago
Hi
No problem... But that was not the issue anyway, right? So, I really recommend to update to the last stable release and try again (perhaps there has been some change with regards to the version you are using), and maybe also try with shorter names.
I'm not a programmer, so unfortunately I cannot further help here... we need a specialist :)
comment:5 by , 9 years ago
Hi again,
I tried last stable grass gis (wingrass) 7.0.3, code revision 67691 and t.rast.aggregate work correctly now. Thank you for your help and time :).
comment:6 by , 9 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
args argument text file