In previous chapter we discussed about the various data sources. Now after you have acquired the data. Data could be in various file formats you cannot directly put it into model. The HEC accepts only DSS as the input. In DSS(data storage system) files you can put Gridded as well as time series data. To interact with these files we have HEC-DSSVue, and to see animation of the gridded data, we have HEC GridUtilities. Other softwares have their own file formats in which they take data e.g. MIKE has dfs0.
Now our motive is to convert the data files to DSS. Before that we should have some idea about few file formats.
.NC (NetCDF)
The NC files contains one or more types of data. You have to extract the exact variable which contains data of your interest.
GRIB or GRIB2
Grib and Grib2 files are another form but the data is highly compressed. This is in use nowadays. There are various grib decoders available on internet. Though I have my own code to do so, as it hardly needs one second to execute.
Tiff
Tif is a simple grid format containing only one type of gridded data. But the size of files is relatively larger as compared to NC and grib.
Sizes Comparison Grib<NC<Tif
There are other formats of files which are available such as HDF, BIN etc. But the concept is same.
When you are working with gridded data there are few things you should keep in mind.
1. Resolution
Resolution refers to the size of a single pixel(smallest unit) of gridded data. Higher resolution means more pixel density and better data.
TRMM/GFS/ECMWF Resolution
GPM/IMD Forecast Resolution
2. Frequency
Frequency of data plays a vital role as if frequency is less even good resulution data becomes unusable and may give bad results.
3. Latency
Latency is the time difference between each data burst or simply how often the data is updated according to the real time.
Latency is real time data is not preferred however latency in forecasted data is somewhat tolerable.
ModelBuilder
Every GIS platform provides way to automate tasks. ArcGIS has ModelBuilder QGIS also provides something similar which automates the repetitive tasks.
Scripts
To process data you need
a GIS platform or you can make your own scripts. Even with GIS platform the
system is not much faster as doing everything manually takes a lot of effort
and time. There are two things here which will ease the process
i) Modelbuilder in
ArcGIS/QGIS
ii) Customised scripts
I
have worked on both and will tell you about both of them in advanced chapters.
Using scripts you can schedule in windows task scheduler and they will work on
their own everyday. Your Data will be acquired, processed, analyzed and your
model will be run automatically everyday. Even it will email and text you about
the progress.
You can write your own scripts to do the same. If you have some knowledge of any language that's all you need. Scripts gives you more flexibility and advantages over the Modelbuilder or similar tools. A python scripts looks like:
In scripting you are free to choose any language, however some languages has easier way to do a certain job than the rest. I use python, batch scripts, javascript, excel macros etc. to automate my system.
The scripts you can add to user interface of software and schedule it or run them one by one. One type of data may contain one or more script. The figure below download ARG (Automatic Rain Gauge) and AWS (Automatic Weather Station) data and convert them into DSS. Different scripts have different jobs and different steps.
The DSS records made after the processing are shown in figure below:
With this I will end this chapter. In next chapter I will discuss about the models required to make an inflow, level forecasting model( or you can say a Real Time Decision Support System).
Comments
Post a Comment