This article describes how to create a new pipeline to receive Earth observation data at regular intervals from the EarthCache dashboard.
In this section, you will learn how to:
|< Previous: review how to select an output||Next: learn how to get results and code >|
Defining the settings for your first pipeline
To select the settings for your pipeline, simply head over to the Settings tab.
Date range: Archival and tasking
Pipelines can be set to look for data in the past, such as images of a forest before a wildfire, or data to be collected in the future, for example the same forest next year to look at vegetation re-growth.
Data that has been previously collected is often referred to as “archival data”. Delivery times for archival data through EarthCache range from 6 minutes to up to 48 hours, based on the data provider. We are working hard to continue to improve delivery times for you.
Entering a start date
Entering a start date is mandatory. This will be the date after which EarthCache will look for data matching the rest of your criteria. For example, if you select November 8th, this mean EarthCache will try and return all images that match your criteria taken on or after November 8th.
Entering an end date (optional)
If you want to stop collecting images after a certain date, simply enter an end date by selecting it in the drop-down calendar. If you enter a start date on November 8th and an end date on November 30th, the platform will only look for data collected by satellites between November 8th and November 30th.
Not entering an end date means the platform will continue to search for data until the pipeline is edited to add an end date or deactivated. Once the pipeline has returned all available results collected until the present day, it will continue to look for future data at the interval selected, returning the best results at the specified interval.
Read: How to set up webhooks to get notified of new results >
You can select the frequency at which your pipeline will attempt to return results, whether it’s every day, once a week, or every 60 days. This is called “interval”.
For example, for a 1 day interval, with a start date of November 8th, the platform try and return one image per day matching the location, bands, resolution, budget, and cloud parameters. If there are multiple results within your interval, the platform will select the best result and return that.
Interval slack is the number of days you are willing to wait before you receive your data. In some cases, satellite providers have lead times of several days before they are able to provide data. With longer interval slack, you will have more sources available to you. With a shorter interval slack, you will receive your data faster, but we may not have as many options to give you the best possible result based on your parameters.
For example, if you need data every 4 days and require your data within two days of the end of each interval, you would set your interval to 4 and your interval slack to 2. When no interval slack is specified, we default to 3 days, which allows us enough time to receive virtually all available imagery data.
Setting an interval slack
To access the interval advanced panel, simply click on the gear icon next to the interval field. In the interval advanced panel, you will be able to select both an interval and an interval slack.
In order to distinguish certain features through optical data, you might require a maximum threshold of cloud cover you are comfortable with. If you select a maximum cloud cover, EarthCache will only return data that contains less clouds than your specified threshold.
Note: certain areas of the world are often covered in clouds. In order to find acceptable imagery for the area, you might need to expand your time range. For example, Seattle (WA) is covered in heavy clouds more than 50% of the time.
What is the difference between commercial and open data?
Through EarthCache, you can access both commercial and open datasets. Open datasets are part of educational and scientific missions financed by governments across the world to help make remote-sensed data more accessible to the public. As such, the data collected has been made available for free by the programs and a number of such datasets can also be accessed for free through EarthCache.
EarthCache also offers access to very-high resolution data from commercial sources. To obtain data from commercial sources, simply input the resolution you need in your call, or select the range you are comfortable with within the EarthCache dashboard.
Open data resolution (10m - 30m)
To select open data (free), simply select a resolution between 10m and 30m. Within our fair usage policy, each user can download up to 50GB of open data for free every month. The 50GB will be reset at the beginning of the month.
At 10m resolution, 50GB is roughly equivalent to 1,000,000 km2
Commercial data resolution (40cm - 5m)
To select high resolution data, a budget will need to be entered.
Current market rates are:
- 1m: $8/km2
- 50cm: $16/km2
- 40cm: $24/km2
You can find the size of your area of interest by simply heading over to the location tab and looking for the area box on the top right of the map.
You will be contacted to confirm data purchase. You will never be charged without your authorization. You can contact email@example.com at any point if you would like assistance.
To view a complete list of all available data sources, associated bands, and resolutions, please consult our Available Data page.
Tags allow for custom labelling of pipelines. This allows for quick searchers within your list of configured pipelines. After completing a tag, please press tab or enter to create another one.
Output format is the file type that is provided for the download. GeoTiff is the default output. We also provide PNG.