Start here
Northmere hosts three kinds of scientific data, and nearly all of it is large. The documentation is organised around the four things people do with it: find, fetch, read and cite.
- Find
- Use the catalogue for the largest collections, or the STAC interface to search the full index of 2,184 collections by area, time range and variable.
- Fetch
- The bulk download guide covers aria2 manifests, curl, rclone and the Globus-style transfer endpoint, with advice on connection counts.
- Read
- The file format notes list the formats in use (Zarr, NetCDF-4, CRAM, Parquet, cloud-optimised GeoTIFF) and the libraries that open them.
- Cite
- The citation guide shows the exact reference to use for each collection version.
A first download in five minutes
- Pick a small collection, for example SAT-2074 (5.8 TB) or CLM-1177 (3.4 TB), and open its page.
- Download the
INDEX.txtin the first directory. It is a few kilobytes and needs no account. - Request a Researcher account if you want the full signed manifest. Verification takes under one working day.
- Run the aria2 command from the collection page and leave it running. Interrupted transfers resume where they stopped.
- Verify with
sha256sum -c SHA256SUMSinside the target directory.
Before you start a multi-terabyte transfer, check your free disk space and the monthly allowance of your tier. A 40 TB collection will not fit in a 12 TB Researcher allowance, but it can be fetched in slices: every collection is split into directories of 1 to 8 TB that you can request separately.
Service limits at a glance
| Tier | Connections | Throughput | Monthly egress | Signed URL lifetime |
|---|---|---|---|---|
| Open | 4 | 400 Mbit/s | 500 GB per IP | none |
| Researcher | 16 | 2 Gbit/s | 12 TB per user | 7 days |
| Associate | 32 | 5 Gbit/s | 300 TB pooled | 14 days |
| Full | 64 | 10 Gbit/s | 2 PB pooled | 30 days |
Getting help
Support requests go through the contact page. Please include the collection code, the exact command and the first error line. Operational problems are posted on the status page before they are answered individually.