Jon and Sandy - SuperMAG

[rweigel]

My recollection is that a few years ago, we were a few weeks from Sandy having his server running.

[jvandegriff]

Sandy - cloud storage is filling up, so SuperMAG given deadline; SuperMAG PI is now on-board with doing this!!

We will use 1 min data and SuperMAG programmer will help.

[rweigel]

Another motivation to getting this done: We are starting to see people build HAPI servers that pass through SuperMAG data:

https://spaceweather.knmi.nl/viewer/?layout=omni_ae_intermagnet#start=2017-09-07T06:00:00&end=2017-09-09T18:00:00

Really, the source should be a SuperMAG HAPI server.

[jvandegriff]

Update from Sandy - approval from PI; now just technical issues left (where to run server); will also add to HelioCloud

[rweigel]

Notes:

1. The fill of 0 is wrong: see https://hapi-server.org/servers-dev/#server=SuperMAG&dataset=data_AAA&parameters=N&start=2020-01-01T00:00Z&stop=2020-01-03T00:00Z&return=image&format=hapiplot&style=svg; it should be 999999.0
2. A sample time range of one month for 1-minute data is too long - a data request takes > 30 seconds. Most browsers timeout after ~120 seconds, and I can imagine an overloaded server often failing on a sample request. I suggest 3 days.
3. The data_ in the dataset name is redundant (because I know it is data ...) and makes matching with INTERMAGNET tedious. Eventually, we'll have WDC data that does not use data_. However, this is what SuperMAG uses internally, so perhaps it is better to be internally consistent.
4. Will non-baseline removed data be available from SuperMAG? Or, will the removed baseline be provided in metadata? Sometimes one wants to know this when comparing.

[rweigel]

Sandy - "data.ucar.edu" already hapi-ized? Look into.

[rweigel]

Sandy needs to ask SuperMAG to run code.

[jvandegriff]

Use naming scheme like existing HAPI servers:

Intermagnet: aae/definitive/PT1M/xyzf

WDC: aaa/PT1M/xyz

SuperMAG does not have definitive / provisional, etc, so use:
station/cadence/coords

[jvandegriff]

I talked to the SuperMAG PI, who is still supportive and I conveyed the sense of urgency or else a pass-through server would be done, which he still does not want. He offered to take a few slides from us about HAPI at SuperMAG, and include them in his presentations, including at the upcoming ISWAT meeting February 10-14, so this should now be our new target for deployment, not Space Weather Workshop in March.

I'll be at Space Weather Workshop (with several hats on), but one is definitely to promote HAPI and connect with NOAA, both at SWPC and at NESDIS (who houses the data used by SWPC).

If we are planning an amalgamation app for ground magnetometer data, would an early release be able to be shown at Space Weather Workshop?

[sandyfreelance]

SuperHAPI server up but has a network timeout bug (prob an APL network handoff issue), diagnosing Thurs & Monday.

• Implemented station names as 'AAA/PT1M/xyz' instead of 'data_AAA' to match wdcapi spec
• Implemented shorter time range for samples

• baselining options still tbd

So a sample query is:
http://supermag.jhuapl.edu/hapi/data?dataset=AAA/PT1M/xyz&start=2020-01-01T00:00Z&stop=2020-01-01T01:00Z