Two million data points a day, and counting

Every day, the MetOceanView service ingests and serves up to our clients more than 2 million unique data points. These are modelled and observed data providing vital marine weather information to users.

The MetOceanView platform displays forecast and historical data for a range of locations. Clients worldwide use the site to access the results from customised wave and hydrodynamic models, helping them make important decisions to maximise safety and improve efficiency.   

“MetOceanView provides a phenomenal amount of information for a wide range of clients,” explains Andre Lobato, who works on data management for MetOceanView. “In order to run such a system, the platform has to process an enormous amount of data.”

Model and observational data are ingested into MetOceanView to provide a complete picture of ocean weather conditions for our clients.

Model and observational data are ingested into MetOceanView to provide a complete picture of ocean weather conditions for our clients.

“Every day, we ingest about 2.25 million discrete data points. More than 2 million of these are unique rows of modelled data from global weather and marine models. In addition to modelled data, we continuously incorporate satellite, lightning, weather station, wave buoy, current meter and tide gauge data as part of the operational infrastructure behind MetOceanView. Some of these data, like METAR stations, NOAA-NDBC buoys, NOAA-MADIS, Himawari 8, GOES and MODIS satellite images are displayed directly on the MetOceanView interface. Others are shown to provide comparisons with our modelled data - e.g. wave buoy data displayed on a graph comparing observed to forecasted wave height.

“Real-time lightning data at times add a huge number of additional observations. Provided through Blitzen (TOA and GPATS), each single lightning strike constitutes a discrete observation. This means that on some days we incorporate millions of lightning data points per day, displaying real-time strikes for Australia, New Zealand and Europe.

Example of one-hour real-time lightning observations for Wednesday 29 March as shown in MetOceanView. Red dots represent clusters of lightning strikes.

Example of one-hour real-time lightning observations for Wednesday 29 March as shown in MetOceanView. Red dots represent clusters of lightning strikes.

We also use observational data to calibrate and validate our meteorological, wave and hydrodynamic forecast models. Observed data can also be used to directly improve our near-real-time forecasts, and can result in significant accuracy gains.

“All this information comes from a variety of sources. Much of the data used in MetOceanView are from our own models and instruments, but some observational data come from external providers. Some of it is private, for example where clients have observations that can help improve the models for their locations.  

“Ingesting such quantities of data requires a range of techniques. Often we have to process the raw information coming in to make it useable for our internal databases. We have designed our systems so that they can handle any data format.

“Ultimately, our clients use MetOceanView as a one-stop-shop for their marine weather information needs. The data we incorporate are valuable to our clients because they help them gain the complete picture of the atmospheric and marine conditions at their site. Good data visualised in an easy-to-understand format allows informed decision-making, which makes for safer operations and increased efficiency, and that is what MetOceanView is all about.”

For more information about MetOceanView, watch our introduction video here, see www.metoceanview.com or email enquiries@metocean.co.nz