Skip to content

Data

Data acquired with NDSF vehicles are valuable digital resources that enable scientific exploration by not only cruise participants but to a diverse community of scientists and educators. Recognizing the importance of these resources, NDSF is re-envisioning its data management practices to establish a modern, scalable, and sustainable data infrastructure that meets the needs of at-sea vehicle operations teams and sea-going scientists as well as shore-side data managers and the broader community. This group includes software engineers and sea-going data personnel as well as shoreside data management and geoinformatics specialists. Questions and comments can be directed to NDSF Information.

Data Policy

The NDSF Data Policy clarifies the roles and responsibilities of the facility and the science party with respect to data stewardship and archiving both at-sea and post-cruise. The Chief Scientist and all PIs should be familiar with this policy in advance of the cruise. The policy also includes guidance on the use, accreditation, and copyright of video and photo imagery which is especially important to understand if media personnel are taking part in your cruise.

NDSF is responsible for providing a single copy of the full data distribution to the Chief scientist and to the shoreside WHOI archive.

Additional copies for distribution among the science party is at the discretion of the Chief Scientist and can be created onboard by the science party using their own media. See “Storage Media Recommendations” to ensure format compatibility.

Vehicle Dive Metadata

Alvin Data

Data Deliverables

The data package received at the conclusion of an Alvin cruise is a collection of real-time automated logs, observer logs and photos, raw data collections, vehicle video, and post-processed products. The document on the right is intended to describe the contents and organization of the package so groups can process and use it more efficiently after the cruise.

Alvin Dive Log & Metadata

Operations Summary

Please click the link below to view the Operations Summary for Alvin from 1964 to 2024.

Alvin Frame-Grabber

The Alvin Frame-Grabber system provides the NDSF community with online access to Alvin's video imagery co-registered with vehicle navigation and attitude data for shipboard analysis, planning of deep submergence research cruises and synoptic review of data post-cruise.

Addition Data

Jason Data

This section is currently under construction. Please contact ndsf_info@whoi.edu if you have any questions.

Jason Dive Log & MetaData

Operations Summary

Please click the link below to view the Operations Summary for the towed vehicles Jason, Jason II, Medea, DSL120, DSL120a, Argo1, Argo2, Angus, CMARC, TowCam and Oceanic Explorer from 1972 to 2023.

Note: This mega-spreadsheet contains a wealth of Jason dives information; data are not QA-ed.

Jason Data Products

A multi-faceted vehicle, Jason is capable of carrying a wide variety of sampling apparatus and of performing many different activities. We provide several documents describing NDSF-maintained data systems and the content that Jason users will receive from them.

Navigation

Jason's primary navigation sources are a Doppler velocity log (DVL) in combination with a fiber optic gyroscope (FOG). This information is usually augmented with georeferenced information from an ultra-short baseline (USBL) system. During a dive both histories are displayed in real time and logged by in-house softwares navest and NavG. The two histories are mathematically merged in post-dive processing to yield an improved history. For those dives that are out of DVL range, a smoothed USBL-based product is delivered.

We recommend bringing to your cruise previously collected navigation and bathymetry, with historical station positions, to increase the efficiency and productivity of your dive time. Jason navigators can guide you in the creation of underlays and waypoints and will incorporate this information into operational software for use in real time. Prior to your cruise, the NDSF data manager can assist you finding information and processing content.

Imagery

Jason carries a variety of video and still camera systems, generally described in the Jason User Manual. Please discuss your project's needs with the Jason team during the pre-cruise process. The Jason control van houses a variety of imagery capture systems that have differing behaviors depending on source and time of development.

Constant Video Recording

Three video cameras ("pilot", "brow", and "science") are captured by duplicate direct-to-hard drive recorders. Video and accompanying files are clipped at a definable time, usually 15 minutes. The codec is h.264, which provides excellent viewing quality but is unsuitable for production video editing. File volumes are about 2.4GB per hour per camera. The processing of these clips requires about 15 minutes per hour of dive time. Video and subtitle files are merged in post-processing to yield a video with optional overlays, playable using software such as vlc.

Video Highlights

For those moments that may be processed for outreach, select activities of the dive can optionally be captured from the watchstander's choice of high definition (1080i) pilot, brow, or ultra HD (2160p/4K) science camera streams. These clips are compressed using the Apple ProRes422 family of codecs (2.7GB/3 minutes for high definition, 15GB/3 minutes ultra definition) and contain embedded time code. The resulting file is a .mov file type, playable by QuickTime and editable using NLE software such as Final Cut or Premiere.

Stills

Several sources of still images from Jason cameras are now available.

  1. Simultaneous still grabbed images from two of the video streams from the science, pilot, or brow, or still image cameras, as selected by science watchstanders. The system produces color RGB TIFF images at 1920x1080 x 8 bits of quantization.
  2. Standard still camera. Sulis 4k (new March 2018), which produces 5968x3352 images in jpeg format.
    • Still images from the various Insite mini-Zeus cameras (SciCam, BrowCam, PilotCam), with the control set to a periodic sampling period. The uncertainties of controlling the pan and tilt mounts and the camera zooms may demand complex post-processing, but this offers an alternative when a downlooking configuration cannot be supported.
    • A machine vision camera & dscam software that is alternatively used for periodic (~10 seconds) downlooking shots, from which Bayer-coded TIFFs, color TIFFS, and color jpegs are produced. By pre-cruisePhoto survey systems.

Jason lighting is now improved by use of LED technology, but some limitations on camera configurations exist. In particular, downlooking captures require use of strobes that can be flashed no more quickly than every ~10 seconds. Please be sure to discuss photographic requirements in the pre-cruise process.

All image filenames denote image capture time. A post-processed product merges vehicle navigation to image filename at the time the image was captured (.ppfx file).

Logging of Events

We are phasing in a new event log/dive summary system called sealog, and run sealog along with the Jason Virtual Van. Sealog provides an event logger interface, and both systems capture event science party commentary with sensor data and video grabs. Event hot buttons ease comment entry and standardize vocabulary. Currently we deliver VirtualVan content. We suggest that prior to the cruise you obtain or develop a hot button list according to the standards of your research community. Some vocabularies can be obtained from http://marinemetadata.org/guides/vocabs

Bathymetry

Jason can carry a 400 kHz Reson Seabat 7125 multibeam sonar. Use of the Seabat 7125 should be arranged in the pre-cruise process, and the chief scientist may wish to bring personnel who specialize in the production of bathymetry maps. The Jason at-sea data processor will produce a quick-look gridded product that is based on a first-cut renavigation and multibeam soundings that have been processed using automated scripts.

Oceanographic Sensors

Jason's additional standard sensors include a dedicated pressure sensor from ParoScientific, an Aanderaa 4330 optical optode, a magnetometer, Reson SVP, and an RTD-based temperature probe. The Seabird Seacat19v2 CTD is typically used in a free-run ASCII output mode: alternate uses of the CTD should be addressed prior to the cruise and may require a dedicated watchstander from the science party.

Deliverables

The cruise data package will be placed on hard drive and will consist of

  1. all raw Jason sensor
  2. video as described
  3. still images
  4. tables (.csv) of sensors data, subsampled and
  5. metadata and
  6. Post-processed navigation
  7. Quick-look multibeam maps and raw multibeam ping files, when
  8. VirtualVan, provided by two methods:
    1. network-independent html hierarchy that can be viewed using a web
    2. after the cruise on password-protectable server at http://4dgeo.whoi.edu/jason

Sentry Data

Data Deliverables

The data package received at the conclusion of a Sentry cruise is a collection of all Sentry data and the best “at-sea” data products produced by the Sentry team. The document on the right is intended to summarize the organization of this data with an emphasis on providing the client with ready access to the files needed for their research and reporting requirements.

Sentry Dive Log & Metadata

Sentry Metadata is automatically generated through the dive directory. This is a self-QA-ed file, and a source of the all-vehicle KML file.

Dive report from cruises without data embargo can be found below.

Data Access At-Sea

Alvin

Sealog, vehicle sensor and navigation data, and representative proxy video products will be retrieved from the submersible and made available on the ship within a few hours of a dive. This data may be accessed and downloaded via network drives residing on the main Atlantis network. The full resolution raw video package requires more time to transfer and may not be available until the following day. Raw video will be staged on an independently networked computer cluster, i.e., with no connections to other ship networks. Full video products may be accessed via the work area in Atlantis's  Main Lab, which offers Apple Mac workstations for viewing, copying, and editing video. The default setting is to provide non-passworded access to all video products. If this is not suitable to the investigator, please raise the issue in pre-cruise discussions.

The Apple Mac workstations provided on the ship offer Thunderbolt and USB connections. Users may also download via and ethernet connection direct to the Synology video server.

Jason

The Jason data processor will provide intermediary versions of the data package (excepting video highlights) throughout the cruise via a NAS share.