Data Products

Data collected during a dive is delivered to the science party via the Alvin  data engineer, with support from Atlantis’s  Shipboard Scientific Support Group (SSSG).

Unless otherwise stated below, Alvin data is delivered as originally collected during the dive, with modest post-dive treatment beyond organization into a cruise data package. Alvin operations are not funded for significant data post-processing. If you have data related questions, please contact the NDSF Data Management (ndsf_info@whoi.edu).

Video 

Alvin cameras currently support recordings in SD, HD and Ultra HD (4K) formats
up to 2160p60, depending on the source selected. The two science observers, using their provided iPad interface, select which camera video streams are routed to their respective in-sphere hard drive recorders.

At the time of recording, camera sources are captured using the ProRes422LT codec and are stored to computer file (.mov) as the raw delivered product. Additional recorders record and store H.264 encoded proxy video in 1080p30 format, and support near real-time processing by the imaging server. In-sphere processing allows for repackaged proxy videos, and embedded subtitles displaying dive metadata, to be available for review by the science party within a few hours of the vehicle returning to the surface.

Automated clip duration for both ProRes and proxy video is set at 15 minutes, however ultimate video clip file sizes are determined by the in-sphere observers and their camera selection behavior.

Sealog 

Observers are encouraged to use the NDSF Sealog event logger interface to enter scientific event commentary and capture co-registered sensor data and video grabs during their dive. The Sealog application is also utilized to collect periodic autosnaps of images and metadata to fill in the narrative of the dive between any manual user events. The result is a web browsable dive summary, available onboard during the cruise and ashore following the cruise.

During dives, event hot buttons ease comment entry and standardize vocabulary. We suggest that prior to the cruise you obtain or develop a hot button list according to the standards of your research community.

Still images 

The Alvin sphere is equipped with a variety of still image capture methods to allow for a diversity of perspectives to be captured and recorded during the dive. Images are captured from:

• The Sealog application, which as noted above, will produce video frame grabs on a timed interval, as well as when manual events are logged in the system.
• Inside the video system user interface, where observers can manually trigger image captures from their live camera controls.
• Digital still cameras provided to the pilot and observers for both in-sphere and through-viewport documentation.

Navigation 

Alvin’s primary inertial navigation sensors consist of a Doppler velocity log (DVL) paired with a fiber optic gyroscope (FOG). These and other navigation sensors are logged by the in-sphere navigation computer to a single hourly ASCII text file. A complementary navigation dataset is logged by a surface navigation computer in Atlantis’s  Top Lab. The cruise package includes the records from both logging stations. Georeferenced vehicle navigation information is produced by the Sonardyne Ranger2 ultra-short baseline (USBL) system, which is based in Top Lab.

At the conclusion of every dive,  Alvin-specific navigational and sensor data is transferred from the submersible to the support ship’s network for further processing. The aggregate navigation ASCII files are parsed by sensor type during the data offload process and files for individual sensors are created. These sensor-specific files can be viewed and processed using programs like a text editor, Matlab or Excel.

Technicians on board the ship also execute a dedicated post-processing script that performs an initial ‘grooming’ of the navigation data and generates an output file that merges inertial and USBL navigation data. Real-time position estimates are improved as a result of the post-processing.

The basic characteristics of this post-processed navigation file (.scc) are as follows:

• 1Hz ASCII file containing post-processed navigation data and a selection of other sensor data collected during the dive
• Typical data: date, time, latitude, longitude, depth, pressure, heading, and altitude. Other sensor data that may be made available includes magnetometer data, CTD data (conductivity, temperature, sound velocity) and data from other sensors that may be resident on the submersible.
• SCC file names contain both the dive number and the date.

Summary plots offering status validation and visual comparisons of the various navigation solutions are also provided.

More detailed mapping requirements associated with specialized equipment (e.g. multibeam, photo mosaics, etc.) may require processing expertise that is not routinely provided by the Alvin team. Specific needs for additional technical assistance should be communicated as part of the pre-cruise planning process.

Vehicle sensors and logs 

When possible, all non-navigational sensor records are collected and logged by Alvin’s  in-sphere Command and Control computer and organized into a single hourly ASCII text file. This aggregate file is delivered with the data package and is also parsed by sensor type during the data offload process, creating records for individual sensor types.

Instrumentation requiring manufacturer-specific software, such as multibeam data and scanning sonars, are run from the in-sphere Science computer and are logged in their native file formats.

Data availability on  Atlantis 

Sealog, vehicle sensor and navigation data, and representative proxy video products will be retrieved from the submersible and made available on the ship within a few hours of a dive. This data may be accessed and downloaded via network drives residing on the main Atlantis network. The full resolution raw video package requires more time to transfer and may not be available until the following day. Raw video will be staged on an independently networked computer cluster, i.e., with no connections to other ship networks. Full video products may be accessed via the work area in Atlantis’s  Main Lab, which offers Apple Mac workstations for viewing, copying, and editing video. The default setting is to provide non-passworded access to all video products. If this is not suitable to the investigator, please raise the issue in pre-cruise discussions.

The Apple Mac workstations provided on the ship offer Thunderbolt and USB connections. Users may also download via an ethernet connection direct to the Synology video server.

Data delivery 

Alvin will produce significant volumes of data, primarily due to its video products. Volume estimates are 2-3TB of raw ProRes video, 100-200GB for a proxy video subset, and 10GB of non-video data per dive. All data will be presented on external USB hard drives, which are offered by Alvin operations to the chief scientist so that she/he can carry the data products back to a home institution for transfer to enterprise-quality storage. Video data will be separate from the other data types and will be on drives formatted with the ExFAT filesystem. A copy of all Alvin cruise data will also be transferred to WHOI’s Data Library and Archives.

When the data package is delivered at the end of the cruise, the chief scientist will be asked to:
(1) review a form that describes the package inventory and acknowledges delivery of the package.

(2) assign embargo duration on the various data components (NSF policy states a maximum of 2-years).

(3) name institutions for whom acknowledgement is due when cruise data is used for outreach or commercial purposes.

Please review updated NSF and NDSF data policy for more information.

Media recommendations 

Questions often arise concerning compatibility between shipboard data systems and science party equipment. Quite often these questions are about the file system used to format the transport disks given to the science party, e.g., FAT32, NTFS, ExFAT, and APFS. Another frequent question concerns the connection types offered on the shipboard equipment.

Alvin video drives are formatted with ExFAT because of its capabilities and intercompatibility with recent versions of MacOSX, linux, and Windows. We recommend that users plan to bring drives compatible with a variety of file systems, suitable for use with the data products and delivery methods planned for use during the cruise.