Data collected during a dive is delivered to the science party via collaboration between the Alvin operations team and Atlantis’s Shipboard Scientific Support Group (SSSG).
Unless otherwise stated below, Alvin data is delivered as originally collected during the dive, without post-dive treatment beyond organization into a cruise data package. Alvin operations are not funded for data post-processing. If you have data related questions, please contact the NDSF Data Management (email@example.com).
Video from all cameras on Alvin is either native 1080i/60 or is upconverted to that standard (1920x1080 pixels). The two science observers select which video streams are routed to the two in-sphere hard drive recorders.
At the time of recording the sources are compressed by the Apple ProRes422LT codec and are captured to computer file (.mov) as the raw delivered product. A proxy product (50% scaling, h.264 codec, 5% filesize) is created in post-processing to provide a version for viewing on personal devices. To view these proxies, which are not playable by Apple Quicktime Player or Windows Media Player, users should install an open source player such as VLC or mplayer on their personal computers. Atlantis’s SSSG technicians offer a VLC installer.
Video clip file sizes are determined by the in-sphere observers and their camera selection behavior. The choice to view and record uninterrupted from one source can lead to very large files (>100GB), and this must be accommodated in your transport and long-term storage planning.
Alvin’s primary still image source is the externally-mounted SubC 1Cam Alpha, which captures images on a timed interval (default 30 secs) throughout the dive. Resolution of these images is 6544 x 3680 x 8 bit and file output is .jpg.
The observers and pilot are provided with digital still cameras for both in-sphere and through viewport documentation.
The Framegrabber logging system captures 1920 x 1080 .jpg stills from the currently-viewed video stream on a timed interval.
Alvin’s primary navigation sensors are a Doppler velocity log (DVL) paired with a fiber optic gyroscope (FOG). These and other navigation sensors are logged by the in-sphere navigation computer to a single hourly ASCII text file. This aggregate file is parsed by sensor type during the data offload process and files for individual sensor types are created. Among these is a simple .csv file that can be viewed and processed using programs like a text editor, Matlab or Excel.
At the conclusion of every dive, Alvin-specific navigational and sensor data is transferred from the submersible to the support ship’s network for further processing. Technicians on board the ship execute a dedicated post-processing script that performs an initial ‘grooming’ of the various data and generates an output file that merges navigation and sensor data. Real-time position estimates are improved as a result of the post-processing. Maps and data reports can then be generated using the post-processed data.
The basic characteristics of the post-processed file (.scc) are as follows:
- 1Hz ASCII file containing post-processed navigation data and a selection of other sensor data collected during the dive
- Typical data: date, time, latitude, longitude, depth, pressure, heading, and altitude. Other sensor data that may be made available includes magnetometer data, CTD data (conductivity, temperature, depth) and data from other sensors that may be resident on the submersible.
- SCC file names contain both the dive number and the date of the dive
Complementary navigation datasets are logged by Alvin’s internal navigation system and equipment in Atlantis’s Top Lab. Georeferenced information is usually produced by the Sonardyne Ranger ultra-short baseline (USBL) system, which is based in Top Lab. The cruise package includes the records from both of these logging stations.
More detailed mapping requirements associated with specialized equipment (e.g. multibeam sonar, etc.) may require processing expertise that is not routinely provided by the Alvin team. Specific needs for additional technical assistance should be communicated as part of the pre-cruise planning process.
When possible, all sensor records are produced and collected in text format, organized hourly. Alvin’s Command and Control computer collects sensor records into a single large file, which is delivered with the data package. This aggregate file is parsed by sensor type during the data offload process, creating files for individual sensor types.
Data Availability on Atlantis
Raw video and sensor data will be retrieved from the submersible and made available on the ship within six hours of a dive. Some portions of the video package may require more time to transfer. Video will be staged on an independently networked computer cluster, i.e., with no connections to other ship’s networks. Video products may be accessed via the work area in Atlantis’s Main Lab, which offers Apple Mac Pro workstations for viewing, copying, and editing video. The default setting is to provide non-passworded access to all video products. If this is not suitable to the investigator, please raise the issue in pre-cruise discussions. All other data types will be available from servers on Atlantis’s main network.
The Apple Mac Pro workstations provided on the ship offer Firewire800 and USB 3.0 connections. They do not offer Thunderbolt. Users may also download via Ethernet.
Alvin will produce significant volumes of data, primarily due to its video products. Volume estimates are 700GB of video and 20GB of non-video data per dive. All data will be presented on external USB hard drives, which are loaned by Alvin operations to the chief scientist so that she/he can carry the data products back to a home institution. Video data will be separate from the other data types, and will be on drives formatted with the ExFAT filesystem. You are asked to promptly return all loaned drives to WHOI after copying content. A copy of all Alvin cruise data will also be transferred to WHOI’s Data Library and Archives.
When data package is delivered at the end of the cruise, chief scientist will be asked to:
(1) review a form that describes the package inventory and provides details for returning the drives to WHOI and acknowledge delivery of the package.
(2) assign embargo duration on the various data components (NSF policy states maximum 2-years).
(3) name institutions for whom acknowledgement is due when cruise data is used for outreach or commercial purposes.
Questions often arise concerning compatibility between shipboard data systems and science party equipment. Quite often these questions are about the file system used to format the transport disks given to the science party, e.g., FAT32, NTFS, HFS, ExFAT. Another frequent question concerns the connection types offered on the shipboard equipment.
Alvin video drives are formatted with ExFAT because of its capabilities and compatibility with recent versions of MacOSX, linux, and Windows. Unfortunately, ExFAT may be incompatible with older operating systems. We recommend that users plan to bring drives (to be) formatted in ExFAT, and make sure your favorite operating system understands this format.