SPICE Archive Preparation Guide (``The NAIF Way'') =========================================================================== Last revised on 2019 DEC 23 by B. V. Semenov. The revision history is provided in ``Appendix 2''. Overview -------------------------------------------------------- This document describes the structure and contents of the PDS SPICE data sets produced by NAIF and the process that NAIF follows to put them together. First it talks about the approach that NAIF recommends for creating and augmenting SPICE data sets. Then it summarizes the steps of the archive preparation process and explains in detail each of the steps -- preparing data, writing documentation, labeling data, generating indexes, and checking and finishing up the archive. Along the way the document refers to numerous examples and describes the tools (programs and scripts) that NAIF uses in this process. These examples and some of the tools are provided in the package accompanying this document to be used as a reference by SPICE archive data set producers. For your convenience a checklist of actions to be taken is provided at the end of this document; please augment it with any additional steps specific to your SPICE archive, print it, and use it during your archive SPICE archive preparation. Applicability -------------------------------------------------------- This SPICE Archive Guide is written for anyone who will be preparing a new or augmented SPICE data archive for submission to the NAIF Node of the PDS. However, following these standards is highly encouraged for any entity involved in archiving SPICE ancillary data at some other archive facility. Accompanying Examples and Tools -------------------------------------------------------- This document is a part of the package that NAIF provides as a reference for SPICE archive data set producers. The package also contains the following directories and files: -- scripts used in the data set preparation process (located under ``scripts'' directory). NAIF uses these scripts on a Sun but they should probably work on any Unix workstation. All C-shell scripts provided in the package use C-shell installed as ``/bin/csh''; all Perl scripts provided in the package use Perl installed as ``/usr/bin/perl''. -- examples of internal comments for various kernel types and missions (located under the ``examples/comments'' directory) -- examples of scripts used to pre-process kernels produced in operations before including them in the archive (located under the ``examples/scripts'' directory) -- examples of MAKLABEL template files for various missions (located under the ``examples/templates'' directory) -- examples of list files used as input to the labeling and index generation scripts for various missions (located under the ``examples/listfiles'' directory) -- examples of meta-information files presented as complete snapshots of actual data sets without the data (located under the ``examples/datasets'' directory) Motivation -------------------------------------------------------- This SPICE Archive Preparation Guide describes SPICE archive standards in great detail. Some of the standards may seem rather ``picky'' or unnecessary, and indeed there are a few items included that are not really used/useful. Also, a few standards will be changed when the NAIF Node migrates to the PDS4 (PDS 2010) standards. (These are the standards that will also be adopted by the consortium of agencies comprising the International Planetary Data Alliance.) But adhering to all of these details is critical to the current and future use of archived SPICE data, especially to achieve interoperability across national archives, and, to facilitate use of archived SPICE data in data search, retrieval and processing tools that are, or will be, part of archive systems. It is imperative that archive preparers carefully check and re-check all components of an archive -- whether it is a new one or an augmentation to an existing one -- before it is submitted for ingestion. NAIF provides a checklist, recommendations and a variety of tools to assist one in the validation. These can help a great deal, but there is much that only the archive preparer can do. NAIF's Approach to SPICE Data Set Preparation -------------------------------------------------------- NAIF's approach to creating SPICE data sets can be summarized by this statement: All SPICE data for a given mission are archived as UNIX text and binary files(1) in a single, accumulating data set on a single virtual volume(2) having the same directory structure(3), the same set of meta information files(4), data file labels with the same structure(5), and data set index files with the same structure(6) as all SPICE data sets produced by NAIF(7). Each of the points identified in this statement using a number in parentheses is briefly explained below: 1. All SPICE data are archived as UNIX text and binary kernels, meaning that all text kernels have lines terminated by only and all binary kernels are in BIG-IEEE (big-endian) binary format. Following this requirement ensures consistency across all SPICE data sets and usability of the data for both future users and the archive distribution and manipulation tools at the NAIF Node of the PDS. To facilitate conversion of text kernels with lines terminated by and binary kernels in LTL-IEEE (little-endian) format to the required format NAIF provides a utility program named BINGO, available on the ``Utilities'' page on the NAIF web site. 2. All SPICE data for a given mission get archived in a single accumulating data set on a single virtual volume. There is no need to break SPICE data for a given mission into multiple data sets based on the mission phases (as was done by NEAR), kernel types (as was done in other data sets produced by other nodes), data producers, or in any other way. Doing so leads to duplication of data, extra effort in preparing multiple marginally different copies of meta-information files, and, most importantly, complicates the lives of future users who, in most cases, prefer to get all SPICE data for a mission from one place. The DATA_SET_ID for this single data set is usually set to ``sc-t-SPICE-6-V1.0'' where ``sc'' is the spacecraft acronym and ``t'' is the target identification (see PDS Std Ref Chapter 6). For example for MGS it is ``MGS-M-SPICE-6-V1.0'', for Cassini -- ``CO-S/J/E/V-SPICE-6-V1.0'', for DS1 -- ``DS1-A/C-SPICE-6-V1.0'', and so on. All SPICE data sets -- even those for the missions that have ended -- are accumulating, which means that if/when additional SPICE data becomes available, this data is added to the same data set. For on-going missions the data that is being added is usually covering the next period of time; for past missions the data that is being added usually provides a better trajectory solution, a better attitude estimate, an instrument parameters update, or fixes liens. New additions are called releases and get documented in the release catalog files (a new release object is added for each release) and in the index file (includes RELEASE_ID column). Each data set resides on a single volume because splitting it into multiple volumes has the same effect as splitting all SPICE data for the mission into multiple data sets -- it makes using the data harder with no real gains. Moreover, the past need to split data sets into smaller chunks due to limited capacity of physical media is no longer the case (at least for SPICE data sets, which are not very large). The VOLUME_ID for this single volume is normally set to ``scSP_1000'' where ``sc'' is the spacecraft acronym. For example for MGS it is ``MGSP_1000'', for Cassini -- ``COSP_1000'', for DS1 -- ``DS1SP_1000'', and so on. This single volume is called virtual because it is not restricted to a specific physical media (CD, DVD, Blueray, etc.). Instead it is intended to reside in a specially named directory on a PDS-D server. The name of this directory is derived from the DATA_SET_ID and the VOLUME_ID by lowercasing them and replacing ``/'' with ``_''. For example, the name of the directory for the MGS data set (MGS-M-SPICE-6-V1.0, MGSP_1000) is ``mgs-m-spice-6-v1.0/mgsp_1000'', for Cassini (CO-S/J/E/V-SPICE-6-V1.0, COSP_1000) -- ``co-s_j_e_v-spice-6-v1.0/cosp_1000'', for DS1 (DS1-A/C-SPICE-6-V1.0, DS1SP_1000) -- ``ds1-a_c-spice-6-v1.0/ds1sp_1000'', and so on. 3. All SPICE data sets have the following PDS-compliant directory structure: ds/vol ds/vol/catalog contains PDS catalog files ds/vol/data contains kernel sub-directories ds/vol/data/ck contains CKs ds/vol/data/ek contains EKs ds/vol/data/fk contains FKs ds/vol/data/ik contains IKs ds/vol/data/lsk contains LSKs ds/vol/data/pck contains PCKs ds/vol/data/sclk contains SCLKs ds/vol/data/spk contains SPKs ds/vol/document contains documents ds/vol/extras contains value-added, non data items ds/vol/extras/mk contains meta-kernels ds/vol/index contains index files ds/vol/software contains instructions for getting sw where the ``ds/vol'' part is formed as described above (for example, for MGS it is ``mgs-m-spice-6-v1.0/mgsp_1000''). the ``extras'' directory may contain additional sub-directories to store additional value-added, non-kernel files such as ORBNUM files (see MRO, MGS, or Cassini data sets for examples). the ``document'' directory may (but does not have to) contain additional sub-directories for additional documents or document sets (see Cassini data set for examples). To create this directory structure for the first release of a data set NAIF developed a script called ``make_pds_directories.csh''. 4. All SPICE data have the following meta-information files, most of which are required by PDS standards: ds/dsindex.lbl PDS-D index label ds/dsindex.tab PDS-D index table ds/vol/aareadme.htm aareadme in text format ds/vol/aareadme.lbl aareadme label ds/vol/aareadme.txt aareadme in HTML format ds/vol/catalog/catinfo.txt ``catalog'' dir. contents desc ds/vol/catalog/insthost.cat instrument host catalog file ds/vol/catalog/mission.cat mission catalog file ds/vol/catalog/person.cat personnel catalog file ds/vol/catalog/ref.cat references catalog file ds/vol/catalog/release.cat release catalog file ds/vol/catalog/spice_hsk.cat resource catalog file ds/vol/catalog/spice_inst.cat SPICE instrument catalog file ds/vol/catalog/spiceds.cat SPICE data set catalog file ds/vol/data/*/*.lbl detached labels, 1 per kernel ds/vol/data/ck/ckinfo.txt ``ck'' dir. contents desc ds/vol/data/ek/ekinfo.txt ``ek'' dir. contents desc ds/vol/data/fk/fkinfo.txt ``fk'' dir. contents desc ds/vol/data/ik/ikinfo.txt ``ik'' dir. contents desc ds/vol/data/lsk/lskinfo.txt ``lsk'' dir. contents desc ds/vol/data/pck/pckinfo.txt ``pck'' dir. contents desc ds/vol/data/sclk/sclkinfo.txt ``sclk'' dir. contents desc ds/vol/data/spk/spkinfo.txt ``spk'' dir. contents desc ds/vol/document/docinfo.txt ``document'' dir contents desc ds/vol/document/lblinfo.txt label location/naming desc ds/vol/document/onlabels.txt label structure desc ds/vol/errata.txt errata information ds/vol/extras/extrinfo.txt ``extras'' dir contents desc ds/vol/extras/mk/mkinfo.txt ``mk'' dir. contents desc ds/vol/index/index.lbl index label ds/vol/index/index.tab index table ds/vol/index/checksum.lbl checksum table label ds/vol/index/checksum.tab checksum table ds/vol/index/indxinfo.txt ``index'' dir contents desc ds/vol/software/softinfo.txt info for getting SPICE sw ds/vol/voldesc.cat volume catalog file Additional catalog files may be provided if there is more than one instrument host (see Deep Impact data set) and/or two or more target catalog files need to be included (see DS1 and SDU data sets). Additional ``*info.txt'' files may be included if additional directories are present under ``document'' and/or ``extras'' (see Cassini and MRO data sets). The ``make_pds_directories.csh'' script, in addition to creating directories, creates a zero size placeholder file for each of the required meta-information files. 5. All SPICE kernels included in all SPICE data sets are labeled with detached labels generated by the MAKLABEL utility program available from NAIF. These labels include the following keywords: PDS_VERSION_ID = PDS3 RECORD_TYPE = RECORD_BYTES = ^SPICE_KERNEL = MISSION_NAME = SPACECRAFT_NAME = DATA_SET_ID = KERNEL_TYPE_ID = PRODUCT_ID = PRODUCT_CREATION_TIME = PRODUCER_ID = MISSION_PHASE_NAME = PRODUCT_VERSION_TYPE = PLATFORM_OR_MOUNTING_NAME = START_TIME = STOP_TIME = SPACECRAFT_CLOCK_START_COUNT = SPACECRAFT_CLOCK_STOP_COUNT = TARGET_NAME = INSTRUMENT_NAME = NAIF_INSTRUMENT_ID = SOURCE_PRODUCT_ID = NOTE = OBJECT = SPICE_KERNEL INTERCHANGE_FORMAT = KERNEL_TYPE = DESCRIPTION = END_OBJECT = SPICE_KERNEL END In addition to being provided as a separate file each data file's label is included at the top of the file's comment area (for binary kernels) or at the top of the file's embedded comments (for text kernels). To run MAKLABEL to generate detached labels and to add these labels to file comments NAIF developed a script called ``label_them_all.pl''. 6. All SPICE data sets include two index files -- the standard PDS data set index in the ``index'' directory and the so-called PDS-D index provided under the ``'' directory. The standard index is always called ``index.tab'' and is accompanied by a label file called ``index.lbl''. The PDS-D index is always called ``dsindex.tab'' and is accompanied by a label file called ``dsindex.lbl''. Both index files have the same set of columns: START_TIME STOP_TIME FILE_SPECIFICATION_NAME DATA_SET_ID PRODUCT_CREATION_TIME RELEASE_ID RELEASE_DATE KERNEL_TYPE_ID PRODUCT_ID VOLUME_ID To generate both indexes using information from the data file labels NAIF developed a script called ``xfer_index.pl''. All SPICE data sets also include an MD5 checksum table file. This file is always called ``checksum.tab'' and is accompanied by a label file called ``checksum.lbl''. Both the table file and the label file are located in the ``index'' directory. To generate these files NAIF uses the script called ``mkpdssum.pl'' developed by the Small Bodies Node of the PDS. 7. While experts on PDS standards can (and did during peer-reviews) find a number of things about SPICE data sets that need improvement or even correcting, NAIF continues to carry on with the archiving approach that it established and polished over 10+ years of creating SPICE data sets. Applying this approach without major deviations results in data sets that truly look and feel the same from mission to mission. This helps both the users of the data who can count on finding archives with the same structure, and the NAIF node staff who in most cases are the people providing expert advice about SPICE data sets. For the reasons noted above, please carefully follow the instructions provided in this Guide and use the tools supplied with this Guide. Once in the archive, no kernels or meta-kernels are removed from it or replaced with different versions with the same name. Instead, new files superseding already archived files are added to the archive. These kernels are given distinct names and the fact that they supersede previously archived kernels is reflected in the meta information files (``spiceds.cat'', ``*info.txt'', meta-kernels). Process Overview -------------------------------------------------------- The process of preparing a SPICE data set includes the following steps: 1. collecting and preparing data 2. putting together descriptions 3. labeling data 4. generating index files 5. checking and finishing up 6. if needed, packaging and delivering the data set to NAIF Each of these steps is described in a separate section below. Step 1: Preparing Data -------------------------------------------------------- Ideally a SPICE data set for a mission should include a comprehensive set of kernels allowing a scientist to compute geometry for any of the mission instruments in regards to any of the mission targets at all applicable times during the mission. To achieve this the data set would normally include all types of kernels needed to provide ephemerides (SPKs), orientation (PCK) and shape (PCK or DSK) of targets, trajectory (SPK) and orientation (CK) of the spacecraft, orientation (CK) and geometric parameters (IK) of the instruments, definitions of the spacecraft and instrument frames (FK), and data for various time conversions (LSK and SCLK). If the project produced Event Kernel (EK) files of any kind, they should also be included in the archive. Identifying Data Usually identifying data that should go into the archive is not very difficult, especially when SPICE kernels were produced and used by the project during operations. In such cases the following kernels used during operations -- independent of whether they were produced by the project teams or obtained from NAIF or other sources -- should be included in the archive: -- LSK * latest generic LSK file -- SPKs * planetary ephemeris SPK file(s), officially accepted by the project * any natural satellite ephemeris SPK file(s), officially accepted by the project * latest SPK file(s) for other types of mission targets (e.g. comets, asteroids) * latest reconstructed spacecraft trajectory SPK file(s) * target and/or spacecraft trajectory SPK files produced by science team(s) (for example Gravity or Radio science); if any * latest ground stations locations SPK file(s); if any * latest structures/instrument locations SPK file(s); if any * predicted SPKs; only if they are needed as gap-fillers for reconstructed data, or must be archived for the record; if any * nominal/special SPKs; only if they are needed to complete the position chains (such as MER-at-landing site SPKs); if any -- PCK * latest generic text PCK file officially used by the project * latest generic binary PCK file(s) officially used by the project * latest project-specific PCK file(s) providing the rotational, shape and possibly additional constants for all of the mission targets; if any -- SCLK * latest spacecraft on-board clock(s) correlation SCLK file(s) * additional latest SCLK files if the project and/or science teams produced special SCLKs for instrument or other hardware clocks or if more than one kind of SCLK kernel was made for the same clock; if any * special SCLK file implementing mean local time or other SCLK-like time systems (see MER for examples); if any -- CK * the latest reconstructed spacecraft attitude CK files * possibly latest predicted spacecraft attitude CK files if they provide a reasonably good prediction and are needed to fill gaps in the reconstructed CK files and/or for some other reason * latest reconstructed spacecraft appendage (solar arrays, HGA, etc.) attitude CK files; if any * possibly latest predicted spacecraft appendage (solar array, HGA, etc.) attitude CK files if they provide a reasonably good prediction and are needed to fill gaps in the reconstructed CK files and/or for some other reason * latest reconstructed instrument orientation CK files for each of the articulating instruments; if any * possibly latest predicted instrument orientation CK files if they provide a reasonably good prediction and are needed to fill gaps in the reconstructed CK files and/or for some other reason * spacecraft and/or instrument CK files produced by science teams as part of C-smithing or other pointing reconstruction process; if any * nominal/special CK files, only if they are needed to complete the orientation chains (for example, an instrument parking position orientation CK file); if any -- FK * latest version of the main mission FK file * latest version of special mission FK files (separate lading site FKs, etc); if any * latest versions of separate instrument FK files; if any * latest versions of generic FK files for natural bodies (Moon, Earth, etc); if any * latest versions of multi-mission dynamic frames FK files; if any -- IK * latest IK file for each of the instruments * latest IK file for auxiliary s/c subsystems, the data from which might be used for science purposes (antennas, star trackers, horizon sensors, etc); if any -- EK * PEF2EK-type sequence and command dictionary EK files (see SDU, Deep Impact for examples); if any * database EK files (see CLEM for examples); if any * CASSINI-style sequence, noise, plan, status EK files (see CASSINI for examples); if any * ENB EK files (see MGS, SDU for examples); if any -- DSK * latest DSK file (or files if multiple kernels with different resolutions and/or for different parts of the surface were produced) for each of the mission targets; if any While no mission produces all kernels from the list above, most missions produce kernels of all types (maybe except EKs) and most of these kernels are needed to compute observation geometry for the mission instruments and, therefore, should be included in the archive. Once the types of kernels that should go into the archive have been identified it is usually fairly easy to decide which actual individual kernels belonging to each ``category'' should be included. Considering these points may help to make this selection: -- For the kernel types that don't cover specific time intervals, cover the whole mission and/or change rarely during the mission -- such as planetary, satellite, structures SPKs, DSK, LSK, PCK, FK, IK, and SCLK -- the latest version of each file at the time of archive preparation should be included. For the first archive release all latest kernels of these types should be included, while for subsequent releases only those kernels that had been updated or improved compared to the already archived files should be included. For example, if the project initially used the Martian Satellite Ephemerides MAR033 SPK file (which was included in the first archive release) but later switched to using the MAR066 SPK file, the MAR066 SPK file should be added to the archive at the next release opportunity. Another example is when the main project FK file was updated to include improved instrument alignment data; if this happened it should be added to the next archive release. -- For the kernel types that provide data for specific time intervals that are normally much shorter than the whole duration of the mission -- such as spacecraft SPK, spacecraft, structure, and instrument orientation CKs, and EK -- the set of files providing the complete coverage for the applicable interval should be included. If the archive preparation takes place at the end of the mission then all kernels of these types needed to provide data coverage for the whole mission should be included. If the mission is on-going and data is added to the archive at regular increments (releases), each intended to cover a specific time interval, then each release should contain the set of these files providing complete coverage for the interval of interest. -- In most cases including duplicate data should be avoided. For example, if the project is producing two strings of reconstructed spacecraft orientation CK files from the same telemetry input (daily ``quick look'' files and weekly ``final'' files) only the ``final'' CK files should be included. Another example is if the project used the same generic LSK file under two different names -- its original name and a short-cut default name, -- which is done sometimes to simplify operations infrastructure, then only the file with the original, actual name should be included in the archive. There are a few cases in which duplicate data should be included. The most common of these cases is when the data comes from two different producers, for example two sets of reconstructed spacecraft trajectory SPK files, one generated by the project NAV team and the other by the Gravity team. In such cases a determination of which set is ``better'' usually cannot be made and both sets should be archived. -- Normally it is also not advisable to include obsolete or superseded data. There are numerous examples of cases when a kernel produced and used for some period in operation becomes obsolete when another version of the same data is released at a later time. The most common of these cases are predicted and quick-look reconstructed spacecraft trajectory SPK files that get superseded by the final reconstructed solution, and earlier versions of SCLK kernels that get superseded by the later versions. Exceptions to this suggestion include cases when superseded data is applicable as gap-filler (for example, predicted CKs used to fill gaps in telemetry based reconstructed CKs) or when an obsolete version needs to be archived to provide consistent access to other archived data (for example archiving an earlier version of SCLK that was used to make a predicted CK also included in the archive). -- No kernel file or meta-kernel file already in the archive should ever be removed or replaced with a new version with the same name. Instead, any kernel or meta-kernel file added to the archive should have a name that is distinct from the names of all files already in the archive. If a kernel file supersedes one or more files already in the archive, this fact should be reflected in the corresponding ``*info.txt'' file and/or ``spiceds.cat'' file and another version of the meta-kernel(s) should be created including this kernel file instead of the kernel file(s) that it supersedes. Collecting and Preparing Data Once the data files have been identified it makes sense to collect them in a single area (``work area'') because frequently the kernels need to be pre-processed before they can go into the archive. Such pre-processing may involve merging or sub-setting files, renaming files to make their names PDS compliant, and augmenting files with internal comments. It should also include validating the final products that will go into the archive. The work area can (BUT does not have to) be structured as the final archive. It can physically reside on more than one workstation BUT having it on single workstation may simplify pre-processing and validation tasks. It does not have to include kernels that don't require pre-processing (merging, renaming or additional comments) and can go into the archive ``as is'' BUT including these kernels might also simplify pre-processing and validation tasks that require multiple kernel types. The kernels that do need to be pre-processed should be copied or ``binary FTP''-ed or ``scp''-ed to the work area. The ways in which the files should be modified usually include one or more of the following: -- Merging Files The data from two or more files may need to be merged together for a number of reasons: to reduce the number of files included in the archive, to eliminate gaps in coverage at the file boundaries, to produce a file that segregates data pieces that must be used together, or to integrate data from updated un-official versions of a file into the official version. Merging to reduce the number of files is usually desirable for the project-generated CKs or SPKs covering short periods of time, for example daily or weekly files, when these files are not very large in size. Merging such files together into a single file covering the whole archive release time span -- monthly, tri-monthly, etc. -- or a few files covering parts of that span will result in substantially reduced number of files, which in turn will reduce the amount of processing needed to put this data into the archive and make access to the archive data more efficient. Merging files to eliminate gaps at file boundaries is usually desirable when the project generates a large number of CK files of the same kind with short coverages not overlapping each other. If the merged file is created from these individual CKs in such a way that data from multiple source segments is aggregated together in the new set of segments, the gaps at the original file boundary times will not be present in the new file. Merging files to produce a file that aggregates data pieces that must be used together in one place may be needed when the spacecraft trajectory SPK and the target ephemeris SPK used to determine it are delivered by the project in two separate files. When this happens it leaves a possibility for the users to use the spacecraft data with a different target trajectory resulting in the wrong relative geometry being computed. This situation happens very rarely but it needs to be checked and addressed. Merging files to integrate data from an updated un-official version of a file into the official version is usually needed when science teams keep a local copy of the main project FK and change it by modifying alignment of a previously defined frame(s) and/or introducing a new frame(s) for their instruments. It is important to inquire about such ``local'' updated copies and, if they exist, collect them and carefully incorporate the data from them into the new version of the official project FK file. When selecting how many files to merge together the size of the merged file should be one of the factors to consider. While SPICE does not impose a ``hard'' limit of a number of megabytes under which this size should be kept -- except, of course, for the 2.1 GB which is the limit for 4-byte integer address space, -- is it probably wise to keep the file size under 200-300 MB. NAIF distributes a few utility programs that can be used to merge various types of kernels. SPKMERGE provided in the generic Toolkit can be used to merge SPK files. DAFCAT and CKSMRG available on the NAIF server (http://naif.jpl.nasa.gov/naif/utilities.html) can be used to merge CK files. In some cases NAIF puts together scripts wrapped around these merge utilities to facilitate file merge tasks that have to be repeated for each archive release. Some of these scripts are included as examples in the ``examples/scripts'' directory of the package accompanying this document. -- Sub-setting Files Sub-setting source files to produce archival files with reduced scope or coverage is needed very rarely. In general it is better to include data files with coverage that extends beyond the current archive release interval rather than to try ``chopping'' the file's coverage to line up with that boundary. But if the project archiving policies or other considerations require such ``lining up'' the SPKMERGE utility (provided in the generic Toolkit) can be used to subset SPK files and the CKSLICER utility (available on the NAIF server) can be used to subset CK files. -- Augmenting Files with Comments It is absolutely crucial that every kernel included in the archive contains comprehensive internal comments describing its contents, source(s) of the data, applicability of the data, etc. This means that all kernels intended for the archive -- binary and text ones, those that should be archived ``as is'' and those that were created by merging or sub-setting other files -- should be checked to verify that they contain adequate comments and, if not, augmented with such comments. The subsection ``Internal Comments'' in the section ``Step 2: Preparing Descriptions'' discusses what should be addressed in the comments for various kernel types and points to numerous comment examples provided in the package accompanying this document. In binary kernels internal comments reside in the special area of the file called the ``comment area''. The comments provided in this area can be accessed -- displayed, added, or deleted -- using the COMMNT utility program. To add new comments to a binary kernel file that does not have any comments, one would first write a text file containing these comments and then add the contents of this file to the comment area using ``commnt -a''. To replace existing comments in a binary kernel file, one would first view existing comments using ``commnt -r'' (or save them to a text file ``commnt -e''), write a text file containing new comments (or edit the text file containing existing comments), delete existing comments from the file using ``commnt -d'', and finally add new or updated comments to the file using ``commnt -a''. In text kernels comments are located at the top part of the file, up to the first ``\begindata'' token on a line by itself, and in the file sections delimited by ``\begintext'' and ``\begindata'' tokens, each on a line by itself. Any number of comment sections intermixed with the data sections can be included in the file. Modifying comments in a text file can be done using any text editor. When modification are made to the file comments, the file version should be increased and the scope of the comment modifications should be mentioned in the version section of the comments. Comments in both binary and text kernels should contain only printable ASCII characters (no TABs); it is also strongly recommended that comment lines should be no longer than 80 characters. While it is not possible to automate writing comments -- as with any other documentation this is the task that needs to be done by the person who puts the archive together by hand or by ``recruiting'' the people/teams who provided the data -- it is certainly possible to automate generating comments for a string of files of a certain type using a template and inserting these comments into the files. The example merge scripts mentioned above each contain steps for creating comments from a template and adding these comments to the output file. -- Renaming Files The names of the files to be included in the archive must comply with the expanded ISO 9660 Level 2 requirement adopted by PDS. Thy must have 36.3 form -- 1-36 character long name + 1-3 character long extension -- and must consist of letters (a-z), digits (0-9) and underscores (_). All names that don't comply with this requirement must be changed. NAIF recommends that the files are named using lowercase letters (as is done for the majority of the PDS-D data sets) rather than using upper case letters that were the requirement during the CD era. The only requirement that NAIF imposes in addition to this general PDS requirement is that the extensions of the kernel files must follow the established convention for SPICE kernels: Binary SPKs .bsp Binary PCKs .bpc Binary DSKs .bds Binary CKs .bc Binary Sequence EKs .bes Binary Database EKs .bdb Binary Plan EKs .bep Text PCKs .tpc Text IKs .ti Text FKs .tf Text LSKs .tls Text SCLKs .tsc Text Notebook EKs .ten Text Meta Kernels .tm NAIF also strongly recommends that the names of all mission specific kernels start with the acronym of the spacecraft or the mission (if a data file contains data for more than one spacecraft associated with the same mission). For example, the names of MGS kernels start with ``mgs_'', the names of MER-1 kernels start with ``mer1_'', and so on. Validating Data Although the majority of the source kernels (both those that go into the archive ``as is'' and those that have been used to make the merged archive files) have been used in operations and have been validated by this use, the final complete set of archival files must be validated by checking the files' coverages, data scope, correctness of comments, data accessibility, integrity, and consistency. The following validation approaches complementing each other are suggested: -- summarizing individual binary kernels (binary SPK, DSK, CK, PCK, EK) and meta-kernels using BRIEF, DSKBRIEF, CKBRIEF, and SPACIT utilities to verify that they are accessible, provide data for the right set of bodies/structures, and have expected coverage -- summarizing FOV definitions in IKs -- directly or via meta-kernels -- using OPTIKS to verify that the IKs are accessible and provide data for the right set of instruments/detectors -- checking comments in the kernels -- both text and binary -- for completeness, correctness and consistency with the summaries of the data produced by summary tools -- comparing files with similar data (for example spacecraft SPKs from different producers) and examining differences to see that they look reasonable; for SPK files this can be done using the SPKDIFF utility, for CK and FK files this can be done using the FRMDIFF utility -- comparing later versions of kernels that need to be added to the archive with already archived earlier versions; for text kernels this can be done by analyzing differences shown by Unix utilities ``diff'' or ``tkdiff'' -- comparing merged archival products with the source operational files; for SPK files this can be done using the SPKDIFF utility, for CK files this can be done using the FRMDIFF utility -- checking file data integrity by running utilities like SPY (currently works only on SPK files) -- writing an application to compute geometry using the archival data and comparing that geometry to known values, for example from the geometry keywords in the science data labels; ideally such computations should be done for each of the instruments, for the quantities that require data from kernels of all types to be accessed, and over the whole span covered by the archive or a particular archive release -- asking the project SPICE users to re-run some of the geometry computations that they have done with source operational files using the final set of kernels and verify that they obtained the same results While some of the validation tasks can be scripted (for example checking coverage based on file summaries or running SPY to check file data integrity), many others have to be done by hand (for example assessing comments in new version of text kernels) in many cases making validation a time and effort consuming activity. Still, the person preparing the archive should try to give his/her best effort to make sure that each archive release contains the complete set of files (in terms of scope and coverage) that are well documented with internal comments. Value-Added Elements The NAIF archiving approach requires meta-kernels and, if available, orbit number (ORBNUM) files to be included in the data sets in addition to SPICE kernels. Since neither meta-kernels nor ORBNUM files are SPICE kernels, they are considered ``value-added'' archive elements and, for this reason, reside in the ``extras' directory of the data set and don't need to be labeled or included into indexes. -- Meta Kernels Meta-kernel files (a.k.a ``furnsh'' files) provide lists of the archived kernels included in the data set suitable for loading into a SPICE-based application via the high level SPICE data loader routine FURNSH. Using meta-kernels makes it easy to load, with one call, a comprehensive SPICE data collection for a given period, which, given that SPICE data sets can contain large number of files, is extremely helpful for future users. For missions with a small number of archived kernels NAIF recommends creating a single meta-kernel providing data for the whole mission. The name of this meta-kernel should follow the ``mmm_vnn.tm'' pattern where ``mmm'' is the mission acronym and ``nn'' is the version number. If/when new kernels are added to the data set, a meta-kernel with the next version number, including the new kernels and leaving out superseded kernels should be created and added to the archive. For missions with a large number of archived kernels NAIF recommends creating a set of meta-kernels each covering one year of the mission. The names of these meta-kernels should follow the ``mmm_yyyy_vnn.tm'' pattern where ``mmm'' is the mission acronym, ``yyyy'' is the year covered by this data, and ``nn'' is the version number. If/when new kernels are added to the data set, meta-kernels for all applicable years with the next version number, including the new kernels and leaving out superseded kernels should be created and added to the archive. Examples of meta-kernels are provided in the ``extras/mk'' directories under data sets provided in the ``examples/datasets'' directory of the package accompanying this document. -- ORBNUM Files ORBNUM files can be generated for orbiter type missions using NAIF's ORBNUM utility program. They provide orbit numbers and orbit start times along with a number of derived parameters at these times. If ORBNUM files are (or can be generated) for a mission, they should be included in the archive. Examples of ORBNUM files are provided in the ``extras/orbnum'' directories under data sets provided in the ``examples/datasets'' directory of the package accompanying this document. If the project produces other value-added files closely related to kernels and ``insists'' on archiving them, these files can also be added to the archive's ``extras'' directory. For example, the CASSINI project produces comparison plots and pointing correction plots for its reconstructed and C-smithed CK files. CASSINI requests these plots be included in the archived data set. NAIF neither objects to nor recommends practices like this. Step 2: Preparing Descriptions -------------------------------------------------------- As for any other PDS data set each of the SPICE data sets includes a large number of meta-information files describing the data, meta-data, volume contents, volume directory structure, and so on. Preparing many (but not all) of these files is very easily done by starting with a file of the same kind from an exiting data set and changing just a few words in it (such as mission name and acronym, publication dates, etc). The same approach -- starting with or using an existing file as an example -- will also work best for putting together the few files that do require more effort, those describing the actual data included in the data set. Spending the effort to prepare adequate descriptions of the data is essential to guarantee that the data can be used correctly and efficiently by future users. NAIF requires that SPICE data sets provide kernel data descriptions in the following three ``places'' in the data set: -- internal comments included in the kernels -- ``*info.txt'' files provided in the kernel type subdirectories under ``data'' directory -- ``data/ck/ckinfo.txt'', ``data/ek/ekinfo.txt'', etc. -- SPICE data set catalog file ``spiceds.cat'' provided in the ``catalog'' directory of the data set Descriptions given in each of these ``places'' have different purposes and levels of detail. The ``comments'' in a particular file provide the most detailed and comprehensive information about the file that they document; the ``*info.txt'' files describe the naming scheme and usage priority of the collection of files of a particular kernel type in the data sub-directory in which they reside; ``spiceds.cat'' provides a high level overview of the data set, covering briefly the types of information provided in, the source data of, and the accuracy of each kind of kernels included in the data set. Descriptions that are expected to be included in the ``comments'', ``info.txt'' file and ``spiceds.cat'' are explained in greater detail in the next few sub-sections of this document. The sub-section following them covers other documents that should be included in the data set -- aareadme files, other catalog files, ``*info.txt'' files, etc. General Note About Meta-information File Format All descriptions and meta-information files provided in SPICE data sets -- internal comments in binary and text kernels, catalog files, and ``*info.txt'' files -- must be prepared as plain ASCII text and must not include any non-printing characters (TABs are not permitted). The lines in the comments and meta-information files must be no longer than 78 characters. The only exception from this requirement is the ``aareadme.htm'' file that may have lines containing HTML references extending beyond the 78 character limit. Internal Comments It is crucial that every kernel included in the archive contains comprehensive internal comments that describe: -- contents of the file -- version and revision history -- status and purpose of the file -- source(s) of the data (including names of the original files if the file was created by merging or sub-setting other files) -- processing that was done on the data -- setup parameters and output logs for utility(ies) used to create the file -- applicability of the data -- data coverage -- data accuracy -- other kernels needed to use the file -- references -- data producer and contact information The comments for a particular file should address all of the categories from this list that are applicable to the kind of data stored in the file. The best approach to writing comments for a SPICE kernel is to start with the comments from a kernel of the same type containing the same or similar kind of information and modify these comments to describe the file in hand. The package accompanying this document contains an ``examples/comments'' directory with numerous examples of internal comments for each kernel type. These comments should be used as a reference or even the starting point for comments for the kernels intended for archiving. *info.txt Files in Kernel Directories Each of the ``*info.txt'' files located in the kernel type subdirectories under the ``data'' directory -- ``data/ck/ckinfo.txt'', ``data/ek/ekinfo.txt'', etc. -- provides brief general information about kernels of that type and describe the naming scheme and use priority for the collection of kernel files contained in the subdirectory where this ``*info.txt'' file resides. The best approach to writing these files is to start with the versions from one of more recent SPICE data sets (for example, the MRO data set provided under the ``examples/datasets'' directory of this package) and modify these versions to address the kernels that are being archived. Some sections of these files -- ``General Information ...'' and ``Kernel File Details'' -- may not need any modifications at all while the other sections -- those describing the file naming convention, file loading priority (if there is such a section), contact information, and other mission specific information -- will need to be re-written to describe the file sets at hand. The values of the PUBLICATION_DATE and NOTE keyword in the attached label at the top of the file will also need to be updated. Well written ``*info.txt'' files put together for the first release may not need to be modified for future releases unless new kinds of kernels not reflected in them get added to the data set. SPICE Data Set Catalog File ``spiceds.cat'' located under the ``catalog'' directory of the data set provides a high level overview of the data set, covering briefly the type of information provided in, the source of, and the accuracy of each kind of kernel included in the data set. Starting with the ``spiceds.cat'' from a recently archived SPICE data set (for example for MRO) the following parts of it should be modified to adapt it to a new mission: -- values of LABEL_REVISION_NOTE, DATA_SET_ID, DATA_SET_NAME, DATA_SET_TERSE_DESC, ABSTRACT_DESC, CITATION_DESC, START_TIME, STOP_TIME, and DATA_SET_RELEASE_DATE. Modifications required to these keywords are rather obvious and come down to changing the mission acronym and setting dates to applicable values. To ensure that the data set can be ingested into the NSSDC deep archive holdings, the START_TIME and STOP_TIME keywords must be set to properly formatted time tags, for example corresponding to the begin and end times of the coverage of the reconstructed SPK or CK files included in the archive. Neither ``N/A'' nor ``UNK'' values are allowed for these two keywords. -- some paragraphs in the description provided in the DATA_SET_DESC keyword, specifically those in the ``Data Set Overview'' section. Most other paragraphs normally provide generic descriptions and can be left ``as is''; still, it is wise to check them to see if they contain any mission specific bits and change those as needed. -- description provided in the CONFIDENCE_LEVEL_NOTE keyword. This is where most of the writing work needs to be done. This description should be re-written to address the kinds of data provided by kernels of each type in the data set being prepared. When doing this NAIF suggests one maintain the structure of the original description (section breakdown, paragraph structure, etc.) and keep the closing sentences of each block (``More information about ...'') intact. -- DATA_SET_MISSION, DATA_SET_TARGET and DATA_SET_HOST objects at the bottom of the file. Modifications required to the keywords in these objects are rather obvious and come down to changing the mission and target names and instrument host acronym. If a mission has more than one target or includes more than one instrument host, additional objects should be added to the file to reflect that (see ``spiceds.cat for Deep Impact for examples). While any recent ``spiceds.cat'' file would be a good starting point it is recommended to look at the ``spiceds.cat'' files for other missions because, while following the same overall structure, they may contain kernel descriptions (in the CONFIDENCE_LEVEL_NOTE value) that may serve as a better template for the kernels being archived. A well written ``spiceds.cat'' put together for the first release may not need to be modified for future releases unless new kinds of kernels not reflected in it get added to the data set. Other Documents in SPICE Data Sets The ``*info.txt'' files in data subdirectories and ``spiceds.cat'' make up about a half of the description files that should be present in the data set. Recommendations for preparing the other half, specifically these files: aareadme.htm aareadme.lbl aareadme.txt errata.txt voldesc.cat catalog/catinfo.txt catalog/insthost.cat catalog/mission.cat catalog/person.cat catalog/ref.cat catalog/release.cat catalog/spice_hsk.cat catalog/spice_inst.cat document/docinfo.txt document/lblinfo.txt document/onlabels.txt extras/extrinfo.txt extras/mk/mkinfo.txt index/indxinfo.txt software/softinfo.txt are outlined file-by-file below. As with the data ``*info.txt'' and ``spiceds.cat'' files, the recommended approach to modifying most of these files is to start with a version from one of the recently archived SPICE data sets (for example, MRO) and change it while keeping the same structure, description layout and keywords to apply to the set of kernels for the new mission. Note that some of the files in this list -- mission, instrument host and references catalogs files -- simply need to be obtained from the project's person/group that is responsible for putting them together. -- aareadme.lbl Change mission acronym in DOCUMENT_NAME and NOTE; change dates in PUBLICATION_DATEs -- aareadme.txt Change mission specific information in the title and ``Introduction'' section; leave ``as is'' descriptions in ``File Formats'' and ``Volume Contents''; if needed, modify directory tree to add and/or remove lines that don't apply to the structure of the data set; change ``Whom to Contact for Information'' and ``Cognizant Persons'' as needed -- aareadme.htm Duplicate changes made in aareadme.txt; -- errata.txt Change mission acronym in NOTE and date in PUBLICATION_DATE; change the list of values of the INSTRUMENT_NAME keyword to match the values in the labels in the new data set (this change is best done after the labels have been generated and these values are known); additional errata related to missing and/or incomplete data or liens may be added to this file if needed; -- voldesc.cat change label revision note; change target name in VOLUME_SERIES_NAME; change mission specific items -- name and acronym -- in VOLUME_SET_NAME, VOLUME_SET_ID, VOLUME_NAME, VOLUME_ID, MRO-M-SPICE-6-V1.0 and DESCRIPTION; change all keywords in the DATA_PRODUCER object as needed; if additional catalog files are present (more than one instrument host and/or target), add pointers to them in the CATALOG object (see the Deep Impact data set for examples); if any of the catalog file names changed, make sure that they are used in the corresponding pointers -- catalog/insthost.cat Obtain this catalog file from the responsible person/group; there may be more than one instrument host associated with a mission; if so, more than one instrument host catalog file will need to be present and they will have to have different names (make sure that ``voldesc.cat'' and ``catalog/catinfo.txt'' reflect that) -- catalog/mission.cat Obtain this catalog file from the responsible person/group; -- catalog/ref.cat Obtain this catalog file from the responsible person/group; -- catalog/person.cat Change information in this file as needed; add/remove objects if more/fewer data providers/preparers should be included -- catalog/release.cat Change LABEL_REVISION_NOTE; for the first release, leave only one object with RELEASE_ID = 0001; set its DATA_SET_ID to the data set's ID; change mission acronym in DISTRIBUTION_TYPE; change DESCRIPTION as needed; for each subsequent release add one release object to the file with the next sequential ID and change RELEASE_ID, RELEASE_PARAMETER_TEXT, and DESCRIPTION as needed while keeping all other keywords the same; -- catalog/spice_hsk.cat Change LABEL_REVISION_NOTE; assuming that the data set will be delivered to NAIF, change DATA_SET_ID and RESOURCE_ID in both objects to use correct data set ID, change mission name and data set ID in ``Basic Browser'' RESOURCE_LINK, and change the data set and volume levels in the ``NAIF On line Archives'' RESOURCE_LINK; -- catalog/spice_inst.cat Change LABEL_REVISION_NOTE; change mission acronym in INSTRUMENT_HOST_ID; -- catalog/catinfo.txt Change date in PUBLICATION_DATE; change volume ID (nnnSP_1000) in NOTE and text; change mission and spacecraft acronyms and names in the text; change ``Contact Information''; if additional catalog files were added to the catalog directory, chance text to reflect that (see the Deep Impact data set for an example) -- document/lblinfo.txt Change date in PUBLICATION_DATE; replace mission acronym in NOTE, title, and descriptions; change contact information; -- document/onlabels.txt It is best to update this file when a label for a SPICE product has been generated; change date in PUBLICATION_DATE; change mission in the NOTE; cut-and-paste actual label into the ``Example label'' section; update mission-specific keyword descriptions in the ``Definition of Keywords/Values for SPICE Kernels'' section using information from the example label (pay special attention to description of DATA_SET_ID, SPACECRAFT_CLOCK_START_COUNT, and SPACECRAFT_CLOCK_START_COUNT) -- document/docinfo.txt Change date in PUBLICATION_DATE; replace mission acronym in NOTE, title, and descriptions; change contact information; if more documents or subdirectories were added to the ``document'' directory, change description to reflect that (see the CASSINI data set for examples) -- extras/mk/mkinfo.txt Change date in PUBLICATION_DATE; change mission name and acronym in the title and descriptions; modify file naming description as needed (see SDU, MRO, and MER data set for different examples of meta-kernel naming); modify the ``guidelines'' paragraph and bulleted list to explain how the list of kernels were picked (see the MGS, MRO, MER, SDU, Deep Impact data sets for examples); change contact information; -- extras/extrinfo.txt Change date in PUBLICATION_DATE; change mission name and acronym in the title and descriptions; change contact information; if additional files or subdirectories are present in ``extras'' change the description to reflect that (see MRO and CAS data sets for examples) -- index/indxinfo.txt Change date in PUBLICATION_DATE; (that's all) -- software/softinfo.txt Change date in PUBLICATION_DATE; (that's all) When so many small changes need to be made by hand in so many files it is easy to overlook some corrections that have to be done. Checking the files that have been updated after all updates have been made is essential to catch that. The simplest ways to do such checking are using ``tkdiff'' (or a similar difference visualization tool) to compare the new version with the original version, and using ``grep'' to look for various tokens that should and should not appear in the new description such as new and old mission acronyms, new and old data set IDs, publication dates, and so on. Step 3: Labeling Data -------------------------------------------------------- Once the complete set of kernels that should go into the archive is available the next step is to generate PDS labels for each of these files. This step includes collecting files in the ``staging area'', creating the MAKLABEL template, creating the kernel list file for input to the ``label_them_all.pl'' script, and running the script to generate labels. To be fair it must be mentioned that labels generated by the process described in this section raised a number of concerns by archived SPICE data set peer-reviewers and other members of the PDS community. These concerns are discussed in ``Appendix 1'' of this document. Staging data ``Staging area'' should have the same directory structure as the archive. The ``make_pds_directories.csh'' can be used to make the required directory structure during initial staging area setup. Kernels of each type should be collected (by copying, moving, scp'ing , etc) in the corresponding subdirectory under the ``data'' directory. For example, all SPK files should go into ``data/spk'', all CKs into ``data/ck'' and so on. The files must be staged with the names under which they will appear in the archive. Preparing MAKLABEL Template The labels for the kernels are generated by the MAKLABEL program available from the utilities page of the NAIF Web server, invoked by the ``label_them_all.pl'' script. MAKLABEL creates labels with the same set of keywords and structure for all SPICE kernel types: PDS_VERSION_ID = PDS3 RECORD_TYPE = RECORD_BYTES = ^SPICE_KERNEL = MISSION_NAME = SPACECRAFT_NAME = DATA_SET_ID = KERNEL_TYPE_ID = PRODUCT_ID = PRODUCT_CREATION_TIME = PRODUCER_ID = MISSION_PHASE_NAME = PRODUCT_VERSION_TYPE = PLATFORM_OR_MOUNTING_NAME = START_TIME = STOP_TIME = SPACECRAFT_CLOCK_START_COUNT = SPACECRAFT_CLOCK_STOP_COUNT = TARGET_NAME = INSTRUMENT_NAME = NAIF_INSTRUMENT_ID = SOURCE_PRODUCT_ID = NOTE = OBJECT = SPICE_KERNEL INTERCHANGE_FORMAT = KERNEL_TYPE = DESCRIPTION = END_OBJECT = SPICE_KERNEL END The program automatically generates values for many of these keywords using information that it collects from the files. For example, the program assigns the file name to the ^SPICE_KERNEL and PRODUCT_ID keywords, sets PRODUCT_CREATION_TIME to current CPU time, sets START_TIME and STOP_TIME to coverage begin and end times obtained by examining the data, sets RECORD_TYPE, RECORD_BYTES, KERNEL_TYPE_ID, INTERCHANGE_FORMAT, and KERNEL_TYPE based on the kernel type, and so on. Not all of the keywords can be populated with values using file data and not all automatically generated values are acceptable. For these keywords MISSION_NAME = SPACECRAFT_NAME = DATA_SET_ID = PRODUCER_ID = MISSION_PHASE_NAME = PRODUCT_VERSION_TYPE = PLATFORM_OR_MOUNTING_NAME = TARGET_NAME = INSTRUMENT_NAME = SOURCE_PRODUCT_ID = NOTE = values have to be set using a special setup file called template. While the MAKLABEL User's Guide provides complete details on template format and content and should be the ultimate source of this information, one concept from it needs to be repeated here to make information provided later in this section understandable. This is the concept of the ``default'' template section and ``optional'' template sections. The default section of the template contains keyword=value pairs that apply to all labels while the optional sections contain keyword-value pairs that apply to the output label only when the option tag is provided on MAKLABEL's command line. The default section appears at the top of the template file before the first option tag line, which is a line that starts with ``--'' followed by a tag consisting of uppercase letters, numbers and ``_''. An optional section appears between one option tag line and the next option tag line and all keywords specified in it are placed in the output label if the option tag from the leading line is specified on the MAKLABEL command line. For example, this small template MISSION_NAME = "GALACTIC" PRODUCER_ID = "UNK" --NAIF PRODUCER_ID = "NAIF" tells MAKLABEL to set MISSION_NAME to "GALACTIC" in all labels (because it appears in the default section), and to set PRODUCER_ID to "UNK" when no option tags are specified in the command line (because it appears in the default section) and to set it to "NAIF" when the NAIF option tag is specified on the command line (because it also appears in the optional section with tag NAIF). Brief guidelines on how to set up a template for the keywords that cannot (or should not) be set using the file data are provided below. While reading these guidelines, it may be helpful to look at the examples of MAKLABEL templates for various missions that are included in the ``examples/templates'' directory of this package. -- MISSION_NAME MISSION_NAME must be set in the default section to the value of MISSION_NAME specified in ``mission.cat''. (See MRO template for examples.) -- DATA_SET_ID DATA_SET_ID must be set in the default section to the value of DATA_SET_ID specified in ``spiceds.cat''. (See MRO template for examples.) -- TARGET_NAME TARGET_NAME must be set in the default section to the name or the list of names of the mission's primary target(s) as specified in the target catalog files. (See MRO, DS1 and SDU templates for examples.) If the mission does not have a primary target or it has too many targets, TARGET_NAME can be set in the default section to "N/A". (See the CASSINI template for an example.) -- SOURCE_PRODUCT_ID SOURCE_PRODUCT_ID can be set in the default section to "N/A". -- NOTE NOTE can be set in the default section to "See comments in the file for details". -- PRODUCT_VERSION_TYPE Since the majority of archived kernels usually contain reconstructed data, PRODUCT_VERSION_TYPE should be set in the default section to "ACTUAL". An optional section with tag PREDICT setting PRODUCT_VERSION_TYPE to "PREDICT" can be included to allow putting this value into the labels for files with predicted data. (See MRO template for examples.) -- SPACECRAFT_NAME If a single spacecraft is associated with the mission, SPACECRAFT_NAME must be set in the default section to the value of INSTRUMENT_HOST_NAME specified in ``insthost.cat''. (See MRO template for examples.) If more than one spacecraft is associated with the mission, SPACECRAFT_NAME must be set in the default section to a list of values of INSTRUMENT_HOST_NAME specified in all instrument host catalog files. Then a set of optional sections, in each of which SPACECRAFT_NAME is set to an individual INSTRUMENT_HOST_NAME, should be added; the spacecraft acronyms can be used as tags for these sections. One of these tags should be used when MAKLABEL is run to make labels for kernels that apply for only one spacecraft. For the kernels that apply to all spacecraft, these tags will be omitted resulting in the list of all spacecraft names specified in the default section being put into the label. (See the Deep Impact template for examples.) -- PRODUCER_ID A set of optional sections, in each of which PRODUCER_ID is set to a particular producer ID, must be set up; the producer acronyms can be used as tags for these sections. One of these tags should be used when MAKLABEL is run to make labels for kernels created by a particular producer. (See the MRO template for examples.) -- INSTRUMENT_NAME and PLATFORM_OR_MOUNTING_NAME A set of optional sections, one for each instrument, setting INSTRUMENT_NAME to the value of INSTRUMENT_NAME from that instrument's catalog file and setting PLATFORM_OR_MOUNTING_NAME to the name of the platform on which the instrument is mounted, must be set up; the instrument acronyms can be used as tags for these sections. One of these tags should be used when MAKLABEL is run to make labels for kernels that apply to a particular instrument (normally only for IKs). (See the Cassini template for examples.) -- MISSION_PHASE_NAME MISSION_PHASE_NAME should be set in the default section to "N/A" to be used for any kernels that cover the whole mission or for which the notion of coverage is not applicable. (See the MRO template for examples.) Then a set of optional sections, one for each of the mission phases defined in the DESCRIPTION section of ``mission.cat'', setting MISSION_PHASE_NAME to a particular mission phase name, must be set up; an abbreviated mission phase name can be used as tags for these sections. One of these tags should be used when MAKLABEL is run to make labels for kernels the coverage of which falls completely within that mission phase. (See the MRO template for examples.) Additional optional sections setting MISSION_PHASE_NAME to lists containing names for two or more adjacent mission phases may need to be created if some kernels have coverage that spans mission phase boundaries. (See MRO template for examples.) Usually the MAKLABEL template prepared and used to label files in the first release can be used without changes for all subsequent releases. In some cases though, for example when data from a new data producer need to be added to the archive or when an additional mission phase was added to the mission time line, the template may need to be augmented with additional optional sections. Preparing Kernel List File After the MAKLABEL template has been prepared, the next step is to put together a kernel list file that, as the name suggests, lists all kernels that will be added to the archive in the release that is being prepared. The list file is the main input to the ``label_them_all.pl'' script and, in addition to listing kernels, provides a set of MAKLABEL options (tags from the template) and a description for each of the files as well as some general information such as the data set ID, release ID, etc. (Note that a concatenation of all kernels lists for an archive is used as the primary input to the script that generates index files, ``xfer_index.pl'', discussed later in this document.) The kernel list file for a particular release must have the following content and structure: DATE = SPACECRAFT = NAIFER = PHONE = EMAIL = VOLUMEID = RELEASE_ID = RELEASE_DATE = EOH FILE = MAKLABEL_OPTIONS = DESCRIPTION = FILE = MAKLABEL_OPTIONS = DESCRIPTION = ... FILE = MAKLABEL_OPTIONS = DESCRIPTION = The first five keywords -- DATE, SPACECRAFT, NAIFER, PHONE, EMAIL -- and the ``EOH'' end-of-the-header marker are optional and are included to provide identification information. These keywords are not used by the ``label_them_all.pl'' or ``xfer_index.pl'' scripts. They are a ``carry over'' required by an earlier incarnation of NAIF's archive scripts. The VOLUMEID, RELEASE_ID, and RELEASE_DATE keywords are required and must be set as follows: VOLUMEID must be set to the lowercased version of the volume ID (for example ``mgsp_1000'' for MGS), RELEASE_ID must be set to the release ID number (for example ``0001'' for release 1), and RELEASE_DATE must be set to the date on which the data will be released to the public (for example ``2007-07-27'' for July 27, 2007.) The rest of the kernel list file must provide triplets of lines, one for each of the files that constitute the release, containing FILE, MAKLABEL_OPTIONS, and DESCRIPTION keywords. The FILE line must always be first, followed by the MAKLABEL_OPTIONS line followed by the DESCRIPTION line. The FILE keyword must provide the file name relative to the volumes's root directory (for example ``data/spk/de403.bsp''). The MAKLABEL_OPTIONS keyword must provide all MAKLABEL option tags applicable to the file named in the preceding FILE keyword. The option tags must be delimited by one or more spaces and will be passed ``as is'' to the MAKLABEL program. If no options are applicable to a file, MAKLABEL_OPTIONS can be set to blank but the line containing it must still be present in the list file, following the FILE keyword line. The DESCRIPTION keyword must provide a brief description of the file; this description will be inserted in the file label to replace the generic description generated by the MAKLABEL program. The value must be on the same line as the keyword and must not ``spill'' over onto the next line(s). The length of the description is not limited. DESCRIPTION can be set to blank but the line containing it must still be present in the list file, following the MAKLABEL_OPTIONS keyword line. When description is set to blank, "N/A" is placed in the label. The list file may contain blank lines as long as they are not placed between the lines in each of the triplets. Normally the kernel list files are kept in the data set root directory of the staging area. The package accompanying this document contains the ``examples/listfiles'' directory with numerous examples of kernel list files for various missions. These list files can be used as references or even as the starting point for preparing kernel list files for a new archive. Running the Labeling Script Once a kernel list file has been prepared the ``label_them_all.pl'' script can be run to generate labels for staged kernels. Before doing that it should be verified that each of the staged text kernels has an architecture/type token -- ``KPL/LSK'', ``KPL/SCLK'', ``KPL/PCK'', and so on -- on the first line of the file. It is required for the MAKLABEL and ARCHTYPE programs, as described in the programs' User's Guides. If the architecture/type token is missing in the file, it must be added using a text editor. It should also be verified that all binary files have format -- BIG-IEEE or LTL-IEEE -- native to the machine on which the script will be run. The BFF utility program available on the ``Utilities'' page on the NAIF web site can be used to display the binary format of one or more binary kernels, like this: > bff Any kernels that have a binary format different from the native binary format of the workstation (the VERSION program provided with the toolkit displays the native format when run with the ``-a'' option) must be converted to native format prior to being labeled. The BINGO utility program available on the ``Utilities'' page on the NAIF web site can be used to do such conversions. The script must be started from the volume's root directory as follows: > label_them_all.pl