Index Page
Converting and Porting SPICE Binary Data Files

Table of Contents 


   Converting and Porting SPICE Binary Data Files
      Abstract
      Revisions
      Introduction
      Binary File Format
      Reading Non-native Binary Data Files
      Restrictions on Using Non-native Binary Data Files

   Utility Programs for Porting SPICE Files.
      The Two Conversion Methods
         TOBIN and TOXFR -- Command Line Based File Conversion.
         Examples Using TOBIN and TOXFR.
         SPACIT -- Interactive File Conversion.


Top

Converting and Porting SPICE Binary Data Files




Last revised on 2017 MAR 10 by N. J. Bachman.



Top

Abstract



SPICE provides a mechanism and a set of utilities to facilitate moving SPICE binary data files from one computing environment to another.



Top

Revisions



March 10, 2017

January 13, 2002

    The contents of this guide were extensively revised to reflect the inclusion of the DAF run-time binary file format translation capability into the SPICE toolkit.



Top

Introduction



This document describes issues related to moving SPICE binary data files from one computing environment to another. Often a SPICE data file (or kernel) is generated within one particular environment for use in another. NAIF has developed software that allows users to read binary SPICE data files generated on most supported platforms by simply transferring the file directly. In a few cases, ``transfer format'' software, available in the SPICE Toolkit, must be used. The remainder of this document discusses when binary kernels may be exchanged directly, what restrictions exist in using files from other computing environments, and the mechanisms for exchanging data with users of environments that do not support this capability.



Top

Binary File Format



SPICE binary data files are Fortran direct access unformatted files, i.e., they contain data in a raw binary form. The specifics of this form, henceforth referred to as binary file format, are determined by characteristics of the computing environment used to generate the file. Factors such as word size and floating point representation control exactly how raw data are placed into the file at the byte level. Computing environments that share the same binary file format may freely exchange SPICE binary data files as though the original files were produced in their environment. Files whose binary file format agrees with the local computing environment are referred to as native format files, or just native files. Similiarly those whose format disagrees, are commonly referred to as non-native.

The collection of computing environments for which SPICE toolkits exist (often called supported environments) utilize four distinct binary file formats: Big Endian IEEE, Little Endian IEEE, Vax D-Floating, and Vax G-Floating. The breakdown by computing environment is as follows: 

 
   Big Endian IEEE:
   ---------------
      HP:
         HP Fortran, HP C
      Macintosh:
         Absoft Fortran, Metrowerks Codewarrior C
      SGI:
         N32/O32 Fortran, MIPS C (N32/O32 ABIs)
      Sun:
         Solaris Fortran, Solaris Native C, Solaris GCC
 
   Little Endian IEEE:
   ------------------
      PC:
         Linux G77, Compaq Visual Fortran, Lahey Fortran 95,
         Linux GCC, Microsoft Visual C++
      Alpha (Digital Unix):
         Digital Fortran, Digital C
 
   Vax D-Floating:
   --------------
      Alpha (VMS):
         Digital Fortran
      VAX:
         Digital Fortran
 
   Vax G-Floating:
   --------------
      Alpha (VMS):
         Digital Fortran
For example, a SPICE binary data file generated on an HP using Fortran can be moved directly to a Sun using CSPICE with the native C compiler (or for that matter any supported compiler). As the table above illustrates, these systems utilize the same binary file format, making such an exchange possible.



Top

Reading Non-native Binary Data Files



The preceding section implies that users of computing environments utilizing dissimiliar binary file formats may not exchange binary files directly. In general this is true, however, the Toolkit software on most supported computing environments includes a run-time binary file format translation, or binary translation, capability. This enables these computing environments to read some non-native binary files, with some restrictions noted in the section that follows.

The diagram below illustrates which computing environments are capable of reading non-native files. The arrows in the diagram indicate the direction in which the files may be exchanged. 

 
    ----------              ----------
   | Big      |----------->| Little   |
   | Endian   |            | Endian   |
   | IEEE     |<-----------| IEEE     |
    ----------              ----------
 
 
    ----------              ----------
   | Vax      |            | Vax      |
   | D-       |            | G-       |
   | Floating |            | Floating |
    ----------              ----------
As the above diagram illustrates, any Big Endian IEEE supported computing environment is able to read binary files generated on Little Endian IEEE environments. The reverse is also true, namely that Little Endian IEEE environments are capable of reading binary files produced on Big Endian IEEE environments.

However, users on these systems may not read binaries produced in either Vax D-Floating or Vax G-Floating environments. For file exchange between these environments the SPICE Toolkit provides the mechanisms discussed in the following chapter, using the ``SPICE Transfer format.''



Top

Restrictions on Using Non-native Binary Data Files



The binary translation capability provides access to non-native binary files on most supported environments. However, there are several restrictions that require noting:

    -- As of Toolkit Version N0066 this capability is implemented both for DAF-based files (SPK, CK, and binary PCK) and DAS-based files (DSK, EK, DBK, type 1 star catalogs).

    -- The binary translation capability provides ``read-only'' access to non-native binary files. Attempts to add, alter, or remove comments from non-native binary files (using tools such as COMMNT) will fail. Also, appending or altering the data contents of non-native binary files (using MKSPK or other mission specific tools) will also fail.

    -- Using non-native files may impact the performance of SPICE based applications. The impact is highly dependent on the particular computing environment and the usage scenario. Typically for SPK files the speed decrease is somewhere between 5 and 10 percent, but may be as large as a factor of 2 or 3 for highly read intensive scenarios. For DSK files a factor of 3 is to be expected.

If the ``read-only'' nature of access or the performance degradation are issues, then converting the non-native binary files into native format is recommended. This can be accomplished using TOBIN and TOXFR (tools included in the SPICE Toolkit) in the following fashion:

First convert the non-native binary to transfer format using TOXFR: 

    > toxfr non-native.bsp transfer.xsp
Then convert the transfer format file to native binary format using TOBIN: 

    > tobin transfer.xsp native.bsp
TOXFR, on environments that support binary translation, can create transfer files from the supported non-native binary file formats. For example, TOXFR on a Sun can make a transfer format file from a PC binary. However, the same version of TOXFR may not make a transfer format file from any file originating from either of the Vax formats.



Top

Utility Programs for Porting SPICE Files.




If it is the case that one of the computing environments generating or using files is not supported for binary translation, several tools are delivered with the SPICE Toolkit to address this issue.

In these situations, the portability of SPICE binary data files is achieved by using an intermediate file called a ``transfer'' file. A transfer file contains an environment independent representation of the data from a binary SPICE file that consists only of ASCII characters, and hence is easily moved from one computing environment to another. There are three steps involved in moving the information in a SPICE binary data file from a host, or originating, computing environment to a target, or destination, computing environment.

    1. Convert the SPICE binary data file into the environment independent transfer file in the host computing environment. The utility programs described later in this document provide this capability.

    2. Move the environment independent transfer file to the target computing environment. (Typically using ASCII mode of FTP).

    3. Convert the environment independent transfer file into a SPICE binary data file in the target computing environment. The utility programs described later in this document provide this capability.



Top

The Two Conversion Methods



There are two methods for converting SPICE data files from one of their formats to the other; an interactive method and a command line, ``or batch,'' method. The interactive method makes use of the SPICE Toolkit utility program SPACIT, and the command line method makes use of the SPICE Toolkit utility programs TOXFR and TOBIN which convert files from the binary format to the transfer format and from the transfer format to the binary format, respectively. (Using TOXFR and TOBIN is usually faster than using SPACIT.)



Top

TOBIN and TOXFR -- Command Line Based File Conversion.



The utility programs TOBIN and TOXFR provide a command line, or ``batch,'' oriented method for converting portable transfer files into SPICE binary kernel files and converting SPICE binary kernel files into portable transfer files, respectively. The name ``tobin'' is obviously derived from ``to binary,'' as in ``convert to binary''. The name ``toxfr'' is, possibly not so obviously, derived from ``to transfer,'' as in ``convert to transfer.''

TOBIN converts a SPICE transfer format file into its binary format for use in a particular computing environment. TOBIN requires at least the name of the input transfer file, which is the first argument on the command line. Optionally, a name for the output binary file may be specified as the second argument on the command line. If a name for the output file is provided it overrides the automatic naming conventions used by the program.

If only the name of the input transfer file is provided on the command line, TOBIN will generate a name for the output binary kernel file that has the same base name as the input transfer file and an appropriate filename extension, based on the filename extension of the transfer file. If TOBIN does not recognize the filename extension of the input transfer file, or there is no filename extension on the input transfer file, a filename extension of '.bin' will be used as the filename extension of the output binary file. If a file having the same name as the output file already exists, TOBIN signals an error and stops. We assume that it is bad form to overwrite or replace an existing file.

TOBIN recognizes the transfer filename extensions ``.xc'', ``.xee'', ``.xpc'', and ``.xsp'' associated with transfer files for SPICE data products. These are converted to the SPICE binary kernel filename extensions ``.bc'', ``.bee'', ``.bpc'', and ``.bsp,'' respectively. Also recognized are the old transfer file extensions ``.tc'', ``.tee'', ``.tpc'', and ``.tsp,'' which are converted into the appropriate filename extensions for a binary kernel file.

TOXFR converts a SPICE binary kernel file into a transfer file format that is used to transfer, or port, the binary data to a different computing environment. TOXFR requires at least the name of the input binary file, which is the first argument on the command line. Optionally, a name for the output transfer file may be specified as the second argument on the command line. If a name for the output file is provided on the command line it overrides the automatic naming conventions used by the program.

If only the name of the input binary file is provided on the command line, TOXFR will generate a name for the output transfer file that has the same base name as the input binary file and an appropriate filename extension, based on the filename extension of the binary file. If TOXFR does not recognize the filename extension of the input binary file, or there is no filename extension on the input binary file, a filename extension of '.xfr' will be used as the filename extension of the output transfer file. If a file having the same name as the output file already exists, TOXFR signals an error and stops. Again, we assume that it is bad form to overwrite or replace an existing file.

TOXFR recognizes the filename extensions ``.bc'', ``.bee'', ``.bpc'', and ``.bsp'' which are associated with SPICE binary kernel files. These are converted to the transfer filename extensions ``.xc'', ``.xee'', ``.xpc'', and ``.xsp,'' respectively.

As mentioned previously, it my not be necessary to convert a SPICE binary kernel file into the transfer file format when moving the data from one computing environment to another. If the binary file formats in each of the computing environments are identical, the same binary file may be used without modification. The direct exchange of SPICE binary kernel files is known to work when moving the files among Sun Sparcstation, HP 9000, SGI, and Macintosh computing environments. This is possible because they all use standard UNIX file formats and IEEE representations for floating point numbers.



Top

Examples Using TOBIN and TOXFR.



TOBIN and TOXFR each require the name of the file to be converted and, optionally, the name of the file to be produced, e.g., 

   prompt > tobin thisfile.xsp
and 

   prompt > tobin thisfile.xsp thisfile.bsp
would both produce the file `thisfile.bsp' in the current directory.

In the first example, the name for the binary file produced is derived from the name of the transfer file, using the full base name and replacing the letter `x' in the file name extension with the letter `b', to indicate that the file is a binary file. In the second example, the name for the binary file produced is taken directly from the command line.

    -- Example 1: Convert the transfer file 'ephem.xsp' into a binary kernel file. 

    > tobin ephem.xsp
    This will create the binary kernel file 'ephem.bsp'.

    -- Example 2: This is the same as Example 1, except we explicitly set the output filename on the command line. 

    > tobin ephem.xsp myephem.bsp
    This will create the binary kernel file 'myephem.bsp'.

    -- Example 3: For this example, we provide a filename extension on the input transfer file that TOBIN does not recognize. 

    > tobin mydata.ttt
    This will create the binary file 'mydata.bin'.

    -- Example 4: Convert the binary kernel file 'ephem.bsp' into a transfer file. 

    > toxfr ephem.bsp
    This will create the transfer file 'ephem.xsp'.

    -- Example 5: This is the same as Example 1, except we explicitly set the output filename on the command line. 

    > toxfr ephem.bsp myephem.xsp
    This will create the transfer file 'myephem.bsp'.

    -- Example 6: For this example, we provide a filename extension on the input binary file that TOXFR does not recognize. 

    > toxfr mydata.bbb
    This will create the transfer file 'mydata.xfr'.



Top

SPACIT -- Interactive File Conversion.



The SPACIT utility program is an interactive, menu driven program that provides a file conversion capability, among other things. The SPACIT main menu has an option for converting a SPICE binary kernel file into a portable transfer file, and an option for converting a transfer file into a binary kernel file. 

            SPACIT Options
 
      ( Q ) Quit.
      ( L ) Log SPACIT output to a file.
      ( T ) Convert transfer file to binary file.
      ( B ) Convert binary file to transfer file.
      ( S ) Summarize binary file.
      ( R ) Read comment area of binary file.
 
      Option:
To convert a SPICE binary kernel file into a portable transfer file, select option `B' from the SPACIT main menu. Prompts for the input and output filenames will be given, and then the binary kernel file will be converted. Similarly, to convert a portable transfer file into a SPICE binary kernel file, select option `T' from the SPACIT main menu.

Please see the SPACIT User's Guide, spacit.ug, for complete details on the use of this program.