-------------------------------------------------------- -------------------------------------------------------- Reference Guide for making SPICE bundles with NPB -------------------------------------------------------- -------------------------------------------------------- What do you need to create a PDS4 SPICE archive? - The SPICE Kernels - NAIF PDS4 Bundler (NPB) - PDS4 Validate -------------------------------------------------------- Steps to create a SPICE PDS4 archive -------------------------------------------------------- Step 1 – Install the necessary software -------------------------------------------------------- - NPB - PDS4 Validate Step 2 – Setup your workspace -------------------------------------------------------- kernels/ - the SPICE kernels that will be collected to generate the bundle in the recommended sub-directory structure. working/ - Houses the NPB configuration files, release plans, and NPB run by product files for each release. working/document/ - contains the archive PDS4 artifacts that must be given as input: the archive description (spiceds) file and the readme file staging/ - This directory is used to store the files generated by the pipeline for the archive increment during the run. ‘mission’/ –> insight/ -This directory is the final output directory of the process, where the bundle is generated. This directory should contain the files already present in the archive (after R1). npb_insight |-- kernels |-- staging |-- insight |-- working |-- document Step 3 – Create the necessary files -------------------------------------------------------- Release Plan - ex. insight_release_01.plan - A simple text file - Lists all kernels for this release - Must be named like – insight_release_01.plan NPB Config File - ex. insight_release_01.xml - This is an XML file that sets up the directory locations, kernel locations, kernel file patterns etc… https://nasa-pds.github.io/naif-pds4-bundler/42_npb_configuration_file.html Archive Description File – spiceds_v001.html This will need to be created for release 1 and new versions will be added as needed for subsequent releases. Readme.txt - This will need to be created for release 1 and remain the same. No updates should be needed. npb_insight |-- kernels |-- staging |-- insight |-- working `-- insight_release_01.plan <- input, kernels to archive `-- insight_release_01.xml <- input, configuration file |-- document `-- spiceds_v001.html `-- readme.txt More information - https://nasa-pds.github.io/naif-pds4-bundler/32_step_2_npb_setup.html Step 4 – Do a test/staging NPB run -------------------------------------------------------- Starting from your main working directory – not within a sub-directory Ex. % source ~/virtenvs/npb/bin/activate.csh % set REL = 01 naif-pds4-bundler /path/to/NPB_config.xml -p /path/to/release.plan -l -v -f staging -p PLAN = uses the provided release plan to gather the correct kernels -l = creates a log file -v = verbose – prints log to the screen as it runs -f staging = tells NPB to stop at staging – not to create the final bundle % naif-pds4-bundler working/insight_release_"$REL".xml -p working/insight_release_"$REL".plan -l -v -f staging npb_insight |-- kernels |-- staging | |-- insight_spice |-- insight |-- working `-- PDS4_PDS_1500.sch <- output, publicly available PDS4 Schematron `-- PDS4_PDS_1500.xsd <- output, publicly available PDS4 Schema `-- insight_release_01.checksum <- output `-- insight_release_01.file_list <- output `-- insight_release_01.kernel_list <- output `-- insight_release_01.log <- output `-- insight_release_01.plan <- input, kernels to archive `-- insight_release_01.validate_config <- output `-- insight_release_01.xml <- input, configuration file |-- document `-- spiceds_v001.html `-- readme.txt More information - https://nasa-pds.github.io/naif-pds4-bundler/33_step_3_running_npb.html Step 5 – Run it for real -------------------------------------------------------- You can run as many staging runs as you want/need *BUT* you need to reset your workspace accordingly! VERY IMPORTANT ** Make sure all test run files have been deleted! ** ** Clear the staging directory ** Remove the NPB created files from the working directory insight_release_01.file_list insight_release_01.checksum insight_release_01.kernel_list insight_release_01.log insight_release_01.validate_config Run NPB without ‘-f staging’ Do this from your main working directory – not within a sub-directory Ex. % naif-pds4-bundler working/insight_release_"$REL".xml -p working/insight_release_"$REL".plan -l -v Check the log for warnings and errors! npb_insight |-- kernels |-- staging | |-- insight_spice |-- insight | |-- insight_spice | ... `-- working `-- insight_release_01.checksum <- output `-- insight_release_01.file_list <- output `-- insight_release_01.kernel_list <- output `-- insight_release_01.log <- output `-- insight_release_01.plan <- input, kernels to archive `-- insight_release_01.validate_config <- output `-- insight_release_01.xml <- input, configuration file -- document `-- spiceds_v001.html `-- readme.txt More information - https://nasa-pds.github.io/naif-pds4-bundler/33_step_3_running_npb.html Step 6 – PDS4 Validate -------------------------------------------------------- Validate the entire bundle (insight_spice) you just created! To use Validate on a complete bundle– % validate /path/to/bundle –c /path/to/validate_config –R pds4.bundle –r /path/for/validate_report.txt % set REL = 01 % validate insight/insight_spice –c working/insight_release_"$REL".validate_config -R pds4.bundle -r working/validate_"$REL"_report_`date +'%Y-%m-%d'`.txt Validate will run , print the number of seconds it took to execute and leave the report file for you to review. The bottom of the report will show the count of passes/fails. More information - https://nasa-pds.github.io/naif-pds4-bundler/34_step_4_checking_the_result.html Step 7 – SPICE checks of full archive -------------------------------------------------------- Check the Meta-Kernel --- % cd insight_spice/spice_kernels/mk % set MK = insight_v01.tm % echo $MK Shows SPK Coverage for all files ( -a) , center of motion (-c), in UTC (-utc) , tabular format (-t) % brief -a -t -c -utc $MK Shows SPK coverage for insight Spacecraft % brief -a -t -utc --155 $MK Displays all CK frames (-a) with frame names (-n) in tabular format (-t) in UTC (-utc) relative to other frames (-rel) % ckbrief -a -t -rel -n -utc $MK Checks for gaps in CK coverage % frmdiff -k $MK -t dumpg -f1 J2000 -t1 insight_SPACECRAFT -f YYYY-MM-DDTHR:MN:SC ::RND -------------------------------------------------------- Steps to add SPICE PDS4 Release 2 to the archive -------------------------------------------------------- Step 1 – Update the necessary software - if needed -------------------------------------------------------- Step 2 – Start with your previous setup as created for release 1 -------------------------------------------------------- Make sure to clear out staging/ npb_insight |-- kernels |-- staging |-- insight | |-- insight_spice | ... `-- working `-- insight_release_01.checksum `-- insight_release_01.file_list `-- insight_release_01.kernel_list `-- insight_release_01.log `-- insight_release_01.plan `-- insight_release_01.validate_config `-- insight_release_01.xml -- document `-- spiceds_v001.html `-- readme.txt Step 3 – Create the necessary files for release 2 -------------------------------------------------------- Release Plan - A simple text file - Lists all kernels for release 2 - Must be named like – insight_release_02.plan If needed -- - Make a new version of the Archive Description File - Starting with your previous file as the base… % cp spiceds_v001.html spiceds_v002.html % vi spiceds_v002.html – spiceds_v002.html Examples of needing a new version.. to include new types of files Removing/adding ‘Errata’ items etc.. NPB Config File - Starting with release 1 config …. % cp insight_release_01.xml insight_release_02.xml % vi insight_release_02.xml - Need updated path to the archive description (spiceds) with the updates for release 2 NOTE - If no updates were needed then this attribute () can be removed. - Double check the workspace and kernel paths are accurate - Check filename patterns, add any missing ones in ‘Kernel List’ npb_insight |-- kernels |-- staging |-- insight | |-- insight_spice <- exactly the same from release 1 |-- working `-- insight_release_01.checksum `-- insight_release_01.file_list `-- insight_release_01.kernel_list `-- insight_release_01.log `-- insight_release_01.plan `-- insight_release_01.validate_config `-- insight_release_01.xml `-- insight_release_02.plan <- input, kernels to archive `-- insight_release_02.xml <- input, configuration file -- document `-- spiceds_v001.html `-- readme.txt Step 4 – Do a test/staging NPB run -------------------------------------------------------- Do this from your main working directory – not within a sub-directory Ex. % source ~/virtenvs/npb/bin/activate.csh % set REL = 02 naif-pds4-bundler /path/to/NPB_config.xml -p /path/to/release.plan -l -v -d files -f staging -p PLAN = uses the provided release plan to gather the correct kernels -l = creates a log file -v = verbose – prints log to the screen as it runs -d files = makes diff html files to compare the current release to the previous release -f staging = tells NPB to stop at staging – not to create the final bundle % naif-pds4-bundler working/insight_release_"$REL".xml -p working/insight_release_"$REL".plan -l -v -d files -f staging Be sure the review the diff files in a web browser! npb_insight |-- kernels |-- staging | |-- insight_spice <- has only release 2 files |-- insight | |-- insight_spice <- exactly the same from release 1 |-- working `-- insight_release_01.checksum `-- insight_release_01.file_list `-- insight_release_01.kernel_list `-- insight_release_01.log `-- insight_release_01.plan `-- insight_release_01.validate_config `-- insight_release_01.xml `-- insight_release_02.checksum <- output `-- insight_release_02.file_list <- output `-- insight_release_02.kernel_list <- output `-- insight_release_02.log <- output `-- insight_release_02.plan <- input, kernels to archive `-- insight_release_02.validate_config <- output `-- insight_release_02.xml <- input, configuration file -- document `-- spiceds_v001.html `-- readme.txt Step 5 – Run it for real -------------------------------------------------------- Make sure all test run files have been deleted! – VERY IMPORTANT Clear the staging directory *** Remove the NPB created files from the working directory*** insight_release_02.file_list insight_release_02.kernel_list insight_release_02.checksum insight_release_02.log insight_release_02.validate_config diff_* Run NPB without ‘-f staging’ Do this from your main working directory – not within a sub-directory Ex. % naif-pds4-bundler working/insight_release_"$REL".xml -p working/insight_release_"$REL".plan -l -v -d files npb_insight |-- kernels |-- staging | |-- insight_spice <- has only release 2 files |-- insight | |-- insight_spice <- has release 1 and release 2 files - full archive |-- working `-- insight_release_01.checksum `-- insight_release_01.file_list `-- insight_release_01.kernel_list `-- insight_release_01.log `-- insight_release_01.plan `-- insight_release_01.validate_config `-- insight_release_01.xml `-- insight_release_02.checksum <- output `-- insight_release_02.file_list <- output `-- insight_release_02.kernel_list <- output `-- insight_release_02.log <- output `-- insight_release_02.plan <- input, kernels to archive `-- insight_release_02.validate_config <- output `-- insight_release_02.xml <- input, configuration file -- document `-- spiceds_v001.html `-- readme.txt Step 6 – PDS4 Validate -------------------------------------------------------- Validate the entire Bundle (insight_spice) that has release 1 and release 2! To use Validate on a complete bundle– validate /path/to/bundle –c /path/to/validate_config –R pds4.bundle –r /path/for/validate_report.txt % set REL = 02 % validate insight/insight_spice –c working/insight_release_"$REL".validate_config -R pds4.bundle -r working/validate_"$REL"_report_`date +'%Y-%m-%d'`.txt Step 7 – SPICE checks of full archive -------------------------------------------------------- Check the Meta-Kernel --- % cd insight_spice/spice_kernels/mk % set MK = insight_v02.tm % brief -a -t -utc --155 $MK % frmdiff -k $MK -t dumpg -f1 J2000 -t1 insight_SPACECRAFT -f YYYY-MM-DDTHR:MN:SC ::RND -------------------------------------------------------- Resources -------------------------------------------------------- NPB Documentation --- https://nasa-pds.github.io/naif-pds4-bundler/ SpiceyPy --- https://spiceypy.readthedocs.io/en/stable/ PDS4 Validate Tool --- https://nasa-pds.github.io/validate/index.html