Page tree
Skip to end of metadata
Go to start of metadata

Introduction

This document records a history of all issues found with CaSSIS data. These issues more relate to the implementation of the PDS standard and in particular the xml label files rather than the data files.

CaSSIS Raw data is being ingested into the PSA daily and is processed by a data processing pipeline (DPS) at ESAC. Currently the latest version of all Raw data files are version 2.0, even if this is the first version, as this assignment was given to all data following a bulk re-processing at ESAC in late 2019/early 2020 and is now assigned operationally to all new data. 

Since version 2.0 of the data has been produced there have been ongoing improvements and bug fixes to the data processing, most significantly following the first CaSSIS data review in April 2020. This means for example that a file produced in May 2020 will contain improvements compared to one produced in February 2020 but both will be data version 2.0, therefore it is also useful to look at the Data Processing System (DPS) version in the file (see below).

The CaSSIS team do not deliver calibrated data in PDS format but instead use an internal xml based format. At ESAC a parser is run to convert the CaSSIS data into PDS4 files and hence the calibrated products are also assigned version 2.0 currently.

All accumulated improvements will be applied in the next bulk re-processing expected at the end of 2020 and over the summer of 2020, due to the accumulated issues with this version, all version 1.0 data will be deleted from the PSA.

Contents:

Data Delivery Status

  • Raw data without full (geometry) labels are generated daily
  • The labels are updated with the geometry and instrument status information following deliveries to the ExoMars team at ESAC from the CaSSIS team at Bern
  • CaSSIS deliveries from Bern also include calibrated framelets and stitched calibrated products
  • All products have associated browse products except for the calibrated framelets, here the stitched product is considered the reference
  • Currently CaSSIS have delivered calibrated products up to February 22nd 2020

Following the first CaSSIS data review in April 2020, all CaSSIS products are public six months after observation date.

Current Issues

The table below lists the known current and previous issues with the CaSSIS data. For the fixed issues please take note of the DPS version and the date. The date should be used for guidance and may be out by a few days as the release is deployed. The DPS version can be found in the Mission Area of the label e.g.

<psa:Processing_Context>
                <psa:processing_software_title>EM16 Data Processing System</psa:processing_software_title>
                <psa:processing_software_version>3.6-RC3</psa:processing_software_version>
</psa:Processing_Context>


Ref.Issue

Affects

Version

Status

Internal

Reference

C3Currently there are multiple problems associated with the 'on the fly' generated collection and bundle inventory files. The issues are understood, how to implement to the fix is in analysis.2.0In analysisP1381
C4The .csv inventory files do not end with CRLF as required by the PDS standard2.0Fixed in PSA 5.10P1462
C5PDS is changing its LID convention for instruments. This will require a change from
  • urn:esa:psa:context:instrument:cassis.tgo
to
  • urn:esa:psa:context:instrument:tgo.cassis

Coordination is required to make this update as it has an impact on ingestion and the PSA database. Coordination between all missions and instruments is required. 

2.0Will be done with next bulk reprocessing in late 2020A
C8

Currently

  • Object_Orientation_RA_Dec

is specified. This needs to be corrected to:

  • Object_Orientation_North_East
2.0Pending input from CaSSIS teamA
C11

Currently all files have:

  • <psa:instrument_pointing_mode>No pointing</psa:instrument_pointing_mode
  • <psa:instrument_pointing_description>No pointing</psa:instrument_pointing_description>

It is being studied if we can insert either 'targeted' and 'ride-along' pointing types plus an appropriate description into these attributes.

2.0In Analysis, low priorityD
C14While it is not a requirement for CaSSIS calibrated data to work with ISIS, what is required to make it work with ISIS is under study.2.0In analysis by CaSSIS teamA





Fixed Issues


Ref.Issue

Affects

Version

When FixedData Processing Fix Version
Version 2 Issues
F2.1NASA URLs were quoted as the http address in CaSSIS labels. NASA has supported both http and https but are planning to phase out http. CaSSIS data now quotes the https address.2.02020-01-20DPS-3.4
F2.2

The CASSIS calibrated product should contain 'Calibrated' instead of 'Raw' in the Processing Level attribute of the PDS4 label.

Note: This issue affected deliveries from CaSSIS in the range April-September 2018.

2.02020-01-30DPS-3.5
F2.3The current context lid_reference
  • urn:nasa:pds:context:target:calibration.non_science

has been deprecated by PDS. The new context/LID will be:

  • urn:nasa:pds:context:target:calibrator.non_science
2.02020-04-06DPS-3.8
F2.4

<Internal_Reference>
<lid_reference>urn:esa:psa:context:target:planet.mars</lid_reference>
<reference_type>data_to_document</reference_type>
</Internal_Reference>

Should be:

<Internal_Reference>
<lid_reference>urn:esa:psa:context:target:planet.mars</lid_reference>
<reference_type>data_to_target</reference_type>
</Internal_Reference>

2.02020-05-11DPS-3.9
F2.5

The pipeline was setting 'Center' for all the pixel intercepts. The order of the pixel intercepts is the order of the corners in our xml headers is the following: (left upper, right upper, right lower, left lower)

2.02020-05-11DPS-3.9
F2.6

In the <DERIVED_HEADER_DATA Description="Derived data from HK"> section. The field geometry_reference_time_utc is now correctly implemented.

2.02020-05-11DPS-3.9
F2.7CaSSIS data should be SingedLSB2 very old files used SignedByte, more recent UnsignedMSB2 (see O1.4)2.02020-05-11DPS-3.9
F2.8

In the CaSSIS data files from Cruise phase, a NASA PDS target logical identifier (LID) was not available for Landolt SA 104 and Theta Ceti

Therefore the following local context with LIDs were generated:

  • urn:esa:psa:context:target:star.landolt_sa_104 and 
  • urn:esa:psa:context:target:star.hd_100889

Since then official PDS context with LIDs have been created: 

  • urn:nasa:pds:context:target:star.sa_104 and
  • urn:nasa:pds:context:target:star.tet_crt

The CaSSIS observations of these objects will be updated to use the PDS LIDs in future.

2.02020-05-11DPS-3.9
F2.9

Labels currently do not have:

  • <disp:Display_Settings>,

which provides Display_Direction. 

2.02020-06-10DPS-3.10
F2.10A request has been made to add descriptions to HK fields. These are listed in the EAICD, adding descriptions to the data products is under study.2.02020-06-10DPS-3.10
F2.11In the FTP the orbit range does not match the orbits. For example, directory with: /Orbit_Range_2201_2300/ actually contain directories to orbits 2200 -> 2299.2.02020-06-10DPS-3.10





Version 1 Issues
O1.1Version 1 does not have accurate times associated with the observations, hence in some cases the version 2 files will show a different orbit number.1.0

O1.2In very old Raw data files the lid_reference:
  • urn:esa:psa:context:target:planet.mars

was implemented. This was soon replaced by the PDS version:

  • urn:pds:nasa:context:target:planet.mars
1.02017-10
O1.3The data type in old Raw products is incorrectly specified as SignedByte, newer products have specified UnsignedMSB21.02018-11-20DPS-2.4.2
O1.4In old Raw products the logical identifier has an illegal capital letter T in the time part of the LID.1.02018-12-13DPS 2.5
O1.5In old Raw products the record length in the label may not be correct and be lower than the location of the last field1.02018-12-13DPS 2.5
O1.6The file size specified in old Raw product labels is incorrect and is lower than the actual file size1.0

O1.7

In very old Raw data the context LID reference is incorrect:

  • urn:esa:psa:context:instrument.cassis

It has been changed to the correct reference:

  • urn:esa:psa:context:instrument:cassis.tgo
1.0

O1.8

When specifying multiple schematrons, old versions of data files only specify the schematypens once but this needs to be done  each time a schematron is specified e.g.

<?xml-model href="http://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1800.sch" href="http://pds.nasa.gov/pds4/geom/v1/PDS4_GEOM_1700_1401.sch" href="http://psa.esa.int/psa/em16/v1/PDS4_PSA_EM16_1000.sch" schematypens="http://purl.oclc.org/dsdl/schematron"?>

should be:

<?xml-model href="http://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1800.sch" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<?xml-model href="http://pds.nasa.gov/pds4/geom/v1/PDS4_GEOM_1700_1401.sch" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<?xml-model href="http://psa.esa.int/psa/em16/v1/PDS4_PSA_EM16_1000.sch" schematypens="http://purl.oclc.org/dsdl/schematron"?>

1.02019-05-08DPS-2.9
O1.9

Old Raw files have 

  • <type>Calibration</type>

This has been deprecated by NASA and for CaSSIS has been replaced by:

  • <type>Calibration Files</type>
1.02019-06-19
  • No labels