Page tree
Skip to end of metadata
Go to start of metadata

Attendees

Apologies

Discussion items

ItemNotes
Robustness Testing

RC explained this this followed on from the email sent to the group on 2018-04-27. The SOC would like to understand the scope of testing of VMs currently in place. A motivation is to understand how confident we can be of having usable LL products in operations in the face of imperfect inputs.

No one had any general remarks to make so we proceeded to go round the instruments representatives asking for their status w.r.t. this testing. The discussion often made reference to the categories described in 2016-01-26 SOWG #8 Low Latency Pipeline Splinter - Public Copy.

KS for EUI indicated that they only currently address TC2 in the tests included with the VM. He indicated that he saw little difficulty in addressing the other categories. He indicated that out-of-order data should not present a problem as they started from the assumption that sorting was necessary. Similarly gaps did not require sophisticated handling - e.g. if the first part of an image is available then a low quality image can be constructed, but if only the last half is available and so the header data missing then no useful product can be produced.

No one from EPD attended the telecon.

Gill was sitting in for Gethyn re SWA and updated us on some status issues, but was not aware of which types of tests were already in place.

For PHI Martin informed us that Lucas Guerrero continues to work on the production of FITS files. The current main issue is performance. Adding robustness testing can't yet be a priority. RC invited PHI to wrap even a very slow pipeline into a VM if they can find the time. This might let us iron out other issues in parallel.

For SoloHI Lynn described how their VM first produces intermediate 'science files' which are then checked at the time of FITS generation to see whether useful products can be created, or whether the request should be treated as 'failed'. She said that earlier VMs did come with tests covering gaps, but that these were not included with the latest version. RC indicated the usefulness of retaining these tests in the suite. Lynn said she would add them back in.

For Metis, Maurizzio that there were no further steps taken yet with the VM. Their time is taken up with trying to handle decompression. The pipeline is designed to take into account the different issues the tests address, but only really TC1 is tested for now. He noted the fact that if even a single packet is lost then Metis cannot produce a product at all. This is considered a permanent situation. The company which created the compression system has completed their contract and is no longer involved.

For SPICE, Terje informed us that the Oslo team now have the technical note from RAL describing the compression! This is also important for tests. The current VM does not correctly handle missing packets in compressed data - the products will be incorrect. RC asked if all data will be compressed; Terje answered that no, some will be compressed, some not. SPICE are sorting the data, so expect ordering not to be an issue. They have basic tests including for duplicated packets. But none of the tests delivered with the VM includes bad data. Their top priority for the moment is to implement the functionality which was waiting for the tech note. This may take a couple of months. Terje is currently optimistic about implementing decompression. But robustness testing has to wait for now.

For MAG Isaias confirmed that he considered his VM to be robust by design to all the test categories described, but that only about 40% of those are actually currently tested. He will incorporate the remaining 60% in the next version of the VM.

Isaias also confirmed that all MAG packets are sent to the SSMM in their order of creation.

At this point RC communicated Chris's conclusion that instruments should, if they request more than one APID for low-latency processing, expect to receive telemetry with the different APIDs interleaved according to an overall time ordering (ground reception time for nominal daily pass requests, OBT for gap-filling requests).

There was no representative from RPW or STIX.

FM suggested having a mechanism to document which categories of tests are covered by each provided test, and potentially give details of the nature of each input dataset (e.g. request_... contains gap starting at OBT x and ending at OBT y). Action on FM (in discussion with RC / SOC) to produce a concrete proposal on this.

IC asked if the SOC was planning to conduct independent acceptance tests on the VMs. RC & FM confirmed that there were no such plans. There were considered to be insufficient SOC resources for this. AW considered that it would be very difficult for the SOC to create the appropriate inputs.

LLDPDD deadline.

RC and AdeG clarified that it was becoming difficult for the SOC that for some instruments the LLDPDDs were still far from complete.

As Terje had to leave the first instrument addressed was SPICE - but just to confirm that in that case the LLDPDD was in good condition.

As this point Anik made the aside that the SOC would be requesting an update on the status of the DPDD documents for science data at the SOWG (email on this pending).

Anik asked Metis if there was progress on the LLDPDD, and MP responded that they were still working on an internal document which includes similar information, and he would attempt to provide a LLDPDD as an extract from this document for the SOWG. He asked if it was essential that the LLDPDD follow the provided format. AdeG responded that while the content was the most critical, it was very useful that the document follow the agreed format because it was to be available to the other instruments as well and a common format made interpretation much easier, and because the SOC will be basing the LL02 DPDDs on the LL01 document.

Anik then mentioned that the SOC has a LLDPDD from PHI, which contains a list of products, but that that list doesn't fit in the allocated downlink. She asked if an update would be possible by the SOWG. MK said that he could not promise without first talking to Johann. Martin thought he had sent some example ('RAW') FITS files for the scenarios from Andreas, but Anik was not sure she had seen these. MK said he would resend.

For MAG, Isaias indicated that the SOC table (screen-shared by RC) showing LLDPDD status was out of date in MAG's case. AW recognized this fact; he also mentioned that there one more modification needed to the document.

VM Status - engineering.

We briefly ran through this category for any items which had not come up when discussing robustness testing.

Lynn mentioned that she was currently working on the problems recently identified by RC and AdeG in the recent VM delivery. She is limited right now by the machine she normally uses being rebuilt. But she plans to provide a new VM including robustness testing before the SOWG.

Metis indicted that there was not an immediate prospect of a VM delivery - it depends on when they solve the decompression problems. Perhaps not before the SOWG.

VM product statusThere was no discussion of this - instead there will be an update at the planned LL Splinter at the upcoming SOWG in July (see AoB).
AoBThis group will next meet at a splinter from the July 2018 SOWG. This willbe on Monday the 9th, probably in the afternoon. Details to follow.

Preparatory Actions

Christopher James Watson to confirm whether packets from different APIDs are interleaved in response when multiple requested.

New Actions

Fernando Martin Porqueras to propose mechanism for documenting the tests delivered with a VM.