Store log output in datamodel extension

[stscijgbot-jp]

Issue JP-3773 was created on JIRA by Tyler Pauly:

JWSTDMS-485 documents the desire for an HST-like trailer file which stores the processing output log during science product generation. JSDP-2860 is tracking SDP's work to generate such an external file, while this ticket is to explore the storage of the log output of the CAL pipeline in a datamodel extension/object/attribute. Such an attribute already appears to exist and is in use for exactly this purpose in Roman processing, such that this work should be straightforward to implement on the JWST side; the primary blocker/investigation remaining is in understanding what information needs to be scrubbed to maintain IT security. From DMS planning, scrubbed information should be:

- Node/machine names
- IP addresses
- usernames who have access to those machines
- filesystems paths

[stscijgbot-jp]

Comment by David Law on JIRA:

Adding a note that this is a indeed a feature still requested by users; I was discussing this at a science meeting just this week.

[stscijgbot-jp]

Comment by Pey-Lian Lim on JIRA:

Do you dump tokens and such? Might wanna scrub that out too.

Is this new extension name defined already? If so, where is the specs? Jdaviz might be interested because we have a Metadata plugin.

[stscijgbot-jp]

Comment by Katie Kaleida on JIRA:

This approach seems like the best way to meet the need of a trailer file, as described in JWSTDMS-485, but without the potential difficultly of needing to scrub any information that should not be available to the general public that the SDP solution would have.  CAL should proceed forward with working this ticket JP-3773 ticket in priority order in the 11.3 development timeframe.  We can then collect feedback from the community if this is meeting their needs, or if an HST-like trailer file is really needed.

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

Implemented in #9211 - this method attempts to scrub any mention of an IP address, the user ID or the host name.

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

Documentation added in #9257