DM-48194: Deploy a dev prompt processing service for LSSTCam-ImSim #4168
Conversation
hsinfang
force-pushed
the
tickets/DM-48194
branch
5 times, most recently
from
11c1266 to
51176d7
Compare
applications/prompt-proto-service-lsstcamimsim/values-usdfdev-prompt-processing.yaml
Outdated
Show resolved
Hide resolved
| # Expect to need roughly n_detector × request_latency / survey_cadence pods | ||
| # But we do not have the compute yet. This will be adjusted. | ||
| autoscaling.knative.dev/max-scale: "200" |
There was a problem hiding this comment.
I think we have values for everything in the formula? Certainly 200 is much too low.
There was a problem hiding this comment.
For now I'm setting it to 800, based on 189detector*120s/30sec=756.
But expect this to be revised later.
Currently we have 28*44= 1232 cores on d-nodes for OR5.
| # @default -- None, must be set | ||
| preprocessing: "" | ||
| # -- Skymap to use with the instrument | ||
| skymap: "lsst_cells_v1" |
There was a problem hiding this comment.
I assume this is the DC2 skymap? Is patchesPerImage = 16 (which I assume was copied from ComCamSim) still valid?
There was a problem hiding this comment.
The original plan is to really use lsst_cells_v1.
Then we found out that the DC2's DC2_cells_v1 and lsst_cells_v1 are identical, just different name.
Only lsst_cells_v1 exists in repo embargo_or5 today so I'd keep it for now. Later we might change it depending on which name is chosen for actual OR5.
There was a problem hiding this comment.
I wasn't thorough. Turns out that what DC2_cells_v1 and lsst_cells_v1 really are depend on the exact repo; notes here, though their configs are very very close.
While what we will do with embargo_or5 and the actual OR5 remains unclear, our existing templates in s3://rubin-pp-dev-users/central_repo_2 have data id skymap=DC2_cells_v1. I'll just use DC2_cells_v1 for now.
hsinfang
force-pushed
the
tickets/DM-48194
branch
7 times, most recently
from
9b94679 to
2562976
Compare
The service is started with mostly configs from ComCam. Larger cache via refcatsPerImage is used because otherwise it cannot store all refcat inputs in cache for upload.py test. More tuning is expected later.
Only a handful of LSSTCam-imSim detectors are used in the small upload.py test. This config allows sending fanned-out messages only for those detectors.
hsinfang
force-pushed
the
tickets/DM-48194
branch
from
2562976 to
d57a3d6
Compare
No description provided.