Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shell-script for updating autodeploy xars on build time #3

Closed
wants to merge 62 commits into from
Closed

shell-script for updating autodeploy xars on build time #3

wants to merge 62 commits into from

Conversation

bwbohl
Copy link
Contributor

@bwbohl bwbohl commented May 26, 2021

This adds a shell script that will be copied into the container and executed if the docker --build-arg UPDATE_XARS=true. Submitting the $VERSION it will fetch apps.xml from eXist-db's public repo and fetch the latest XAR-versions for this eXist version.

@bwbohl
Copy link
Contributor Author

bwbohl commented May 26, 2021

open for discussion though still see some room for improvement ;-)

with information on new build-arg UPDATE_XARS
@bwbohl
Copy link
Contributor Author

bwbohl commented May 26, 2021

at the moment the packages to be pulled are hard coded, would it be better to iterate through the contents in the auto deploy directory to see which packages are in autodeploy? My impression is though, that this is quite stable over all version of eXist-db, or am I mistaken?

Copy link
Owner

@peterstadler peterstadler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for the PR!
Indeed, it's a great idea to fetch the latest XARs on image build and this should be added. In addition to the minor issues I added as comments, a major improvement would be to provide the list of XARs as a build ARG in the Dockerfile and then iterate over this list in the update-xars script. Do you think that might be possible?

@bwbohl
Copy link
Contributor Author

bwbohl commented May 27, 2021

accidentally closed my tab before submitting a longish comment: wrap up:

  • make update-xars.sh self sufficient so it can be run standoff in order to prepare a folder that can be added to docker runs auto deploy folder: docker run … -v PATH:/opt/exist/autodeploy
  • alternatively add a build param $ADD_XARS_DIR that would be added to auto deploy, too if you want to add some custom XARs.
  • if the definition of which XARs to deploy should be submitted via a list of e.g. XAR-names (and then fetched by script from repo):

bwbohl added 3 commits May 31, 2021 14:51
add build ARG for updating autodeploy xars on build time

Update README.md

with information on new build-arg UPDATE_XARS

update xars on build
@bwbohl
Copy link
Contributor Author

bwbohl commented May 31, 2021

@peterstadler please test the list of XARs is fixed for now; have to checkout how to submit an array as docker ARG for individualised XAR list.
But the main goal of updating the XARs on docker build should be achieved.

@bwbohl
Copy link
Contributor Author

bwbohl commented May 31, 2021

btw closes #2

@bwbohl
Copy link
Contributor Author

bwbohl commented May 31, 2021

@peterstadler actually there's one more thing I'd like to change: make the scrip runnable in standalone by:

  • allow $DIR to be set by $3 in calling update-xars.sh
if [[ -d "$3" ]]
then
   DIR="$3"
   CLEANUP=false
elif [[ "$3" != "" ]]
then
   DIR=`mkdir -p "$3"`
else
   DIR=`mktemp -d`
fi

and

# delete temporary dir
if [[ "$CLEANUP" = "false" ]]
then
    echo "XARs are located at: $DIR"
else
    echo "copy updated XARs to autodeploy folder"
    cp "$DIR"/*.xar ${EXIST_HOME}/autodeploy/
    rm -Rf "$DIR"
fi

Is that ok with you?

@peterstadler
Copy link
Owner

Hi @bwbohl, just a reminder what we decided at our last conference call:

  • provide the list of XARs as a build ARG in the Dockerfile and then iterate over this list in the update-xars script
  • add more comments in the update-xars script for standalone mode

add some documentation and user feedback
check second argument
update setting XAR repo URL
update setting download folder
update user feedback
fix fail on more than one argument
fix fail on creating custom download directory
add documentation comments
* always create temporary download directory
* if $3 is set use as $PUBIR
* allow fourth argument if prune then delete XARs in $PUBDIR before copying
* Add documentation
* Add help
* allow package names  to be submitted as argument
* reorder arguments
* completely reworked parsing of arguments to parsing of options
* instead of relying on a specific order or $ARGS, e.g. `-v 5.2.0` can be used to pass the eXist-db version
* ALERT: documentation still out of sync
* redefine bold and normal typeface shortcuts
* update documentation and help
bwbohl added 16 commits November 1, 2021 22:54
add documentation comments
* always create temporary download directory
* if $3 is set use as $PUBIR
* allow fourth argument if prune then delete XARs in $PUBDIR before copying
* Add documentation
* Add help
* allow package names  to be submitted as argument
* reorder arguments
* completely reworked parsing of arguments to parsing of options
* instead of relying on a specific order or $ARGS, e.g. `-v 5.2.0` can be used to pass the eXist-db version
* ALERT: documentation still out of sync
* redefine bold and normal typeface shortcuts
* update documentation and help
* for each XAR download check if file downloaded successfully, else exit
remove some trailing whitespaces and unnecessary line breaks
* add some documentation
* add license
@bwbohl
Copy link
Contributor Author

bwbohl commented Nov 22, 2021

everything's works fine so far but as the base-image has changed will have to do some update work and obviously with the new default packages for exist v5.3.0 we definitely need to get the default list from the exist repo (or somewhere else e.g. maintain a list here)

bwbohl added 3 commits May 3, 2022 08:41
before this commit, there was a default list for older versions of eXist-db; now the list gets populated based on the install.xml in the exist-db repo
@bwbohl bwbohl marked this pull request as ready for review May 3, 2022 08:45
@bwbohl bwbohl requested a review from peterstadler May 3, 2022 08:45
@bwbohl bwbohl changed the title add build ARG for updating autodeploy xars on build time shell-script for updating autodeploy xars on build time May 3, 2022
@bwbohl
Copy link
Contributor Author

bwbohl commented May 3, 2022

@peterstadler finally I found the time to implement a dynamic population of the default XAR_LIST based on the eXist-db $VERSION and https://raw.githubusercontent.com/eXist-db/exist/eXist-"$VERSION"/exist-installer/src/main/izpack/install.xml

@bwbohl
Copy link
Contributor Author

bwbohl commented May 3, 2022

btw: only adds functx if it is in the default deploys of the eXist-db version; so it closes #3 but functx might be missing for older db version that did not include in the default deploys

@bwbohl
Copy link
Contributor Author

bwbohl commented May 25, 2022

ping

@peterstadler
Copy link
Owner

Hi @bwbohl, sorry for the delay …
I was just having a look and trying it out: Seems we need to install libxml2-utils (for xmllint) and some other dependencies. So, trying to keep the image size small, I thought of downloading the xars in an initial downloader step in the Dockerfile. We'd then copy over only the xars, not the tools; or do you need the script update-xars.sh in a running container?
If you're ok, I'd try to refactor this to make it work.

@bwbohl
Copy link
Contributor Author

bwbohl commented Oct 19, 2022

yeah, you're probably right. The way you describe is just the way I use it e.g. in https://git.uni-paderborn.de/baz-ga/docker-website/-/blob/main/Dockerfile

Should I create a docker-xar-download (or similar) Docker image that you could just use as FROM in your Dockerfile?

@baz-ga baz-ga closed this by deleting the head repository Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants