Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add version handling #103

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

add version handling #103

wants to merge 2 commits into from

Conversation

ksarink
Copy link
Collaborator

@ksarink ksarink commented Jun 18, 2022

check for upload and remove not working

check for upload and remove not working
@@ -152,6 +161,8 @@ def checkpresent(self, key):
return False

def transfer_store(self, key, local_file):
if not self.dataset_version == ':latest':
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and line 205 are not working.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it seems to work. Only datalad push is ignoring Exceptions?

Copy link
Collaborator Author

@ksarink ksarink Jun 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When no exception is raised datalad push actually works and a DRAFT is created on dataverse.

The log is exactly the same when it fails. It still says it succeeded but nothing changed on dataverse:

publish(ok): . (dataset) [refs/heads/git-annex->origin:refs/heads/git-annex 62d6084..a5c49a4]                                                                                                 
publish(ok): . (dataset) [refs/heads/master->origin:refs/heads/master 424537b..37ed9ae]                                                                                                       
action summary:                                                                                                                                                                               
  publish (ok: 2)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to work when the files handled by annex are pushed
It does not work when it's a git remote.

Steps to reproduce:
git clone "datalad-annex::?type=external&externaltype=dataverse&encryption=none&url=http://localhost:8080/&doi=doi:10.5072/FK2/JSUZ6P" directory
cp filexy directory ``cd directory
datalad save && datalad push --to origin

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose that is misleading, because you pushed that before, right? Hence, git-annex knows the file is there already and doesn't even try to TRANSFER-STORE. Needs to be fresh and with no pushed content yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify: If you look and the results, it's only two: one for each branch pushed. No content pushed whatsoever, and no error reported from annex. Hence this push call didn't even try.

Copy link
Collaborator Author

@ksarink ksarink Jun 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah there are no new files since only the datalad metadata gets uploaded. Which are just these two files which are changed when new files are saved into the datalad instance.
Screenshot from 2022-06-18 14-48-19

I am not sure if this really is not an issue

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that doesn't look right indeed. I'll have a look at it on Monday!

Thanks a ton so far!

@ksarink ksarink requested review from mih and bpoldrack June 18, 2022 13:28
@ksarink
Copy link
Collaborator Author

ksarink commented Jun 18, 2022

Oh and this should be tested on a dataverse with a sibling created. I was only able to test it on dataverse datasets which is either used for the git files or the annex files. Not both

@ksarink ksarink marked this pull request as ready for review June 18, 2022 14:15
@mih
Copy link
Member

mih commented Mar 1, 2023

Just to leave a note: I have resumed development and I am planing to look into this soon. I have also noticed the improper behavior of datalad push elsewhere. It reports success when an individual git push (same remote) of a single branch actually causes a non-zero exit because of an inappropriate API call by the special remote.

I have started to RF the code quite a bit, so I am not sure how easy it would be to simply reapply your changes. However, I will make sure to try and credit you nevertheless should I have to implement this in another way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants