Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Enhanced missing local storage error #427

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Pijukatel
Copy link
Contributor

Enhanced missing local storage error

Closes: #412

@github-actions github-actions bot added this to the 110th sprint - Tooling team milestone Mar 10, 2025
@github-actions github-actions bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Mar 10, 2025
@Pijukatel Pijukatel requested review from vdusek and janbuchar and removed request for vdusek March 10, 2025 14:38
@Pijukatel Pijukatel marked this pull request as ready for review March 10, 2025 14:38
@Pijukatel Pijukatel requested a review from vdusek March 10, 2025 15:49
Copy link
Contributor

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few questions about the decorator.

I'm also wondering if we're not using a sledgehammer to crack a nut.

From the issue description:

Yeah, it was reported by me.

I did some tests and to me it seems to be working as expected.

Cool 👍 As discussed in the Slack thread, there is no actual bug, it's just that mixing remote and local development makes things really confusing. So updating the documentation makes sense.

On top of that, how about updating the error message? If:

I access storage by ID, and
I'm in local environment, and
the storage does not exist, then
show an error mentioning that the issue might be that the storage is remote?

So this should be really just about updating the error message. I would rather do it directly rather than introduce this decorator-obscuration magic.

It seems like this should be only about updating the error message. Maybe I would rather do it directly.

) -> Any:
try:
return await function(self=self, id=id, name=name, force_cloud=force_cloud)
except Exception as e:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to catch all exceptions?

Comment on lines +68 to +70
e.args = (
f'{e.args[0]} (If you are trying to retrieve a remote storage, use `force_cloud=True` argument.)',
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't e.args contain more than one item? If so, am I correct that we are discarding them?

TFun = TypeVar('TFun', bound=Callable[..., Any])


def _add_local_storage_error_hint(function: TFun) -> TFun:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't we move the decorator to a dedicated private module, perhaps apify._utils?

)
raise

return cast(TFun, wrapper)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need the cast? Doesn't functools.wraps already take care of preserving the function's type?

@Pijukatel
Copy link
Contributor Author

So this should be really just about updating the error message. I would rather do it directly rather than introduce this decorator-obscuration magic.

It seems like this should be only about updating the error message. Maybe I would rather do it directly.

I am not sure this is even necessary. I raised the question in the slack, but no one responded there, so this is just the same question more aloud :-)

This error originates from crawlee and that is not aware of the force_cloud argument which is added by sdk. So we can't have this extra message being added at the place where the exception is originally raised.

I am not saying that inside that decorator it is nice, but I think in general it is correct approach to detach such additional error message enhancement (If needed at all?!) to decorator so that it is not polluting the main function logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Actor.open_dataset with getData() method produces an RuntimeError
2 participants