avoid racyness from the polling library #15856
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The library we're using unfortunately has a race condition that's especially worse when using polling to check for new changes of the file. Avoid this racyness by sleeping for a bit for now, but really this should be fixed upstream ideally.
Changelog[uncommitted] (2024-04-03)Bug Fixes
|
justinvp
approved these changes
Apr 3, 2024
Comment on lines
+1459
to
+1467
| // The tail library we're using is racy when shutting down. | ||
| // If it gets the shutdown signal before reading the data, it | ||
| // will just shut down before finding the EoF. This problem | ||
| // is exacerbated on Windows, where we use the poller, which | ||
| // polls for changes every 250ms. Sleep a little bit longer | ||
| // than that to ensure the tail library had a chance to read | ||
| // the whole file. On OSs that don't use the poller we still | ||
| // want to try to avoid the problem so we sleep for a short | ||
| // amount of time. |
There was a problem hiding this comment.
Should we include a link to nxadm/tail#67 in the comment?
There was a problem hiding this comment.
Good idea, done ✔️
Yup, I was planning to do that next week after the hackathon is over, since I was going to prioritize the hackathon, but also really wanted to improve the CI situation first. |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Apr 11, 2024
Tentative changelog: ### Features - [cli] Support always fully qualifying stack names in CLI output [#15857](#15857) - [cli] Add --no-plugins and --no-dependencies to `pulumi install` [#15910](#15910) - [docs] Implement Java constructor syntax examples [#15805](#15805) - [sdk/nodejs] Make TypeScript and ts-node optional peer dependencies to allow using user specified versions [#15622](#15622) - [sdk/{nodejs,python}] Allow apply to have unknown values during updates [#15898](#15898) - [sdk/python] Adds 'typeChecker' runtime option to the Python language host [#15725](#15725) ### Bug Fixes - [auto] Tolerate missing stack and bookkeeping files in ProgramTest. [#15863](#15863) - [cli] Fix a panic when user's home directory could not be looked up [#15872](#15872) - [cli] Fix some commands that didn't respect `--disable-integrity-checking` - [programgen/dotnet] Removes trailing whitespace from emitted DependsOn resource option expressions [#15892](#15892) - [auto/go] Avoid flakyness when reading the event log from pulumi commands [#15856](#15856) - [sdk/go] Fix Provider and Providers options in Go transform functions [#15885](#15885) - [sdk/nodejs] Handle serialization of aliases for well known native functions [#15873](#15873) - [sdk/nodejs] Correctly serialise functions whose code would make use of reserved identifiers [#15879](#15879) - [sdk/nodejs] Serialize function values obtained from Function.bind [#15887](#15887) - [sdk/python] Improve types of getters in Python SDK [#15865](#15865) - [sdkgen/{dotnet,go}] Fixes SDK-generation when referencing shared types in config variables [#15772](#15772) ### Miscellaneous - [sdk/nodejs] Update builtin module list for function serialization [#15830](#15830) - [sdk/nodejs] Set package.json engines to node >= 18 [#15845](#15845)
Merged
github-merge-queue bot
pushed a commit
that referenced
this pull request
Apr 12, 2024
### Features - [cli] Support always fully qualifying stack names in CLI output [#15857](#15857) - [cli] Add --no-plugins and --no-dependencies to `pulumi install` [#15910](#15910) - [docs] Implement Java constructor syntax examples [#15805](#15805) - [sdk/nodejs] Make TypeScript and ts-node optional peer dependencies to allow using user specified versions [#15622](#15622) - [sdk/{nodejs,python}] Allow apply to have unknown values during updates [#15898](#15898) - [sdk/python] Adds 'typeChecker' runtime option to the Python language host [#15725](#15725) ### Bug Fixes - [auto] Tolerate missing stack and bookkeeping files in ProgramTest. [#15863](#15863) - [cli] Fix a panic when user's home directory could not be looked up [#15872](#15872) - [cli] Fix some commands that didn't respect `--disable-integrity-checking` [#15911](#15911) - [programgen/dotnet] Removes trailing whitespace from emitted DependsOn resource option expressions [#15892](#15892) - [auto/go] Avoid flakyness when reading the event log from pulumi commands [#15856](#15856) - [sdk/go] Fix Provider and Providers options in Go transform functions [#15885](#15885) - [sdk/nodejs] Handle serialization of aliases for well known native functions [#15873](#15873) - [sdk/nodejs] Correctly serialise functions whose code would make use of reserved identifiers [#15879](#15879) - [sdk/nodejs] Serialize function values obtained from Function.bind [#15887](#15887) - [sdk/python] Improve types of getters in Python SDK [#15865](#15865) - [sdkgen/{dotnet,go}] Fixes SDK-generation when referencing shared types in config variables [#15772](#15772) ### Miscellaneous - [sdk/nodejs] Update builtin module list for function serialization [#15830](#15830) - [sdk/nodejs] Set package.json engines to node >= 18 [#15845](#15845)
Merged
github-merge-queue bot
pushed a commit
that referenced
this pull request
Apr 15, 2024
### Features
- [auto/{go,nodejs,python}] Add support for continue-on-error parameter
of the destroy command to the Automation API
[#15921](#15921)
- [cli] Support always fully qualifying stack names in CLI output
[#15857](#15857)
- [cli] Add --no-plugins and --no-dependencies to `pulumi install`
[#15910](#15910)
- [docs] Implement Java constructor syntax examples
[#15805](#15805)
- [sdk/nodejs] Make TypeScript and ts-node optional peer dependencies to
allow using user specified versions
[#15622](#15622)
- [sdk/{nodejs,python}] Allow apply to have unknown values during
updates
[#15898](#15898)
- [sdk/python] Add 'typeChecker' runtime option to the Python language
host
[#15922](#15922)
[#15725](#15725)
### Bug Fixes
- [auto] Tolerate missing stack and bookkeeping files in ProgramTest
[#15922](#15922)
[#15863](#15863)
- [cli] Fix a panic when user's home directory could not be looked up
[#15872](#15872)
- [cli] Fix some commands that didn't respect
`--disable-integrity-checking`
[#15911](#15911)
- [programgen/dotnet] Remove trailing whitespace from emitted DependsOn
resource option expressions
[#15922](#15922)
[#15892](#15892)
- [auto/go] Avoid flakyness when reading the event log from pulumi
commands
[#15856](#15856)
- [sdk/go] Fix Provider and Providers options in Go transform functions
[#15885](#15885)
- [sdk/nodejs] Handle serialization of aliases for well known native
functions
[#15873](#15873)
- [sdk/nodejs] Correctly serialise functions whose code would make use
of reserved identifiers
[#15879](#15879)
- [sdk/nodejs] Serialize function values obtained from Function.bind
[#15887](#15887)
- [sdk/python] Improve types of getters in Python SDK
[#15865](#15865)
- [sdkgen/{dotnet,go}] Fix SDK-generation when referencing shared types
in config variables
[#15772](#15772)
[#15922](#15922)
### Miscellaneous
- [sdk/nodejs] Update builtin module list for function serialization
[#15830](#15830)
- [sdk/nodejs] Set package.json engines to node >= 18
[#15845](#15845)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The library we're using unfortunately has a race condition that's especially worse when using polling to check for new changes of the file. Avoid this racyness by sleeping for a bit for now, but really this should be fixed upstream ideally.
I opened a upstream issue for this in nxadm/tail#67. I think we should merge this in the meantime, as it makes this work, even though it's very hacky.
I tested this in #15853, where I could no longer reproduce the failure we've seen before.
Fixes #15659
Fixes #15235