-
Notifications
You must be signed in to change notification settings - Fork 581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bears/python: Add ShebangBear #1937
base: master
Are you sure you want to change the base?
Conversation
bears/python/ShebangBear.py
Outdated
|
||
def run(self, filename, file): | ||
""" | ||
Ensure that the file begins with the Shebang operator. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove indentation here, should be
"""
Ensure that the ...
"""
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok 👍
bears/python/ShebangBear.py
Outdated
diff = Diff(file) | ||
mes = '' | ||
|
||
if '#!' in file[0]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just check for #!/usr/bin/env
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
having #! in the first line don't necessarily measn that it was intended to be the shebang operator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so you just add the shebangs if #!/usr/bin/env
it is not found
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The referenced issue requires that coala enforces using the #!usr/bin/env
(#1932).
Additionally, I am not familiar with the unix flags so i don't know whether #! can be used for purposes other than the shebang operator.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A shebang basically specifies the shell for the command to call the interpreter (like python
, ruby
, php
, or another UNIX shell). We want to enforce the usage of using /usr/bin/env
to get the proper path of the interpreter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yukiisbored so first of all, the bear should be moved from the python bears directory.
Now, let's discuss the case where the user uses something like #!/usr/bin/python
, how can we check that the user doesn't use the general form of the shebang operator and then deduce that script is written in python(or any other language) ?
Do you think that the current check can be improved somehow?
Thanks 😃
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AMR-KELEG You can just grab the binary name (without the path) and add that at the end of /usr/bin/env
.
ex.
#!/usr/bin/python
->#!/usr/bin/env python
#!/usr/bin/python3
->#!/usr/bin/env python3
#!/usr/bin/asuka --intepreter-mode --no-warnings
->#!/usr/bin/env asuka --intepreter-mode --no-warnings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I think the fitting category is between shell
and general
(depends on how you look at it ;))
bears/python/ShebangBear.py
Outdated
self, | ||
mes, | ||
diff.affected_code(filename), | ||
diffs={filename: diff}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and tests for this bear
bears/python/ShebangBear.py
Outdated
mes = '' | ||
|
||
if '#!' in file[0]: | ||
diff.modify_line(1, '#!/usr/bin/env') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you don't want #!/usr/bin/env
as your shebang without any binary right? That's just a directory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, afaik env
is used to run a program inside a modified environment (from env(1)
GNU coreutils).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sils Yes, you're right.
The modified line should include the binary.
#!/usr/bin/env
isn't enough.
The modified line should look like: #!/usr/bin/env python
.
@yukiisbored What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AMR-KELEG It's ok. Please don't wait for our response just do the changes needed. We're pretty busy with a lot of stuff and we don't want to have this PR delayed just because of us~
bears/general/ShebangBear.py
Outdated
return | ||
|
||
diff = Diff(file) | ||
mes = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume this is message, use message
or msg
or something that is more clear as a message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok
bears/general/ShebangBear.py
Outdated
if '#!' in file[0]: | ||
binary = file[0].split('bin/', 1)[1] | ||
diff.modify_line(1, '#!/usr/bin/env {}'.format(binary)) | ||
mes = 'Use the /usr/bin/env.' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use 'Use /usr/bin/env {}'.format(binary)
instead ?
bears/general/ShebangBear.py
Outdated
message = None | ||
|
||
if '#!' in file[0]: | ||
binary = file[0].split('bin/', 1)[1].strip('\n') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please use os.path.basename
instead of this? Since, it causes problems if it's /usr/sbin/
, /sbin/
, /usr/local/sbin
, etc.
bears/general/ShebangBear.py
Outdated
""" | ||
Ensure that the file uses the generic Shebang operator. | ||
""" | ||
if len(file) == 0 or '#!/usr/bin/env' in file[0]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also noticed an empty list is False
and I think it's better to do it that way.
https://stackoverflow.com/questions/53513/best-way-to-check-if-a-list-is-empty#53522
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think .startswith('#!/usr/bin/env')
is better...
bears/general/ShebangBear.py
Outdated
binary = os.path.basename(file[0]).strip('\n') | ||
new_line = '#!/usr/bin/env {}'.format(binary) | ||
diff.modify_line(1, new_line) | ||
message = 'Use the {}.'.format(repr(new_line)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find this message to be unclear especially for users who don't know what is /usr/bin/env
. Any ideas? I'm not sure how to phrase it as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Use /usr/bin/env
to call the program" and then a more info "/usr/bin/env
is preferable to be used because blah blah blah ..." is better I think...
bears/general/ShebangBear.py
Outdated
diff = Diff(file) | ||
message = None | ||
|
||
if '#!' in file[0]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as https://github.com/coala/coala-bears/pull/1937/files#r129461801 but just check for #!
.
bears/general/ShebangBear.py
Outdated
return | ||
|
||
diff = Diff(file) | ||
message = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find both of the above to be tricky when reading it. I think it's better to put it inside the if statement below and just yield it inside it as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically,
if "check if it starts with a shebang":
diff = Diff(file)
...
message = "message goes here"
...
yield ...
Also, Don't forget to squash your commits. Since this is only one "change". |
Please address @yukiisbored 's comments after which we can have another iteration. |
@yukiisbored @yash-nisar I know that my last commit was (a long long) time ago but could you please review the pull request again? |
@gitmate-bot rebase
|
Hey! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost! |
Automated rebase was successful! |
@Makman2 Here we go 😄 |
Ensure that the file uses the generic Shebang operator. | ||
""" | ||
if not file or file[0].startswith('#!/usr/bin/env'): | ||
return |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should also return a result if the file is empty, saying there's no shebang?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe without a patch, but at least saying it's missing ;)
(oh would be so cool if we would have custom result actions, then we could write one where the user can input immediately the name of the binary... but we don't have it yet 😢)
Ensure that the file uses the generic Shebang operator. | ||
""" | ||
if not file or file[0].startswith('#!/usr/bin/env'): | ||
return |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you also add a newline here? I think it improves readability 👍
diff = Diff(file) | ||
message = None | ||
binary = os.path.basename(file[0]).strip('\n') | ||
new_line = '#!/usr/bin/env {}'.format(binary) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you are using #!/usr/bin/env
quite often, I think it's worth to put it inside a variable as a constant 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is only written twice here..
Putting it inside a variable would be only beneficial if we will need to change it later..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's also for consistency 👍
Everything that's used more than once shouldn't be inlined ;)
if file[0].startswith('#!'): | ||
diff = Diff(file) | ||
message = None | ||
binary = os.path.basename(file[0]).strip('\n') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
don't you have to strip off the #!
part in file[0]
?
So in the end you have file[0][2:]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if i understood your comment but the basename extracts the binary file out of the path.
https://docs.python.org/2/library/os.path.html#os.path.basename
Why would i strip the #! if i don't care what the first part of the path is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but is os.path.basename
able to handle it? As the path starts after #!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/python/cpython/blob/3.6/Lib/posixpath.py#L142
The method just extracts all the characters after the last separator.
IMO, we shouldn't strip off the #!
.
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those characters don't belong to the path, that's why I'm asking for it, as conceptually it's not "completely right". I would do file[0][2:]
for the sake of comprehensibility ;)
diff.modify_line(1, new_line) | ||
message_desc = ('This eliminates the limitations caused by systems' | ||
' that have non-standard file system layout') | ||
message = 'Use the {}.\n{}.'.format(repr(new_line), message_desc) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I'm not that convinced of the message, but can't think of a better one right now... we'll discuss it again ;)
diff.modify_line(1, new_line) | ||
message_desc = ('This eliminates the limitations caused by systems' | ||
' that have non-standard file system layout') | ||
message = 'Use the {}.\n{}.'.format(repr(new_line), message_desc) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But please inline message_desc
, I think it's just an unnecessary extra variable ;) Then you don't have to use two placeholders ;)
|
||
def run(self, filename, file): | ||
""" | ||
Ensure that the file uses the generic Shebang operator. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can be precise and describe the idea of using /usr/bin/env
;)
message_desc), | ||
line=1, | ||
severity=RESULT_SEVERITY.NORMAL, | ||
file=get_testfile_path(file_name))], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also assert for the diff? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I couldn't figure out how to assert it.
https://github.com/coala/coala/blob/master/coalib/results/Result.py#L161
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Makman2 could you please give me a hint?
Thanks 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just use the diff
param in the result here 👍
self.check_validity(self.uut, | ||
['#!/usr/bin/env python \n print hello world']) | ||
self.check_validity(self.uut, | ||
['']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think an empty file should be written as an empty list, that's how python handles it 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe you should put those into test files too? Up to you :)
The |
AUTHORS = {'The coala developers'} | ||
AUTHORS_EMAILS = {'[email protected]'} | ||
LICENSE = 'AGPL-3.0' | ||
CAN_FIX = {'Syntax'} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm not sure if it's syntax... do we have a better fitting keyword?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Formatting?
I think syntax is the better choice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You aren't correcting syntax errors... hmm... Could be that you have to create a new keyword, maybe Shebang
itself works?
You could maybe get some ideas from the community ;)
@@ -0,0 +1,33 @@ | |||
import os |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add a newline here to separate system imports from others ;)
unack 90a7bd6 |
Closes #1932
For short term contributors: we understand that getting your commits well
defined like we require is a hard task and takes some learning. If you
look to help without wanting to contribute long term there's no need
for you to learn this. Just drop us a message and we'll take care of brushing
up your stuff for merge!
Checklist
them.
individually. It is not sufficient to have "fixup commits" on your PR,
our bot will still report the issues for the previous commit.) You will
likely receive a lot of bot comments and build failures if coala does not
pass on every single commit!
After you submit your pull request, DO NOT click the 'Update Branch' button.
When asked for a rebase, consult coala.io/rebase
instead.
Please consider helping us by reviewing other peoples pull requests as well:
cobot mark wip <URL>
to get it outof the review queue.
The more you review, the more your score will grow at coala.io and we will
review your PRs faster!