Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(#139): do not check for host endianness in IP conversion #443

Merged
merged 2 commits into from
Jun 7, 2024

Conversation

rectified95
Copy link
Contributor

@rectified95 rectified95 commented Jun 5, 2024

Description

Function determineEndian() always returns the same value since the host byte order is always little-endian, so it can be skipped.
What's more, the byte order of IP address values passed by eBPF programs doesn't need to be changed, because Golang's net.IP type expects big-endian byte arrays.

Details:

We always enter this if-branch binary.LittleEndian.PutUint32(ip, nn) which looks like it performs conversion TO little-endian, while it actually expects to receive input already IN little-endian.
Note, the input number's bytes are kept in the same order - this function merely converts a uint32 to a byte array. The result is we convert a big-endian number to a big-endian byte array.

func (littleEndian) PutUint32(b []byte, v uint32) {
	_ = b[3] // early bounds check to guarantee safety of writes below
	b[0] = byte(v)
	b[1] = byte(v >> 8)
	b[2] = byte(v >> 16)
	b[3] = byte(v >> 24)
}

Crucially, net.IP expects big-endian byte arrays. For example, IP 15.14.0.0 (0x0F0E0000 in host order, and 0x00000E0F in network order) should be passed in as [0F, 0E, 00, 00] (most significant byte at the lowest memory address) and so doesn't need inverting. Example here: https://play.golang.com/p/ykiP6mLB_gG

In summary, we currently do not perform byte order conversion and shouldn't start to, either in GO or eBPF code.

Related Issue

Fixes #139

This PR looks almost identical to #161 which was rejected because it didn't include byte order conversion in eBPF, but turns out that's not needed. It also uses the correct uint-to-array conversion functions.

Checklist

  • I have read the contributing documentation.
  • I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
  • I have correctly attributed the author(s) of the code.
  • I have tested the changes locally.
  • I have followed the project's style guidelines.
  • I have updated the documentation, if necessary.
  • I have added tests, if applicable.

@rectified95 rectified95 changed the title Remove check for host endianness in IP conversion. fix(#139): do not check for host endianness in IP conversion Jun 5, 2024
@rectified95 rectified95 marked this pull request as ready for review June 6, 2024 00:57
@rectified95 rectified95 requested a review from a team as a code owner June 6, 2024 00:57
@rbtr rbtr added type/fix Fixes something area/plugins scope/S Change is Small labels Jun 6, 2024
@nddq
Copy link
Contributor

nddq commented Jun 7, 2024

Relates to #95

@nddq nddq added this pull request to the merge queue Jun 7, 2024
Merged via the queue into microsoft:main with commit 97723dc Jun 7, 2024
25 checks passed
Carpe-Wang pushed a commit to Carpe-Wang/retina that referenced this pull request Jun 11, 2024
…microsoft#443)

Function `determineEndian()` always returns the same value since the
host byte order is always little-endian, so it can be skipped.
What's more, the byte order of IP address values passed by eBPF programs
doesn't need to be changed, because Golang's `net.IP` type expects
big-endian byte arrays.

**Details:**

We always enter this if-branch `binary.LittleEndian.PutUint32(ip, nn)`
which looks like it performs conversion TO little-endian, while it
actually expects to receive input already IN little-endian.
Note, the input number's bytes are kept in the same order - this
function merely converts a uint32 to a byte array. The result is we
convert a big-endian number to a big-endian byte array.

```
func (littleEndian) PutUint32(b []byte, v uint32) {
	_ = b[3] // early bounds check to guarantee safety of writes below
	b[0] = byte(v)
	b[1] = byte(v >> 8)
	b[2] = byte(v >> 16)
	b[3] = byte(v >> 24)
}
```
Crucially, `net.IP` expects big-endian byte arrays. For example, IP
`15.14.0.0` (`0x0F0E0000` in host order, and `0x00000E0F` in network
order) should be passed in as `[0F, 0E, 00, 00]` (most significant byte
at the lowest memory address) and so doesn't need inverting. Example
here: https://play.golang.com/p/ykiP6mLB_gG

In summary, we currently do not perform byte order conversion and
shouldn't start to, either in GO or eBPF code.

Fixes microsoft#139

This PR looks almost identical to
microsoft#161 which was rejected because
it didn't include byte order conversion in eBPF, but turns out that's
not needed. It also uses the correct uint-to-array conversion functions.

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

Signed-off-by: Igor Klemenski <[email protected]>
Signed-off-by: Carpe-Wang <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jun 12, 2024
# Description

Partially closes #95.
#443 will remove the usage of
`unsafe` in `utils_linux`

## Related Issue

If this pull request is related to any issue, please mention it here.
Additionally, make sure that the issue is assigned to you before
submitting this pull request.

## Checklist

- [ ] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [ ] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [ ] I have correctly attributed the author(s) of the code.
- [ ] I have tested the changes locally.
- [ ] I have followed the project's style guidelines.
- [ ] I have updated the documentation, if necessary.
- [ ] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed


![image](https://github.com/microsoft/retina/assets/28567936/b2bd8aa5-e659-44e6-85d7-e67b3b1a43fa)
## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
matmerr pushed a commit to matmerr/retina that referenced this pull request Jul 3, 2024
…microsoft#443)

# Description

Function `determineEndian()` always returns the same value since the
host byte order is always little-endian, so it can be skipped.
What's more, the byte order of IP address values passed by eBPF programs
doesn't need to be changed, because Golang's `net.IP` type expects
big-endian byte arrays.

**Details:**

We always enter this if-branch `binary.LittleEndian.PutUint32(ip, nn)`
which looks like it performs conversion TO little-endian, while it
actually expects to receive input already IN little-endian.
Note, the input number's bytes are kept in the same order - this
function merely converts a uint32 to a byte array. The result is we
convert a big-endian number to a big-endian byte array.

```
func (littleEndian) PutUint32(b []byte, v uint32) {
	_ = b[3] // early bounds check to guarantee safety of writes below
	b[0] = byte(v)
	b[1] = byte(v >> 8)
	b[2] = byte(v >> 16)
	b[3] = byte(v >> 24)
}
```
Crucially, `net.IP` expects big-endian byte arrays. For example, IP
`15.14.0.0` (`0x0F0E0000` in host order, and `0x00000E0F` in network
order) should be passed in as `[0F, 0E, 00, 00]` (most significant byte
at the lowest memory address) and so doesn't need inverting. Example
here: https://play.golang.com/p/ykiP6mLB_gG

In summary, we currently do not perform byte order conversion and
shouldn't start to, either in GO or eBPF code.
## Related Issue

Fixes microsoft#139 

This PR looks almost identical to
microsoft#161 which was rejected because
it didn't include byte order conversion in eBPF, but turns out that's
not needed. It also uses the correct uint-to-array conversion functions.

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

Signed-off-by: Igor Klemenski <[email protected]>
matmerr pushed a commit to matmerr/retina that referenced this pull request Jul 3, 2024
# Description

Partially closes microsoft#95.
microsoft#443 will remove the usage of
`unsafe` in `utils_linux`

## Related Issue

If this pull request is related to any issue, please mention it here.
Additionally, make sure that the issue is assigned to you before
submitting this pull request.

## Checklist

- [ ] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [ ] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [ ] I have correctly attributed the author(s) of the code.
- [ ] I have tested the changes locally.
- [ ] I have followed the project's style guidelines.
- [ ] I have updated the documentation, if necessary.
- [ ] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed


![image](https://github.com/microsoft/retina/assets/28567936/b2bd8aa5-e659-44e6-85d7-e67b3b1a43fa)
## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugins scope/S Change is Small type/fix Fixes something
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't check endieness on the host
4 participants