Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rewrite Linux syscalls generation / Migrate to 64-bit time #21440

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Rewrite Linux syscalls generation / Migrate to 64-bit time #21440

wants to merge 6 commits into from

Conversation

The-King-of-Toasters
Copy link
Contributor

Changes by Arnd Bergmann have migrated all supported architectures to use a table for their syscall lists. This removes the need of calling a C pre-processor and simplifies the logic dramatically.
The side effect is the number of names changed on targets that use the "generic" table. That list (located under scripts/syscall.tbl) adds the _time64 suffix to syscalls taking a timespec64 on 32-bit targets. Similarly, the _time32 suffix has been removed.

The result is a lot of breakage in our Linux wrappers, which makes me worried that the logic for determining the proper timespec to use was subtly broken all this time. Should be a good chance to finish #4726 - we only have 14 years after all...

@alexrp
Copy link
Member

alexrp commented Sep 18, 2024

The result is a lot of breakage in our Linux wrappers, which makes me worried that the logic for determining the proper timespec to use was subtly broken all this time.

This situation is highly confusing because timespec and kernel_timespec are not the same thing, and the latter is usually what you want. But yes, the time stuff needs to be cleaned up and the standard library probably just wholesale switched to time64. (I can't think of a good reason why we would want to maintain time32 support?)

@The-King-of-Toasters
Copy link
Contributor Author

Kinda scary that the CI passed 🙃 - do we not test for time overflow?

@The-King-of-Toasters The-King-of-Toasters changed the title Rewrite Linux syscalls generation Rewrite Linux syscalls generation / Migrate to 64-bit time Sep 29, 2024
@The-King-of-Toasters
Copy link
Contributor Author

Time to finish #4726, starting with stat.
For any readers, expect a lot of breakage until this is ready for review.

alexrp
alexrp reviewed Oct 2, 2024
View reviewed changes
lib/std/os/linux.zig Outdated Show resolved Hide resolved
alexrp
alexrp reviewed Oct 2, 2024
View reviewed changes
lib/std/os/linux.zig Outdated Show resolved Hide resolved
@The-King-of-Toasters
Rewrite generate_linux_syscalls to be completely table based
5c46c77 
Changes by Arnd Bergmann have migrated all supported architectures to
use a table for their syscall lists. This removes the need of calling a
C pre-processor and simplifies the logic dramatically.
@The-King-of-Toasters
Syscalls: Update list for 6.11
5a4e51d 
The generic syscall table has different names for syscalls that take a
timespec64 on 32-bit targets, in that it adds the `_time64` suffix.
Similarly, the `_time32` suffix has been removed.

I'm not sure if the existing logic for determining the proper timespec
struct to use was subtly broken, but it should be a good chance to
finish ziglang#4726 - we only have 14 years after all...

In other news:

- x86_64 gets `uretprobe`, a syscall to speed up returning BPF probes.
- Hexagon gets `clone3`, but don't be fooled: it just returns ENOSYS.
@The-King-of-Toasters
Target: Bump Linux minimum requirements for kernel+glibc
a16b32a 
As per ziglang#21738, the minimum kernel has been bumped to 5.11, and glibc to
2.34. The maximum has also been updated to the new 6.11 release.
@The-King-of-Toasters
Linux: Always use time64 APIs
c11471c 
`std.os.linux` has been reworked to use 64-bit time APIs on all targets.
This required auditing all types used by the Kernel ABI for each of the
supported targets, along with other some rewrites in `std.posix`.

= `std.os.linux`

The `Stat` structures for each target are now merged together into the
`KernelStat` struct that switches on the target, similar to existing
types. Targets that have a `stat64` structure have been modified to make
that it's default, and all definitions have been re-written to match
the kernel headers exactly using the `c_` int types.

Of course, newer linux ports don't even implement the `stat(2)` family,
instead requiring userspace to wrap it using `statx(2)`. Thus, a new
`Stat` type has been created to hold information from both APIs. The new
public variable `has_fstatat` has also been introduced, so that callers
can check if the current target at least implements `fstatat(2)`, and to
use `statx(2)` if not. The `major`, `minor` and `makedev` functions have
also been ported over to make the translation from `statx(2)` possible.

== `timespec`

The KernelStat `(a|c|m)time` fields are no longer defined as
`timespecs`, since their signedness and bit size vary by target.
Instead, the new `timespec.makeTimespec` function is used, which does
some comptime checks on the time types before performing `@intCasts` on
each field.

Speaking of, the `timespec` type has been redefined to be the same as
`__kernel_timespec`, as it is the modern 64-bit type that the kernel is
using going forward. Since some syscalls (e.g. `timerfd_(get|set)time`)
require the `timespec64` type, it has been added as well. Note that the
only difference between it and `__kernel_timespec` is that `ts_nsec` is
defined as a `c_long`, thus explicit padding fields are added and zeroed
out for 32-bit targets to avoid issued with uninitialised memory.

== Misc.

- The VDSO `clock_gettime` symbol now points to the proper 64-bit
  verrsion for each arch.
- The `Time64` struct has been created to hold all the proper `time64`
  syscalls for each target.

= `std.posix`

- Add `fstatatLinux` that either uses `fstatat` or `statx` depending on
  the value of `linux.has_fstatat`.
- Move `std.os.fstat(at)_wasi` into the `std.posix.fstat(at)Wasi` in
  light of ziglang#21023.
- Move the libc path for `fstatat` into the new function `fstatatC`.
- Mark `fstatatZ` as `inline` since the logic is simplified.
@The-King-of-Toasters
std.c: Unify largefile and time64 support on Linux
05b6515 
This commit follows the work done in `std.os.linux`, in that the `Stat`,
`time_t` and `timespec` types have been audited against the libc
definitions and fixed appropriately.

Targeting the `largefile` and `time64` functions require linking to
specific symbols. In order not to over-complicate `std.posix`, which
already uses `lfs64_abi`, the logic for selecting the right function has
been moved into `std.c`. These functions are imported from the new file
`vlfts.zig` along with the two options `largefile_abi` and `time64_abi`.
This allows `std.c` to select the proper symbol for e.g. fstatat, which
could be one of the following:

- `fstatat`.
- `fstatat64`.
- `__fstatat64_time64`.
- `fstatat_time64`.

Simple, isn't it...
@The-King-of-Toasters
glibc_compat: Refer to lfs64 symbols
f5567d5
@The-King-of-Toasters The-King-of-Toasters marked this pull request as ready for review October 29, 2024 11:38
@The-King-of-Toasters
Copy link
Contributor Author

I'm marking this ready for review as I believe the aarch64 runners are simply using an older version of glibc. While that is fixed, I'd like some feedback on my changes.

@alexrp
Copy link
Member

alexrp commented Nov 2, 2024

Looks like I caused a merge conflict here with #21860.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexrp alexrp alexrp left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@The-King-of-Toasters @alexrp