Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(vote-program): initializeAccount instruction #563

Draft
wants to merge 30 commits into
base: harnew/system-program
Choose a base branch
from

Conversation

dadepo
Copy link
Contributor

@dadepo dadepo commented Feb 11, 2025

No description provided.

Copy link

codecov bot commented Feb 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

see 1 file with indirect coverage changes

@0xNineteen 0xNineteen changed the title feat(vote, consensus): Implement InitializeAccount instruction of the vote program feat(consensus): Implement InitializeAccount instruction of the vote program Feb 11, 2025
@0xNineteen 0xNineteen changed the title feat(consensus): Implement InitializeAccount instruction of the vote program feat(consensus): vote-program : InitializeAccount instruction Feb 11, 2025
@0xNineteen 0xNineteen changed the title feat(consensus): vote-program : InitializeAccount instruction feat(consensus): vote-program [initializeAccount instruction] Feb 11, 2025
@0xNineteen 0xNineteen changed the title feat(consensus): vote-program [initializeAccount instruction] feat(vote-program): initializeAccount instruction Feb 11, 2025
yewman and others added 22 commits February 12, 2025 13:35
* Add docusaurus

* Add ci.png

---------

Co-authored-by: Alex Hutchison <[email protected]>
* RPC server: io_uring upgrade
Separates the server into two parts: the context, and the work pool;
the context contains everything generally needed to run the server,
the work pool contains a statically polymorphic implementation for
a pool to dispatch the actual work to.
In doing this, we also separate certain things out into a few different
files.
The RPC server context API has been modified slightly to reflect this,
and the work pool directly exposed, for now.

* Don't use file-as-struct

* Run style script, respect line length limit

* Improve accept failure handling & update TODOs
* Handle potentially failing/cancelling of `accept_multishot` by
  re-queueing it, based on the `IORING_CQE_F_MORE` flag.
* Revise/simplify the queueing logic for the `accept_multishot` SQE.
* Resolve the EINTR TODO panics, returning a catch-all error value
  indicating it as a bad but non-critical error.
* Update the `a: ?noreturn` `if (a) |*b|` TODO, adding that it's solved
  in 0.14; it should be resolved after we update to 0.14.
* Unify EAGAIN panic message.

* Add TODO to remove hacky-ish workaround

* Use `self: Type` convention

* A few minor fixups and improvements

* Simplify test, make server socket nonblocking
On MacOS, on basic WorkPool, this means we now need to manually set the
accepted socket's flags to the right things, ie, blocking, as opposed
to the server socket's nonblocking mode.
Means we also have to handle EAGAIN a bit differently in the io_uring
backend, but that's a fine tradeoff.

* server.zig -> server/lib.zig

* Segregate out the basic backend
And re-organize some methods based on that change

* Re-organize server module

* Update LOGGER_SCOPE

* Simplify & improve io_uring backend error handling

* De-scope `accept_flags`

* Simplify `can_use` for linux cross-compilation

* (io_uring) Rework error handling, add timeout
Do not exit for *any* errors that are specific to the
related connection, simply free them and continue to the next CQE.

Specifically in the case of `error.SubmissionQueueFull`, instead of
immediately failing, we instead first try to flush the submission queue
and then try again to submit; if it fails a second time, that means
despite flushing the submission queue, it somehow still failed, so
we panic, since this indicates something is *very* wrong.

This also eliminates the `pending_cqes_buf`, since there is actually
no situation in which `consumeOurCqe` returns an error, and we resume
work afterwards - either we process all the received CQEs, or we hard
exit - this was already essentially the case before, now it's more
obvious.

For the main submit, we now wait for at least 1 connection, but we
also add a timeout SQE to make it terminate if we don't receive a
connection or completion of another task for 1 second; this alleviates
the busy loop that was running before.

* (io_uring) Remove multishot_accept_submitted
Also slightly refactor error sets.
Now instead of checking to see if we need to set a flag to re-queue
the multishot accept, we just pass in the server context on init
and queue it, which now makes sense since the context and workpool
are separate.

* (io_uring) Simplify new entry creation
Also add fix for rebase

* Misc fixups

* Re-organize alias/import

* General restructure
* Move more specific functions to the only files they're used.
* Move the `serve*` functions outside of `Context`, making them
  free functions which just accept the context and work pool.
* Remove `acceptAndServeConnection`; originally this was required to
  be able to nicely structure the unit test, and used to be more
  integrated, however it no longer makes sense as a concept.
* Inline `handleRequest` into the basic backend.
* Make the `acceptHandled` function, moved into the basic backend,
  guarantee the specified `sync` behavior, and inline `have_accept4`.
* Appropriately re-export the relevant parts of the server API.
* Added top level doc comments.

* Re-oorganize loggers & scopes

* Refactor `build_options` imports

* Add `no_network_tests` build option
And disable the rpc server test when it is enabled

* Update circleci with `-Dno-network-tests`
* 'trusted_validator' -> 'trusted-validator' on the CLI.
* Update doc references to 'trusted_validator' -> 'trusted-validator'.
* Add _option suffix to option variables missing it.
* Remove newline from gossip_host_option description.
* Use `: T = .{` instead of `= T{` for option variable inits.
* Consolidate common groupings of options into arrays for re-use.
* init

* fixes

* more fixes

* add docs for docs

* more

* more

* fix ci

* move g-ignore to docs/

* update
* Revert "fix(prometheus): prometheus cannot connect to sig's metrics (#531)"

This reverts commit 3e0043e.

* prometheus: use HTTP 1.1 instead of 1.0
* Bufferpool: add CachedRead.borrowSlice

* Accountsdb: add bufferpool field

* use ReadHandle in accountsdb

* get it building

* wip - debugging

* make a mess

* comment all test failures

* add probes

* test and fix ReadHandle's bincode

* misc fixes

* cleanup

* fix line lengths

* make freq atomic

* all tests passing / misc fixes

* add atomic linear fifo

* use threadlocal io_uring

* frames metadata: use atomics for all fields

* make style check happy

* small cleanup

* add flags for benchmark profiling

* rename iterator to byteIterator, add chunked Iterator

* replace iterator with faster one; simplified code

* remove probes

* address comments

* remove .inner on ReadHandle

* misc cleanup

* handle full queue, rollback rcs on error

* blocking io: rc cleanup on error path

* remove allocation from hot path

* line length check

* seek files to 0 before reading all

* use *const ReadHandle

* split out owned and unowned buffers

* remove useless errors from read + related functions

* store and document file_length on accountfiles

* move file_map append

* init_allocator -> allocator

* cloneAndTrack: duplicate cached read handles

* add TODO for io_uring cli flag

* cleanup some io_uring code

* split out FrameManager and properly address thread safety

* remove atomicstack

* make style check happy

* improve threadsafety / add comments on it

* bugfix: use preadall

* batch some lock usages / add comments

* remove .size, factor out Metadata, remove some atomics

* rework locking / fix threadsafety issue / temp regress io_uring

* make fast path / reduce contention

* squashme

* concurrency rework

* bring back io_uring

* simplify io_uring & wait less

* remove dead code

* reduce read calls

* default to blocking io

* align frames

* removed dead code

* style check

* fix bad diff

* remove noinlines used for debugging

* correctly update populated on read

* remove incorrect asserts (triggers rarely, not actually a bug)

* remove dead code

* naming & comments improvements

* small naming/style fixes

* move constants to top

* some renames / style changes

* minor style fixes

* macos: correct page size

* test some errdefers / increase codecov
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

7 participants