Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No ability to deploy without downtime #1004

Open
mikkpokk opened this issue Mar 2, 2025 · 12 comments
Open

No ability to deploy without downtime #1004

mikkpokk opened this issue Mar 2, 2025 · 12 comments

Comments

@mikkpokk
Copy link

mikkpokk commented Mar 2, 2025

Octane Version

2.8.1

Laravel Version

10.48.28

PHP Version

8.3.16

What server type are you using?

Swoole

Server Version

6.0.1

Database Driver & Version

No response

Description

I'm using Envoyer to deploy a Laravel Octane application, taking advantage of its zero-downtime deployment features.

However, Octane currently does not support zero-downtime deployment because it does not follow symlink directories. It always remains in the actual directory (instead of the symlinked one) where the Octane server was initially started. When the outdated release directory is deleted, Octane continues running in that location, causing errors on every request and resulting in 500 response codes on the live application.

Error thrown:

Warning: require(/var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/bootstrap.php): Failed to open stream: No such file or directory in /var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/swoole-server on line 18
Fatal error: Uncaught Error: Failed opening required '/var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/bootstrap.php' (include_path='.:/usr/bin/[email protected]/8.3.16/share/[email protected]/pear') in /var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/swoole-server:18
Stack trace:
#0 /var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/swoole-server(95): {closure}(Array)
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 0)
#2 /var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/swoole-server(170): Swoole\Server->start()
#3 {main}
  thrown in /var/www/domain.com/releases/202502010200023/vendor/laravel/octane/bin/swoole-server on line 18
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 1)
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 3)
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 2)
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 4)
#1 [internal function]: {closure}(Object(Swoole\Http\Server), 5)

Steps To Reproduce

Use any 0-downtime deployment or test it manually using the following instructions:

  1. Use the Swoole driver as an example.
  2. cd one directory up from the project's base path.
  3. Create a current symlink directory for your project using the command: ln -nsf ./octane-project-test ./current
  4. Start the Octane server: php ./current/artisan octane:start
  5. Copy your project to another directory: cp -R ./octane-project-test ./octane-project-test-new
  6. Activate the new release: ln -nsf ./octane-project-test-new ./current
  7. Reload the Octane server: php ./current/artisan octane:reload
  8. Remove the original project directory: rm -rf ./octane-project-test
Copy link

github-actions bot commented Mar 3, 2025

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!

@EduardoMateos
Copy link

Hello! We are experiencing the same issue. Every time we deploy a very small project, there is a brief downtime, and open connections are abruptly closed, returning 500 errors.

drwxr-sr-x 16 www-data www-data 4096 Mar 4 06:09 api-backend-build-1606
drwxr-sr-x 16 www-data www-data 4096 Mar 4 06:15 api-backend-build-1607
drwxr-sr-x 16 www-data www-data 4096 Mar 4 06:25 api-backend-build-1608
lrwxrwxrwx 1 www-data www-data 40 Mar 4 06:26 lastbuild -> /var/www/releases/api-backend-build-1608

It is not possible to run php artisan octane:reload because it does not detect that it is running in the directory with the symbolic link. So, we reload Supervisor instead, but this closes connections and returns some errors.

We haven't found a better way to handle this at the moment. We deploy to a server in a very simple way and do not consider using load balancers or deploying servers in Docker for this client.

@mikkpokk mikkpokk changed the title No ability to deploy without 0-downtime No ability to deploy without downtime Mar 4, 2025
@mikkpokk
Copy link
Author

mikkpokk commented Mar 12, 2025

@EduardoMateos It's important to mention that in Laravel, you should keep the storage directory outside the release directory and symlink it to the release directory to avoid various issues. However, it still won't work with Swoole until a fix is released.

@crynobone @taylorotwell The fix is in PR #1009. I manually tested all scenarios, including those without the zero-downtime deployment strategy, and all tests passed.

Verified on both MacOS and Linux.

@mikkpokk
Copy link
Author

Whoever is interested in using this ability right now, before the official Octane release, can add the following extra lines to composer.json before the "require" directive.

composer.json

    ...
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/mikkpokk/octane"
        }
    ],
    ...

@jihadismail8
Copy link

I don't know about you guys , but I do have around 2k requests per seconds and am using octane .
I made my own docker compose file and docker file . Everytime I after development I push the docker image to my docker registry then I on my VPS I just pull it and run it , the downtime is extremely minimal

@mikkpokk
Copy link
Author

I don't know about you guys , but I do have around 2k requests per seconds and am using octane . I made my own docker compose file and docker file . Everytime I after development I push the docker image to my docker registry then I on my VPS I just pull it and run it , the downtime is extremely minimal

At a volume of 2,000 req/s, that downtime (ranging from 5s to nearly 30s with Docker) is already considered a disaster.

@jihadismail8
Copy link

with health checks and Nginx using docker DNS , whenever the new container is up and healthy the traffic is forwarded to it , the downtime is <1s .
for some projects it might be a disaster , ofc would be even better with out any downtime , for me for now its not critical and it works ( updates usually at nights )

@EduardoMateos
Copy link

EduardoMateos commented Mar 13, 2025

Hello! In case it's helpful, when I deploy a new version to avoid downtime, I do the following:

  1. First, I run the new version of Laravel Octane on a random port.
  2. I update the Nginx configuration to point to this random port.
  3. I keep the old Laravel Octane service alive for 60 seconds.
  4. I terminate the old service.

I have this process automated with a script that runs in the pipeline after the tests.

In my case, my project is a simple API and doesn't use storage.

As other users have mentioned, it might be more interesting to do it with Docker.

@mikkpokk
Copy link
Author

mikkpokk commented Mar 13, 2025

Hello! In case it's helpful, when I deploy a new version to avoid downtime, I do the following:

  1. First, I run the new version of Laravel Octane on a random port.
  2. I update the Nginx configuration to point to this random port.
  3. I keep the old Laravel Octane service alive for 60 seconds.
  4. I terminate the old service.

I have this process automated with a script that runs in the pipeline after the tests.

In my case, my project is a simple API and doesn't use storage.

As other users have mentioned, it might be more interesting to do it with Docker.

That's a well-known, complex workaround on the internet - complex because you have to dynamically modify and reload both the Nginx and Supervisor configurations. Not to mention the temporary overhead: for one minute, your server tries to handle double the number of workers. If your server is under heavy load, this may cause it to freeze.

Swoole itself is like PM2 and a Node.js server combined - two in one. However, Swoole is not responsible for autostart or autorestarts; that must be handled via Supervisor.

Docker doesn't reduce real server downtime - in fact, it does the opposite. If your Docker setup runs across multiple servers (load balancers), visitors will encounter fewer 500 errors because a percentage of visitors (including automated load-balancer health checks) will trigger an "unhealthy" signal when hitting an unresponsive server. The load balancer will then stop directing traffic to that server until it becomes healthy again. This may create a false impression that it's a bulletproof deployment strategy - but it is not. Moreover, it can still cause resource shortages due to deployment.

Other than that, I don't see why people spent a couple of years using complex workarounds instead of simply reporting and fixing the issue directly in the Octane configuration. At least now it's resolved, and no workarounds are needed.

EDIT: Even if you don't use storage yourself, Laravel framework and Octane does. That's why it's important to keep it symlinked. You lose state and log files otherwise.

@inikoo
Copy link

inikoo commented Mar 20, 2025

h,ey @mikkpokk i already reported this 2 months ago , #996 , but i am using roadrunner. you think your PR #1009 can be ported to roadrunner? (and frankenphp)

ok, just for the record i solved this hacky way, in the mean time that PRs are done:

STEP 1 : create a normal directory (not a symbolic link),

let say that directory is: /home/aiku/aiku/anchor/

as part of your deployment actions copy your project sources there.
e.g. i using deployer, just find the equivalent in envoy or whatever you using

desc('Sync octane anchor');
task('deploy:sync-octane-anchor', function () {
    run("rsync -avhH --delete {{release_path}}/ {{deploy_path}}/anchor/octane");
});

STEP 2: then your octane in supervisor conf file, the octane start command should be run in that directory

;etc/supervisor/aiku-production-octane.conf
[program:aiku-octane-production_boro]
process_name=%(program_name)s
command=/usr/bin/php8.3  /home/aiku/aiku/anchor/octane/artisan octane:start -q --workers=32
... 

STEP 3 , in your config/octane.php you must add this(in may case is roadrunner you need to find out the correct config for swoole, and frankenphp )

'roadrunner' => [
'command' => env('OCTANE_ROADRUNNER_WORKER_PATH', base_path('vendor/bin/roadrunner-worker')),
]

then put in your .env
OCTANE_ROADRUNNER_WORKER_PATH=../../../../../home/aiku/aiku/anchor/octane/vendor/bin/roadrunner-worker

now you can run octane:reload with no downtime 🥳

@mikkpokk
Copy link
Author

@inikoo Hey, you can try implementing fixes for RoadRunner and/or FrankenPHP using ideas from my PR and push a new PR for FrankenPHP and/or RoadRunner if you have the knowledge to develop and test those servers.

I don't know much about RoadRunner or FrankenPHP, and I don't have the ability or knowledge to test them thoroughly.

@inikoo
Copy link

inikoo commented Mar 21, 2025

@inikoo Hey, you can try implementing fixes for RoadRunner and/or FrankenPHP using ideas from my PR and push a new PR for FrankenPHP and/or RoadRunner if you have the knowledge to develop and test those servers.

I don't know much about RoadRunner or FrankenPHP, and I don't have the ability or knowledge to test them thoroughly.

i could test in RoadRunner, but i can't implement fixes for RoadRunner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants