🧱 Better infrastructure

[jbelien]

Yes, this project is still maintained!

The current issue is that our (OpenStreetMap Belgium) server and the infrastructure (SQLite) worked perfectly fine for a few region but is undersized for the number of regions we have now!

I gave it some thought and here are a few suggestions/ideas:

• Migrate to https://platform.sh/ (hosting recommended by Symfony)
• Migrate to https://forge.laravel.com/ (hosting recommended by Laravel)
• Migrate to https://vapor.laravel.com/ (serverless hosting recommended by Laravel)
• Migrate to (upcoming) https://cloud.laravel.com/

We could also rewrite the whole application using other language(s) but that would require too much work.

You're more than welcome to contribute to this discussion if you have experience with one of those solutions.

[jbelien]

Platform.sh

The OSM Welcome Tool is developed using Symfony and Platform.sh is the hosting platform "linked" to Symfony: https://symfony.com/cloud/

The Symfony integration with Platform.sh has been created by the Symfony team and Fabien Potencier, Symfony project lead and founder, as a fully-managed platform optimized for you!

Rely on our Symfony expertise to offer a unique experience for anyone who wants to code and develop with Symfony. We conceived the Platform.sh's Symfony integration to cover all your needs while developing a project with Symfony without moving from your console.

The best way to deploy a Symfony project is Platform.sh.
-- Source: https://symfony.com/cloud/features

Infrastructure

Provides all necessary features: app, PostgreSQL, queue/worker, cron job, ...

You don't seem to be have control on the hosting provider, this is handled for you based onnthe region you choose ; available regions (and providers): https://docs.platform.sh/development/regions.html

Pricing

https://platform.sh/pricing/

I can't immediately figure out how much it would cost us but I'm guessing something around 30 USD/month.

Laravel Forge

Infrastructure

Provides all necessary features: app, PostgreSQL, queue/worker, cron job, ...

You seem to be able to choose your hosting provider (which requires an account in that/those provider(s) ; providers: DigitalOcean, Akamai, Vultr, Amazon, Hetzner, ...

Pricing

https://forge.laravel.com/#pricing

• 19 USD/month (without database backup)
• 39 USD/month

Seems to NOT include server cost from provider (see above).

Laravel Vapor

Infrastructure

Provides all necessary features: app, PostgreSQL, queue/worker, cron job, ...

Provider: Amazon

Pricing

https://vapor.laravel.com/

39 USD/month + AWS usage cost (pay-as-you go)

[jbelien]

If we go for Laravel Forge or Vapor, might be worth migrating from Symfony to Laravel.
It shouldn't be too much work and would allow us to enable features like Laravel Octane.
This could be done in a second phase.

[atomoil]

I’m here cos @joostschouppe mentioned this to the local chapters & communities working group.

I’m very out of touch regarding modern PHP frameworks but it looks like you have a docker container so if that’s up to date then I wonder whether you’ve considered using a cloud provider (eg Amazon) and hosting via the container (eg https://aws.amazon.com/getting-started/hands-on/deploy-docker-containers/ ). Afaik all cloud providers offer somekind of docker/container option.

I think it should end up a bit cheaper but I appreciate it would be more/differently complex to get up and running in the first place.

[xivk]

I have quite some experience with hosting docker containers using Hetzner Cloud (https://www.hetzner.com/cloud/), probably even cheaper compared to Amazon and should be stable enough for this usecase. We host all our company stuff like that.

I would be willing to try this with @jbelien if he thinks this is a good enough option. But it's up to him, I also am very out of touch with PHP.

[jbelien]

Thank you both!

Docker containerization was indeed a plan I had a while back (see #766).
I did deploy the OSM Welcome Tool (using Docker) on DigitalOcean App Platform because performances were really catastrophic (see osmbe/community_planning#41 (comment) for more details).
I must admit I didn't try another provider because OSMBE uses DigitalOCean as hosting provider for the rest of our tools and I didn't want to have to manage 2 separate hosting providers.

I'm still convinced that Docker could be an option but looking at the time I could invest in this these last few months, I'm looking now for a "fully managed" solution.

I'm currently running tests on Plaform.sh and Laravel Forge that look promising (basically push update to main branch and wait for redeploy).
After a few tests, there is already a clear difference between the 2:

• Platform.sh seems to be a fully PaaS : choose resources and services you need (hosted in a specific region) and you get a monthly invoice for what you have
• Laravel Forge seems more like a (really) nice dashboard to help you manage your server ; you connect Laravel Forge to a VM (I would choose DigitalOcean droplet), and it install and manages everything you need on that server. You get a monthly invoice from Laravel Forge + a monthly invoice from your hosting provider

I'm close to have a version of the app running on Platform.sh. I'll keep you updated.

[jbelien]

Test on Platform.sh

https://2-x-rai4ihq-b62c5e7ddypzg.fr-3.platformsh.site/

Daily automated update setup for Belgium, Berlin, and Brazil as test.

Price estimation:

Test 1: Belgium + Brazil ✅
Test 2: Belgium + Brazil + 24 regios in Asia ✅
Test 3 all regions ✅

[raphaelpe]

Hello, I logged in to the website but the website is giving me an error.

I'm from Brazil and I started using this tool on the origin server and used it to invite new members of the community but it was no longer updated.

[jbelien]

@raphaelpe Ah yes, I also get an error... Most likely due to a dependency upgrade. I'll have a look 👍

[raphaelpe]

[jbelien]

Issue is fixed in 102a13b

[jbelien]

Since yesterday evening, https://welcome.osm.be/ is pointing to the new instance.

I'll copy the "historical" data from the previous instance to the new one in the next few days.

[jbelien]

Okay, it still needs a bit of work.
Updates often go over the memory limit (screenshot show manual updates).
Manual updates don't seem to be killed but cronjob well.
I'll try to improve the process to reduce the memory usage.

[jbelien]

I've refactored the process (see adacee1) and it uses way less memory.
Asia is being automatically updated since I've deployed the improvement.
I'll keep an eye on it the next few days.

[jbelien]

Seems to work fine! 🎉

I've tried to restore the data from the "old" Welcome Tool.
Unfortunately it's really a lot of data and we reach an Out Of Memory error when trying to open a region.
I'll try to solve that issue when I have a bit more time.