Skip to content

Unable to use DBT-CLI using the default postgres dbt image #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bbouretdev opened this issue Oct 8, 2024 · 3 comments
Closed

Unable to use DBT-CLI using the default postgres dbt image #149

bbouretdev opened this issue Oct 8, 2024 · 3 comments
Assignees
Labels
area/plugin Plugin-related issue or feature request bug Something isn't working

Comments

@bbouretdev
Copy link

Expected Behavior

Using "ghcr.io/kestra-io/dbt-postgres:latest" container image inside a DBT-CLI task (io.kestra.plugin.dbt.cli.DbtCLI) should install a working postgres adapter version, enabling DBT commands to run properly.

Actual Behaviour

This stacktrace appears on flow execution: "Error importing adapter: No module named 'dbt.adapters.postgres'".

It seems to work when using "beforeCommands: pip install dbt-postgres" instead of "containerImage" as a workaround

Steps To Reproduce

No response

Environment Information

  • Kestra Version:
  • Plugin version:
  • Operating System (OS / Docker / Kubernetes):
  • Java Version (If not docker):

Example flow

No response

@bbouretdev bbouretdev added the bug Something isn't working label Oct 8, 2024
@kestrabot kestrabot bot added this to Issues Oct 8, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Issues Oct 8, 2024
@anna-geller anna-geller added the area/plugin Plugin-related issue or feature request label Oct 8, 2024
@Ben8t
Copy link
Member

Ben8t commented Oct 10, 2024

AFAIK, matters of updating ghcr.io/kestra-io/dbt-postgres:latest on our side

@anna-geller
Copy link
Member

it was updated 4 days ago, so everything works on CI side https://github.com/kestra-io/plugin-dbt/actions/runs/11212170143

@Ben8t can you try to reproduce on the same postgres you run kestra on with docker compose?

@Ben8t Ben8t self-assigned this Oct 11, 2024
@Ben8t
Copy link
Member

Ben8t commented Oct 18, 2024

Indeed last builds shoudl have fixed
I'm able to run dbt.CLI with Postgres containerImage successfully:

id: transform
namespace: kestra.weather

concurrency:
  behavior: QUEUE
  limit: 1

tasks:
  - id: git
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/Ben8t/dbt_weather.git
        branch: main

      - id: dbt
        type: io.kestra.plugin.dbt.cli.DbtCLI
        containerImage: ghcr.io/kestra-io/dbt-postgres:latest
        profiles: |
          dbt_weather:
            outputs:
              dev:
                type: postgres
                host: "{{ secret('POSTGRES_HOST') }}"
                user: "{{ secret('POSTGRES_USERNAME') }}"
                password: "{{ secret('POSTGRES_PASSWORD') }}"
                port: 5432
                dbname: data
                schema: weather
                threads: 8
                connect_timeout: 10
            target: dev
        commands:
          - dbt deps
          - dbt build
      
triggers:
  - id: listen-flow
    type: io.kestra.plugin.core.trigger.Flow
    conditions:
      - type: io.kestra.plugin.core.condition.ExecutionFlowCondition
        namespace: kestra.weather
        flowId: load
      - type: io.kestra.plugin.core.condition.ExecutionStatusCondition
        in:
          - SUCCESS

image

Closing for now. @bbouretdev do not hesitate to give a full reproducer if you still face an issue here

@Ben8t Ben8t closed this as completed Oct 18, 2024
@github-project-automation github-project-automation bot moved this from Backlog to Done in Issues Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugin Plugin-related issue or feature request bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

3 participants