Authentication Issues (gcloud auth application-default login)

[HampB]

Expected behavior

Sorry for an issue without much details - i'll happily help to investigate. But im quite stuck on where to begin. It might also just be an user error, would be happy to get some instructions then :-)

---

Running compilation and other actions in the vscode extension sometimes throws "Reauthentication is needed. Please run gcloud auth application-default login to reauthenticate." even tough im still authenticated.
When this happens, i can still run dbt commands outside of poweruser, hence there should not be a auth issue.

When this happens, doing gcloud auth application-default login also wont work. It's stuck in some unauthed state and a reload of vscode is needed to get working again.

Actual behavior

Should not need to authenticate when auth is still valid.

Steps To Reproduce

Unfortunatly i'm currently not sure how to reproduce the issue, i'll add additional info if i find a way to reproduce.

Log output/Screenshots

Here's the traceback that dumps in the compiled preview window:

Traceback (most recent call last):
  File ".../dbt_common/clients/jinja.py", line 529, in catch_jinja
    yield
  File ".../dbt_common/clients/jinja.py", line 579, in render_template
    return template.render(ctx)
  File ".../dbt_common/clients/jinja.py", line 164, in render
    return self.environment.handle_exception()
  File ".../jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 19, in top-level template code
  File ".../jinja2/sandbox.py", line 401, in call
    return __context.call(__obj, *args, **kwargs)
  File ".../dbt/clients/jinja.py", line 82, in __call__
    return self.call_macro(*args, **kwargs)
  File ".../dbt_common/clients/jinja.py", line 323, in call_macro
    return macro(*args, **kwargs)
  File ".../jinja2/runtime.py", line 784, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 5, in template
  File ".../jinja2/sandbox.py", line 401, in call
    return __context.call(__obj, *args, **kwargs)
  File ".../dbt/adapters/bigquery/impl.py", line 309, in get_relation
    table = self.connections.get_bq_table(database, schema, identifier)
  File ".../dbt/adapters/bigquery/connections.py", line 522, in get_bq_table
    return client.get_table(self.table_ref(database, schema, identifier))
  File ".../google/cloud/bigquery/client.py", line 1170, in get_table
    api_response = self._call_api(
  File ".../google/cloud/bigquery/client.py", line 843, in _call_api
    return call()
  File ".../google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
  File ".../google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
  File ".../google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File ".../google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File ".../google/cloud/_http/__init__.py", line 482, in api_request
    response = self._make_request(
  File ".../google/cloud/_http/__init__.py", line 341, in _make_request
    return self._do_request(
  File ".../google/cloud/_http/__init__.py", line 379, in _do_request
    return self.http.request(
  File ".../google/auth/transport/requests.py", line 533, in request
    self.credentials.before_request(auth_request, method, url, request_headers)
  File ".../google/auth/credentials.py", line 239, in before_request
    self._blocking_refresh(request)
  File ".../google/auth/credentials.py", line 202, in _blocking_refresh
    self.refresh(request)
  File ".../google/oauth2/credentials.py", line 409, in refresh
    ) = reauth.refresh_grant(
  File ".../google/oauth2/reauth.py", line 349, in refresh_grant
    raise exceptions.RefreshError(
google.auth.exceptions.RefreshError: Reauthentication is needed. Please run `gcloud auth application-default login` to reauthenticate.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../vscode-dbt-power-user/dist/dbt_core_integration.py", line 751, in _compile_node
    compiled_node = self.sql_compiler.compile(self.dbt)
  File ".../dbt/task/sql.py", line 45, in compile
    return self.compiler.compile_node(self.node, manifest, {}, write=False)
  File ".../dbt/compilation.py", line 566, in compile_node
    node = self._compile_code(node, manifest, extra_context)
  File ".../dbt/compilation.py", line 424, in _compile_code
    node.compiled_code = jinja.get_rendered(
  File ".../dbt/clients/jinja.py", line 144, in get_rendered
    rendered = render_template(template, ctx, node)
  File ".../dbt_common/clients/jinja.py", line 578, in render_template
    with catch_jinja(node):
  File ".../contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File ".../dbt_common/clients/jinja.py", line 543, in catch_jinja
    raise CompilationError(str(e), node) from e
dbt_common.exceptions.base.CompilationError: Compilation Error in sql_operation t_718285a667d04d92ba18ef007daf91bb (from remote system.sql)
  Reauthentication is needed. Please run `gcloud auth application-default login` to reauthenticate.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../vscode-dbt-power-user/dist/dbt_core_integration.py", line 731, in _compile_sql
    node = self._compile_node(server_node)
  File ".../vscode-dbt-power-user/dist/dbt_core_integration.py", line 758, in _compile_node
    raise Exception(str(e))
Exception: Compilation Error in sql_operation t_718285a667d04d92ba18ef007daf91bb (from remote system.sql)
  Reauthentication is needed. Please run `gcloud auth application-default login` to reauthenticate.

Operating System

Windows 11 v 10.0.26100

dbt version

1.9.2

dbt Adapter

bigquery 1.9.1

dbt Power User version

v0.54.0

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[mdesmet]

My guess is that the command if you run it in the terminal updates some env variables that are not picked up by the running process. How does gcloud auth application-default login work?

[HampB]

That could make sense @mdesmet
Running the command generates a file containing "application_default_credentials.json" in the location of the env GOOGLE_APPLICATION_CREDENTIALS. as can be seen here auth/application-default/login

Running gcloud auth application-default login starts of with the logging:

The environment variable [GOOGLE_APPLICATION_CREDENTIALS] is set to:
  [C:\Users\username\AppData\Roaming\gcloud\application_default_credentials.json]
Credentials will still be generated to the default location:
  [C:\Users\username\AppData\Roaming\gcloud\application_default_credentials.json]
To use these credentials, unset this environment variable before
running your application.