[Bug]: OpenAI API key changes not reflected across multiple calls

[LennartSchmidtKern]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

I am using graphrag in a FastAPI backend service where a request can create a new custom index. The OpenAI API keys appear to be cached in memory across requests. Even though configurations are set dynamically from a template each time (with a possible new key), and the correct key is present just before entering build_index function, the requests still use an outdated key. This appears in the following scenario: First, an incorrect api key is used, which results in an 401 error as expected. The next time a correct key is used, which still results in the same error showing an preview of the previous incorrect key. Logging the settings going into build_index shows the intended key.

My configuration explicitly sets caching to "none", but the documentation is unclear on whether this affects the overall caching behavior of the LLM client. Restarting the FastAPI server clears the issue, suggesting that the key is being held in memory rather than being refreshed dynamically. This might also be related to the use of the fnllm library as far as I understand.

Steps to reproduce

No response

Expected Behavior

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

• GraphRAG Version:
• Operating System:
• Python Version:
• Related Issues:

[natoverse]

This is by design. It's not explicitly a caching issue (your key is not written to the cache), but is related to the cache and to threading. We use a singleton ModelFactory to register model instances so we can manage rate limiting with parallel API calls. The instance name is hard-coded per workflow step ("extract_graph" for example) to ensure the cache instance matches between runs. So this should be a first-one-wins situation, which is validated in your notes. I'll put it in the backlog to investigate a different approach, but we may not be able to get to it for awhile.