Update models.py -added Groq API integration #995
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
feat: add GroqApiModel class for Groq API integration This commit introduces the GroqApiModel class that mirrors the structure and usage of HfApiModel from the smolagents library. The new class leverages the Groq API endpoint to support chat completions with configurable parameters such as temperature, max tokens, top-p, and streaming responses. Key changes: - Added GroqApiModel with a default model_id of "qwen" (overridable via parameter). - Supports both streaming and non-streaming responses. - Implements a similar interface to HfApiModel for consistency within the codebase. This implementation facilitates seamless integration with Groq's API and offers flexibility in choosing different models.
|
Hey, cool stuff. Is it more beneficial than the https://docs.litellm.ai/docs/providers/groq which already is there? |
|
Hey there, Nopes, not so beneficial, It just took me some time to explore that we can also integrate groq using LiteLLM (as I was not familiar with LiteLLM), so just thought of adding it as Groq is very widely used and it didn't have a connector of its own, so decided to add it. |
|
Since Groq is open ai compatible, curious what would be the nuances between using Was just looking at: https://console.groq.com/docs/openai |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: add GroqApiModel class for Groq API integration
This commit introduces the GroqApiModel class that mirrors the structure and usage of HfApiModel from the smolagents library. The new class leverages the Groq API endpoint to support chat completions with configurable parameters such as temperature, max tokens, top-p, and streaming responses.
Key changes:
This implementation facilitates seamless integration with Groq's API and offers flexibility in choosing different models.