Skip to main content
Open In ColabOpen on GitHub

SambaStudio

SambaNova 的 Sambastudio 是一个允许您训练、运行批量推理任务以及部署在线推理端点来运行您自行微调的开源模型的平台。

caution

您目前所在的页面记录了如何将 SambaStudio 模型用作 文本补全模型。我们建议您使用 聊天补全模型。

您可能正在寻找的是 SambaStudio 聊天模型。

概述​

集成详情​

类包本地可序列化JS 支持包下载量包最新版本
SambaStudiolangchain_community❌beta❌PyPI - DownloadsPyPI - Version

本示例将介绍如何使用 LangChain 与 SambaStudio 模型进行交互。

设置​

凭据​

部署模型需要 SambaStudio 环境。请访问 sambanova.ai/products/enterprise-ai-platform-sambanova-suite 获取更多信息。

您需要 部署一个端点 并设置 SAMBASTUDIO_URL 和 SAMBASTUDIO_API_KEY 环境变量：

import getpass
import os

if "SAMBASTUDIO_URL" not in os.environ:
    os.environ["SAMBASTUDIO_URL"] = getpass.getpass()
if "SAMBASTUDIO_API_KEY" not in os.environ:
    os.environ["SAMBASTUDIO_API_KEY"] = getpass.getpass()

安装​

集成位于 langchain-community 包中。我们还需要安装 sseclient-py 包，这是运行流式预测所必需的。

%pip install --quiet -U langchain-community sseclient-py

实例化​

from langchain_community.llms.sambanova import SambaStudio

llm = SambaStudio(
    model_kwargs={
        "do_sample": True,
        "max_tokens": 1024,
        "temperature": 0.01,
        "process_prompt": True,  # set if using CoE endpoints
        "model": "Meta-Llama-3-70B-Instruct-4096",  # set if using CoE endpoints
        # "repetition_penalty":  1.0,
        # "top_k": 50,
        # "top_logprobs": 0,
        # "top_p": 1.0
    },
)
API Reference:SambaStudio

调用​

现在我们可以实例化我们的模型对象并生成聊天补全：

input_text = "Why should I use open source models?"

completion = llm.invoke(input_text)
completion
"Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:\n\n1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.\n2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.\n3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.\n4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.\n5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.\n6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.\n7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.\n8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.\n9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.\n10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.\n11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.\n12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.\n13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.\n14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.\n15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.\n\nOverall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair."
# Streaming response
for chunk in llm.stream("Why should I use open source models?"):
    print(chunk, end="", flush=True)
Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:

1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.
2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.
3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.
4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.
5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.
6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.
7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.
8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.
9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.
10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.
11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.
12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.
13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.
14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.
15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.

Overall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair.

链式调用​

from langchain_core.prompts import PromptTemplate

prompt = PromptTemplate.from_template("How to say {input} in {output_language}:\n")

chain = prompt | llm
chain.invoke(
    {
        "output_language": "German",
        "input": "I love programming.",
    }
)
API Reference:PromptTemplate
'In German, you can say:\n\n"Ich liebe das Programmieren."\n\nHere\'s a breakdown of the sentence:\n\n* "Ich" means "I"\n* "liebe" is the verb "to love" in the present tense, first person singular (I love)\n* "das" is the definite article "the"\n* "Programmieren" is the noun "programming"\n\nSo, "Ich liebe das Programmieren" literally means "I love the programming".\n\nIf you want to make it sound more casual, you can say:\n\n"Ich liebe\'s Programmieren."\n\nThe apostrophe in "liebe\'s" is a contraction of "liebe es", which is a more informal way of saying "I love it".\n\nAlternatively, you can also say:\n\n"Programmieren ist meine Leidenschaft."\n\nThis sentence means "Programming is my passion". Here, "Programmieren" is the subject, "ist" is the verb "to be" in the present tense, and "meine Leidenschaft" means "my passion".'

API 参考​

有关 SambaStudio llm 功能和配置的详细文档，请访问 API 参考：https://python.langchain.com/api_reference/community/llms/langchain_community.llms.sambanova.SambaStudio.html