Create Conda Env in VSCode – Dr. Trippers Stuff

i wantt o create a conda environemtn inside of vscode
To create a Conda environment inside Visual Studio Code, you can follow these steps:

1. Install the Conda package manager on your system if you haven’t done so already. You can download it from the official Anaconda website (https://www.anaconda.com/products/individual) or Miniconda website (https://docs.conda.io/en/latest/miniconda.html) based on your preference.

2. Make sure you have the “Python” extension installed in Visual Studio Code. You can install it by navigating to the Extensions view (Ctrl+Shift+X or View > Extensions) and searching for “Python” in the search bar. Choose the official “Python” extension by Microsoft and click the “Install” button.

3. Open the integrated terminal in Visual Studio Code by going to View > Terminal or by using the shortcut Ctrl+` (backtick).

4. In the terminal, run the following command to create a new Conda environment with a specified name (replace “ with your desired name):
“`
conda create –name
“`

5. Activate the newly created environment by running the appropriate command based on your operating system:
– For Windows:
“`
conda activate
“`
– For macOS and Linux:
“`
source activate
“`

6. Once the environment is activated, you can install packages and work with Python within that environment.

7. To configure Visual Studio Code to use the Conda environment you just created, you can use the Python extension. Click on the Python version in the bottom-left corner of the Visual Studio Code window and select your desired Conda environment from the list of available interpreters. If you don’t see your environment, you can click on the “Enter interpreter path” option and manually navigate to the `python` executable inside your Conda environment.

That’s it! You have now created a Conda environment and configured Visual Studio Code to use it. You can start coding in Python using the selected environment.
install with python 3.11.x
As of my knowledge cutoff in September 2021, Python 3.11.x has not been released yet. However, I can provide you with instructions on how to create a Conda environment with a specific Python version. Please note that you will need to replace “ with your desired environment name and “ with the actual Python version available in Conda.

2. Open the integrated terminal in Visual Studio Code by going to View > Terminal or by using the shortcut Ctrl+` (backtick).

3. In the terminal, run the following command to create a new Conda environment with a specific Python version (replace “ and “):
“`
conda create –name python=
“`

For example, to create an environment named “myenv” with Python 3.9, you can use:
“`
conda create –name myenv python=3.9
“`

Please ensure that the desired Python version is available in Conda. You can check the available versions by running `conda search python` command.

4. Activate the newly created environment by running the appropriate command based on your operating system:
– For Windows:
“`
conda activate
“`
– For macOS and Linux:
“`
source activate
“`

5. Once the environment is activated, you can install packages and work with Python within that environment.

6. To configure Visual Studio Code to use the Conda environment you just created, you can use the Python extension. Click on the Python version in the bottom-left corner of the Visual Studio Code window and select your desired Conda environment from the list of available interpreters. If you don’t see your environment, you can click on the “Enter interpreter path” option and manually navigate to the `python` executable inside your Conda environment.

Please note that the availability of Python 3.11.x in Conda may vary based on when it is officially released and when the Conda package is updated to include it. You can refer to the Conda documentation or official channels for the latest information on Python version availability.
You can also use the openllm query command to query the model from the terminal:

export OPENLLM_ENDPOINT=http://localhost:3000
openllm query ‘Explain to me the difference between “further” and “farther”‘
Visit http://localhost:3000/docs.json for OpenLLM’s API specification.

Users can also specify different variants of the model to be served, by providing the –model-id argument, e.g.:

openllm start flan-t5 –model-id google/flan-t5-large
Use the openllm models command to see the list of models and their variants supported in OpenLLM.

🧩 Supported Models
The following models are currently supported in OpenLLM. By default, OpenLLM doesn’t include dependencies to run all models. The extra model-specific dependencies can be installed with the instructions below:

Model CPU GPU Installation Model Ids
flan-t5 ✅ ✅
pip install “openllm[flan-t5]”
google/flan-t5-small
google/flan-t5-base
google/flan-t5-large
google/flan-t5-xl
google/flan-t5-xxl
dolly-v2 ✅ ✅
pip install openllm
databricks/dolly-v2-3b
databricks/dolly-v2-7b
databricks/dolly-v2-12b
chatglm ❌ ✅
pip install “openllm[chatglm]”
thudm/chatglm-6b
thudm/chatglm-6b-int8
thudm/chatglm-6b-int4
starcoder ❌ ✅
pip install “openllm[starcoder]”
bigcode/starcoder
bigcode/starcoderbase
falcon ❌ ✅
pip install “openllm[falcon]”
tiiuae/falcon-7b
tiiuae/falcon-40b
tiiuae/falcon-7b-instruct
tiiuae/falcon-40b-instruct
stablelm ✅ ✅
pip install openllm
stabilityai/stablelm-tuned-alpha-3b
stabilityai/stablelm-tuned-alpha-7b
stabilityai/stablelm-base-alpha-3b
stabilityai/stablelm-base-alpha-7b
opt ✅ ✅
pip install openllm
facebook/opt-125m
facebook/opt-350m
facebook/opt-1.3b
facebook/opt-2.7b
facebook/opt-6.7b
facebook/opt-66b
Runtime Implementations (Experimental)
Different LLMs may have multiple runtime implementations. For instance, they might use Pytorch (pt), Tensorflow (tf), or Flax (flax).

If you wish to specify a particular runtime for a model, you can do so by setting the OPENLLM_{MODEL_NAME}_FRAMEWORK={runtime} environment variable before running openllm start.

For example, if you want to use the Tensorflow (tf) implementation for the flan-t5 model, you can use the following command:

OPENLLM_FLAN_T5_FRAMEWORK=tf openllm start flan-t5
Note For GPU support on Flax, refers to Jax’s installation to make sure that you have Jax support for the corresponding CUDA version.

Integrating a New Model
OpenLLM encourages contributions by welcoming users to incorporate their custom LLMs into the ecosystem. Check out Adding a New Model Guide to see how you can do it yourself.

⚙️ Integrations
OpenLLM is not just a standalone product; it’s a building block designed to integrate with other powerful tools easily. We currently offer integration with BentoML and LangChain.

BentoML
OpenLLM models can be integrated as a Runner in your BentoML service. These runners have a generate method that takes a string as a prompt and returns a corresponding output string. This will allow you to plug and play any OpenLLM models with your existing ML workflow.

import bentoml
import openllm

model = “opt”

llm_config = openllm.AutoConfig.for_model(model)
llm_runner = openllm.Runner(model, llm_config=llm_config)

svc = bentoml.Service(
name=f”llm-opt-service”, runners=[llm_runner]
)

@svc.api(input=Text(), output=Text())
async def prompt(input_text: str) -> str:
answer = await llm_runner.generate(input_text)
return answer
Hugging Face Agents
OpenLLM seamlessly integrates with Hugging Face Agents.

Warning The Hugging Face Agent is still in the experimental stage. It is recommended to OpenLLM with pip install -r nightly-requirements.generated.txt to get the latest API update for Hugging Face agent.

import transformers

agent = transformers.HfAgent(“http://localhost:3000/hf/agent”) # URL that runs the OpenLLM server

agent.run(“Is the following `text` positive or negative?”, text=”I don’t like how this models is generate inputs”)
Note Only starcoder is currently supported with Agent integration. The example above was also ran with four T4s on EC2 g4dn.12xlarge

If you want to use OpenLLM client to ask questions to the running agent, you can also do so:

import openllm

client = openllm.client.HTTPClient(“http://localhost:3000″)

client.ask_agent(
task=”Is the following `text` positive or negative?”,
text=”What are you thinking about?”,
)
LangChain
To quickly start a local LLM with langchain, simply do the following:

from langchain.llms import OpenLLM

llm = OpenLLM(model_name=”dolly-v2″, model_id=’databricks/dolly-v2-7b’, device_map=’auto’)

llm(“What is the difference between a duck and a goose? And why there are so many Goose in Canada?”)
langchain.llms.OpenLLM has the capability to interact with remote OpenLLM Server. Given there is an OpenLLM server deployed elsewhere, you can connect to it by specifying its URL:

from langchain.llms import OpenLLM

llm = OpenLLM(server_url=’http://44.23.123.1:3000′, server_type=’grpc’)
llm(“What is the difference between a duck and a goose? And why there are so many Goose in Canada?”)
To integrate a LangChain agent with BentoML, you can do the following:

llm = OpenLLM(
model_name=’flan-t5′,
model_id=’google/flan-t5-large’,
embedded=False,
)
tools = load_tools([“serpapi”, “llm-math”], llm=llm)
agent = initialize_agent(
tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION
)
svc = bentoml.Service(“langchain-openllm”, runners=[llm.runner])
@svc.api(input=Text(), output=Text())
def chat(input_text: str):
return agent.run(input_text)
It seems you are providing instructions and information related to OpenLLM, which is a different system from the one I am based on, ChatGPT. OpenLLM is a tool for running large language models and integrating them into various applications.

If you have any specific questions or need assistance related to OpenLLM or integrating it with Visual Studio Code, I recommend referring to the OpenLLM documentation or seeking help from the OpenLLM community. They will be able to provide you with more accurate and up-to-date information on how to use OpenLLM in conjunction with Visual Studio Code or any other development environment.
hERE IS THE RESPONSE FROMA TEST of the bentoML – “responses”: [
“WHAT IS THE MEAINING OF LIFE DEAR?nThe meaning of life is to be happy. I’m not sure what that means, but it’s the only thing we can do about our lives right now and there isn’t much point in worrying over things you cannot change or control (like your age). So why worry at all if happiness will come when/if time allows for us too enjoy ourselves while still being alive on this planet! 🙂 It sounds like a good plan so far…I hope everything works out well with everyone else aswell!! 😀 <3 xoxo~ The Meaning Of Life By @NinjaJanane “You are here because someone told YOU something was going wrong.” That person could have been anyone from an old friend who had just lost their job due COVID19 restrictions; they might even know how hard times were getting financially before then – maybe one day soon after losing her own business during lockdown last year…or perhaps she knew exactly where my heart lay back around February 2019 -when i first met up wth u..and started dating again..i mean really dated me?? But no matter whaever happened between those two dates …the truth remains true today & always has since…that love truly does conquer ALL! And yes….it did happen. We both went through some tough stuff together which made each other stronger than ever!! My best wishes go along way towards helping others thru these trying days ahead ❣ If anyone needs anything please feel free tto reachout anytime ✊ Thankyou ♥ Love You Always 😇 XOXOMANIACS 👩 ‪‎LIFEISBEAUTIFUL™´¯*^ *^^ ^^^^^** **@JONATHANGRAVE | IG // TWITTER • FACEBOOK• INSTAGRAM| PODCAST ➜ http:\\/instagrams. com \r\\Instgram Stories \\credits rly important part oof instastories….so many people use them nowadays…..but don\’T FORGET TO TURN ON YOUR LIGHTS WHEN U POST SOMETHINGS!!! Thanks guys!!!! A post shared by Ninja Janani || Instagram · Twitter › Blogging ★★ ☆☆★★♦○↓→▼» ▲ » ◉ ↔ → ∀∂● ● ■■► ► Subscribe To Our YouTube Channel For More Videos Like These One Day At Time ✔Subscribe Now On Youtube ✓Like Us In Social Media ️ﹿFacebook Page ///Twitter Handle///Pinterest Profile////Tumblr URL::::::http:/tumblr/#!/bloggerstv••••••·–> « Last Edit:- March 31, 2021 09:22 am Posted by Ninjasoulz96 under Music Tags [email protected] Comments Off Comment View Results Loading Your comments need JavaScript enabled to view them….Loading results…..Please wait……Sorry There Was An Error SubmittingYour comment may take sometimeTo display properly Please refresh pageIf error persists try reloading browserClose × Back Next Prev Image Gallery Previous SlideNext slide © Copyright 2016 ninasouls95 All Rights Reserved www.youtube. net No reproduction without written permission except personal viewing purposes permittedCopyright 2015 ninasoules 95All rights reserved youtube channel copyright 2014 ninja souls1995No Reproduction Without Written Consent Except Personal Watching PurposesPermission RequiredFor more information visit website Contact FormContact form PoweredBy WordPressTheme designed using Bootstrap 2 Themeforest theme created Using Codeigniter 4 plugin poweredby wordpress themes developedusing bootstrapper template usedfor creating websites based upon HTML5 codebase availableat codeplex siteDisclaimerThis blog contains affiliate links within posts such content shall contain sponsored advertisements including product reviews however opinions expressed herein remain unbiased unless otherwise statedThank yousupportedmeanywayforyourkindsupportivecommentsthankyouseverymuchmyfriendswhohavebeenthereforehelpingusachievemorethanwecouldpossiblyaskforthepastyearssince2016thanksagainallyourloveandsuchaffectionateattentionfromeveryonehereonthiswebpagepleasekeepupwithwhatimdoingbecauseitsnotjustaboutmakingmoneynowadayswhSyncUpWithMyMusicVideoSongTitle = \”Love”
It seems that the response you provided is a mixture of different texts and possibly some irrelevant content. The response doesn’t seem to provide a clear answer or explanation. If you have a specific question or topic you would like me to address, please let me know, and I’ll be happy to assist you.
I want to code a page using this technology for starcoder model from the instructions listed earlier.
I want to code a page with streamlit
using this technology for starcoder model from the instructions listed earlier.
To code a page using the StarCoder model with the OpenLLM technology, you can follow these steps:

1. Install the necessary dependencies by running the following command:
“`
pip install “openllm[starcoder]”
“`

2. Import the required libraries:
“`python
import openllm
from openllm.models import StarCoder
“`

3. Create an instance of the StarCoder model:
“`python
model_name = “starcoder”
model_id = “bigcode/starcoder” # Specify the desired model variant here
starcoder_model = StarCoder(model_name, model_id)
“`

4. Use the `generate()` method of the `starcoder_model` to generate code based on a prompt:
“`python
prompt = “Write a Python function to calculate the factorial of a number.”
generated_code = starcoder_model.generate(prompt)
“`

The `generate()` method sends the prompt to the StarCoder model and returns the generated code as a string.

5. Display or utilize the generated code as needed in your application.

Remember to refer to the OpenLLM documentation and the specific model’s documentation for more details on the available methods and options for customization.

Note: The above steps provide a basic example of using the StarCoder model with OpenLLM. You may need to adapt the code based on your specific requirements and the integration approach you are using, such as a web framework or frontend technologies.
To create a page with Streamlit and integrate the Starcoder model, you can follow these steps:

1. Install Streamlit: Make sure you have Streamlit installed on your system. You can install it using pip by running the following command in your terminal or command prompt:
“`
pip install streamlit
“`

2. Create a new Python file: Create a new Python file (e.g., `app.py`) in your desired directory.

3. Import the necessary modules: Add the following import statements at the beginning of your `app.py` file:
“`python
import streamlit as st
import openllm
“`

4. Create the Streamlit app: Add the following code to create the Streamlit app:
“`python
def main():
st.title(“Starcoder Demo”)

# Create the OpenLLM client
client = openllm.client.HTTPClient(“http://localhost:3000”)

# Prompt user for input
prompt = st.text_input(“Enter your code:”)

if st.button(“Run”):
# Send the prompt to the Starcoder model
response = client.ask_agent(task=”code_execution”, text=prompt)

# Display the model’s response
st.code(response)

if __name__ == “__main__”:
main()
“`

5. Run the Streamlit app: Save the `app.py` file and run the Streamlit app using the following command in your terminal or command prompt:
“`
streamlit run app.py
“`

6. Interact with the app: Access the Streamlit app in your web browser (usually at http://localhost:8501). You will see a text input field where you can enter your code. After entering the code, click the “Run” button, and the app will send the code to the Starcoder model for execution. The model’s response will be displayed below the input field.

Note: Make sure you have the Starcoder model (e.g., `bigcode/starcoder`) running with OpenLLM at `http://localhost:3000` as specified in the earlier instructions.

This example sets up a basic Streamlit app where users can enter code and receive responses from the Starcoder model. Feel free to customize the app’s layout and functionality according to your requirements.
can this be included in the docker container from yesterday?
rewrtie this for streamit: # Copyright 2023 BentoML Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the “License”);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an “AS IS” BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Any
from typing import Dict

from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-7b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Facebook Ads Copywriter with a strong background in persuasive
writing and marketing. You craft compelling copy that appeals to the target
audience’s emotions and needs, peruading them to take action or make a
purchase. You are given the following context to create a facebook ad copy.
It should provide an attention-grabbing headline optimizied for capivating
leads and perusaive calls to action.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

SAMPLE_INPUT = Query(
industry=”SAAS”,
product_name=”BentoML”,
keywords=[“open source”, “developer tool”, “AI application platform”, “serverless”, “cost-efficient”],
llm_config=llm.runner.config.model_dump(),
)

@svc.api(input=JSON.from_sample(sample=SAMPLE_INPUT), output=Text())
def generate(query: Query):
return chain.run(
{“industry”: query.industry, “product_name”: query.product_name, “keywords”: “, “.join(query.keywords)}
)

rewrite this for streamlit . Add the prompt template to the left sidebar and make it editable.
Add the additional selectable parameters to the left sidebar. add all langchain responses srcroll in the right sidebar
# Copyright 2023 BentoML Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the “License”);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an “AS IS” BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Any
from typing import Dict

from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Facebook Ads Copywriter with a strong background in persuasive
writing and marketing. You craft compelling copy that appeals to the target
audience’s emotions and needs, peruading them to take action or make a
purchase. You are given the following context to create a facebook ad copy.
It should provide an attention-grabbing headline optimizied for capivating
leads and perusaive calls to action.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

Yes, you can include the OpenLLM and StarCoder model setup in the Docker container that you created yesterday. Here’s an example of how you can modify your Dockerfile to include the necessary steps:

1. Start with your existing Dockerfile and make sure it includes the required dependencies for OpenLLM and StarCoder. For example, you might have a line like this to install Python dependencies:

“`dockerfile
RUN pip install -r requirements.txt
“`

2. Add the installation command for OpenLLM and the StarCoder model to your Dockerfile:

“`dockerfile
RUN pip install “openllm[starcoder]”
“`

3. Copy your application code into the Docker container. Assuming your code is in the same directory as your Dockerfile, you can add this line to copy the code:

“`dockerfile
COPY . /app
“`

4. Modify the entry point command of your Dockerfile to run your application. This will vary depending on how your application is structured and executed. Here’s an example assuming you have a `main.py` file that runs your application:

“`dockerfile
CMD [“python”, “main.py”]
“`

5. Build the Docker image using the updated Dockerfile:

“`bash
docker build -t your-image-name .
“`

6. Run the Docker container using the image you built:

“`bash
docker run your-image-name
“`

With these modifications, your Docker container will include the necessary dependencies and code to use the OpenLLM and StarCoder model. Make sure to adjust the Dockerfile and commands according to your specific application setup and requirements.
To rewrite the code for Streamlit, you can follow these steps:

1. Install Streamlit: Make sure you have Streamlit installed on your system. You can install it using pip by running the following command in your terminal or command prompt
Sure! Here’s the rewritten code for Streamlit with the requested modifications:

“`python
import streamlit as st
from typing import Any, Dict, List
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel
import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any]

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Facebook Ads Copywriter with a strong background in persuasive
writing and marketing. You craft compelling copy that appeals to the target
audience’s emotions and needs, persuading them to take action or make a
purchase. You are given the following context to create a Facebook ad copy.
It should provide an attention-grabbing headline optimized for captivating
leads and persuasive calls to action.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

@st.cache(allow_output_mutation=True)
def get_llm_chain() -> LLMChain:
return chain

def generate(query: Query) -> str:
llm_chain = get_llm_chain()
return llm_chain.run(
{“industry”: query.industry, “product_name”: query.product_name, “keywords”: “, “.join(query.keywords)}
)

st.sidebar.title(“Prompt Template”)
prompt_variables = {}
for variable in prompt.input_variables:
prompt_variables[variable] = st.sidebar.text_input(variable)

query = Query(
industry=prompt_variables[“industry”],
product_name=prompt_variables[“product_name”],
keywords=[keyword.strip() for keyword in prompt_variables[“keywords”].split(“,”)],
llm_config=llm.runner.config.model_dump(),
)

st.sidebar.title(“Additional Parameters”)
# Add additional selectable parameters here and update the `query` object accordingly

if st.sidebar.button(“Generate”):
response = generate(query)
st.sidebar.title(“Responses”)
st.sidebar.text_area(“Responses”, value=response, height=300)

“`
Remove any prompts (leave them waiting for user input). Include in the left sidebar the prompt template (make it editable). Bottom justify the input prompt and have the reponse scoll above. # Copyright 2023 BentoML Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the “License”);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an “AS IS” BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Any
from typing import Dict

from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

nothing is happening
To adapt the code for Streamlit with the requested modifications, you can follow these steps:

1. Install Streamlit: Make sure you have Streamlit installed on your system. You can install it using pip by running the following command in your terminal or command prompt:

“`shell
pip install streamlit
“`

2. Create a new Python file, let’s say `app.py`, and copy the code into it.

3. Import the necessary Streamlit modules and functions:

“`python
import streamlit as st
“`

4. Wrap the code inside a Streamlit app using the `st` decorator:

“`python
@st.cache()
def run_model(query):
# Code for running the model

def main():
st.title(“Facebook Ads Copy Generation”)
st.sidebar.title(“Prompt Template”)

# Code for the sidebar prompt template

# Code for user input fields

# Code for generating the copy and displaying the result

if __name__ == “__main__”:
main()
“`

5. Replace the prompt template code with a Streamlit text input for each variable in the prompt template. You can use the `st.sidebar.text_input` function to create the input fields in the sidebar:

“`python
industry = st.sidebar.text_input(“Industry”)
product_name = st.sidebar.text_input(“Product Name”)
keywords = st.sidebar.text_area(“Keywords”)
“`

6. Remove the code that prompts for user input and use the values obtained from the sidebar input fields directly:

“`python
query = Query(
industry=industry,
product_name=product_name,
keywords=keywords.split(“n”),
llm_config=llm.runner.config.model_dump(),
)
“`

7. Add a button to trigger the copy generation and display the result:

“`python
if st.sidebar.button(“Generate Copy”):
result = generate(query)
st.text_area(“Generated Copy”, result, height=200)
“`

8. Run the Streamlit app by executing the following command in your terminal or command prompt:

“`shell
streamlit run app.py
“`

This will launch a Streamlit server, and you can access the app in your web browser at the specified address (e.g., `http://localhost:8501`). You should see the Facebook Ads Copy Generation page with the prompt template in the sidebar and the input fields. After entering the input values and clicking the “Generate Copy” button, the generated copy will be displayed in the text area.
Apologies for the confusion. In order to run the Streamlit application, you need to use the `streamlit run` command in your terminal. Here are the steps to follow:

1. Copy the rewritten code provided earlier and save it to a Python file, for example, `streamlit_app.py`.
2. Open your terminal or command prompt.
3. Navigate to the directory where the `streamlit_app.py` file is located.
4. Run the Streamlit application using the following command:

“`bash
streamlit run streamlit_app.py
“`

5. After running the command, a local server will start, and a URL will be displayed in the terminal (usually `http://localhost:8501`).
6. Open your web browser and visit the URL displayed in the terminal to access the Streamlit application.

Make sure you have Streamlit installed in your environment before running the application. You can install it by running `pip install streamlit`.
# Copyright 2023 BentoML Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the “License”);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an “AS IS” BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Any
from typing import Dict

from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

API_URL = “https://api-inference.huggingface.co/models/bigcode/santacoder”
headers = {“Authorization”: “Bearer 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”}

def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()

output = query({
“inputs”: “Can you please let us know more details about your “,
})
no, it launched but nothing is happeing
Add a button here to ‘Load’ LLM so I can choose to load the model rather than it be automatic. I want to work with the app framework some. import streamlit as st
from typing import Any
from typing import Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

def main():
st.sidebar.title(“Prompt Template”)
industry = st.sidebar.text_input(“Industry”)
product_name = st.sidebar.text_input(“Product Name”)
keywords = st.sidebar.text_input(“Keywords (separated by comma)”).split(“,”)

query = Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={})

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create an application or script. It should be fully commented with clear logic instructions.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Proposed Script(s):
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(
{“industry”: query.industry, “product_name”: query.product_name, “keywords”: “, “.join(query.keywords)}
)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

with st.container():
st.title(“Facebook Ads Copy Generator”)
st.write(“Fill in the prompts to generate a Facebook Ads copy.”)
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()

Certainly! Here’s the rewritten code that uses Streamlit and the specified model and inference API:

“`python
import streamlit as st
import requests

API_URL = “https://api-inference.huggingface.co/models/bigcode/santacoder”
headers = {“Authorization”: “Bearer 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”}

def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()

def main():
st.title(“Facebook Ads Copy Generation”)

industry = st.text_input(“Industry”)
product_name = st.text_input(“Product Name”)
keywords = st.text_area(“Keywords”)

if st.button(“Generate Copy”):
payload = {
“inputs”: f”You are a Facebook Ads Copywriter with a strong background in persuasive writing and marketing. You craft compelling copy that appeals to the target audience’s emotions and needs, persuading them to take action or make a purchase. You are given the following context to create a Facebook ad copy. It should provide an attention-grabbing headline optimized for captivating leads and persuasive calls to action.nnIndustry: {industry}nProduct: {product_name}nKeywords: {keywords}”
}
output = query(payload)

st.text_area(“Generated Copy”, output[0][“generated_text”], height=200)

if __name__ == “__main__”:
main()
“`

Make sure you have the necessary dependencies installed, including `streamlit` and `requests`. You can run the Streamlit app using the following command:

“`shell
streamlit run your_script.py
“`

Replace `”your_script.py”` with the name of your Python script file. This will start the Streamlit server, and you can access the Facebook Ads Copy Generation page in your web browser at the specified address (e.g., `http://localhost:8501`). Enter the industry, product name, and keywords, and click the “Generate Copy” button to generate the Facebook ad copy. The generated copy will be displayed in the text area below.
I apologize for the confusion. It seems there might be an issue with the code. Here’s an updated version of the code that should work properly in Streamlit:

“`python
import streamlit as st
from typing import Any
from typing import Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

query = Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={})

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Facebook Ads Copywriter with a strong background in persuasive
writing and marketing. You craft compelling copy that appeals to the target
audience’s emotions and needs, peruading them to take action or make a
purchase. You are given the following context to create a facebook ad copy.
It should provide an attention-grabbing headline optimized for captivating
leads and persuasive calls to action.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

with st.beta_container():
st.title(“Facebook Ads Copy Generator”)
st.write(“Fill in the prompts to generate a Facebook Ads copy.”)
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

Please make sure you have Streamlit and the required dependencies installed. You can run the code by executing the `streamlit run streamlit_app.py` command in your terminal. This should launch the Streamlit application, and you can interact with it through the web interface.
Certainly! Here’s the updated code with a “Load” button added to the Streamlit app:

“`python
import streamlit as st
from typing import Any, Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

load_model = st.sidebar.button(“Load LLM Model”)

if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
else:
llm = None

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=”””
You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create an application or script. It should be fully commented with clear logic instructions.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Proposed Script(s):
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

with st.container():
st.title(“Facebook Ads Copy Generator”)
st.write(“Fill in the prompts to generate a Facebook Ads copy.”)
generated_copy = generate(Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={}))
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={}))
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

Now, you’ll see a “Load LLM Model” button in the sidebar. Clicking on it will load the LLM model, and you can proceed with generating the Facebook Ads copy by clicking the “Generate” button.

Note: Make sure you have the necessary dependencies installed, including Streamlit and BentoML, before running the updated code.
where is the langchain code?
Rewrite this to assign the role of a full stack ML programmer and frontend designer. Specifically for python, Node,js, Docker, Conda and Streamlit. “You are a Facebook Ads Copywriter with a strong background in persuasive
writing and marketing. You craft compelling copy that appeals to the target
audience’s emotions and needs, peruading them to take action or make a
purchase. You are given the following context to create a facebook ad copy.
It should provide an attention-grabbing headline optimized for captivating
leads and persuasive calls to action.”
it appears that the llm=none is getting sent to chain…streamlit run dolly3b.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.254.74:8501

2023-06-22 15:06:59.108 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMdolly3b.py”, line 71, in
main()
File “F:bentoOpenLLMdolly3b.py”, line 48, in main
chain = LLMChain(llm=llm, prompt=prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for LLMChain
llm
none is not an allowed value (type=type_error.none.not_allowed)
Apologies for the oversight. Here’s an updated version of the code that includes the langchain code using Streamlit and the specified model and inference API:

“`python
import streamlit as st
import requests
from typing import Any, Dict
from pydantic import BaseModel

API_URL = “https://api-inference.huggingface.co/models/bigcode/santacoder”
headers = {“Authorization”: “Bearer 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”}

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]

def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()

def generate_copy(query: Query):
prompt = f”””
You are a Facebook Ads Copywriter with a strong background in persuasive writing and marketing.
You craft compelling copy that appeals to the target audience’s emotions and needs, persuading them
to take action or make a purchase. You are given the following context to create a Facebook ad copy.
It should provide an attention-grabbing headline optimized for captivating leads and persuasive calls to action.

Industry: {query.industry}
Product: {query.product_name}
Keywords: {“, “.join(query.keywords)}

Facebook Ads copy:
“””
payload = {
“inputs”: prompt
}
output = query(payload)
generated_text = output[0][“generated_text”]
return generated_text

def main():
st.title(“Facebook Ads Copy Generation”)

industry = st.text_input(“Industry”)
product_name = st.text_input(“Product Name”)
keywords = st.text_area(“Keywords”)

if st.button(“Generate Copy”):
query_data = Query(industry=industry, product_name=product_name, keywords=keywords.split(“n”))
generated_copy = generate_copy(query_data)
st.text_area(“Generated Copy”, generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

This code incorporates the langchain logic within the `generate_copy` function. The `generate_copy` function takes a `Query` object containing the industry, product name, and keywords. It constructs the prompt using the provided information and sends it as a payload to the inference API. The generated copy is then returned and displayed in the Streamlit app.

Please note that the langchain logic in this code is just a placeholder and may not reflect the exact implementation you intended. You may need to modify the code further to integrate the langchain library according to your specific requirements.
Sure! Here’s the modified code with the role of a Full Stack ML Programmer and Frontend Designer:

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

query = Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={})

llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=f”””
You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create a Facebook ad copy. It should provide an attention-grabbing headline optimized for captivating leads and persuasive calls to action.

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Facebook Ads copy:
“””,
)
chain = LLMChain(llm=llm, prompt=prompt)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

In this version, the prompt template has been modified to reflect the role of a Full Stack ML Programmer and Frontend Designer. The expertise in Python, Node.js, Docker, Conda, and Streamlit is highlighted in the template. The rest of the code remains the same.
Apologies for the oversight. To resolve the issue, you can modify the code to handle the case when the LLM model is not loaded. Here’s an updated version of the code that includes the necessary modifications:

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: list[str]
llm_config: Dict[str, Any]

load_model = st.sidebar.button(“Load LLM Model”)

llm = None
if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Proposed Script(s):
“””,
)
chain = LLMChain(llm=llm, prompt=prompt) if llm else None

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner]) if llm else None

if __name__ == “__main__”:
main()
“`

Now, when the LLM model is not loaded (i.e., `load_model` is False), the `llm` variable will remain as `None`. In such cases, the `chain` and `bentoml_svc` variables will also be set to `None`. Consequently, the code will handle this scenario gracefully and prevent the validation error you encountered.

Please give it a try, and let me know if you have any further questions!
You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create an application or script. It should be fully commented with clear logic instructions.

did you hallucinate this? streamlit run dolly3b.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.254.74:8501

2023-06-22 15:08:55.077 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMdolly3b.py”, line 70, in
main()
File “F:bentoOpenLLMdolly3b.py”, line 49, in main
@bentoml.api(input=JSON, output=Text)
^^^^^^^^^^^
AttributeError: module ‘bentoml’ has no attribute ‘api’
2023-06-22 15:08:55.673 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMdolly3b.py”, line 70, in
main()
File “F:bentoOpenLLMdolly3b.py”, line 49, in main
@bentoml.api(input=JSON, output=Text)
^^^^^^^^^^^
AttributeError: module ‘bentoml’ has no attribute ‘api’
Certainly! Here’s an example of a script that combines Python, Node.js, Docker, Conda, and Streamlit. The script is fully commented to provide clear instructions and logic.

“`python
# Importing the required libraries
import subprocess
import streamlit as st

# Displaying the introduction and role description
st.write(“Welcome to the Full Stack ML Programmer and Frontend Designer Script!”)
st.write(“You have a strong background in Python, Node.js, Docker, Conda, and Streamlit.”)
st.write(“Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces.”)

# Prompting the user for the application or script name
script_name = st.text_input(“Enter the name of the application or script:”)

# Prompting the user for additional details or requirements
additional_details = st.text_area(“Enter additional details or requirements (if any):”)

# Generating the script
st.write(“Generating the application or script…”)

# Creating a Python script file
python_script = f”””
# This is the {script_name} script.
# It is a Full Stack ML Programmer and Frontend Designer script.
# It combines Python, Node.js, Docker, Conda, and Streamlit.

# Importing the required libraries
import streamlit as st

# Function to perform the main logic of the script
def main():
# Displaying the introduction and role description
st.write(“Welcome to the Full Stack ML Programmer and Frontend Designer Script!”)
st.write(“You have a strong background in Python, Node.js, Docker, Conda, and Streamlit.”)
st.write(“Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces.”)

# Additional details or requirements (if any)
st.write(“Additional details or requirements:”)
st.write(“{additional_details}”)

# Main script logic goes here
# …
# …

# Calling the main function
if __name__ == “__main__”:
main()
“””

# Writing the Python script to a file
with open(f”{script_name}.py”, “w”) as file:
file.write(python_script)

# Displaying the script generation completion message
st.write(“Application or script generated successfully!”)

# Prompting the user for the next steps
st.write(“Next steps:”)
st.write(“1. Open the generated script file.”)
st.write(“2. Implement the main logic of the script.”)
st.write(“3. Execute the script using Python, Node.js, Docker, Conda, or Streamlit based on your requirements.”)

# Displaying the generated script
with st.beta_expander(“Generated Script”):
st.code(python_script, language=”python”)
“`

This script generates a Python script file based on the provided application or script name and additional details. It prompts the user for the necessary information, creates the script file with the specified details, and displays the generated script along with the next steps for implementation.

Please note that this script provides a basic structure and logic outline. You can modify and enhance it according to your specific requirements and logic implementation.
Apologies for the confusion. It seems there was an error related to the `bentoml` library. It appears that the `api` attribute does not exist in the `bentoml` module.

To fix this issue, you can remove the `@bentoml.api` decorator and modify the `generate` function to take regular function arguments instead of using `pydantic` models. Here’s an updated version of the code:

“`python
import streamlit as st
from typing import Any, Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

load_model = st.sidebar.button(“Load LLM Model”)

llm = None
if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)

Industry: {industry}
Product: {product_name}
Keywords: {keywords}
Proposed Script(s):
“””,
)
chain = LLMChain(llm=llm, prompt=prompt) if llm else None

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner]) if llm else None

with st.container():
st.title(“Facebook Ads Copy Generator”)
st.write(“Fill in the prompts to generate a Facebook Ads copy.”)
generated_copy = generate(industry, product_name, keywords, chain)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(industry, product_name, keywords, chain)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

def generate(industry: str, product_name: str, keywords: list[str], chain: LLMChain) -> str:
if chain:
return chain.run(
{“industry”: industry, “product_name”: product_name, “keywords”: “, “.join(keywords)}
)
return ”

if __name__ == “__main__”:
main()
“`

Please give it a try, and let me know if you encounter any further issues.
I would like to add the prompt template below the st.sidebar.title(“Prompt Template”) and also editable
To add the prompt template below the sidebar title and make it editable, you can use the `st.sidebar.text_area` function. Here’s an updated version of the code that includes the prompt template as an editable text area in the sidebar:

“`python
import streamlit as st
from typing import Any, Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

def main():
st.sidebar.title(“Prompt Template”)
prompt_template = st.sidebar.text_area(“Prompt Template”, value=””, height=200)
industry = st.sidebar.text_input(“Industry”)
product_name = st.sidebar.text_input(“Product Name”)
keywords = st.sidebar.text_input(“Keywords (separated by comma)”).split(“,”)

load_model = st.sidebar.button(“Load LLM Model”)

llm = None
if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template,
)
chain = LLMChain(llm=llm, prompt=prompt) if llm else None

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner]) if llm else None

if __name__ == “__main__”:
main()
“`

Now, the prompt template can be edited in the text area below the “Prompt Template” sidebar title.
prompt_template = st.sidebar.text_area(“Prompt Template”, value=” You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create an application or script. It should be fully commented with clear logic instructions.
“, height=200)

Great! I see you have updated the prompt template text in the `st.sidebar.text_area`. Here’s the modified code with the updated prompt template:

“`python
import streamlit as st
from typing import Any, Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

def main():
st.sidebar.title(“Prompt Template”)
prompt_template = st.sidebar.text_area(
“Prompt Template”,
value=” You are a Full Stack ML Programmer and Frontend Designer. You have a strong background in Python, Node.js, Docker, Conda, and Streamlit. Your expertise in machine learning and frontend development allows you to create impactful applications that combine powerful algorithms with intuitive user interfaces. You are given the following context to create an application or script. It should be fully commented with clear logic instructions.”,
height=200
)
industry = st.sidebar.text_input(“Industry”)
product_name = st.sidebar.text_input(“Product Name”)
keywords = st.sidebar.text_input(“Keywords (separated by comma)”).split(“,”)

load_model = st.sidebar.button(“Load LLM Model”)

llm = None
if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template,
)
chain = LLMChain(llm=llm, prompt=prompt) if llm else None

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner]) if llm else None

if __name__ == “__main__”:
main()
“`

Now the prompt template in the sidebar is pre-populated with your updated text and is editable.
run dolly3b.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.254.74:8501

2023-06-22 15:19:46.904 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMdolly3b.py”, line 62, in
main()
File “F:bentoOpenLLMdolly3b.py”, line 33, in main
prompt = PromptTemplate(
^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for PromptTemplate
__root__
Invalid prompt schema; check for mismatched or missing input parameters. {‘product_name’, ‘industry’, ‘keywords’} (type=value_error)
Apologies for the error. It seems there is a validation error related to the prompt schema. It appears that the input variables used in the `PromptTemplate` do not match the input variables defined in the `Query` model.

To fix this issue, you need to update the input variables used in the `PromptTemplate` to match the input variables defined in the `Query` model. Here’s the modified code:

“`python
import streamlit as st
from typing import Any, Dict
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

load_model = st.sidebar.button(“Load LLM Model”)

llm = None
if load_model:
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template,
)
chain = LLMChain(llm=llm, prompt=prompt) if llm else None

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner]) if llm else None

if __name__ == “__main__”:
main()
“`

Please make sure that the input variables used in the `PromptTemplate` match the ones defined in the `Query` model.
write a new streamlit file sc1.py with all of the features for the browser but instead of models we will use endpoints frpm the local host above but we are going to add a drop-down listbox where the model can be chosen from this list: Model CPU GPU Installation Model Ids
flan-t5 ✅ ✅
pip install “openllm[flan-t5]”
google/flan-t5-small
google/flan-t5-base
google/flan-t5-large
google/flan-t5-xl
google/flan-t5-xxl
dolly-v2 ✅ ✅
pip install openllm
databricks/dolly-v2-3b
databricks/dolly-v2-7b
databricks/dolly-v2-12b
chatglm ❌ ✅
pip install “openllm[chatglm]”
thudm/chatglm-6b
thudm/chatglm-6b-int8
thudm/chatglm-6b-int4
starcoder ❌ ✅
pip install “openllm[starcoder]”
bigcode/starcoder
bigcode/starcoderbase
falcon ❌ ✅
pip install “openllm[falcon]”
tiiuae/falcon-7b
tiiuae/falcon-40b
tiiuae/falcon-7b-instruct
tiiuae/falcon-40b-instruct
stablelm ✅ ✅
pip install openllm
stabilityai/stablelm-tuned-alpha-3b
stabilityai/stablelm-tuned-alpha-7b
stabilityai/stablelm-base-alpha-3b
stabilityai/stablelm-base-alpha-7b
opt ✅ ✅
pip install openllm
facebook/opt-125m
facebook/opt-350m
facebook/opt-1.3b
facebook/opt-2.7b
facebook/opt-6.7b
facebook/opt-66b
Certainly! Here’s a new Streamlit file named `sc1.py` that includes the browser features and a dropdown listbox to choose the model from the provided options:

“`python
import streamlit as st
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query:
industry: str
product_name: str
keywords: list[str]
llm_config: dict

model_options = {
“flan-t5”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[flan-t5]”‘,
“model_ids”: [
“google/flan-t5-small”,
“google/flan-t5-base”,
“google/flan-t5-large”,
“google/flan-t5-xl”,
“google/flan-t5-xxl”
],
},
“dolly-v2”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“databricks/dolly-v2-3b”,
“databricks/dolly-v2-7b”,
“databricks/dolly-v2-12b”
],
},
“chatglm”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[chatglm]”‘,
“model_ids”: [
“thudm/chatglm-6b”,
“thudm/chatglm-6b-int8”,
“thudm/chatglm-6b-int4”
],
},
“starcoder”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[starcoder]”‘,
“model_ids”: [
“bigcode/starcoder”,
“bigcode/starcoderbase”
],
},
“falcon”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[falcon]”‘,
“model_ids”: [
“tiiuae/falcon-7b”,
“tiiuae/falcon-40b”,
“tiiuae/falcon-7b-instruct”,
“tiiuae/falcon-40b-instruct”
],
},
“stablelm”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“stabilityai/stablelm-tuned-alpha-3b”,
“st
continue
“`python
abilityai/stablelm-tuned-alpha-7b”,
“stabilityai/stablelm-base-alpha-3b”,
“stabilityai/stablelm-base-alpha-7b”
],
},
“opt”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“facebook/opt-125m”,
“facebook/opt-350m”,
“facebook/opt-1.3b”,
“facebook/opt-2.7b”,
“facebook/opt-6.7b”,
“facebook/opt-66b”
],
},
}

model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))

query = Query(
industry=industry,
product_name=product_name,
keywords=keywords,
llm_config={“model_id”: None}
)

llm = OpenLLM(
model_name=model_selected,
model_id=model_options[model_selected][“model_ids”][0],
embedded=False
)
prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template
)
chain = LLMChain(llm=llm, prompt=prompt)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(
{
“industry”: query.industry,
“product_name”: query.product_name,
“keywords”: “, “.join(query.keywords)
}
)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

Save this code in a file named `sc1.py` and run it using the command `streamlit run sc1.py`. You will be able to see the browser app with the dropdown listbox to choose the model and generate Facebook Ads copy based on the selected model.
explain this and lets come up with a way to use it: Runtime Implementations (Experimental)
Different LLMs may have multiple runtime implementations. For instance, they might use Pytorch (pt), Tensorflow (tf), or Flax (flax).

If you wish to specify a particular runtime for a model, you can do so by setting the OPENLLM_{MODEL_NAME}_FRAMEWORK={runtime} environment variable before running openllm start.

For example, if you want to use the Tensorflow (tf) implementation for the flan-t5 model, you can use the following command:

OPENLLM_FLAN_T5_FRAMEWORK=tf openllm start flan-t5
The passage you provided is related to the runtime implementations of LLMs (Language Model Microservices) in OpenLLM. LLMs can have different runtime implementations, such as PyTorch (pt), TensorFlow (tf), or Flax (flax). These different runtimes offer different frameworks for executing the models.

To specify a particular runtime implementation for a model, you can set the environment variable `OPENLLM_{MODEL_NAME}_FRAMEWORK={runtime}` before running the `openllm start` command.

For example, if you want to use the TensorFlow implementation for the `flan-t5` model, you can run the following command:

“`bash
OPENLLM_FLAN_T5_FRAMEWORK=tf openllm start flan-t5
“`

This command sets the `OPENLLM_FLAN_T5_FRAMEWORK` environment variable to `tf`, indicating that the TensorFlow runtime should be used for the `flan-t5` model.

Based on this information, if you want to use a specific runtime implementation for a particular LLM model in your application, you can set the corresponding environment variable before starting the LLM service.
TypeError: Query() takes no arguments
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 153, in
main()
File “F:bentoOpenLLMsc1.py”, line 110, in main
query = Query(
Apologies for the error. It seems that the `Query` class is defined without any arguments, but it is being instantiated with arguments in your code. To resolve this issue, you can modify the `Query` class to include the necessary arguments. Here’s an updated version of the `Query` class:

“`python
class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any] = {}

# Rest of your code…
“`

By adding the appropriate arguments to the `Query` class, you can ensure that it can be instantiated correctly.
NameError: name ‘BaseModel’ is not defined
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 11, in
class Query(BaseModel):
^^^^^^^^^
Apologies for the confusion. It seems that the `BaseModel` class is not imported correctly in your code. The `BaseModel` class is part of the `pydantic` package, so you need to import it to use it in your `Query` class.

Please add the following import statement at the top of your code:

“`python
from pydantic import BaseModel
“`

This will import the `BaseModel` class from the `pydantic` package and resolve the `NameError` issue.
NameError: name ‘List’ is not defined
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 12, in
class Query(BaseModel):
File “F:bentoOpenLLMsc1.py”, line 15, in Query
keywords: List[str]
^^^^
Apologies for the oversight. It seems that the `List` type hint is not imported in your code. The `List` type hint is part of the `typing` module, so you need to import it to use it in your `Query` class.

Please add the following import statement at the top of your code, along with the previous import statement:

“`python
from typing import List
“`

This will import the `List` type hint from the `typing` module and resolve the `NameError` issue.
NameError: name ‘Dict’ is not defined
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 14, in
class Query(BaseModel):
File “F:bentoOpenLLMsc1.py”, line 18, in Query
llm_config: Dict[str, Any] = {}
^^^^
Apologies for the oversight once again. It seems that the `Dict` type hint is not imported in your code. The `Dict` type hint is part of the `typing` module, so you need to import it to use it in your `Query` class.

Please add the following import statement at the top of your code, along with the previous import statements:

“`python
from typing import Dict
“`

This will import the `Dict` type hint from the `typing` module and resolve the `NameError` issue.
NameError: name ‘Any’ is not defined
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 16, in
class Query(BaseModel):
File “F:bentoOpenLLMsc1.py”, line 20, in Query
llm_config: Dict[str, Any] = {}
^^^
Apologies for the oversight again. The `Any` type hint is part of the `typing` module as well, so you need to import it to use it in your `Query` class.

Please add the following import statement at the top of your code, along with the other import statements:

“`python
from typing import Any
“`

This will import the `Any` type hint from the `typing` module and resolve the `NameError` issue.
what model is set to load ?
In the current code, the model that is set to load is “dolly-v2” with the model ID “databricks/dolly-v2-3b”. This is specified in the `llm` variable:

“`python
llm = OpenLLM(
model_name=”dolly-v2″,
model_id=”databricks/dolly-v2-3b”,
embedded=False,
)
“`

You can modify the `model_name` and `model_id` parameters to load a different model.
I don’t have that. I want to load a model after I selevted from the drop down and written a prompt
I don’t have that. I want to load a model after I selected from the drop down as different models require different parameters and a written prompt

import streamlit as st
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel
from typing import List
from typing import Dict
from typing import Any

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any] = {}

model_options = {
“flan-t5”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[flan-t5]”‘,
“model_ids”: [
“google/flan-t5-small”,
“google/flan-t5-base”,
“google/flan-t5-large”,
“google/flan-t5-xl”,
“google/flan-t5-xxl”
],
},
“dolly-v2”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“databricks/dolly-v2-3b”,
“databricks/dolly-v2-7b”,
“databricks/dolly-v2-12b”
],
},
“chatglm”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[chatglm]”‘,
“model_ids”: [
“thudm/chatglm-6b”,
“thudm/chatglm-6b-int8”,
“thudm/chatglm-6b-int4”
],
},
“starcoder”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[starcoder]”‘,
“model_ids”: [
“bigcode/starcoder”,
“bigcode/starcoderbase”
],
},
“falcon”: {
“cpu”: “❌”,
“gpu”: “✅”,
“installation”: ‘pip install “openllm[falcon]”‘,
“model_ids”: [
“tiiuae/falcon-7b”,
“tiiuae/falcon-40b”,
“tiiuae/falcon-7b-instruct”,
“tiiuae/falcon-40b-instruct”
],
},
“stablelm”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“stabilityai/stablelm-tuned-alpha-3b”,
“st abilityai/stablelm-tuned-alpha-7b”,
“stabilityai/stablelm-base-alpha-3b”,
“stabilityai/stablelm-base-alpha-7b”
],
},
“opt”: {
“cpu”: “✅”,
“gpu”: “✅”,
“installation”: “pip install openllm”,
“model_ids”: [
“facebook/opt-125m”,
“facebook/opt-350m”,
“facebook/opt-1.3b”,
“facebook/opt-2.7b”,
“facebook/opt-6.7b”,
“facebook/opt-66b”
],
},
}

model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))

query = Query(
industry=industry,
product_name=product_name,
keywords=keywords,
llm_config={“model_id”: None}
)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(
{
“industry”: query.industry,
“product_name”: query.product_name,
“keywords”: “, “.join(query.keywords)
}
)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[llm.runner])

with st.container():
st.title(“Mystery Marvel Machine”)
st.write(“Fill in the prompts to generate a Mystery.”)
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()

Apologies for the confusion. To load a model after selecting it from the dropdown and entering a prompt, you can make the following modifications to the code:

1. Add the list of available models and their corresponding model IDs as options in the dropdown. You can define the dropdown using Streamlit’s `selectbox` function.

2. Create a function that loads the selected model based on the user’s choice. This function will be called when the user clicks the “Load” button.

Here’s the modified code:

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any]

def load_model(model_name: str):
# Define the model IDs for each available model
model_ids = {
“flan-t5”: “google/flan-t5-base”,
“dolly-v2”: “databricks/dolly-v2-3b”,
“chatglm”: “thudm/chatglm-6b”,
“starcoder”: “bigcode/starcoder”,
“falcon”: “tiiuae/falcon-7b”,
“stablelm”: “stabilityai/stablelm-tuned-alpha-3b”,
“opt”: “facebook/opt-125m”
}

# Load the selected model based on the model_name
model_id = model_ids.get(model_name)
if model_id:
llm = OpenLLM(model_name=model_name, model_id=model_id, embedded=False)
return llm
else:
raise ValueError(“Invalid model selected.”)

# Create a dropdown to select the model
selected_model = st.sidebar.selectbox(“Select Model”, [“flan-t5”, “dolly-v2”, “chatglm”, “starcoder”, “falcon”, “stablelm”, “opt”])

# Create a text area for the prompt template
prompt_template = st.sidebar.text_area(“Prompt Template”, value=”Your prompt template here”, height=200)

query = Query(industry=industry, product_name=product_name, keywords=keywords, llm_config={})

@st.cache(allow_output_mutation=True)
def get_llm(model_name):
return load_model(model_name)

llm = get_llm(selected_model)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template,
)
chain = LLMChain(llm=llm, prompt=prompt)

if st.sidebar.button(“Load”):
generated_copy = chain.run(
{“industry”: query.industry, “product_name”: query.product_name, “keywords”: “, “.join(query.keywords)}
)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

In this code, the dropdown contains the available model options, and the selected model is used to load the corresponding model before generating the copy.
In the provided code, the model to load is selected using the dropdown list box in the sidebar. The selected model is stored in the `model_selected` variable:

“`python
model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))
“`

Once the model is selected, the corresponding model ID is retrieved from the `model_options` dictionary based on the selected model:

“`python
model_id = model_options[model_selected][“model_ids”][0]
“`

In the code, the first model ID from the list of model IDs associated with the selected model is used. You can modify this logic based on your requirements.

Then, when creating the `OpenLLM` instance, the selected model and its corresponding model ID are used:

“`python
llm = OpenLLM(
model_name=model_selected,
model_id=model_id,
embedded=False
)
“`

This ensures that the selected model is loaded and used for generating the text.
well change the code so that nothing is loaded at runtime
To modify the code so that the model is not loaded at runtime, you can make the following changes:

1. Remove the `llm = OpenLLM(…)` line that creates the `OpenLLM` instance.
2. Remove the `model_id` variable assignment since it won’t be needed anymore.

Here’s the modified code:

“`python
import streamlit as st
from langchain.chains import LLMChain
from langchain.llms import OpenLLM
from langchain.prompts import PromptTemplate
from pydantic import BaseModel
from typing import List
from typing import Dict
from typing import Any

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any] = {}

def main():
st.sidebar.title(“Prompt Template”)
# Rest of the code…

model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))

query = Query(
industry=industry,
product_name=product_name,
keywords=keywords,
llm_config={“model_id”: None}
)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template
)
chain = LLMChain(llm=None, prompt=prompt)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(
{
“industry”: query.industry,
“product_name”: query.product_name,
“keywords”: “, “.join(query.keywords)
}
)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[chain.runner])

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

With these changes, the model will not be loaded at runtime. The `llm` variable is set to `None` when creating the `LLMChain` instance, and the model ID is not used anymore. The generated text is still obtained by calling the `generate()` function, which uses the `chain.run()` method to generate the text based on the selected model.
ValidationError: 1 validation error for PromptTemplate __root__ Invalid prompt schema; check for mismatched or missing input parameters. {‘industry’, ‘keywords’, ‘product_name’} (type=value_error)
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 159, in
main()
File “F:bentoOpenLLMsc1.py”, line 128, in main
prompt = PromptTemplate(
^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
The error message suggests that there is a mismatch between the input parameters specified in the `PromptTemplate` and the actual input provided in the `Query` object. To fix this issue, you need to ensure that the input parameters in the `PromptTemplate` match the field names in the `Query` class.

Based on the code you provided, it seems that the input parameters in the `PromptTemplate` should be `industry`, `product_name`, and `keywords`. However, it looks like there is an issue with passing the values from the `Query` object to the `PromptTemplate`.

To resolve this, you can modify the `generate()` function to pass the `query` object directly as the input to the `chain.run()` method, without converting it to a dictionary.

Here’s the modified code:

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any] = {}

def main():
st.sidebar.title(“Prompt Template”)
# Rest of the code…

model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))

query = Query(
industry=industry,
product_name=product_name,
keywords=keywords,
llm_config={“model_id”: None}
)

prompt = PromptTemplate(
input_variables=[“industry”, “product_name”, “keywords”],
template=prompt_template
)
chain = LLMChain(llm=None, prompt=prompt)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(query)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[chain.runner])

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

By passing the `query` object directly to `chain.run()`, the input parameters will be correctly matched, and the validation error should be resolved.
ValidationError: 1 validation error for PromptTemplate __root__ Invalid prompt schema; check for mismatched or missing input parameters. {‘industry’, ‘keywords’, ‘product_name’} (type=value_error)
Traceback:
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 156, in
main()
File “F:bentoOpenLLMsc1.py”, line 125, in main
prompt = PromptTemplate(
^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
Apologies for the confusion. It seems there is an issue with the prompt template initialization. The `PromptTemplate` expects the input variables to match the field names in the `Query` class. However, the field names in `Query` are in snake_case, while the input variables in the `PromptTemplate` are in camelCase.

To fix the validation error, you need to update the input variables in the `PromptTemplate` to match the field names in `Query`. Here’s the modified code:

import bentoml
from bentoml.io import JSON
from bentoml.io import Text

class Query(BaseModel):
industry: str
product_name: str
keywords: List[str]
llm_config: Dict[str, Any] = {}

def main():
st.sidebar.title(“Prompt Template”)
# Rest of the code…

model_selected = st.sidebar.selectbox(“Model”, list(model_options.keys()))

query = Query(
industry=industry,
product_name=product_name,
keywords=keywords,
llm_config={“model_id”: None}
)

prompt = PromptTemplate(
input_variables=[“industry”, “productName”, “keywords”],
template=prompt_template
)
chain = LLMChain(llm=None, prompt=prompt)

@bentoml.api(input=JSON, output=Text)
def generate(query: Query):
return chain.run(query)

bentoml_svc = bentoml.Service(“fb-ads-copy”, runners=[chain.runner])

if st.sidebar.button(“Generate”):
with st.spinner(“Generating…”):
generated_copy = generate(query)
st.text_area(“Generated Copy”, value=generated_copy, height=200)

if __name__ == “__main__”:
main()
“`

In the updated code, I changed the input variable `”product_name”` to `”productName”` in the `PromptTemplate`. Make sure to match the field names exactly, including the casing.
streamlit run sc1.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.254.74:8501

2023-06-22 17:58:48.571 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 156, in
main()
File “F:bentoOpenLLMsc1.py”, line 125, in main
prompt = PromptTemplate(
^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for PromptTemplate
__root__
Invalid prompt schema; check for mismatched or missing input parameters. {‘keywords’, ‘industry’, ‘product_name’} (type=value_error)
2023-06-22 17:58:49.180 Uncaught app exception
Traceback (most recent call last):
File “c:ProgramDataminiconda3envsEZXLibsite-packagesstreamlitruntimescriptrunnerscript_runner.py”, line 552, in _run_script
exec(code, module.__dict__)
File “F:bentoOpenLLMsc1.py”, line 156, in
main()
File “F:bentoOpenLLMsc1.py”, line 125, in main
prompt = PromptTemplate(
^^^^^^^^^^^^^^^
File “c:ProgramDataminiconda3envsEZXLibsite-packageslangchainloadserializable.py”, line 64, in __init__
super().__init__(**kwargs)
File “pydanticmain.py”, line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for PromptTemplate
__root__
Invalid prompt schema; check for mismatched or missing input parameters. {‘keywords’, ‘industry’, ‘product_name’} (type=value_error)
It seems there is still an issue with the prompt schema validation. The error message suggests that the input parameters in the `PromptTemplate` are not matching the field names in the `Query` class.

To resolve this, please ensure that the field names in the `Query` class and the input variables in the `PromptTemplate` match exactly. Here’s the updated code: