Limin's Machine Learning Lab – openai API with LocalAI

Using openai API with locally running LLMs.

import time
import openai

openai.api_base = "http://localhost:8080/v1"
openai.api_key = "dummy api key"
model = "nous-hermes-13b.ggmlv3.q4_0.bin"
prompt = "Who is Stephen Hawking? Please write one paragraph."

start_time = time.perf_counter()

response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=1000,
    temperature=0.0
)

elapsed = time.perf_counter() - start_time


print(response.choices[0].text)
print(f'\nElapsed time: {elapsed}')

Stephen Hawking was a renowned theoretical physicist, cosmologist and author who made groundbreaking contributions to our understanding of the universe, including the now famous theory of black holes and the discovery that black holes emit radiation. He was also known for his work on quantum mechanics, general relativity and the origins of the cosmos. Despite being diagnosed with a rare form of motor neuron disease at the age of 21, he continued to make important contributions to science until his death in 2018.

Elapsed time: 62.64086055383086

prompt = """
What is the sentiment of the following review delimited with triple backticks.

Please limit your answer to one word.

```
I bought a laptop last week. It arrived quickly, \
but had a minor issue. The company promptly replaced it, \
and the new one works perfectly. Impressed with their quick \
response and the laptop's performance.
```
"""

start_time = time.perf_counter()

chat_completion = openai.ChatCompletion.create(
    model=model,
    messages=[{"role": "user", "content": prompt}])

elapsed = time.perf_counter() - start_time

print(chat_completion.choices[0].message)
print(f'\nElapsed time: {elapsed}')

{
  "content": "Positive",
  "role": "assistant"
}

Elapsed time: 26.65933546423912