Update README.md

README.md

@@ -19,4 +19,46 @@ Only use the API reference to understand the syntax of the request.
 
 ## Answer
 {output}
+```
+
+## Example usage
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "substratusai/weaviate-gorilla-v4-random-split"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    load_in_4bit=True,
+    device_map='auto',
+)
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+text = """
+## Instruction
+Your task is to write GraphQL for the Natural Language Query provided. Use the provided API reference and Schema to generate the GraphQL. The GraphQL should be valid for Weaviate.
+
+Only use the API reference to understand the syntax of the request.
+
+## Natural Language Query
+```text Get me the top 10 historical events related to 'World War II', and show the event name, description, year, significant impact, and the names and populations of the involved countries. ```
+
+## Schema
+{ "classes": [ { "class": "HistoricalEvent", "description": "Information about historical events", "vectorIndexType": "hnsw", "vectorizer": "text2vec-transformers", "properties": [ { "name": "eventName", "dataType": ["text"], "description": "Name of the historical event" }, { "name": "description", "dataType": ["text"], "description": "Detailed description of the event" }, { "name": "year", "dataType": ["int"], "description": "Year the event occurred" }, { "name": "hadSignificantImpact", "dataType": ["boolean"], "description": "Whether the event had a significant impact" }, { "name": "involvedCountries", "dataType": ["Country"], "description": "Countries involved in the event" }{ "class": "Country", "description": "Information about countries", "vectorIndexType": "hnsw", "vectorizer": "text2vec-transformers", "properties": [ { "name": "countryName", "dataType": ["text"], "description": "Name of the country" }, { "name": "population", "dataType": ["int"], "description": "Population of the country" }}}
+
+## API reference
+1. Limit BM25 search results Limit the results[] You can limit the number of results returned by a `bm25` search, - to a fixed number, using the `limit: <N>` operator - to the first N "drops" in `score`, using the `autocut` operator `autocut` can be combined with `limit: N`, which would limit autocut's input to the first `N` objects. Limiting the number of results Use the `limit` argument to specify the maximum number of results that should be returned: ```graphql { Get { JeopardyQuestion( bm25: { query: "safety" }, limit: 3 ) { question answer _additional { score } } } } ```
+
+## Answer
+```graphql
+"""
+device = "cuda:0"
+
+inputs = tokenizer(text, return_tensors="pt").to(device)
+# this was needed due to a issue with model not taking token_type_ids
+# inputs.pop("token_type_ids")
+outputs = model.generate(**inputs, max_new_tokens=300)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```