Batch API now helps Embeddings and OpenAI Compatibility
At present we’re extending the Gemini Batch API to assist the newly launched Gemini Embedding mannequin in addition to providing builders the power to leverage the OpenAI SDK to submit and course of batches.
This builds on the preliminary launch of the Gemini Batch API – which permits asynchronous processing at 50% decrease charges for prime quantity and latency tolerant use circumstances.
Batch API Embedding Help
Our new Gemini Embedding Mannequin is already getting used for 1000’s of manufacturing deployments. And now, you may leverage the mannequin with the Batch API at a lot larger price limits and at half the value – $0.075 per 1M enter tokens – to unlock much more value delicate, latency tolerant, or asynchronous use circumstances.
Get began with Batch Embeddings with just a few strains of code:
# Create a JSONL along with your requests:
# {"key": "request_1", "request": {"output_dimensionality": 512, "content material": {"components": [{"text": "Explain GenAI"}]}}}
# {"key": "request_2", "request": {"output_dimensionality": 512, "content material": {"components": [{"text": "Explain quantum computing"}]}}}
from google import genai
consumer = genai.Consumer()
uploaded_batch_requests = consumer.recordsdata.add(file='embedding_requests.jsonl')
batch_job = consumer.batches.create_embeddings(
mannequin="gemini-embedding-001",
src={"file_name": uploaded_batch_requests.identify}
)
print(f"Created embedding batch job: {batch_job.identify}")
# Look forward to as much as 24 hours
if batch_job.state.identify == 'JOB_STATE_SUCCEEDED':
result_file_name = batch_job.dest.file_name
file_content_bytes = consumer.recordsdata.obtain(file=result_file_name)
file_content = file_content_bytes.decode('utf-8')
for line in file_content.splitlines():
print(line)Python
For extra informations and examples go to:
OpenAI compatibility for Batch API
Switching to Gemini Batch API is now as simple as updating a couple of strains of code if you happen to use the OpenAI SDK compatibility layer:
from openai import OpenAI
openai_client = OpenAI(
api_key="GEMINI_API_KEY",
base_url="https://generativelanguage.googleapis.com/v1beta/openai/"
)
# Add JSONL file in OpenAI batch enter format...
# Create batch
batch = openai_client.batches.create(
input_file_id=batch_input_file_id,
endpoint="/v1/chat/completions",
completion_window="24h"
)
# Look forward to as much as 24 hours & ballot for standing
batch = openai_client.batches.retrieve(batch.id)
if batch.standing == "accomplished":
# Obtain outcomes...Python
You’ll be able to learn extra concerning the OpenAI Compatibility layer and batch assist in our documentation.
We’re repeatedly increasing our batch providing to additional optimize the price of utilizing Gemini API, so hold an eye fixed out for additional updates. Within the meantime, completely happy constructing!
