\n<\/p>\n
At present we’re extending the Gemini Batch API to assist the newly launched Gemini Embedding mannequin<\/a> in addition to providing builders the power to leverage the OpenAI SDK<\/a> to submit and course of batches.<\/p>\n This builds on the preliminary launch of the Gemini Batch API<\/a> – which permits asynchronous processing at 50% decrease charges for prime quantity and latency tolerant use circumstances.<\/p>\n<\/div>\n Our new Gemini Embedding Mannequin is already getting used for 1000’s of manufacturing deployments<\/a>. And now, you may leverage the mannequin with the Batch API at a lot larger price limits<\/a> and at half the value – $0.075 per 1M enter tokens – to unlock much more value delicate, latency tolerant, or asynchronous use circumstances.<\/p>\n Get began with Batch Embeddings with just a few strains of code:<\/p>\n<\/div>\n \n Python\n <\/p>\n<\/div>\n For extra informations and examples go to:<\/p>\n<\/div>\n Switching to Gemini Batch API is now as simple as updating a couple of strains of code if you happen to use the OpenAI SDK compatibility layer:<\/p>\n<\/div>\n \n Python\n <\/p>\n<\/div>\nBatch API Embedding Help<\/b><\/h3>\n
# Create a JSONL along with your requests:
\n# {\"key\": \"request_1\", \"request\": {\"output_dimensionality\": 512, \"content material\": {\"components\": [{\"text\": \"Explain GenAI\"}]}}}
\n# {\"key\": \"request_2\", \"request\": {\"output_dimensionality\": 512, \"content material\": {\"components\": [{\"text\": \"Explain quantum computing\"}]}}}
\n
\nfrom google import genai
\n
\nconsumer = genai.Consumer()
\n
\nuploaded_batch_requests = consumer.recordsdata.add(file='embedding_requests.jsonl')
\n
\nbatch_job = consumer.batches.create_embeddings(
\n mannequin=\"gemini-embedding-001\",
\n src={\"file_name\": uploaded_batch_requests.identify}
\n)
\n
\n
\nprint(f\"Created embedding batch job: {batch_job.identify}\")
\n
\n# Look forward to as much as 24 hours
\n
\nif batch_job.state.identify == 'JOB_STATE_SUCCEEDED':
\n result_file_name = batch_job.dest.file_name
\n file_content_bytes = consumer.recordsdata.obtain(file=result_file_name)
\n file_content = file_content_bytes.decode('utf-8')
\n
\n for line in file_content.splitlines():
\n print(line)<\/code><\/pre>\nOpenAI compatibility for Batch API<\/b><\/h3>\n
from openai import OpenAI
\n
\nopenai_client = OpenAI(
\n api_key=\"GEMINI_API_KEY\",
\n base_url=\"https:\/\/generativelanguage.googleapis.com\/v1beta\/openai\/\"
\n)
\n
\n# Add JSONL file in OpenAI batch enter format...
\n
\n# Create batch
\nbatch = openai_client.batches.create(
\n input_file_id=batch_input_file_id,
\n endpoint=\"\/v1\/chat\/completions\",
\n completion_window=\"24h\"
\n)
\n
\n# Look forward to as much as 24 hours & ballot for standing
\nbatch = openai_client.batches.retrieve(batch.id)
\n
\nif batch.standing == \"accomplished\":
\n # Obtain outcomes...<\/code><\/pre>\n