We\u2019re excited to announce that\u00a0Amazon Bedrock Customized Mannequin Import<\/a>\u00a0now helps\u00a0Qwen<\/a> fashions.\u00a0Now you can import customized weights\u00a0for Qwen2, Qwen2_VL, and Qwen2_5_VL architectures, together with fashions like Qwen 2, 2.5 Coder, Qwen 2.5 VL, and QwQ 32B.\u00a0You possibly can deliver your personal custom-made Qwen fashions into Amazon Bedrock and deploy them in a completely managed, serverless atmosphere\u2014with out having to handle infrastructure or mannequin serving.<\/p>\n On this put up, we cowl the right way to deploy Qwen 2.5 fashions with Amazon Bedrock Customized Mannequin Import, making them accessible to organizations wanting to make use of state-of-the-art AI capabilities throughout the AWS infrastructure at an efficient value.<\/p>\n Qwen 2 and a couple of.5 are households of enormous language fashions, out there in a variety of sizes and specialised variants to go well with numerous wants:<\/p>\n Amazon Bedrock Customized Mannequin Import permits the import and use of your custom-made fashions alongside current basis fashions (FMs) by way of a single serverless, unified API. You possibly can entry your imported customized fashions on-demand and with out the necessity to handle the underlying infrastructure. Speed up your generative AI software growth by integrating your supported customized fashions with native Amazon Bedrock instruments and options like Amazon Bedrock Information Bases, Amazon Bedrock Guardrails, and Amazon Bedrock Brokers. Amazon Bedrock Customized Mannequin Import is mostly out there within the US-East (N. Virginia), US-West (Oregon), and Europe (Frankfurt) AWS Areas. Now, we\u2019ll discover how you should utilize Qwen 2.5 fashions for 2 frequent use instances: as a coding assistant and for picture understanding. Qwen2.5-Coder is a state-of-the-art code mannequin, matching capabilities of proprietary fashions like GPT-4o. It helps over 90 programming languages and excels at code era, debugging, and reasoning. Qwen 2.5-VL brings superior multimodal capabilities. In accordance with Qwen, Qwen 2.5-VL isn’t solely proficient at recognizing objects similar to flowers and animals, but in addition at analyzing charts, extracting textual content from pictures, deciphering doc layouts, and processing lengthy movies.<\/p>\n Earlier than importing the Qwen mannequin with Amazon Bedrock Customized Mannequin Import, just be sure you have the next in place:<\/p>\n On this instance, we’ll exhibit the right way to construct a coding assistant utilizing the Qwen2.5-Coder-7B-Instruct mannequin<\/p>\n You’ll use Relying in your mannequin dimension, this might take a couple of minutes. When accomplished, your Qwen Coder 7B mannequin folder will include the next recordsdata.<\/p>\n You can too do that utilizing the AWS Administration Console for Amazon Bedrock.<\/p>\n For those who\u2019re utilizing your personal position, be sure you add the next belief relationship as describes in \u00a0Create a service position for mannequin import<\/a>.<\/p>\n After your mannequin is imported, anticipate mannequin inference to be prepared, after which chat with the mannequin on the playground or by way of the API. Within the following instance, we append Notice that when utilizing the After the mannequin is prepared for inference, you should utilize Chat Playground in Bedrock console or APIs to invoke the mannequin.<\/p>\n Qwen2.5-VL-* affords multimodal capabilities, combining imaginative and prescient and language understanding in a single mannequin. This part demonstrates the right way to deploy Qwen2.5-VL utilizing Amazon Bedrock Customized Mannequin Import and take a look at its picture understanding capabilities.<\/p>\n Obtain the mannequin from Huggingface Face and add it to Amazon S3:<\/p>\n Subsequent, import the mannequin to Amazon Bedrock (both through Console or API):<\/p>\n After the import is full, take a look at the mannequin with a picture enter. The Qwen2.5-VL-* mannequin requires correct formatting of multimodal inputs:<\/p>\n When supplied with an instance picture of a cat (such the next picture), the mannequin precisely describes key options such because the cat\u2019s place, fur coloration, eye coloration, and basic look. This demonstrates Qwen2.5-VL-* mannequin\u2019s capacity to course of visible data and generate related textual content descriptions.<\/p>\n The mannequin\u2019s response:<\/p>\n You should utilize Amazon Bedrock Customized Mannequin Import to make use of your customized mannequin weights inside Amazon Bedrock for supported architectures, serving them alongside Amazon Bedrock hosted FMs in a completely managed approach by way of On-Demand mode. Customized Mannequin Import doesn\u2019t cost for mannequin import. You might be charged for inference based mostly on two components: the variety of energetic mannequin copies and their length of exercise. Billing happens in 5-minute increments, ranging from the primary profitable invocation of every mannequin copy. The pricing per mannequin copy per minute varies based mostly on components together with structure, context size, Area, and compute unit model, and is tiered by mannequin copy dimension. The customized mannequin unites required for internet hosting is determined by the mannequin\u2019s structure, parameter rely, and context size. Amazon Bedrock mechanically manages scaling based mostly in your utilization patterns. If there aren’t any invocations for five minutes, it scales to zero and scales up when wanted, although this may contain cold-start latency of as much as a minute. Extra copies are added if inference quantity constantly exceeds single-copy concurrency limits. The utmost throughput and concurrency per copy is set throughout import, based mostly on components similar to enter\/output token combine, {hardware} kind, mannequin dimension, structure, and inference optimizations.<\/p>\n For extra data, see Amazon Bedrock pricing<\/a>.<\/p>\n To keep away from ongoing expenses after finishing the experiments:<\/p>\n Keep in mind that whereas Amazon Bedrock Customized Mannequin Import doesn\u2019t cost for the import course of itself, you’re billed for mannequin inference utilization and storage.<\/p>\n Amazon Bedrock Customized Mannequin Import empowers organizations to make use of highly effective publicly out there fashions like Qwen 2.5, amongst others, whereas benefiting from enterprise-grade infrastructure. The serverless nature of Amazon Bedrock eliminates the complexity of managing mannequin deployments and operations, permitting groups to concentrate on constructing purposes quite than infrastructure. With options like auto scaling, pay-per-use pricing, and seamless integration with AWS providers, Amazon Bedrock supplies a production-ready atmosphere for AI workloads. The mixture of Qwen 2.5\u2019s superior AI capabilities and Amazon Bedrock managed infrastructure affords an optimum stability of efficiency, value, and operational effectivity. Organizations can begin with smaller fashions and scale up as wanted, whereas sustaining full management over their mannequin deployments and benefiting from AWS safety and compliance capabilities.<\/p>\nOverview of Qwen fashions<\/h2>\n
\n
Overview of Amazon Bedrock Customized Mannequin Import<\/h2>\n
Conditions<\/h2>\n
\n
Use case 1: Qwen coding assistant<\/h2>\n
\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-1.jpeg\")
<\/p>\nQwen\/Qwen2.5-Coder-7B-Instruct<\/code>\u00a0for the remainder of the walkthrough.\u00a0We don\u2019t exhibit fine-tuning steps, however you may also fine-tune<\/a> earlier than importing.<\/p>\n\n
from huggingface_hub import snapshot_download\n\nsnapshot_download(repo_id=\"\u00a0Qwen\/Qwen2.5-Coder-7B-Instruct\", \n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 local_dir=f\".\/extractedmodel\/\")<\/code><\/pre>\n<\/p><\/div>\n\n
config.json<\/code>, generation_config.json<\/code>, tokenizer_config.json<\/code>, tokenizer.json<\/code>, and vocab.json<\/code><\/li>\nsafetensor<\/code> recordsdata and mannequin.safetensors.index.json<\/code><\/li>\nLICENSE<\/code>, README.md<\/code>, and merges.txt<\/code><\/li>\n<\/ul>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-2.jpeg\")
<\/p>\n\n
boto3<\/code>\u00a0or the command line:<\/li>\n<\/ol>\naws s3 cp .\/extractedfolder s3:\/\/yourbucket\/path\/ --recursive<\/code><\/p>\n\n
response = self.bedrock_client.create_model_import_job(\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0jobName=\"uniquejobname\",\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0importedModelName=\"uniquemodelname\",\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0roleArn=\"fullrolearn\",\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0modelDataSource={\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0's3DataSource': {\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0's3Uri':\u00a0\"s3:\/\/yourbucket\/path\/\"\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0}\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0}\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0)\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0<\/code><\/pre>\n<\/p><\/div>\n\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-3.jpeg\")
<\/p>\n\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-4.jpeg\")
<\/p>\n\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-5.jpeg\")
<\/p>\n\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-6.jpeg\")
<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-7.jpeg\")
<\/p>\nPython<\/code> to immediate the mannequin to straight output Python code to checklist gadgets in an S3 bucket. Keep in mind to make use of the appropriate chat template to enter prompts within the format required. For instance, you will get the appropriate chat template for any appropriate mannequin on Hugging Face utilizing under code:<\/p>\nfrom transformers import AutoTokenizer\ntokenizer = AutoTokenizer.from_pretrained(\"Qwen\/Qwen2.5-Coder-7B-Instruct\")\n\n# As a substitute of utilizing mannequin.chat(), we straight use mannequin.generate()\n# However it is advisable to use tokenizer.apply_chat_template() to format your inputs as proven under\nimmediate = \"Write pattern boto3 python code to checklist recordsdata in a bucket saved within the variable `my_bucket`\"\nmessages = [\n\u00a0\u00a0 \u00a0{\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n\u00a0\u00a0 \u00a0{\"role\": \"user\", \"content\": prompt}\n]\ntextual content = tokenizer.apply_chat_template(\n\u00a0\u00a0 \u00a0messages,\n\u00a0\u00a0 \u00a0tokenize=False,\n\u00a0\u00a0 \u00a0add_generation_prompt=True\n)<\/code><\/pre>\n<\/p><\/div>\ninvoke_model<\/code>\u00a0APIs, it’s essential to use the complete Amazon Useful resource Title (ARN) for the imported mannequin. You’ll find the Mannequin ARN within the Bedrock console, by navigating to the Imported fashions part after which viewing the Mannequin particulars web page, as proven within the following determine<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-8.jpeg\")
<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-9.jpeg\")
<\/p>\nUse case 2: Qwen 2.5 VL picture understanding<\/h2>\n
Import Qwen2.5-VL-7B to Amazon Bedrock<\/h3>\n
from huggingface_hub import snapshot_download\n\nhf_model_id = \"Qwen\/Qwen2.5-VL-7B-Instruct\"\n\n# Allow quicker downloads\nos.environ[\"HF_HUB_ENABLE_HF_TRANSFER\"] = \"1\"\n\n# Obtain mannequin regionally\nsnapshot_download(repo_id=hf_model_id, local_dir=f\".\/{local_directory}\")<\/code><\/pre>\n<\/p><\/div>\nresponse = bedrock.create_model_import_job(\n\u00a0\u00a0 \u00a0jobName=job_name,\n\u00a0\u00a0 \u00a0importedModelName=imported_model_name,\n\u00a0\u00a0 \u00a0roleArn=role_arn,\n\u00a0\u00a0 \u00a0modelDataSource={\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0's3DataSource': {\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0's3Uri': s3_uri\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0}\n\u00a0\u00a0 \u00a0}\n)<\/code><\/pre>\n<\/p><\/div>\nCheck the imaginative and prescient capabilities<\/h3>\n
def generate_vl(messages, image_base64, temperature=0.3, max_tokens=4096, top_p=0.9):\n\u00a0\u00a0 \u00a0processor = AutoProcessor.from_pretrained(\"Qwen\/QVQ-72B-Preview\")\n\u00a0\u00a0 \u00a0immediate = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n\u00a0\u00a0 \u00a0\n\u00a0\u00a0 \u00a0response = consumer.invoke_model(\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0modelId=model_id,\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0physique=json.dumps({\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0'immediate': immediate,\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0'temperature': temperature,\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0'max_gen_len': max_tokens,\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0'top_p': top_p,\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0'pictures': [image_base64]\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0}),\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0settle for=\"software\/json\",\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0contentType=\"software\/json\"\n\u00a0\u00a0 \u00a0)\n\u00a0\u00a0 \u00a0\n\u00a0\u00a0 \u00a0return json.hundreds(response['body'].learn().decode('utf-8'))\n\n# Utilizing the mannequin with a picture\nfile_path = \"cat_image.jpg\"\nbase64_data = image_to_base64(file_path)\n\nmessages = [\n\u00a0\u00a0 \u00a0{\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0\"role\": \"user\",\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0\"content\": [\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0{\"image\": base64_data},\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0{\"text\": \"Describe this image.\"}\n\u00a0\u00a0 \u00a0 \u00a0 \u00a0]\n\u00a0\u00a0 \u00a0}\n]\n\nresponse = generate_vl(messages, base64_data)\n\n# Print response\nprint(\"Mannequin Response:\")\nif 'selections' in response:\n\u00a0\u00a0 \u00a0print(response['choices'][0]['text'])\nelif 'outputs' in response:\n\u00a0\u00a0 \u00a0print(response['outputs'][0]['text'])\nelse:\n\u00a0\u00a0 \u00a0print(response)\n\u00a0\u00a0\u00a0\u00a0<\/code><\/pre>\n<\/p><\/div>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ML-18505-image-10.jpeg\")
<\/p>\nThis picture includes a close-up of a cat mendacity down on a gentle, textured floor, probably a sofa or a mattress. The cat has a tabby coat with a mixture of darkish and lightweight brown fur, and its eyes are a hanging inexperienced with vertical pupils, giving it a fascinating look. The cat's whiskers are distinguished and lengthen outward from its face, including to the detailed texture of the picture. The background is softly blurred, suggesting a comfortable indoor setting with some furnishings and probably a window letting in pure gentle. The general environment of the picture is heat and serene, highlighting the cat's relaxed and content material demeanor. <\/code><\/pre>\n<\/p><\/div>\nPricing<\/h2>\n
Clear up<\/h2>\n
\n
Conclusion<\/h2>\n
![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/Ajit.png\")
Ajit Mahareddy<\/strong> is an skilled\u00a0Product and Go-To-Market (GTM) chief\u00a0with over 20 years of expertise in\u00a0Product Administration, Engineering, and Go-To-Market. Previous to his present position, Ajit led product administration constructing\u00a0AI\/ML merchandise\u00a0at main expertise corporations, together with Uber, Turing, and eHealth. He’s keen about\u00a0advancing Generative AI applied sciences\u00a0and\u00a0driving real-world affect with Generative AI.<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2023\/10\/26\/Shreyas-Subramanian-100.jpg\")
Shreyas Subramanian<\/strong> is a Principal Knowledge Scientist and helps prospects through the use of generative AI and deep studying to resolve their enterprise challenges utilizing AWS providers. Shreyas has a background in large-scale optimization and ML and in using ML and reinforcement studying for accelerating optimization duties.<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2024\/06\/16\/yanyan.png\")
Yanyan Zhang<\/strong>\u00a0is a Senior Generative AI Knowledge Scientist at Amazon Internet Companies, the place she has been engaged on cutting-edge AI\/ML applied sciences as a Generative AI Specialist, serving to prospects use generative AI to realize their desired outcomes. Yanyan graduated from Texas A&M College with a PhD in Electrical Engineering. Outdoors of labor, she loves touring, understanding, and exploring new issues.<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/dharini.jpeg\")
Dharinee Gupta<\/strong> is an Engineering Supervisor at AWS Bedrock, the place she focuses on enabling prospects to seamlessly make the most of open supply fashions by way of serverless options. Her staff focuses on optimizing these fashions to ship the most effective cost-performance stability for purchasers. Previous to her present position, she gained intensive expertise in authentication and authorization methods at Amazon, creating safe entry options for Amazon choices. Dharinee is keen about making superior AI applied sciences accessible and environment friendly for AWS prospects.<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2025\/06\/13\/ravi.png\")
Lokeshwaran Ravi<\/strong> is a Senior Deep Studying Compiler Engineer at AWS, specializing in ML optimization, mannequin acceleration, and AI safety. He focuses on enhancing effectivity, lowering prices, and constructing safe ecosystems to democratize AI applied sciences, making cutting-edge ML accessible and impactful throughout industries.<\/p>\n![\"\"](\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2022\/11\/15\/June-Won-.jpg\")