This put up was written with NVIDIA and the authors wish to thank Adi Margolin, Eliuth Triana, and Maryam Motamedi for his or her collaboration.<\/em><\/p>\n

Organizations in the present day face the problem of processing massive volumes of audio information\u2013from buyer calls and assembly recordings to podcasts and voice messages\u2013to unlock invaluable insights. Computerized Speech Recognition (ASR) is a crucial first step on this course of, changing speech to textual content in order that additional evaluation could be carried out. Nevertheless, working ASR at scale is computationally intensive and could be costly. That is the place asynchronous inference on Amazon SageMaker AI is available in. By deploying state-of-the-art ASR fashions (like NVIDIA Parakeet fashions<\/a>) on SageMaker AI with asynchronous endpoints, you may deal with massive audio recordsdata and batch workloads effectively. With asynchronous inference, long-running requests could be processed within the background (with outcomes delivered later); it additionally helps auto-scaling to zero when there\u2019s no work and handles spikes in demand with out blocking different jobs.<\/p>\n

On this weblog put up, we\u2019ll discover host the NVIDIA Parakeet ASR mannequin on SageMaker AI and combine it into an asynchronous pipeline for scalable audio processing. We\u2019ll additionally spotlight the advantages of Parakeet\u2019s structure and the NVIDIA Riva toolkit for speech AI, and focus on use NVIDIA NIM for deployment on AWS.<\/p>\n

NVIDIA speech AI applied sciences: Parakeet ASR and Riva Framework<\/h2>\n
NVIDIA provides a complete suite of speech AI applied sciences, combining high-performance fashions with environment friendly deployment options. At its core, the Parakeet ASR mannequin household represents state-of-the-art speech recognition capabilities, attaining industry-leading accuracy with low phrase error charges (WERs) <\/a>. The mannequin\u2019s structure makes use of the Quick Conformer encoder with the CTC or transducer decoder, enabling 2.4\u00d7 sooner processing than normal Conformers whereas sustaining accuracy.<\/p>\n
NVIDIA speech NIM is a set of GPU-accelerated microservices for constructing customizable speech AI purposes. NVIDIA Speech fashions ship correct transcription accuracy and pure, expressive voices in over 36 languages\u2013best for customer support, contact facilities, accessibility, and world enterprise workflows. Builders can fine-tune and customise fashions for particular languages, accents, domains, and vocabularies, supporting accuracy and model voice alignment.<\/p>\n
Seamless integration with LLMs and the NVIDIA Nemo Retriever<\/a> make NVIDIA fashions best for agentic AI purposes, serving to your group stand out with safer, high-performing, voice AI. The NIM framework delivers these companies as containerized options, making deployment easy by Docker containers that embrace the required dependencies and optimizations.<\/p>\n
This mixture of high-performance fashions and deployment instruments offers organizations with an entire answer for implementing speech recognition at scale.<\/p>\n
Answer overview<\/strong><\/h2>\n
The structure illustrated within the diagram showcases a complete asynchronous inference pipeline designed particularly for ASR and summarization workloads. The answer offers a strong, scalable, and cost-effective processing pipeline.<\/p>\n
$\"\"$ <\/p>\n
Structure elements<\/strong><\/h3>\n
The structure consists of 5 key elements working collectively to create an environment friendly audio processing pipeline. At its core, the SageMaker AI asynchronous endpoint hosts the Parakeet ASR mannequin with auto scaling capabilities that may scale to zero when idle for price optimization.<\/p>\n
\n
The information ingestion course of begins when audio recordsdata are uploaded to Amazon Easy Storage Service (Amazon S3), triggering AWS Lambda capabilities that course of metadata and provoke the workflow.<\/li>\n
For occasion processing, the SageMaker endpoint routinely sends out Amazon Easy Notification Service (Amazon SNS) success and failure notifications by separate queues, enabling correct dealing with of transcriptions.<\/li>\n
Efficiently transcribed content material on Amazon S3 strikes to Amazon Bedrock LLMs for clever summarization and extra processing like classification and insights extraction.<\/li>\n
Lastly, a complete monitoring system utilizing Amazon DynamoDB shops workflow standing and metadata, enabling real-time monitoring and analytics of your complete pipeline.<\/li>\n<\/ol>\n
Detailed implementation walkthrough<\/strong><\/h2>\n
On this part, we’ll present the detailed walkthrough of the answer implementation.<\/p>\n
SageMaker asynchronous endpoint conditions<\/strong><\/h3>\nTo run the instance notebooks, you want an AWS account with an AWS Id and Entry Administration<\/a> (IAM) function with least-privilege permissions<\/a> to handle sources created. For particulars, confer with Create an AWS account<\/a>. You may must request a service quota improve for the corresponding SageMaker async internet hosting situations. On this instance, we’d like one ml.g5.xlarge SageMaker async internet hosting occasion and a ml.g5.xlarge SageMaker pocket book occasion. You may also select a distinct built-in growth setting (IDE), however make certain the setting comprises GPU compute sources for native testing.<\/p>\n
SageMaker asynchronous endpoint configuration<\/strong><\/h3>\n
While you deploy a customized mannequin like Parakeet, SageMaker has a few choices:<\/p>\n
\n
Use a NIM container offered by NVIDIA<\/li>\n
Use a big mannequin inference (LMI) container<\/li>\n
Use a prebuilt PyTorch container<\/li>\n<\/ul>\n
We\u2019ll present examples for all three approaches.<\/p>\n
Utilizing an NVIDIA NIM container<\/strong><\/h4>\nNVIDIA NIM<\/a> offers a streamlined method to deploying optimized AI fashions by containerized options. Our implementation takes this idea additional by making a unified SageMaker AI endpoint that intelligently routes between HTTP and gRPC protocols to assist maximize each efficiency and capabilities whereas simplifying the deployment course of.<\/p>\n
Revolutionary dual-protocol structure<\/strong><\/p>\n
The important thing innovation is the mixed HTTP + gRPC structure that exposes a single SageMaker AI endpoint with clever routing capabilities. This design addresses the widespread problem of selecting between protocol effectivity and have completeness by routinely deciding on the optimum transport methodology. The HTTP route is optimized for easy transcription duties with recordsdata below 5MB, offering sooner processing and decrease latency for widespread use circumstances. In the meantime, the gRPC route helps bigger recordsdata (SageMaker AI real-time endpoints assist a max payload of 25MB) and superior options like speaker diarization with exact word-level timing data. The system\u2019s auto-routing performance analyzes incoming requests to find out file measurement and requested options, then routinely selects essentially the most acceptable protocol with out requiring guide configuration. For purposes that want specific management, the endpoint additionally helps pressured routing by \/invocations\/http<\/strong> for easy transcription or \/invocations\/grpc<\/strong> when speaker diarization is required. This flexibility permits each automated optimization and fine-grained management based mostly on particular software necessities.<\/p>\n
Superior speech recognition and speaker diarization capabilities<\/strong><\/p>\n
The NIM container allows a complete audio processing pipeline that seamlessly combines speech recognition with speaker identification<\/a> by the NVIDIA Riva built-in capabilities. The container handles audio preprocessing, together with format conversion and segmentation, whereas ASR and speaker diarization processes run concurrently on the identical audio stream. Outcomes are routinely aligned utilizing overlapping time segments, with every transcribed section receiving acceptable speaker labels (for instance, Speaker_0, Speaker_1). The inference handler processes audio recordsdata by the entire pipeline, initializing each ASR and speaker diarization companies, working them in parallel, and aligning transcription segments with speaker labels. The output contains the complete transcription, timestamped segments with speaker attribution, confidence scores, and complete speaker rely in a structured JSON format.<\/p>\n
Implementation and deployment<\/strong><\/p>\n
The implementation extends NVIDIA parakeet-1-1b-ctc-en-us NIM container<\/a> as the inspiration, including a Python aiohttp server that seamlessly manages the entire NIM lifecycle by routinely beginning and monitoring the service. The server handles protocol adaptation by translating SageMaker inference requests to acceptable NIM APIs, implements the clever routing logic that analyzes request traits, and offers complete error dealing with with detailed error messages and fallback mechanisms for sturdy manufacturing deployment. The containerized answer streamlines deployment by normal Docker and AWS CLI instructions, that includes a pre-configured Docker file with the required dependencies and optimizations. The system accepts a number of enter codecs together with multipart form-data (advisable for optimum compatibility), JSON with base64 encoding for easy integration eventualities, and uncooked binary uploads for direct audio processing.<\/p>\n
For detailed implementation directions and dealing examples, groups can reference the full implementation and deployment pocket book<\/a> within the AWS samples repository, which offers complete steering on deploying Parakeet ASR with NIM on SageMaker AI<\/a> utilizing the convey your individual container (BYOC) method. For organizations with particular architectural preferences, separate HTTP-only and gRPC-only implementations are additionally accessible, offering less complicated deployment fashions for groups with well-defined use circumstances whereas the mixed implementation provides most flexibility and automated optimization.<\/p>\n
AWS clients can deploy these fashions both as production-grade NVIDIA NIM containers immediately from SageMaker Market or JumpStart, or open supply NVIDIA fashions accessible on Hugging Face<\/a>, which could be deployed by customized containers on SageMaker or Amazon Elastic Kubernetes Service (Amazon EKS). This permits organizations to decide on between absolutely managed, enterprise-tier endpoints with auto-scaling and safety, or versatile open-source growth for analysis or constrained use circumstances.<\/p>\n
Utilizing an AWS LMI container<\/strong><\/h4>\n
LMI containers are designed to simplify internet hosting massive fashions on AWS. These containers embrace optimized inference engines like vLLM, FasterTransformer, or TensorRT-LLM that may routinely deal with issues like mannequin parallelism, quantization, and batching for giant fashions. The LMI container is basically a pre-configured Docker picture that runs an inference server (for instance a Python server with these optimizations) and means that you can specify mannequin parameters by utilizing setting variables.<\/p>\n
To make use of the LMI container for Parakeet<\/a>, we might usually:<\/p>\n
\n
Select the suitable LMI picture<\/strong>: AWS offers totally different LMI pictures for various frameworks. For Parakeet , we would use the DJLServing picture for environment friendly inference. Alternatively, NVIDIA Triton Inference Server (which Riva makes use of) is an possibility if we package deal the mannequin in ONNX or TensorRT format.<\/li>\n
Specify the mannequin configuration<\/strong>: With LMI, we regularly present a model_id (if pulling from Hugging Face Hub) or a path to our mannequin, together with configuration for load it (variety of GPUs, tensor parallel diploma, quantization bits). The container then downloads the mannequin and initializes it with the desired settings. We are able to additionally obtain our personal mannequin recordsdata from Amazon S3 as an alternative of utilizing the Hub.<\/li>\n
Outline the inference handler<\/strong>: The LMI container may require a small handler script or configuration to inform it course of requests. For ASR, this may contain studying the audio enter, passing it to the mannequin, and returning textual content.<\/li>\n<\/ol>\n
AWS LMI containers ship excessive efficiency and scalability by superior optimization methods, together with steady batching, tensor parallelism, and state-of-the-art quantization strategies. LMI containers combine a number of inference backends (vLLM, TensorRT-LLM by a single unified configuration), serving to customers seamlessly experiment and swap between frameworks to search out the optimum efficiency stack in your particular use case.<\/p>\n
Utilizing a SageMaker PyTorch container<\/strong><\/h4>\nSageMaker provides PyTorch Deep Studying Containers (DLCs)<\/a> that include PyTorch and lots of widespread libraries pre-installed. In this instance<\/a>, we demonstrated lengthen our prebuilt container to put in crucial packages for the mannequin. You may obtain the mannequin immediately from Hugging Face in the course of the endpoint creation or obtain the Parakeet mannequin artifacts, packaging it with crucial configuration recordsdata right into a mannequin.tar.gz archive, and importing it to Amazon S3. Together with the mannequin artifacts, an inference.py script is required because the entry level script to outline mannequin loading and inference logic, together with audio preprocessing and transcription dealing with. When utilizing the SageMaker Python SDK to create a PyTorchModel, the SDK will routinely repackage the mannequin archive to incorporate the inference script below \/choose\/ml\/mannequin\/code\/inference.py, whereas maintaining mannequin artifacts in \/choose\/ml\/mannequin\/ on the endpoint. As soon as the endpoint is deployed efficiently, it may be invoked by the predict API by sending audio recordsdata as byte streams to get transcription outcomes.<\/p>\n
For the SageMaker real-time endpoint, we presently permit a most of 25MB for payload measurement. Be sure to have arrange the container to additionally permit the utmost request measurement. Nevertheless, if you’re planning to make use of the identical mannequin for the asynchronous endpoint, the utmost file measurement that the async endpoint helps is 1GB and the response time is as much as 1 hour. Accordingly, you need to setup the container to be ready for this payload measurement and timeout. When utilizing the PyTorch containers, listed below are some key configuration parameters to think about:<\/p>\n
\n
SAGEMAKER_MODEL_SERVER_WORKERS<\/strong>: Set the variety of torch staff that can load the variety of fashions copied into GPU reminiscence.<\/li>\n
TS_DEFAULT_RESPONSE_TIMEOUT<\/strong>: Set the day trip setting for Torch server staff; for lengthy audio processing, you may set it to the next quantity<\/li>\n
TS_MAX_REQUEST_SIZE<\/strong>: Set the byte measurement values for requests to 1G for async endpoints.<\/li>\n
TS_MAX_RESPONSE_SIZE<\/strong>: Set the byte measurement values for response.<\/li>\n<\/ul>\n
Within the instance pocket book, we additionally showcase leverage the SageMaker native session offered by the SageMaker Python SDK. It helps you create estimators and run coaching, processing, and inference jobs regionally utilizing Docker containers as an alternative of managed AWS infrastructure, offering a quick technique to take a look at and debug your machine studying scripts earlier than scaling to manufacturing.<\/p>\n
CDK pipeline conditions<\/strong><\/h3>\n
Earlier than deploying this answer, be sure to have:<\/p>\n
\n
AWS CLI configured<\/strong> with acceptable permissions \u2013 Set up Information<\/a><\/li>\n
AWS Cloud Improvement Package (AWS CDK) put in<\/strong> \u2013 Set up Information<\/a><\/li>\n
Node.js 18+<\/strong> and Python 3.9+<\/strong> put in<\/li>\n
Docker \u2013 Set up Information<\/strong><\/a><\/li>\n
SageMaker endpoint<\/strong> deployed along with your ML mannequin (Parakeet ASR fashions or related)<\/li>\n
Amazon SNS matters<\/strong> created for achievement and failure notifications<\/li>\n<\/ol>\n
CDK pipeline setup<\/strong><\/h3>\n
The answer deployment begins with provisioning the required AWS sources utilizing Infrastructure as Code (IaC) rules. AWS CDK creates the foundational elements together with:<\/p>\n
\n
DynamoDB Desk<\/strong>: Configured for on-demand capability to trace invocation metadata, processing standing, and outcomes<\/li>\n
S3 Buckets<\/strong>: Safe storage for enter audio recordsdata, transcription outputs, and summarization outcomes<\/li>\n
SNS matters<\/strong>: Separate queues for achievement and failure occasion dealing with<\/li>\n
Lambda capabilities<\/strong>: Serverless capabilities for metadata processing, standing updates, and workflow orchestration<\/li>\n
IAM roles and insurance policies<\/strong>: Applicable permissions for cross-service communication and useful resource entry<\/li>\n<\/ul>\n
Atmosphere setup<\/h4>\n
Clone the repository and set up dependencies:<\/p>\n
\n
# Set up degit, a library for downloading particular sub directories\nnpm set up -g degit\n\n# Clone simply the precise folder\nnpx degit aws-samples\/genai-ml-platform-examples\/infrastructure\/automated-speech-recognition-async-pipeline-sagemaker-ai\/sagemaker-async-batch-inference-cdk sagemaker-async-batch-inference-cdk\n\n# Navigate to folder\ncd sagemaker-async-batch-inference-cdk\n\n# Set up Node.js dependencies\nnpm set up\n\n# Arrange Python digital setting\npython3 -m venv .venv\nsupply .venv\/bin\/activate\n\n# On Home windows:\n.venvScriptsactivate\npip set up -r necessities.txt<\/code><\/pre>\n<\/p><\/div>\nConfiguration<\/h4>\nReplace the SageMaker endpoint configuration in bin\/aws-blog-sagemaker.ts:<\/p>\n \n\nvim bin\/aws-blog-sagemaker.ts \n\n# Change the endpoint identify \nsageMakerConfig: { \n endpointName: 'your-sagemaker-endpoint-name', \n enableSageMakerAccess: true \n}<\/code><\/pre>\n<\/p><\/div><\/div>\nYou probably have adopted the pocket book to deploy the endpoint, you need to have created the 2 SNS matters. In any other case, be sure to create the proper SNS matters utilizing CLI:<\/p>\n \n# Create SNS matters\naws sns create-topic --name success-inf\naws sns create-topic --name failed-inf<\/code><\/pre>\n<\/p><\/div>\nConstruct and deploy<\/h4>\nEarlier than you deploy the AWS CloudFormation template, make certain Docker is working.<\/p>\n \n# Compile TypeScript to JavaScript\nnpm run construct\n\n# Bootstrap CDK (first time solely)\nnpx cdk bootstrap\n\n# Deploy the stack\nnpx cdk deploy<\/code><\/pre>\n<\/p><\/div>\nConfirm deployment<\/h4>\nAfter profitable deployment, observe the output values:<\/p>\n \nDynamoDB desk identify for standing monitoring<\/li>\n Lambda operate ARNs for processing and standing updates<\/li>\n SNS subject ARNs for notifications<\/li>\n<\/ul>\nSubmit audio file for processing<\/strong><\/h3>\nProcessing Audio Recordsdata<\/h4>\nReplace the upload_audio_invoke_lambda.sh<\/code><\/p>\n \nLAMBDA_ARN=\"YOUR_LAMBDA_FUNCTION_ARN\"\nS3_BUCKET=\"YOUR_S3_BUCKET_ARN\"<\/code><\/pre>\n<\/p><\/div>\nRun the Script:<\/p>\n AWS_PROFILE=default .\/scripts\/upload_audio_invoke_lambda.sh<\/code><\/p>\n This script will:<\/p>\n \nObtain a pattern audio file<\/li>\n Add the audio file to your s3 bucket<\/li>\n Ship the bucket path to Lambda and set off the transcription and summarization pipeline<\/li>\n<\/ul>\nMonitoring progress<\/h4>\nYou may test the lead to DynamoDB desk utilizing the next command:<\/p>\n \naws dynamodb scan --table-name YOUR_DYNAMODB_TABLE_NAME<\/code><\/pre>\n<\/p><\/div>\nVerify processing standing within the DynamoDB desk:<\/p>\n \nsubmitted<\/strong>: Efficiently queued for inference<\/li>\n accomplished<\/strong>: Transcription accomplished efficiently<\/li>\n failed<\/strong>: Processing encountered an error<\/li>\n<\/ul>\nAudio processing and workflow orchestration<\/strong><\/h3>\nThe core processing workflow follows an event-driven sample:<\/p>\n Preliminary processing and metadata extraction:<\/strong> When audio recordsdata are uploaded to S3, the triggered Lambda operate analyzes the file metadata, validates format compatibility, and creates detailed invocation information in DynamoDB. This facilitates complete monitoring from the second audio content material enters the system.<\/p>\n Asynchronous Speech Recognition:<\/strong> Audio recordsdata are processed by the SageMaker endpoint utilizing optimized ASR fashions. The asynchronous course of can deal with varied file sizes and durations with out timeout issues. Every processing request is assigned a novel identifier for monitoring functions.<\/p>\n Success path processing:<\/strong> Upon profitable transcription, the system routinely initiates the summarization workflow. The transcribed textual content is shipped to Amazon Bedrock, the place superior language fashions generate contextually acceptable summaries based mostly on configurable parameters equivalent to abstract size, focus areas, and output format.<\/p>\n Error dealing with and restoration:<\/strong> Failed processing makes an attempt set off devoted Lambda capabilities that log detailed error data, replace processing standing, and may provoke retry logic for transient failures. This sturdy error dealing with ends in minimal information loss and offers clear visibility into processing points.<\/p>\n Actual-world purposes<\/strong><\/h2>\nCustomer support analytics:<\/strong> Organizations can course of 1000’s of customer support name recordings to generate transcriptions and summaries, enabling sentiment evaluation, high quality assurance, and insights extraction at scale.<\/p>\n Assembly and convention processing:<\/strong> Enterprise groups can routinely transcribe and summarize assembly recordings, creating searchable archives and actionable summaries for contributors and stakeholders.<\/p>\n Media and content material processing:<\/strong> Media corporations can course of podcast episodes, interviews, and video content material to generate transcriptions and summaries for improved accessibility and content material discoverability.<\/p>\n Compliance and authorized documentation:<\/strong> Authorized and compliance groups can course of recorded depositions, hearings, and interviews to create correct transcriptions and summaries for case preparation and documentation.<\/p>\n Cleanup<\/strong><\/h2>\nUpon getting used the answer, take away the SageMaker endpoints to stop incurring further prices. You need to use the offered code to delete real-time and asynchronous inference endpoints, respectively:<\/p>\n \n# Delete real-time inference\nendpointreal_time_predictor.delete_endpoint()\n\n# Delete asynchronous inference\nendpointasync_predictor.delete_endpoint()<\/code><\/pre>\n<\/p><\/div>\nYou also needs to delete all of the sources created by the CDK stack.<\/p>\n \n# Delete CDK Stack\ncdk destroy<\/code><\/pre>\n<\/p><\/div>\nConclusion<\/strong><\/h2>\nThe mixing of highly effective NVIDIA speech AI applied sciences with AWS cloud infrastructure creates a complete answer for large-scale audio processing. By combining Parakeet ASR\u2019s industry-leading accuracy and velocity with NVIDIA Riva\u2019s optimized deployment framework on the Amazon SageMaker asynchronous inference pipeline, organizations can obtain each high-performance speech recognition and cost-effective scaling. The answer leverages the managed companies of AWS (SageMaker AI, Lambda, S3, and Bedrock) to create an automatic, scalable pipeline for processing audio content material. With options like auto scaling to zero, complete error dealing with, and real-time monitoring by DynamoDB, organizations can give attention to extracting enterprise worth from their audio content material slightly than managing infrastructure complexity. Whether or not processing customer support calls, assembly recordings, or media content material, this structure delivers dependable, environment friendly, and cost-effective audio processing capabilities. To expertise the complete potential of this answer, we encourage you to discover the answer and attain out to us if in case you have any particular enterprise necessities and wish to customise the answer in your use case.<\/p>\n\n Concerning the authors<\/h3>\nMelanie Li, PhD,<\/strong> is a Senior Generative AI Specialist Options Architect at AWS based mostly in Sydney, Australia, the place her focus is on working with clients to construct options utilizing state-of-the-art AI\/ML instruments. She has been actively concerned in a number of generative AI initiatives throughout APJ, harnessing the ability of LLMs. Previous to becoming a member of AWS, Dr. Li held information science roles within the monetary and retail industries.<\/p>\n <\/strong>Tony Trinh<\/strong> is a Senior AI\/ML Specialist Architect at AWS. With 13+ years of expertise within the IT {industry}, Tony focuses on architecting scalable, compliance-driven AI and ML options\u2014significantly in generative AI, MLOps, and cloud-native information platforms. As a part of his PhD, he\u2019s doing analysis in Multimodal AI and Spatial AI. In his spare time, Tony enjoys mountaineering, swimming and experimenting with residence enchancment.<\/p>\n Alick Wong<\/strong> is a Senior Options Architect at Amazon Internet Companies, the place he helps startups and digital-native companies modernize, optimize, and scale their platforms within the cloud. Drawing on his expertise as a former startup CTO, he works intently with founders and engineering leaders to drive development and innovation on AWS.<\/p>\n <\/strong>Andrew Smith<\/strong> is a Sr. Cloud Assist Engineer within the SageMaker, Imaginative and prescient & Different staff at AWS, based mostly in Sydney, Australia. He helps clients utilizing many AI\/ML companies on AWS with experience in working with Amazon SageMaker. Outdoors of labor, he enjoys spending time with family and friends in addition to studying about totally different applied sciences.<\/p>\n Derrick Choo<\/strong> is a Senior AI\/ML Specialist Options Architect at AWS who accelerates enterprise digital transformation by cloud adoption, AI\/ML, and generative AI options. He focuses on full-stack growth and ML, designing end-to-end options spanning frontend interfaces, IoT purposes, information integrations, and ML fashions, with a selected give attention to pc imaginative and prescient and multi-modal methods.<\/p>\n <\/strong>Tim Ma<\/strong> is a Principal Specialist in Generative AI at AWS, the place he collaborates with clients to design and deploy cutting-edge machine studying options. He additionally leads go-to-market methods for generative AI companies, serving to organizations harness the potential of superior AI applied sciences.<\/p>\n Curt Lockhart<\/strong> is an AI Options Architect at NVIDIA, the place he helps clients deploy language and imaginative and prescient fashions to construct finish to finish AI workflows utilizing NVIDIA\u2019s tooling on AWS. He enjoys making advanced AI really feel approachable and spending his time exploring the artwork, music, and outside of the Pacific Northwest.<\/p>\n Francesco Ciannella <\/strong>is a senior engineer at NVIDIA, the place he works on conversational AI options constructed round massive language fashions (LLMs) and audio language fashions (ALMs). He holds a M.S. in engineering of telecommunications from the College of Rome \u201cLa Sapienza\u201d and an M.S. in language applied sciences from the College of Laptop Science at Carnegie Mellon College. <\/strong><\/p>\n \n <\/div>\n\n","protected":false},"excerpt":{"rendered":" This put up was written with NVIDIA and the authors wish to thank Adi Margolin, Eliuth Triana, and Maryam Motamedi for his or her collaboration. Organizations in the present day face the problem of processing massive volumes of audio information\u2013from buyer calls and assembly recordings to podcasts and voice messages\u2013to unlock invaluable insights. Computerized Speech […]<\/p>\n","protected":false},"author":2,"featured_media":8225,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[387,6170,1047,266,6168,192,6169,388,1233],"class_list":["post-8223","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-amazon","tag-asr","tag-hosting","tag-models","tag-nim","tag-nvidia","tag-parakeet","tag-sagemaker","tag-speech"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/8223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8223"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/8223\/revisions"}],"predecessor-version":[{"id":8224,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/8223\/revisions\/8224"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/8225"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8223"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}