|
In the present day, we’re asserting the supply of AI21 Labs’ highly effective new Jamba 1.5 household of huge language fashions (LLMs) in Amazon Bedrock. These fashions symbolize a big development in long-context language capabilities, delivering pace, effectivity, and efficiency throughout a variety of functions. The Jamba 1.5 household of fashions consists of Jamba 1.5 Mini and Jamba 1.5 Massive. Each fashions help a 256K token context window, structured JSON output, operate calling, and are able to digesting doc objects.
AI21 Labs is a pacesetter in constructing basis fashions and synthetic intelligence (AI) programs for the enterprise. Collectively, AI21 Labs and AWS are empowering prospects throughout industries to construct, deploy, and scale generative AI functions that resolve real-world challenges and spark innovation by a strategic collaboration. With AI21 Labs’ superior, production-ready fashions along with Amazon’s devoted providers and highly effective infrastructure, prospects can leverage LLMs in a safe surroundings to form the way forward for how we course of info, talk, and study.
What’s Jamba 1.5?
Jamba 1.5 fashions leverage a singular hybrid structure that mixes the transformer mannequin structure with Structured State Area mannequin (SSM) expertise. This progressive strategy permits Jamba 1.5 fashions to deal with lengthy context home windows as much as 256K tokens, whereas sustaining the high-performance traits of conventional transformer fashions. You’ll be able to study extra about this hybrid SSM/transformer structure within the Jamba: A Hybrid Transformer-Mamba Language Mannequin whitepaper.
Now you can use two new Jamba 1.5 fashions from AI21 in Amazon Bedrock:
- Jamba 1.5 Massive excels at complicated reasoning duties throughout all immediate lengths, making it superb for functions that require prime quality outputs on each lengthy and brief inputs.
- Jamba 1.5 Mini is optimized for low-latency processing of lengthy prompts, enabling quick evaluation of prolonged paperwork and information.
Key strengths of the Jamba 1.5 fashions embody:
- Lengthy context dealing with – With 256K token context size, Jamba 1.5 fashions can enhance the standard of enterprise functions, resembling prolonged doc summarization and evaluation, in addition to agentic and RAG workflows.
- Multilingual – Help for English, Spanish, French, Portuguese, Italian, Dutch, German, Arabic, and Hebrew.
- Developer-friendly – Native help for structured JSON output, operate calling, and able to digesting doc objects.
- Velocity and effectivity – AI21 measured the efficiency of Jamba 1.5 fashions and shared that the fashions reveal as much as 2.5X quicker inference on lengthy contexts than different fashions of comparable sizes. For detailed efficiency outcomes, go to the Jamba mannequin household announcement on the AI21 web site.
Get began with Jamba 1.5 fashions in Amazon Bedrock
To get began with the brand new Jamba 1.5 fashions, go to the Amazon Bedrock console, select Mannequin entry on the underside left pane, and request entry to Jamba 1.5 Mini or Jamba 1.5 Massive.
To check the Jamba 1.5 fashions within the Amazon Bedrock console, select the Textual content or Chat playground within the left menu pane. Then, select Choose mannequin and choose AI21 because the class and Jamba 1.5 Mini or Jamba 1.5 Massive because the mannequin.
By selecting View API request, you will get a code instance of how one can invoke the mannequin utilizing the AWS Command Line Interface (AWS CLI) with the present instance immediate.
You’ll be able to comply with the code examples within the Amazon Bedrock documentation to entry obtainable fashions utilizing AWS SDKs and to construct your functions utilizing varied programming languages.
The next Python code instance exhibits how one can ship a textual content message to Jamba 1.5 fashions utilizing the Amazon Bedrock Converse API for textual content technology.
import boto3
from botocore.exceptions import ClientError
# Create a Bedrock Runtime consumer.
bedrock_runtime = boto3.consumer("bedrock-runtime", region_name="us-east-1")
# Set the mannequin ID.
# modelId = "ai21.jamba-1-5-mini-v1:0"
model_id = "ai21.jamba-1-5-large-v1:0"
# Begin a dialog with the person message.
user_message = "What are 3 enjoyable details about mambas?"
dialog = [
{
"role": "user",
"content": [{"text": user_message}],
}
]
strive:
# Ship the message to the mannequin, utilizing a primary inference configuration.
response = bedrock_runtime.converse(
modelId=model_id,
messages=dialog,
inferenceConfig={"maxTokens": 200, "temperature": 0.4, "topP": 1},
)
# Extract and print the response textual content.
response_text = response["output"]["message"]["content"][0]["text"]
print(response_text)
besides (ClientError, Exception) as e:
print(f"ERROR: Cannot invoke '{model_id}'. Motive: {e}")
exit(1)The Jamba 1.5 fashions are good to be used instances like paired doc evaluation, compliance evaluation, and query answering for lengthy paperwork. They’ll simply examine info throughout a number of sources, verify if passages meet particular pointers, and deal with very lengthy or complicated paperwork. You’ll find instance code within the AI21-on-AWS GitHub repo. To study extra about how one can immediate Jamba fashions successfully, take a look at AI21’s documentation.
Now obtainable
AI21 Labs’ Jamba 1.5 household of fashions is mostly obtainable in the present day in Amazon Bedrock within the US East (N. Virginia) AWS Area. Test the full Area record for future updates. To study extra, take a look at the AI21 Labs in Amazon Bedrock product web page and pricing web page.
Give Jamba 1.5 fashions a strive within the Amazon Bedrock console in the present day and ship suggestions to AWS re:Publish for Amazon Bedrock or by your ordinary AWS Help contacts.
Go to our group.aws website to seek out deep-dive technical content material and to find how our Builder communities are utilizing Amazon Bedrock of their options.
— Antje
September 25, 2024 – Up to date screenshot and code instance with optimized inference parameter settings.
