Introducing AI21 Labs Jamba 1.5 Large and Jamba 1.5 Mini on Azure AI Models-as-a-Service

Yina Arenas Thank you for announcing these models. It is certainly exciting to have these available. I am looking forward to taking advantage of their speed and large context lengths. Unfortunately, neither the Jamba 1.5 Large nor Mini have the correct max_token configuration for their APIs. Instead of 256k, they currently are limited to 4k. 

  max_tokens 4096, 

If you attempt a number greater than 4k you will receive an error:

 Error processing Jamba API response: Error: Jamba API request failed: 422 - "Invalid input" "message" "Input should be less than or equal to 4096" "status" 422

Thanks for sharing and I look forward to using them when the API is corrected.