Qwen3-1.7B LiteRT-LM Model

This repository contains LiteRT-LM variants of Qwen/Qwen3-1.7B optimized for on-device text generation.

Available Artifact

File	Quantization Recipe	Context	Size
Qwen3_1.7B.litertlm	dynamic_wi8_afp32	-	2.1 GB
Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm	dynamic_wi4b32_afp32	4096	932 MB

Conversion Notes

Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm is a dynamic INT4 variant (block-32 weights, FP32 activations). It was converted through the LiteRT Torch (litert-torch) path and quantized with AI Edge Quantizer. This artifact incorporates LiteRT-LM GPU graph optimizations, including composite ops for RoPE, fused QKV, and fused Gate/Up projections, and is configured with static prefill memory allocation.

How to Use

Command-Line Interface

1. Install the prerequisites:

   pip install litert-lm

2. Run the command in CLI:

   litert-lm run --from-huggingface-repo=litert-community/Qwen3-1.7B Qwen3_1.7B.litertlm --prompt="Write me a poem on nature"

Edge Gallery App

1. Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
2. Importing the Model: Navigate to the Model manager within the app and click the "+" (plus) icon in the bottom-right corner. Two options will appear:
   • Import from HF (Recommended): Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired .litertlm model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
   • From local model file: First, download the .litertlm model directly to your Android device, OR download it to your computer and push it via ADB (e.g., adb push Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm /sdcard/Download/). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap "Import".

For full details on importing models and other features, see the Edge Gallery App Wiki.

To build the demo app from source, please follow the instructions from the GitHub repository.

Python

1. Install the prerequisites:

   pip install litert-lm huggingface_hub

2. Download the model file:

   from huggingface_hub import hf_hub_download

   model_path = hf_hub_download(
       repo_id="litert-community/Qwen3-1.7B",
       filename="Qwen3_1.7B.litertlm"
   )

3. Run inference:

   import litert_lm

   litert_lm.set_min_log_severity(litert_lm.LogSeverity.ERROR)  # Hide log for TUI app

   with litert_lm.Engine(model_path) as engine:
     with engine.create_conversation() as conversation:
       while True:
         user_input = input("\n>>> ")
         for chunk in conversation.send_message_async(user_input):
           print(chunk["content"][0]["text"], end="", flush=True)

Integration

Ready to integrate this into your product? Get started in the LiteRT-LM documentation.