Microsoft Developer Community Blog

Options
6,892
SumitGhosh on Mar 04 2024 09:00 AM
8,931
Jamesdld23 on Feb 29 2024 11:30 PM
1,381
AaronStark on Feb 29 2024 03:01 PM
36.1K
DhruvChand on Feb 20 2024 09:57 AM
5,111
Asir-Selvasingh on Feb 15 2024 08:45 AM
2,168
AaronStark on Feb 13 2024 11:35 AM
7,982
SumitGhosh on Feb 12 2024 09:00 AM
1,612
SumitGhosh on Feb 05 2024 09:00 AM
2,276
Andreas Helland on Feb 01 2024 11:30 AM
1,393
Jamesdld23 on Jan 31 2024 11:45 PM
1,863
rickydiep on Jan 26 2024 06:23 AM
2,076
KofiForson on Jan 24 2024 04:39 PM
2,198
Andreas Helland on Jan 22 2024 10:18 AM
2,384
Andreas Helland on Jan 10 2024 09:22 AM
3,562
stephaneeyskens on Jan 06 2024 12:24 AM
2,746
Andreas Helland on Jan 03 2024 10:37 AM
35.8K
MeganNesbitt on Jan 02 2024 07:53 AM
1,999
yizha1 on Dec 14 2023 03:46 PM
2,421
Jeffrey Chilberto on Dec 07 2023 07:11 PM
2,745
Andreas Helland on Dec 05 2023 12:44 PM
1,821
carolinauribe on Dec 05 2023 06:00 AM
2,361
DavidHernandez on Dec 04 2023 12:21 AM
4,088
Jamesdld23 on Dec 01 2023 12:09 PM
2,316
FeynmanZhou on Nov 30 2023 12:07 AM
4,568
AaronStark on Nov 27 2023 03:17 PM
9,253
sagar_lankala on Nov 15 2023 08:00 AM
3,011
greenie-msft on Nov 15 2023 08:00 AM
1,715
rickydiep on Nov 15 2023 08:00 AM
3,479
neelaythaker on Nov 13 2023 10:48 AM

Latest Comments

kinfey, your script above will start downloading 2 shards, like this: model.safetensors.index.json: 0%| | 0.00/16.3k [00:00<?, ?B/s] model.safetensors.index.json: 100%|██████████| 16.3k/16.3k [00:00<?, ?B/s]Downloading shards: 0%| | 0/2 [00:00<?, ?it/s]model-00001-of-00002.safetensors: 0%| | 0.00/4....
0 Likes
I execute the command:wasmedge --dir .:. --nn-preload default:GGML:AUTO: Phi-3-mini-4k-instruct-q4.gguf llama-api-server.wasm -p phi-3-chat but the command say: unknown option nn-preload
0 Likes
please follow hugging face guideline microsoft/Phi-3-mini-128k-instruct · Hugging Face import torch from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline torch.random.manual_seed(0) model = AutoModelForCausalLM.from_pretrained( "microsoft/Phi-3-mini-128k-instruct", device_map="cuda"...
0 Likes
I have downloaded phi 3 mini 128k instruct model in 2 safetensors files using model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-128k-instruct",device_map="cuda",torch_dtype="auto",trust_remote_code=True,). How to merge those files for inference (mayne merge is not needed). Just how ...
0 Likes