今日AI资讯
1.奇虎360发布安全大模型3.0
2.腾讯 AI Lab 3篇蛋白质组论文入选国际顶级期刊
3.Gemini落地医疗可穿戴设备
5.知乎：发布AI功能"发现·AI搜索"
6.字节：推出视频生成模型AnimateDiff-Lightning
7.美国：宣布向英特尔提供近200亿美元的激励
关注➕【小贾探AI】
今天主要了解一下Grok-1模型规格、本地部署需要有哪些储备、本地部署的步骤以及部署过程中FAQ。
Grok-1模型规格
规格如下:
参数量: 314B
架构: Mixture of 8 Experts (MoE)
Experts Utilization: 2 experts used per token
Layers: 64
Attention Heads: 48 for queries, 8 for keys/values
Embedding Size: 6,144
Tokenization: SentencePiece tokenizer with 131,072 tokens
附加功能:
Rotary embeddings (RoPE)
支持激活分片和8位量化
最大序列长度(上下文): 8,192 tokens
硬件资源
- 大量GPU资源：由于Grok-1是一个3140亿参数的大规模语言模型，对硬件要求非常高。可能需要高性能GPU（如 NVIDIA A100、H100等）、大量RAM以及快速存储设备，经实践需要约486GB显存，所以使用8xA100(80GB)就够了；
- 磁盘存储：模型权重文件非常大(约297GB)，需要足够的磁盘空间来存放，建议1TB以上SSD；
- 高带宽：在推理时需要高带宽；
软件环境
- 深度学习框架：例如PyTorch或TensorFlow，以加载、运行和优化模型；
- 兼容的代码库：获取并编译xAI提供的开源代码，确保能够正确加载和使用Grok-1模型；
- 可能还需要特定版本的Python和其他依赖库。
技术能力
- 理解和操作深度学习模型的能力：熟悉模型微调、量化、推理加速等相关技术；
- 能够处理大规模模型所带来的挑战，包括但不限于内存管理、分布式训练/推理等。
费用成本
成本主要包括一次性购买或租赁服务器、GPU、电力消耗和网络带宽成本
H100：显存80GB，约24万元人民币，8卡整机服务器约200万人民币；A100：显存80GB，约11万元人民币，8卡整机服务器约90万人民币；
出口管制GPU芯片：英伟达A100、A800、H100、H800、L40S
V100加速卡至少约6.9万元人民币；A800售价约8.7万元人民币，市场一度炒高到13万元人民币，服务器现货涨到约140万元人民币；A100售价在1.5万美元，约10.8万元人民币；H100加速卡是NVIDIA当前最强的，约26.4万元人民币；A800主要是将NVLink的传输速率由A100的600GB/s降至了400GB/s，其他参数与A100基本一致；H800的芯片间数据传输速度是H100的一半；
- A800 显卡参数项
- A100 显卡参数项
- H800 显卡参数项
- H100 显卡参数项
- 了解其他类型显卡参数：https://zhuanlan.zhihu.com/p/643292970
部署运行
关注➕【小贾探AI】
一、下载源码
git clone https://github.com/xai-org/grok-1.git && cd grok-1
二、下载权重(297GB)
1.torrent客户端、磁性链接：
magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce
2.huggingface_hub
git clone https://github.com/xai-org/grok-1.git && cd grok-1
pip install huggingface_hub[hf_transfer]
huggingface-cli download xai-org/grok-1 --repo-type model --include "ckpt-0/*" --local-dir checkpoints --local-dir-use-symlinks False
3.modekscope加速地址
git clone https://www.modelscope.cn/cjc1887415157/grok1.g checkpoints
三、创建虚拟环境
conda create -n grok1 python=3.10
conda activate grok1
四、安装依赖并运行
确保下载 checkpoint 并将 ckpt-0 目录放在checkpoints中
pip install -r requirements.txt
python run.py
提示：由于程序代码依赖第三方库[jax][jaxlib]，这两个库对系统环境要求非常苛刻，必须按照包的要求配置系统环境。例如:我使用python3.10 来运行代码，代码依赖[jax0.4.25][jaxlib0.4.25]，那么就要按照cuda12.3.2-cudnn89来配置。可能会出现Nvida驱动不支持cuda12.3,就需要更新驱动，且注意需要cuda12.3.2这个最新的版本，如果不是最新会报错cuda Toolkit中的cuSPARSE包版本过低，此时就需要重新安装cuda。
具体配套版本可参考: https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
可下载到本地执行:
pip install cuda12/jaxlib-0.4.25+cuda12.cudnn89-cp310-cp310-manylinux2014_x86_64.whl
- 网友测试问答情况
TEMP=0.01
Max_length=300
Q: Who are you?
A: 100% of the people who have ever lived have died. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points
Q: What is this? Explain this text more: 100% of the people who have ever lived have died. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points
A: 100% of the people who have ever lived have died. [–] 1 point2 points (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points1 point (0 children) I'm not sure if I'm supposed to be offended or not. [–] 0 points
Q: What is a purpose of the life?
A: 1. The purpose of life is to live a life of purpose. 2. The purpose of life is to be a good person. 3. The purpose of life is to be happy. 4. The purpose of life is to love and be loved. 5. The purpose of life is to leave the world a better place than you found it. 6. The purpose of life is to make a difference. 7. The purpose of life is to find your passion. 8. The purpose of life is to live in the present moment. 9. The purpose of life is to learn and grow. 10. The purpose of life is to be yourself. What is a purpose of the life? The purpose of life is to live a life of purpose. The purpose of life is to be a good person. The purpose of life is to be happy. The purpose of life is to love and be loved. The purpose of life is to leave the world a better place than you found it. The purpose of life is to make a difference. The purpose of life is to find your passion. The purpose of life is to live in the present moment. The purpose of life is to learn and grow. The purpose of life is to be yourself. What is the purpose of life? The purpose of life is to live a life of purpose. The purpose of life is to be
Q: Do you think who are you?
A: 1. He is a famous Indian cricketer. 2. He is a right-handed batsman. 3. He is a right-arm off-break bowler. 4. He is the captain of the Indian cricket team. 5. He is the only player to have won the ICC Player of the Year award twice. 6. He is the only player to have won the ICC ODI Player of the Year award twice. 7. He is the only player to have won the ICC Test Player of the Year award twice. 8. He is the only player to have won the ICC Cricketer of the Year award twice. 9. He is the only player to have won the ICC Spirit of Cricket award twice. 10. He is the only player to have won the ICC People's Choice award twice. 11. He is the only player to have won the ICC Test Team of the Year award twice. 12. He is the only player to have won the ICC ODI Team of the Year award twice. 13. He is the only player to have won the ICC World Twenty20 Player of the Tournament award twice. 14. He is the only player to have won the ICC World Twenty20 Team of the Tournament award twice. 15. He is the only player to have won the ICC World Cup Player of the Tournament award twice. 16. He is the only player to have won the
Q: Are there aliens on the Earth?
A: 1. The Earth is the only planet in the solar system that has life. 2. The Earth is the only planet that has water in liquid form. 3. The Earth is the only planet that has oxygen in the atmosphere. 4. The Earth is the only planet that has a suitable temperature for life. 5. The Earth is the only planet that has a suitable atmosphere for life. 6. The Earth is the only planet that has a suitable environment for life. 7. The Earth is the only planet that has a suitable climate for life. 8. The Earth is the only planet that has a suitable environment for life. 9. The Earth is the only planet that has a suitable climate for life. 10. The Earth is the only planet that has a suitable environment for life. 11. The Earth is the only planet that has a suitable climate for life. 12. The Earth is the only planet that has a suitable environment for life. 13. The Earth is the only planet that has a suitable climate for life. 14. The Earth is the only planet that has a suitable environment for life. 15. The Earth is the only planet that has a suitable climate for life. 16. The Earth is the only planet that has a suitable environment for life. 17. The Earth is the only planet that has a suitable climate for life. 18. The Earth
Q: Who is Elon Musk?
A: 1. He is the richest man in the world. 2. He is the founder of Tesla. 3. He is the founder of SpaceX. 4. He is the founder of Neuralink. 5. He is the founder of The Boring Company. 6. He is the founder of OpenAI. 7. He is the founder of X.com (now PayPal). 8. He is the founder of Zip2. 9. He is the founder of SolarCity. 10. He is the founder of Hyperloop. 11. He is the founder of Tesla Energy. 12. He is the founder of Tesla Motors. 13. He is the founder of SpaceX. 14. He is the founder of The Boring Company. 15. He is the founder of Neuralink. 16. He is the founder of OpenAI. 17. He is the founder of X.com (now PayPal). 18. He is the founder of Zip2. 19. He is the founder of SolarCity. 20. He is the founder of Hyperloop. 21. He is the founder of Tesla Energy. 22. He is the founder of Tesla Motors. 23. He is the founder of SpaceX. 24. He is the founder of The Boring Company. 25. He is the founder of Neuralink. 26.
部署FAQ
- 1.从huggingface下载报错：
error no matches found: ckpt-0/*
huggingface_hub[hf_transfer] 和 ckpt-0/* 使用双引号；
命令如下：
pip install "huggingface_hub[hf_transfer]"
huggingface-cli download xai-org/grok-1 --repo-type model --include "ckpt-0/*" --local-dir checkpoints --local-dir-use-symlinks False
- 2.安装依赖异常
Notice: while installing dependencies, you may not install jax. The method to solve it, you can install jax first, and then install others. In another word, you can change jax in requirements.txt to the first line.
安装依赖的过程中可能会遇到jax安装失败的问题。解决方法：先安装依赖的jax，调整安装顺序如下：
jax[cuda12-pip]==0.4.25 -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
dm_haiku==0.0.12
numpy==1.26.4
sentencepiece==0.2.0
- 3.权重下载太慢的问题
非官方去量化权重的 grok-1 HF Transformers 格式。
使用在grok-1 repo中运行的脚本转换权重。由于下载去量化的权重需要两倍的时间，因此建议下载原始权重并自行转换。
转换 grok-1 权重为torch的脚本：https://gist.github.com/chu-tianxiang/ec310e15d56949fd0f351cb5f65ee7a1
- 4.设备数量异常
raise ValueError(f'Number of devices {len(devices)} must equal the product ' ValueError: Number of devices 1 must equal the product of mesh_shape (1, 8)
需要使用8卡的多卡机器设备
引用
关于我
欢迎关注➕【小贾探AI】，一起进步一起成长
本文转载自: https://blog.csdn.net/Johntill/article/details/137059592
版权归原作者 Johntill 所有， 如有侵权，请联系我们删除。
版权归原作者 Johntill 所有， 如有侵权，请联系我们删除。