Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions docs/EasterEgg/Android/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ tar -xvzf android-studio-2024.1.1.12-linux.tar.gz
cd android-studio
wget https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip?hl=zh-cn
unzip commandlinetools-linux-11076708_latest.zip\?hl\=zh-cn
export JAVA_HOME=/root/Downloads/android-studio/jbr
export JAVA_HOME=/root/android/android-studio/jbr
cmdline-tools/bin/sdkmanager "ndk;27.0.12077973" "cmake;3.22.1" "platforms;android-34" "build-tools;33.0.1" --sdk_root='sdk'
```

Expand All @@ -39,24 +39,22 @@ cmdline-tools/bin/sdkmanager "ndk;27.0.12077973" "cmake;3.22.1" "platforms;andr
. "$HOME/.cargo/env"
export ANDROID_NDK=/root/android/android-studio/sdk/ndk/27.0.12077973
export TVM_NDK_CC=$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang
export JAVA_HOME=/root/android//android-studio/jbr
export JAVA_HOME=/root/android/android-studio/jbr
export ANDROID_HOME=/root/android/android-studio/sdk
export PATH=/usr/local/cuda-12/bin:$PATH
export PATH=/root/android/android-studio/sdk/cmake/3.22.1/bin:$PATH
```
## 2 转换模型
### 2.1 安装mlc-llm
参考[https://llm.mlc.ai/docs/install/mlc_llm.html](https://llm.mlc.ai/docs/install/mlc_llm.html)
(如果下载很慢可以取消重新运行一下,或者本地下载了拷过去)
参考[https://llm.mlc.ai/docs/install/mlc_llm.html](https://llm.mlc.ai/docs/install/mlc_llm.html),安装`mlc-llm`可能**需要代理**

安装`pytorch`部分也可以使用其他包含`torch`的`conda`环境
```
conda create --name mlc-prebuilt python=3.11
conda activate mlc-prebuilt
conda install -c conda-forge git-lfs
pip install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 transformers sentencepiece protobuf
wget https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_llm_nightly_cu122-0.1.dev1445-cp311-cp311-manylinux_2_28_x86_64.whl
wget https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_ai_nightly_cu122-0.15.dev404-cp311-cp311-manylinux_2_28_x86_64.whl
pip install mlc_ai_nightly_cu122-0.15.dev404-cp311-cp311-manylinux_2_28_x86_64.whl
pip install mlc_llm_nightly_cu122-0.1.dev1445-cp311-cp311-manylinux_2_28_x86_64.whl
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
python -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly-cu122 mlc-ai-nightly-cu122
```
测试如下输出说明安装正确

Expand All @@ -72,17 +70,24 @@ cd mlc-llm
git submodule update --init --recursive
```

### 2.2 转换参数
### 2.2 (可选)转换参数
(如果不想上传到huggingface可以跳过这一步,有公开上传的)

使用 `mlc_llm` 的 `convert_weight` 对模型参数进行转换和量化,转换后的参数可以跨平台使用

```
mkdir -p /root/models/
ln -s /share/new_models/Shanghai_AI_Laboratory/internlm2_5-1_8b-chat /root/models/internlm2_5-1_8b-chat
cd android/MLCChat
export TVM_SOURCE_DIR=/root/android/mlc-llm/3rdparty/tvm
export MLC_LLM_SOURCE_DIR=/root/android/mlc-llm
mlc_llm convert_weight /root/models/internlm2_5-1_8b-chat/ \
--quantization q4f16_1 \
-o dist/internlm2_5-1_8b-chat-q4f16_1-MLC
```
### 2.3 生成配置
### 2.3 (可选)生成配置
(如果不想上传到huggingface可以跳过这一步,有公开上传的)

使用 `mlc_llm` 的 `gen_config` 生成 `mlc-chat-config.json` 并处理 `tokenizer`

出现提示时输入`y`
Expand All @@ -94,12 +99,16 @@ mlc_llm gen_config /root/models/internlm2_5-1_8b-chat/ \
-o dist/internlm2_5-1_8b-chat-q4f16_1-MLC
Do you wish to run the custom code? [y/N] y
```
### 2.4 上传到huggingface
上传这一步需要能访问huggingface,可能需要部署代理
如果没有代理可以直接在接下来的配置中使用如下链接的模型(和文档中的转换方法一样)
### 2.4 (可选)上传到huggingface
上传这一步需要能访问huggingface,可能需要**部署代理**并耗费一定流量

具体方法可以参考网上的大量教程

如果不想上传到huggingface可以跳过这一步,直接在接下来的配置中使用如下链接的模型(和文档中的转换方法一样)
[https://huggingface.co/timws/internlm2_5-1_8b-chat-q4f16_1-MLC](https://huggingface.co/timws/internlm2_5-1_8b-chat-q4f16_1-MLC)
### 2.5 (可选) 测试转换的模型
### 2.5 (可选) 测试转换的模型
在打包之前可以测试模型效果,需要编译成二进制文件

在个人电脑上运行测试代码正常,**InternStudio**上**暂未成功**
```

Expand Down