PRfire项目--大模型农业分析

本地LLM：Ovis-4B/Ovis-2B

环境

宿主系统：windows11

为私有化部署，使用Docker进行环境管理

从头安装

docker pull pytorch/pytorch:2.4.0-cuda12.4-cudnn9-devel
docker run -d  --privileged=true -p 5595:5595 -p 7897:7897 --name PRfire --shm-size 200G --ulimit memlock=-1 --gpus=all -it pytorch/pytorch:2.4.0-cuda12.4-cudnn9-devel /bin/bash
docker attach PRfire

apt update
apt upgrade
apt install git

git clone https://github.com/Puiching-Memory/PRfire.git
cd PRfire
git submodule init
git submodule update

mkdir model
export HF_ENDPOINT=https://hf-mirror.com
pip install huggingface_hub
huggingface-cli download --resume-download AIDC-AI/Ovis2-4B --local-dir ./model/

cd Ovis
pip install -r requirements.txt
pip install -e . -v
pip install flash-attn==2.7.0.post2 --no-build-isolation -v

export GRADIO_SERVER_NAME=0.0.0.0
python ovis/serve/server.py --model_path /workspace/PRfire/model/ --port 5595

使用方法

1. 载入Docker镜像tar包

   docker run -d  --privileged=true -p 5595:5595 -p 7897:7897 --name PRfire --shm-size 200G --ulimit memlock=-1 --gpus=all -it PRfire-Ovis:latest /bin/bash
   

2. 启动Ovis+gradio

   cd /workspace/PRfire/Ovis/
   python ovis/serve/server.py --model_path /workspace/PRfire/model/ --port 5595
   

API

本地浏览器访问：http://0.0.0.0:5595

python

见ovis_run.py

Javascript

npm i -D @gradio/client

import { Client } from "@gradio/client";

const response_0 = await fetch("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png");
const exampleImage = await response_0.blob();
			
const response_1 = await fetch("undefined");
const exampleVideo = await response_1.blob();
			
const client = await Client.connect("http://0.0.0.0:5595/");
const result = await client.predict("/predict", { 
		type: "Image", 
				image: exampleImage, 
				video: exampleVideo, 
		text: "Hello!!", 
});

console.log(result.data);

参数项	类型	选项	注释
type	string	必填	“Image”,"Vedio","TextOnly"
image	Blob/File/Buffer	必填	可选项：null
video	any	必填	可选项：null
text	string	必填

TODO