Skip to content

Commit f07de13

Browse files
authored
Add guidance for speed benchmark (#4537)
* add a guidance for benchmark speed * add training time guidance * fix typos * fix url * fix type
1 parent 186e391 commit f07de13

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

docs/model_zoo.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,10 @@ We also train [Faster R-CNN](https://github.com/open-mmlab/mmdetection/blob/mast
193193

194194
## Speed benchmark
195195

196+
### Training Speed benchmark
197+
198+
We provide [analyze_logs.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/analysis_tools/analyze_logs.py) to get average time of iteration in training. You can find examples in [Log Analysis](https://mmdetection.readthedocs.io/en/latest/useful_tools.html#log-analysis).
199+
196200
We compare the training speed of Mask R-CNN with some other popular frameworks (The data is copied from [detectron2](https://github.com/facebookresearch/detectron2/blob/master/docs/notes/benchmarks.md)).
197201
For mmdetection, we benchmark with [mask_rcnn_r50_caffe_fpn_poly_1x_coco_v1.py](https://github.com/open-mmlab/mmdetection/blob/master/configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_poly_1x_coco_v1.py), which should have the same setting with [mask_rcnn_R_50_FPN_noaug_1x.yaml](https://github.com/facebookresearch/detectron2/blob/master/configs/Detectron1-Comparisons/mask_rcnn_R_50_FPN_noaug_1x.yaml) of detectron2.
198202
We also provide the [checkpoint](http://download.openmmlab.com/mmdetection/v2.0/benchmark/mask_rcnn_r50_caffe_fpn_poly_1x_coco_no_aug/mask_rcnn_r50_caffe_fpn_poly_1x_coco_no_aug_compare_20200518-10127928.pth) and [training log](http://download.openmmlab.com/mmdetection/v2.0/benchmark/mask_rcnn_r50_caffe_fpn_poly_1x_coco_no_aug/mask_rcnn_r50_caffe_fpn_poly_1x_coco_no_aug_20200518_105755.log.json) for reference. The throughput is computed as the average throughput in iterations 100-500 to skip GPU warmup time.
@@ -207,6 +211,17 @@ We also provide the [checkpoint](http://download.openmmlab.com/mmdetection/v2.0/
207211
| [Detectron](https://github.com/facebookresearch/Detectron) | 19 |
208212
| [matterport/Mask_RCNN](https://github.com/matterport/Mask_RCNN/) | 14 |
209213

214+
### Inference Speed Benchmark
215+
216+
We provide [benchmark.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/analysis_tools/benchmark.py) to benchmark the inference latency.
217+
The script benchmarkes the model with 2000 images and calculates the average time ignoring first 5 times. You can change the output log interval (defaults: 50) by setting `LOG-INTERVAL`.
218+
219+
```shell
220+
python toools/benchmark.py ${CONFIG} ${CHECKPOINT} [--log-interval $[LOG-INTERVAL]] [--fuse-conv-bn]
221+
```
222+
223+
The latency of all models in our model zoo is benchmarked without setting `fuse-conv-bn`, you can get a lower latency by setting it.
224+
210225
## Comparison with Detectron2
211226
212227
We compare mmdetection with [Detectron2](https://github.com/facebookresearch/detectron2.git) in terms of speed and performance.

0 commit comments

Comments
 (0)